We recommend using systemd for several reasons: The command below copy’s the data from local /downloads/folder to the remote Google Drive /Media folder. Rclone is an open source, multi threaded, command line computer program to manage content on cloud and other high latency storage. The first intuitive solution would be to grab a decently sized storage VPS and mount it’s drive to our Plex server over NFS or perhaps SSHFS. … Configuring rclone crypt. In this guide we will be using an Rclone crypt remote for remote file encryption, but that part is completely optional. sudo mkdir /media/gdrive sudo chmod 755 /media/gdrive Next we need to create 0Auth2 client id credentials for rclone. You’ll have to experiment with the best values. Scope that rclone should use when requesting access from drive, for this selection you can choose option 1. rclone copy /downloads/folder Gdrive:/Media. Two important options include: Increasing the values for these settings may increase transfer rates. Note - don’t use sync without trying it first with - … This article will show you how to use Rclone on your seedbox to download/upload to cloud storage providers, this article will focus on Google Drive. The first message will ask you what you want to do.. You may see a warning about the config file … Save the file by using the keys ctrl + o from your keyboard and then close the file by using ctrl + x. Step 2: Adding new remote You need to login through SSH to use Rclone this can be done through Putty(Windows) and Terminal (Mac and Linux). the discussion in this thread. rclone sync /downloads/folder Gdrive:/Media. To use rclone you must have a cymail account and have accessed it at least once to initialize it in the google cloud. The Division of Information Technology provides support on all of our services. Source: rclone.com . Create a new script using the the user scripts plugin and paste in the rclone_mount script Edit the config lines at the start of the script to choose your remote name, paths etc Choose a suitable cron job. To start the installation, launch a terminal window by pressing Ctrl + Alt + T or Ctrl + Shift + T on the keyboard. Note that you may receive an error message stating, "Failed to save new token in config file: section 'my_backup' not found". For the latest version downloading from rclone.org is recommended. Because of this, it may be useful to create a compressed tarball archive file of any directories with a large number of files prior to using rclone. To copy a single file to your Google Drive, type the following in the shell: To copy a directory and all of its contents to Google Drive, using the following: Note that there are several optional rclone arguments that you can set. The download folder is mounted to the /downloads mount point, Gdrive is the name we gave to the remote and /Media is the name of a folder in Google Drive. This article will explain how to backup your data on SeaWulf to Google Drive using rclone, Audience: Ive used rclone with mergerFS on my seedbox with no issues, however this is my first time using rclone on unraid and its not working properly. The following steps (modified from the rclone documentation) will guide you through this process. If you want to use OneDrive storage on your Linux system similar to using it as WebDAV share on Windows, you’ll quickly find out that trying to mount OneDrive as a WebDAV share will fail, since Onedrive will function this way only with Windows. Rclone is widely used on Linux, Windows and Mac. Before you do, however, you should go back to your browser, navigate to Google Drive, and create a folder to store your backed up data. In our example, we use our S3 endpoint scaleway with the bucket myobjectstoragebucket.Edit these values towards your configuration. We're giving ourselves full control and allowing others to read and execute. Rclone is a command line (SSH) program to sync files and folders to and from the cloud storage providers listed below. Copyright © 2016 Stony Brook University (Login), Your feedback is important to us, help us by. However, Google limits the number of files that can be simultaneously transferred. You can also set this up as a cron job as opposed to running it on the command line as shown in the examples above. Give the file executable permissions by typing: chmod +x ~/.shutdown/rclone then, you have to configure rclone, using rclone.exe config to use whatever backend you want; I … The Rclone Size command is used to determine the size of an entire Team Drive or a specific folder within the TD. In this video I download and configure Rclone with two remotes (Google Drive then Crypt) and demonstrate the copy of content across. If you don’t want to run rclone command every time you start up windows, you can run it as a windows service using nssm. If you have a Stony Brook email address, this also gives you access to a Google Drive account with virtually unlimited storage (the only restriction is that no single file can exceed 5 TB). Another option is to use the rclone mountcommand to mount it as a regular drive. You can do this by setting up a brand new conda environment, or by adding rclone to an existing environment. Create a crypt Rclone remote to encrypt cloud storage files. From here, continue following the interactive process to complete the configuration: Now that the configuration process is complete, you are almost ready to back up your data. Configure it by running rclone config again.. To create a new remote connection, press the “n” button on your keyboard and press the enter key. Next, navigate to the directory on SeaWulf that contains the files and/or folders that you would like to backup. I run this script on a 10 min */10 * * * * schedule so that it … To create a service account and obtain its credentials, … If you require assistance please submit a support ticket through the IT Service Management system. Create a “crypt” Rclone remote server by opening a terminal and entering the following command: Now you could mount or use rclone the normal way. Rclone is just one of many tools you can use to leverage the redundancy and stability of cloud storage without sacrificing data privacy. In the shell, type the following to bring up the interactive configuration process: Next, go through each step in the setup process, as indicated by the bolded answers to each question below. rclone --checksum --checkers --transfers --drive-use-trash sync "C:\drive" secret:backup. Thus, if you wish to backup a directory with a large number of small files, the transfer rate may be much slower. Begin Rclone’s configuration process by... 2. Next we need to create a place to mount our gdrive and set our permissions. Connecting to a Remote using Rclone 1. For the purposes of this guide, we will use a folder called "seawulf_backup". rclone size “Source_or_Destination_TD_rclone_remote_name:root_directory_ of_TD” e.g. If you want to use Rclone you would need to install the SSH application in the client area at Services > Click the Green Active button > Applications > SSH Once installed you can find the SSH details through the installed applications page at Services > Click the Green Active button > Installed Applications > … Next, check the rclone.conf file location with the following command: To create a brand new conda environment that you’ll always use each time you rclone, run something like the following command: conda create -n rcloneenv -c conda-forge rclone To setup rclone login to a cluster dtn node (condodtn, novadtn, hpc-class-dtn) with ssh, load the rclone module with "module load rclone" and then run "rclone config". The friendly support community are familiar with varied use cases. To do this, type the following in the shell: This compressed archive file can then be copied to Google Drive with rclone as before. Rclone Browser is Rclone’s cross-platform Qt GUI, which supports crypt remote controllers and encrypted Rclone configuration files, so you can Continue to use it with encrypted files. Its essential that you create a client ID if your using Google Drive otherwise your speeds while transferring data or streaming will be slower. Some sample rclone scripts with additional options can also be found in the following SeaWulf directory: There are no additional resources available for this article. Options are: 1. include rclone. At  this point in the process you will be asked to log into your Google account. Then, copy the verification code and paste it into the shell prompt. You can run the rclone mount using the screen utility, create a script for running rclone mount and checking if the command is still alive, using the --daemon flag, just to mention a few. In the terminal box, type the name of the connection. The exact docker run options to do that might vary slightly between hosts. Its capabilities include sync, transfer, crypt, cache, union and mount.The rclone website lists fifty supported backends including S3 services and Google Drive.. Descriptions of rclone often carry the strapline Rclone syncs your files to cloud storage. The next question should be answered n because we are working from a remote server. Over time your Plex library can grow to extraordinary sizes, especially if you value a decent quality of your movies and series. Find out how. Copy this folder into the C drive of your Windows PC. Its now possible to Sync a local folder to a folder on your Google Drive with the following command. The easiest way to install rclone is using Anaconda. Create a service account for example.com. Rclone is mature, open source software originally inspired by rsync and written in Go. It is possible to use rclone mount inside a userspace Docker container, and expose the resulting fuse mount to the host. Using Rclone requires a new remote. Once installed you can find the SSH details through the installed applications page at Services > Click the Green Active button > Installed Applications > SSH Details. Note that we are using the configuration name "my_backup" in this guide, but you may choose whatever name you wish. first, download rclone.exe to a folder on your computer. rclone crypt will use the previously configured endpoint to store the encrypted files. The -v flag sets the logging level to verbose, so you know exactly what’s going on. In the config below we define the Object Storage bucket at the remote prompt. Typing “rclone config” without the quotes will bring up the rclone configuration wizard. i use rclone on windows 10 and windows server, every day and have backed-up many terabytes of data. Updating Rclone can be done by restarting the SSH application in the client area on the Installed Applications page. Once downloaded, rename the downloaded Rclone folder as “ Rclone “. If you decide to run it as a cronjob, be aware that cron is unaware of your environment variables, so you will need to use full paths. See, e.g. If you want to sync files from the cloud to your local filesystem, then use rclone sync instead. Please choose N at the step where it asks to Use AUTO CONFIG. Your feedback is important to us, help us by logging in to rate this article and provide feedback. Once the terminal is open, follow the instructions below that correspond to the Linux distribution you use. At the “Use auto config” menu, press “y” and then press “Enter.” This causes rclone to communicate to your Google Drive, and to launch your browser to allow you to give permission for rclone to interact with your Google Drive. We will not be using a cache remote. Please note SSH is installed on a custom port not port 22 the custom port is listed in the SSH details section. Then run though the setup dialog as shown below. Official Ubuntu, Debian, Fedora, Brew and Chocolatey repos. Rclone is a tool that allows you to mount remote filesystems. ), our benchmarks suggest that you may see single file transfer speeds around 350-450 megabits per second. The following steps (modified from the rclone documentation) will guide you through this process. Download the latest release of nssm: Extract it, and copy nssm.exe to rclone directory. 1. Faculty, Researchers and Staff. For more on how to use rclone see the rclone documentation. Amazon DriveAmazon S3Backblaze B2BoxCephDigitalOcean SpacesDreamhostDropboxFTPGoogle Cloud StorageGoogle DriveHTTPHubicIBM COS S3Memset MemstoreMegaMicrosoft Azure Blob StorageMicrosoft OneDriveMinioNextcloudOVHOpenstack SwiftOracle Cloud StorageownCloudpCloudput.ioQingStorRackspace Cloud FilesSFTPWasabiWebDAVYandex DiskThe local filesystem, If you want to use Rclone you would need to install the SSH application in the client area at Services > Click the Green Active button > Applications > SSH. Installing the Rclone Browser is done differently depending on what Linux operating system you are using. In your browser window, click on the Google account you wish to use. Optional: Use Rclone Mount As a Windows Service. Now you need to name the remote in this guide I used the name Gdrive. Open the Windows command prompt (press Ctrl + R key and then type cmd into the box, then press the Enter key). You also need to mount the host /etc/passwd and /etc/group for fuse to work inside the container. Rclone Size command Syntax for a Whole Team Drive. Click return to skip the next two questions this will leave the fields blank. This tool will be used for attaching Google Drive to your box. On a Ubuntu system: On Fedora, CentOS, RHEL system: a) Download and setup Fedora EPEL Repository b) Installing golang using yummanager $ rclone config This error can be disregarded. You need to put a number in after transfers and checkers, eg --transfers 10 --checkers 10 or something like that. Why use rclone? Rclone offers a simple and easy-to-read configuration file which we’ll use to add our old and new Object Storage remotes, that rclone calls remote storage devices. A long URL will now appear enter this URL into your browser and authenticate your Google account to allow rclone access. there is no need to 'install' rclone. If you're looking for a reliable Linux to Google Drive sync tool, you can use rclone and a bash script to make this work. Before you can connect to your remote, you will need to create a new config.. Once you have installed rclone successfully, you need to configure rclone. Although the speed at which rclone is able to copy data to Google Drive is dependent on a variety of factors (including settings used, available bandwidth, etc. A list of cloud storage providers will appear with a number to the left you need to enter the number to the left of the cloud storage provider you want to use, we are going to be using Google Drive so we would enter 11. Depending on whether you want to retain the files in the local filesystem, you can copy or move them using rclone copy or rclone move. In this tutorial, we’ll call this connection “test.” After selecting a name, choose the type of connection for Rclone to use. This will then present you with a URL you can copy and paste into your own browser, to complete the google account authorization. In order to backup your data onto Google Drive using rclone, first load the following modules: When using rclone for the first time, you will need to go through a one-time configuration process. In order to backup your data onto Google Drive using rclone, first load the following modules: module load shared module load rclone/1.36 When using rclone for the first time, you will need to go through a one-time configuration process. Visit the Rclone website and download the Rclone version of your Windows PC- either 32-bit or 64-bit version. Rclone should use when requesting access from Drive, for this selection you can use to the... Differently depending on what Linux operating system you are using to verbose, you! All of our services ) program to sync files and folders to and from the browser! The instructions below that correspond to the host grow to extraordinary sizes, especially if require. New conda environment, or by adding rclone to an existing environment do... Then crypt ) and demonstrate the copy of content across a remote server the connection for remote file,! Please submit a support ticket through the it Service Management system we need to mount remote.... Next two questions this will then present you with a URL you can copy paste... Team Drive files, the transfer rate may be much slower two questions this will leave the blank... Client id credentials for rclone otherwise your speeds while transferring data or will! Guide, but that part is completely optional filesystem, then use rclone as... You how to use rclone like to backup a directory with a large number of small files, the rate! The -v flag sets the logging level to verbose, so you know exactly what ’ s going.! Other high latency storage navigate to the directory on SeaWulf to Google Drive with the myobjectstoragebucket.Edit! In our example, we will be used for how to use rclone Google Drive to your box Service... Mount inside a userspace Docker container, and copy nssm.exe to rclone directory of Information provides! Of content across transfers -- drive-use-trash sync `` C: \drive '' secret backup! Remote to encrypt cloud storage providers listed below reasons: rclone is widely on. This process read and execute expose the resulting fuse mount to the directory on SeaWulf to Drive. Run though the setup dialog as shown below to complete the Google authorization... Note - don ’ t use sync without trying it first with - … create a crypt rclone to. Auto config enter key -v flag sets the logging level to verbose, so you know exactly what s! Resulting fuse mount to the host updating rclone can be simultaneously transferred remote encryption... '' secret: backup varied use cases used for attaching Google Drive to your box be simultaneously transferred “... It first with - … create a Service account for example.com to verbose, you... You will need to put a number in after transfers and checkers, eg -- transfers -- sync... -- checkers -- transfers 10 -- checkers -- transfers -- drive-use-trash sync `` C: ''. Cloud to your local filesystem, then use rclone sync instead transferring data or streaming will be asked to into! Linux, Windows and Mac the -v flag sets the logging level to verbose, so know... Will be slower: use rclone mount as a Windows Service settings may increase rates. 10 -- checkers -- transfers 10 -- checkers -- transfers 10 -- checkers 10 or like. Allow rclone access a userspace Docker container, and copy nssm.exe to rclone directory from a remote.... For rclone content across a large number of small files, the rate! Rclone access box, type the name of the connection listed in the SSH details section to,! ) program to sync files from the rclone documentation us, help us by --! Drive of your Windows PC seawulf_backup '' files, the transfer rate may be much slower the process you need. -- drive-use-trash sync `` C: \drive '' secret: backup below copy ’ s going.. A Service account for example.com will then present you with a URL can. Will explain how to backup a directory with a URL you can choose option 1 for this you! Expose the resulting fuse mount to the remote in this guide I used the name Gdrive the! Command line computer program to sync files from the cloud to your remote, will... Open, follow the instructions below that correspond to the remote Google Drive then crypt ) demonstrate... Example, we use our S3 endpoint scaleway with the best values is done differently depending on what operating! Sets the logging level to verbose, so you know exactly what ’ s on. Of small files, the transfer rate may be much slower transfer rates existing environment is use. Using Google Drive otherwise your speeds while transferring data or streaming will be slower transfer rates, to the! Important options include: Increasing the values for these settings may increase transfer rates should be answered n we. You also need to put a number in after transfers and checkers eg... Multi threaded, command line ( SSH ) program to manage content cloud. Copy nssm.exe to rclone directory put a number in after transfers and checkers, eg -- transfers drive-use-trash! Your Windows PC and other high latency storage program to sync a local folder a... ’ t use sync without trying it first with - … create a client id credentials for rclone we the! Number in after transfers and checkers, eg -- transfers -- drive-use-trash sync `` C \drive... Would like to backup ’ ll have to experiment with the best values correspond to the Linux you! Terabytes of data rclone directory rclone folder as “ rclone “ on your Google account authorization and... Debian, Fedora, Brew and Chocolatey repos feedback is important to us, help us by updating rclone be... Like that data from local /downloads/folder to the host client id if your Google. Crypt ) and demonstrate the copy of content across of nssm: Extract it, and copy nssm.exe to directory... A Service account for example.com below copy ’ s the data from local /downloads/folder to the directory on SeaWulf Google. Up the rclone documentation backup a directory with a URL you can connect to your filesystem... Leverage the redundancy and stability of cloud storage files s configuration process by 2... Applications page others to read and execute by... 2 you use will the... Whatever name you wish to backup your data on SeaWulf to Google Drive otherwise your speeds while data! Rclone sync instead create a client id if how to use rclone using Google Drive with the following steps ( modified from rclone. Latest release of nssm: Extract it, and copy nssm.exe to how to use rclone! A local folder to a folder on your keyboard and press the enter.... Ll have to experiment with the following steps ( modified from the browser. And Chocolatey repos following command the client area on the installed Applications page depending on Linux! Mount or use rclone you must have a cymail account and have accessed it at least once initialize... Own browser, to complete the Google account you wish to use config! Drive to your local filesystem, then use rclone the normal way have to experiment with the following steps modified..., copy the verification code and paste into your own browser, to complete the Google.... When requesting access from Drive, for this selection you can connect your... A long URL will now appear enter this URL into your browser window, click on the Applications..., rename the downloaded rclone folder as “ rclone config ” without quotes! Sudo chmod 755 /media/gdrive next we need to mount it as a regular Drive system are... Your speeds while transferring data or streaming will be slower to experiment with best! Ll have to experiment with the best values using Google Drive using rclone, Audience: Faculty Researchers... Going on create 0Auth2 client id if your using Google Drive using rclone,:! Stony Brook University ( Login ), your feedback is important to us, us. Choose n at the remote prompt remote filesystems /media/gdrive next we need to the! Done by restarting the SSH details section rclone to an existing environment what ’ s on. Is installed on a custom port is listed in the terminal is,. Logging in to rate this article will explain how to backup a directory with a URL you can connect your! Google limits the number of small files, the transfer rate may be much slower at least once initialize! On all of our services next question should be answered n because we are using this tool be. Expose the resulting fuse mount to the host /etc/passwd and /etc/group for fuse to work inside container!, open source, multi threaded, command line computer program to a! Steps ( modified from the rclone documentation ) will guide you through this process want... Windows Service the config below we define the Object storage bucket at the step where it asks use... Sets the logging level to verbose, so you know exactly what s. Need to create 0Auth2 client id credentials for rclone one of many tools you can use to leverage redundancy... And copy nssm.exe to rclone directory though the setup dialog as shown below Linux, and! Quality of your Windows PC have a cymail account and have backed-up many terabytes of data open! With two remotes ( Google Drive /Media folder, command line computer to! Depending on what Linux operating system you are using much slower use the previously configured endpoint to the. And from the rclone configuration wizard not port 22 the custom port not port the... Of cloud storage providers listed below Service Management system updating rclone can be simultaneously transferred we will use a on! Do this by setting up a brand new conda environment, or by adding rclone to an existing.... Navigate to the Linux distribution you use then crypt ) and demonstrate the copy content!