Using Amazon Cloud Drive with Plex Media Server on Ubuntu… And Encrypting It!

UPDATED – 26 January, 2017

This guide previously made use of acd_cli for mounting Amazon Cloud Drive. It’s been updated to use rClone for both mounting and uploading, as this is not only more efficient, but also much more robust. Using rClone for mounting eliminates the need to sync/remount your ACD mount.

If you’ve already used this guide, you can simply head over to the automatic mounting script guide, update your script with the changes in the guide, and execute it, you should be good to go!

The idea of an “unlimited” Plex server is an exciting one, but there are some bases you’ll want to cover before dumping all your media into the cloud. The biggest issue with keeping your media hosted in the cloud is encryption; you don’t want any storage providers snooping around your files, regardless of their source or content. This guide will get you going, with a focus on security.

Also, with the announcement of Plex’s new “Plex Cloud” offering, I’ve seen quite a few people posting on Reddit’s Plex subreddit for help with using Amazon Cloud Drive (ACD) as a storage provider for their Plex media on their standalone Plex server. There have been some guides floating around the Internet, but some of these guides use a confusing approach (in my opinion) to accomplish a task that’s already a bit confusing for some. I’ve been using the setup detailed in this guide for a few years now, even with multiple servers utilizing the same ACD mount, and I’ve had very few issues. I’ve automated this setup with tools like CouchPotato, Sonarr, and custom scripts in conjunction with Plex Requests to the point where I rarely have to do any maintenance on my Plex servers, and anybody can request media and have it available in minutes without any kind of interaction from me.

If you haven’t set up Plex yet, or you’ve got a fresh installation of Ubuntu, you can use my other guide, Getting Started with Plex to get the prerequisite setup finished first.

In this guide, we’ll be making use of several different projects to get this setup working smoothly, so while a guide can help you get set up, troubleshooting any problems can be difficult and a guide won’t always be helpful. However, if everything is set up in a logical way to begin with, the pain of troubleshooting can eased. Let’s get started on that now!

What you’ll need:

  • Amazon Cloud Drive account (obviously)
  • User account running these tools and Plex (preferably named “Plex”)
  • unzip
  • Python 3 (installed by default on Ubuntu 14.04+)

What we’ll be installing:

How it Will Work:

Basically, we will be using rClone both to read our Amazon Cloud Drive and to write to back ACD, and EncFS to encrypt it all. The real magic happens with UnionFS, as it will merge our read and write directories into one, so we can essentially forget about those other directories and focus on a single path for all our needs, including other tools like CouchPotato and Sonarr.

I’m going to assume your user account running Plex is not root, and is called “Plex” for the sake of convenience. Substitute usernames as necessary here, and stay logged in as this user throughout the guide. We’ll use “sudo” for anything requiring elevated privileges.

Anytime “~” is specified in a directory, it is assumed to be equal to ‘/home/plex’

Step 1 – Creating Necessary Directories

Let’s create the directories we’ll be using to make all this work. For ease of use, I like to put these directories right inside my Plex user’s home directory. So let’s make sure we’re at the right place (again, you should be logged in as your Plex user, not root):

cd ~

Now that we’re in the home directory, let’s create the five directories we’ll be using:

mkdir .acd
mkdir acd
mkdir .local
mkdir local
mkdir media

These directories will be used as follows:

~/.acd – ACD mount containing encrypted media
~/acd – ACD mount decrypted with EncFS
~/.local – local media after being encrypted by EncFS
~/local – where your local/downlaoded media will be copied to
~/media – the UnionFS mount point which will merge ‘~/acd’ and ‘~/local’ directories

Step 2 – Setting up rClone

Install rclone

Start off by making sure you’re in your Downloads directory, then grabbing the rclone package:

cd ~/Downloads
wget http://downloads.rclone.org/rclone-current-linux-amd64.zip

Now, unzip the archive and change directories to it:

unzip rclone-current-linux-amd64.zip
cd rclone-*-linux-amd64

Copy the binary and give root ownership:

sudo cp rclone /usr/sbin/
sudo chown root:root /usr/sbin/rclone
sudo chmod 755 /usr/sbin/rclone

Then install the manpage so you can read all about rclone:

sudo mkdir -p /usr/local/share/man/man1
sudo cp rclone.1 /usr/local/share/man/man1/
sudo mandb

Set Up rclone

Now we’re ready to configure rclone. Once again, this process is different depending on whether you’re using a desktop session or a headless SSH session. If you’re using a terminal in a desktop session (GUI), then simply enter the commands below, otherwise scroll down a tad further to “Using SSH or a headless server”

Using a desktop interface/GUI

In your terminal, simply type the following command to enter the rclone configuration process:

rclone config

This will guide you through a fairly easy process. Let’s go through the steps together:

  • Press “n” to add a new remote connection, then “enter” to proceed
  • You’ll want to name your instance something simple, like “acd” to make it easier to upload files
  • Press “1” for ACD, then “enter” to proceed
  • Press “enter” for the next two fields to use defaults
  • Press “y” to use the “automatic” mode, which will open a browser window. Enter your ACD credentials, and log in. The config process should continue automatically.
  • Press “y” if you can see your token text on the screen
  • Press “q” to exit the config process, we’re done for now

Using SSH or a headless server

If you’re managing a headless Plex server, or you don’t have a desktop environment installed, authenticating rClone gets a bit trickier. I’ve found success using the w3m browser. So lets install that:

sudo apt-get install -y w3m

This will install a text-only terminal-based browser. It should do fine for this purpose, but don’t plan on surfing the web with it.

Now we’ll install a utility called Screen (this is included in later versions of Ubuntu). Screen allows you to have multiple terminal sessions open without worrying about them expiring or closing out. This is useful if you need to access your server remotely with Putty or something similar to execute a script or other process, but can’t have the SSH session open until that script/process is complete. Ending your SSH session also ends whatever script/process you were running. Screen is a great way to prevent that from happening, and lets you reconnect to existing sessions from any other SSH session. So let’s install Screen:

sudo apt-get install -y screen

Sometimes Screen is already installed, but either way it should be now for sure. To open a new terminal session with Screen, just type:

screen

Press “enter” to dismiss the message, and now you’re in a second terminal session. You can exit a Screen session while leaving it active at anytime by pressing “CTRL+A” then “D” to disconnect. Now that we have a second session running, let’s start the rclone config process:

rclone config

Move past these steps the same as the GUI users would have, up to the “use auto-config” step:

  • Press “n” to add a new remote connection, then “enter” to proceed
  • You’ll want to name your instance something simple, like “acd” to make it easier to upload files
  • Press “1” for ACD, then “enter” to proceed
  • Press “enter” for the next two fields to use defaults
  • PAUSE at the auto-config step. Pressing “y” to use the “automatic” mode will attempt to open a browser window. This won’t work, however it will show you a URL to use. Copy this URL, then disconnect this Screen session by pressing “CTRL+A” then “D” which will keep that session active in the background
  • Open the w3m browser to the rclone URL by entering:
w3m http://127.0.0.1:53682/auth
  • Enter your ACD credentials by moving your cursor to the required fields, pressing “enter” and typing your username/password, then pressing “enter” again
  • Place your cursor on “Sign in using our secure server” and press “enter” to sign in. You should see a “success” message
  • Press “q” to exit the w3m browser
  • Type the following command to reconnect to your previous Screen session:
screen -r
  • Press “y” if you can see your token text on the screen
  • Press “q” to exit the config process, we’re done for now
  • Press “CTRL+A” then “K” to permanently kill this Screen session

Just to make sure everything worked correctly, we can run the following command to list the remote accounts configured with rClone, and it should list “acd” under the “Name” heading and “amazon cloud drive” under the “Type” heading:

rclone listremotes

Mounting ACD with rClone

Now we’re ready to mount our Amazon Cloud Drive instance to make it accessible to the server. This is accomplished with one simple command, however you can automate this process (and the rest of this guide) using the scripts over at our other guide here! The command to mount ACD is as follows (don’t forget the ampersand at the end!):

rclone mount acd:Plex /home/plex/.acd &

This will mount your ACD instance in the ‘.acd’ directory in the plex user’s home directory. The ampersand executes the command in the background so we can continue using our terminal. Now we can move on to the encryption phase!

Step 3 – Setting up EncFS

Install EncFS

Now that we have our ACD instance mounted using rClone, let’s make sure it’s encrypted:

sudo apt-get install -y encfs

Set Up and Test Encryption

Then we’ll go ahead and get our directories encrypted. Using EncFS, the first directory in the command is the one you’ll want encrypted, and the second directory is the one you want decrypted. The nice thing about EncFS is you don’t really have to specify the action, if the first directory already contains encrypted files, the second directory will automatically decrypt them. So let’s encrypt our local media directories:

encfs /home/plex/.local /home/plex/local

You should see a prompt from EncFS regarding encryption settings. Use the default and just press “enter” to move on to password creation. Make sure you remember this password! Type it a second time, and your local directories are now encrypted. You can test this by creating the directories Plex will use in the ‘/home/plex/local’ directory. In this guide, we’ll only worry about movies and TV shows, so let’s create a directory for each:

cd ~/local
mkdir Movies
mkdir "TV Shows"

Now let’s list the ‘local’ directory to make sure they’re created:

ls ~/local

And you should see the “movie” and “tv” directories listed. Now, let’s list the encrypted directory:

ls ~/.local

You should see two directories with a bunch of random characters. This confirms our files are being encrypted.

Move Settings File

Now let’s move our EncFS settings file to a more convenient location. First we need to unmount our local media directories. This command is also used to unmount other EncFS directories in the future, though you shouldn’t need to use it very often:

fusermount -u /home/plex/local

Next, we’ll move the “encfs6.xml” file and rename it to something simpler:

mv /home/plex/.local/.encfs6.xml  /home/plex/encfs.xml

Encrypting Directories Using EncFS Configuration Files

This “encfs.xml” file is your encryption “key,” not to be confused with the password. These work in tandem to keep your files secure, so you’ll want to make sure you keep a copy of this file backed up somewhere safe. Now, when mounting an EncFS directory, you’ll need to specify the location of the config file. Anytime you mount an encrypted directory, the command will always start the same, only the directories will change as needed. Let’s go ahead and mount our local media directory again using the config file:

ENCFS6_CONFIG='/home/plex/encfs.xml' encfs /home/plex/.local /home/plex/local

Okay, now let’s do the same for our ACD directories. Enter the following into your terminal:

ENCFS6_CONFIG='/home/plex/encfs.xml' encfs /home/plex/.acd /home/plex/acd

And that’s it for encryption! It seems like a lot, but underneath it all, everything is pretty simple. Four directories, two of which are encrypted mirrors of the others, and a fifth to merge the two decrypted directories together. Let’s move on to merging those directories with UnionFS!

Step 4 – Merging directories using UnionFS-Fuse

Install UnionFS-Fuse

Once again, let’s get UnionFS-Fuse installed;

sudo apt-get install -y unionfs-fuse

Merge Directories Using UnionFS

There’s no other setup required for this part, we just need to specify which directories we want to merge. Those will be the decrypted directories, local and ACD, so Plex can be refreshed to grab new movies/episodes when they’re downloaded without waiting for them to be uploaded, and so other tools like CouchPotato and Sonarr can move files into this directory and they’ll appear to be in the same location as all the ACD media.

unionfs-fuse -o cow /home/plex/local=RW:/home/plex/acd=RO /home/plex/media/

The “RW” and “RO” arguments in that command specify which directory is writable (local directory) and which is read-only (ACD directory). So now anything you copy/move to ‘/home/plex/media’ will actually be in ‘/home/plex/local’ and thus encrypted in ‘/home/plex/.local’, and anything in your ‘/home/plex/acd’ directory will also show up in the ‘/home/plex/media’ directory. We only need to worry about one directory to point Plex libraries at, or to have CouchPotato/Sonarr monitor. This makes life much, much easier! Now, we’re ready to move on to the final phase of this guide; uploading your downloaded media to ACD.

Step 5 – Adding ACD Directories to Plex

Let’s make sure Plex can see the ACD directories now! Hop over to your Plex web portal, and add a new library (or edit an existing one). When choosing your directory, browse to ‘/home/plex/media’ and you should see the “movie” and “tv” directories we created. Choose the respective directory for your library, and that’s it! Anything you upload to ACD will show up in that directory.

If you don’t see the ‘media’ directory when trying to add it through Plex, it could be a permissions issue. One thing you can try is browsing to the ‘/home/plex/acd’ directory from Plex web and seeing if those directories show up. If they do, this is a permissions issue with the UnionFS mount. Unmount it by typing:

fusermount -u /home/plex/media

And then edit  the fuse configuration file:

sudo nano /etc/fuse.conf

Find the line “#user_allow_other” and remove the “#” by pressing the delete key in front of that line. Save the file by pressing “CTRL+X” and “Y” to overwrite. Then, mount the merged UnionFS-Fuse directory by adding the “allow_other” option to the mount command:

unionfs-fuse -o cow,allow_other /home/plex/local=RW:/home/plex/acd=RO /home/plex/media/

Now, you should be able to see the “movie” and “tv” folders inside ‘/home/plex/media’ directory when adding a library in Plex web.

Step 6 – Uploading to ACD with rclone

Now that our directories are set up and encrypted and we have them added to our Plex server, it’s time to start adding media to them.

Upload Local Files/Directories to ACD Using rclone

The syntax for copying files with rclone is “rclone copy sourcedir remote:destinationdir”. In this context, “sourcedir” is your local encrypted directory, “remote” is what you named your ACD instance when setting up rclone (I suggested acd), and “destinationdir” is the directory in your ACD you want to move those files to, which in our case would be inside the “Plex” folder you created in the ACD web portal.

Anyways, on to the meat and potatoes! To upload your local files over to your ACD mount, we’ll be uploading your encrypted local directory to the encrypted ACD mount with the following command:

rclone copy /home/plex/.local acd:Plex

This should copy the files from your local encrypted directory to your encrypted ACD directory, and then you can manually delete them from ‘home/plex/local’ if the copy was successful.

One thing I like to do is upload my movie and TV directories separately by appending those directory names to the ends of the source and destination directories in the rclone move command. You can just list your ‘/home/plex/.local’ directory to get the names of the directories, and you can figure out which is which by using the “touch” command to place a test file in either directory, then listing the encrypted directories to see which has a file in it. Keep note of those directory names then for future reference, and then you can add them into your “rclone move” command rather than using the “rclone copy” command to have these sub-directories automatically deleted after a successful copy. Here’s what my commands look like to upload with rclone (using my encrypted directory names):

rclone copy /home/plex/.local/BZxQ7QeNh8jos9pEzcKXk5vy/ acd:Plex/BZxQ7QeNh8jos9pEzcKXk5vy
rclone copy /home/plex/.local/L,ULJSSLPjETMTzk5OhzCioC/ acd:Plex/L,ULJSSLPjETMTzk5OhzCioC

And then to empty out my local storage while maintaining those parent directories:

rm -Rf "/home/plex/local/Movies/"*
rm -Rf "/home/plex/local/TV Shows/"*

This will maintain a “Movies” and “TV Shows” directory inside the ‘/home/plex/local’ directory for you.

Step 7 – Use Crontab to Mount rClone, EncFS, and UnionFS Automatically (UPDATE)

I’ve added a new post for handling all your mounts automatically! Check out Automatically Mounting acd_cli, EncFS, and UnionFS Mounts at Boot

Wrap-Up

Well, you made it this far, so hopefully everything is working for you and you’re starting to take advantage of that “unlimited” storage! If this guide garners a decent amount of interest, I’ll write another guide on automating the downloading, renaming, moving, and uploading of all your media using scripts with CouchPotato and Sonarr. You can also fully automate the conversion of your media if you just hate transcoding and want all your media in a universal MP4 format. Leave a comment below if you’d find that useful!

Also, if you’re having any issues, you can either leave a comment below or hit me up on Reddit.

If these guides have been helpful and you’d like to help out with funding my blog, you can make a donation here. It’s greatly appreciated!

Advertisements

203 thoughts on “Using Amazon Cloud Drive with Plex Media Server on Ubuntu… And Encrypting It!

  1. I’m stuck @ Step 5 – Adding ACD Directories to Plex

    When I do the #user_allow_other

    Basically permissions get randomly broken somewhere

    can’t even delete/modify adc folder. I’m stuck, help is much appreciated.

    Like

  2. I had to reformat my server but saved the encfs file. How to I use that one to keep everything I have on my acd server? I tried copy and pasting what was in the original one into the new one, but its still creating different encrypted file names for the Movies/TV Show file names.

    Like

    • Copy and pasting the original encfs file now has my server using the same encrypted file names, so I believe I have it working. But it isn’t syncing all of the files I originally uploaded to ACD.

      Like

      • I found the issue. Somehow, the permissions to .acd ended up as:

        drwxr-xr-x 1 root root 0 Feb 24 14:48 .acd

        All my other folders are correct:

        drwxrwxr-x 2 plex plex 4096 Feb 23 01:55 acd

        How do I fix this? Using chmod isn’t changing anything.

        Like

      • If I remove the acd mount, I’m able to sudo chown .acd to myself. I’m unable to do so if acd is mounted to it:
        drwxrwxr-x 2 plex plex 4096 Feb 24 14:47 .acd

        Once I change it, if I run:

        rclone mount acd:Plex /home/plex/.acd &

        I get:

        plex@bbm:~$ rclone mount acd:Plex /home/plex/.acd &
        [1] 1635
        plex@bbm:~$ 2017/02/24 15:02:02 mount helper error: fusermount: mount failed: Operation not permitted
        2017/02/24 15:02:02 Fatal error: failed to mount FUSE fs: fusermount: exit status 1
        c
        c: command not found
        [1]+ Exit 1 rclone mount acd:Plex /home/plex/.acd
        [1]+ Exit 1 rclone mount acd:Plex /home/plex/.acd

        Once I redo it using sudo, it sucessfully mounts, but changes permissions of .acd back to root:
        drwxr-xr-x 1 root root 0 Feb 24 15:02 .acd

        How do I fix this?

        Like

      • It is OK for /.acd/ to be root:root
        Mine is and everything works smoothly.

        Are you still having issues getting this all to work, or were you just worried about that folder flipping to root?

        Like

  3. Great how to thank you!!!
    Did any one already tested with CryFS? EncFS is working perfect but I don’t like that folder structur and file size isn’t encrypted.

    Like

  4. Do you have to install acd_cli as well. Everything is working via your guide but I already have 10TB of content on Amazon. So whilst everything is mounted it doesn’t show my existing files already residing on Amazon.

    Is there a sync command that I need to run in order for it to see what is on Amazon? on my existing box i’m doing everything via acd_cli commands so i’m new to rclone

    Like

      • Yes. It’s all encrypted via encfs. I copied the XML file from my old server. If I build a new server with acd_cli and use the encryption key it all shows up but I’m assuming the acd_cli sync does that part.

        What’s interesting with this setup is if I create the Movies folder in the local directory it’s identical in name to what’s on my Amazon cloud drive so the encryption key is right and no errors occur when I use the ENCFS command. It’s like it needs to sync or something.

        Like

  5. Hey! I have a question

    I had set up a whole server and I have all the files on ACD. I have the encfs xml file, and the password. My question would be, if I were to start anew, how would I modify the encfs steps in order to use the old password and xml file? Do I just introduce the same password, and then move the old xml file in lieu of the new one?

    Basically, I’m doing the whole process again, but I already have files on ACD, how do I connect to them again (I have password and key file).

    Thanks!

    Like

    • been here done this with FOUR different servers.. hahahaha….

      all ya need to do is set up the new server with the same exact directory structure (ie the folders inside plex folder.. movies tv etc..)

      copy your old xml encfs file to the new server

      and its all good : )

      just set it up exactly the same.. all the same steps… only difference is the old xml file

      fyi… the mount scripts have always been problematic for me…. i just set up MobaXterm macros for all the mounting encfs and password stuff….

      my only remaining bugaboo… is how to move and delete files on acd without breaking the sync having to delete nodes and do an old sync etc…. i’ve still yet to figure that out… : (

      Like

      • If you try using rclone for the mount and remove the -ro flag, it’ll be much more reliable and you can delete files from the acd directory as you would normally. They’ll go to the ACD trash.

        Like

      • Are you doing anything special after the mounting? My folders are completely empty when mounted even though the content is on Amazon. I’ve rebuilt my servers numerous times using ACD_CLI and once the XML is copied and acd_cli sync is run everything shows up.

        With this method nothing is showing up so trying to determine what i’m doing wrong or missing

        Like

      • Thanks so much for your reply. I was basically following this guide up until where it says “move settings file”, but instead of moving the newly generated settings file I uploaded the old one. That apparently worked, as I can see the old folder names in my /.local folder.

        I moved on and followed the guide again. I got to step six “Upload Local Files/Directories to ACD using rclone” and when I try the rclone copy /home/plex/.local/”old encrypted folder name” I get “Failed to copy: failed to make directory: HTTP code 409: “409 conflict”: response body: “{\”code\”:\”NAME_ALREADY_EXISTS\”

        For the record, I still can’t see anything on either my encrypted or unencrypted folders when looking at them through filezilla. The files are still in the amazon cloud drive.

        Any idea what may be going on?

        Like

    • This is what I see

      encfs on /home/plex/acd type fuse.encfs (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
      encfs on /home/plex/local type fuse.encfs (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
      unionfs-fuse on /home/plex/media type fuse.unionfs-fuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000,default_permissions)
      acd:Plex on /home/plex/.acd type fuse.rclone (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)

      Like

      • Alright so when you say your folders are empty, are we talking in the media directory or the acd directory? Can you check both? The acd directory should show unencrypted ACD

        Like

  6. Correct. Both the Media and ACD folder is empty. If I create folders with the same name in Local as they are on Amazon they are identical in name so I know the encryption key is right. The only difference with this server to the other ones i’ve built are i’m using acd_cli on the others and use the following folder structure

    .local-sorted
    local-sorted
    .acd-sorted
    acd-sorted
    sorted (fuse)

    I don’t think this is the problem as i’m just changing the name in the ENCFS command. The directories in those folders are the same as your documentation

    TV Shows
    Movies

    I’m going to rebuild it from scratch and see if I have the same issue. Maybe i’m mixing it up somewhere. I’m not that great with knowing the ins and outs of ubuntu \ linux yet

    Like

  7. Hi all
    Newbie question about this – how much space do you need to have locally? For example, I have 3TB of media on a NAS which is nearing end of life. I have a laptop with a 300gb HD which I would like to use with this ACD setup but does media stream direct from ACD or get pulled down on the fly and then get streamed from local source?
    If it’s streamed from ACD, does the stream route via my broadband line to remote friends or does the ACD stream go direct from Amazon to remote friends and thereby cutting my home out of the equation (apart from marking Plex as the item is watched)?

    Thanks

    Like

    • The files are streamed from ACD to your server, then out to the clients. 300GB should be plenty as long as you’re uploading media to ACD as it’s added. Serving remote clients will still depend on your upload speed, as your server will still act as the “middle man” to other people streaming.

      Like

  8. Hi, I´m triying to do the same on a Debian headless server but I´m not able to finish the rclone config beause when I enter mi amazon cloud credentials I get a message that says that I have to enter again my password and type the caracters from a captcha that I can´t see on the w3m browser.
    Is there any way to avoid that captcha?
    Thanks!

    Like

  9. Hey there,

    I followed your guide till “Step 7 – Use Crontab to Mount rClone, EncFS, and UnionFS Automatically (UPDATE)” but after I move my files to ACD I can’t see them in my media folder. Also my /home/plex/acd folder seems to be a little bit weird:

    plex@vpsxxxxx:~$ cd acd
    plex@vpsxxxxx:~/acd$ ls
    ls: cannot open directory ‘.’: No such file or directory
    plex@vps386953:~/acd$ ls -la
    ls: cannot open directory ‘.’: No such file or directory
    plex@vps386953:~/acd$ cd ..
    plex@vps386953:~$ ls -la
    ls: cannot access ‘acd’: No such file or directory
    total 56
    d????????? ? ? ? ? ? acd
    drwxrwxr-x 1 plex plex 0 Mar 14 12:08 .acd
    drwxrwxr-x 4 plex plex 4096 Mar 14 12:09 local
    drwxrwxr-x 4 plex plex 4096 Mar 14 12:09 .local
    drwxrwxr-x 1 plex plex 4096 Mar 14 12:09 media

    Like

  10. i followed the guide and i am stucking in the verification…
    rclone seems to run fine exept the sync isnt working.
    I mean i can upload and download files to acd this works.
    But the files and folders will not synced with the acd folder.
    has someone a hint for me why?

    Like

    • ok now i found out that the mount basically will work but only an inital time… if i move or create files and folders it will not synced to the acd folder… but if i remount the the drive it sees the new files…
      i can too not copy any data to the acd folder…

      Like

  11. Hi!
    Tank you for your excellent guid. I have short question about encfs, “souldn’t it say “ENCFS6_CONFIG=’/home/plex/encfs.xml’ encfs –reverse /home/plex/loca /home/plex/.locall” instead of “ENCFS6_CONFIG=’/home/plex/encfs.xml’ encfs /home/plex/.local /home/plex/local”?
    At least this is how i understand the encfs wiki…
    –reverse
    Normally EncFS provides a plaintext view of data on demand: it stores enciphered data and displays
    plaintext data. With –reverse it takes as source plaintext data and produces enciphered data on-
    demand. This can be useful for creating remote encrypted backups, where you do not wish to keep the
    local files unencrypted.

    Like

  12. Hi!
    Thank you for your excellent guide. I have a short question about encfs, souldn’t it say “ENCFS6_CONFIG=’/home/plex/encfs.xml’ encfs –reverse /home/plex/loca /home/plex/.locall” instead of “ENCFS6_CONFIG=’/home/plex/encfs.xml’ encfs /home/plex/.local /home/plex/local”?
    At least this is how i understand the encfs wiki…
    –reverse
    Normally EncFS provides a plaintext view of data on demand: it stores enciphered data and displays
    plaintext data. With –reverse it takes as source plaintext data and produces enciphered data on-
    demand. This can be useful for creating remote encrypted backups, where you do not wish to keep the
    local files unencrypted.

    Like

  13. Hi there,
    nice guide. Was really helpful. I am facing a problem. Where when I scan the library In plex it starts to work out fine. But after Servers minutes the mountpoint gets disconnected. When I try to acces via ssh. The output is: cannot access. Transport endpoint is not connected. I can clear and reattach to again get access to ACD but the scan didn’t finished by the time. Any ideas on why I get this problem?

    Like

  14. If you got issues with unmount ( “fusermount: failed to unmount /home/plex/local: Device or resource busy”) , use it with force command : fusermount -uz /home/plex/local

    Like

  15. Have you noticed any performance issues with using this method? Since switching from ADC_CLI I’ve noticed the media streaming getting slower and slower to start. At the end of every stream it stops to buffer with literally 20 – 30 seconds left of play. Worked flawlessly for a number of weeks and then gradually just started going downhill.

    I’ve tried amending the mount script with some flags but that doesn’t appear to be solving the issue. Doing some research people are saying to do the following but for some reason every time I run this it won’t mount acd properly

    rclone mount –max-read-ahead 200M –transfers 20 –checkers 40 –read-only –allow-other –acd-templink-threshold 0 –stats 1s –buffer-size 1G acd: /home/plex/.acd &

    I’m wondering if I go back to using ACD_CLI to mount and use rClone for everything else. Not sure how it will play when it comes to syncing and updating though.

    Like

  16. hi there,

    how can I setup that “torrents” go to the “/home/plex/local” directory ?
    Transmission starts to download, but after 2-5 seconds it stops with “file exists” error message.

    Downloading into /home/plex/downloads works normal
    But than I have to copy the files over to ~/local which costs me 2 times HDD space
    Hardlinks are not working for me because of “Invalid cross-device link” error

    😦

    Like

  17. I am having a problem. When I issue the command rm -Rf “/home/plex/local/TV Shows/”* it removes the directories from local but it also breaks Emby and Sickrage which I use. I point them both to the media folder like the setup says but the shows are gone after the rm command. Am I pointing my programs to the wrong location?

    Like

    • Really only because when I first started using ACD as a storage backend, rClone wasn’t capable of doing a “live” mount and I was using acd_cli, which didn’t offer encryption. Though, going through that process made me realize that it’s best in some cases NOT to use built-in encryption with these different mounting programs because what happens when (if) a better alternative to rClone comes around? You’d be handicapped to using rClone because of encryption. So I think using a third-party option keeps you flexible down the road. I can mount my drive on any machine and install encfs to access it. I can’t be guaranteed to get rClone working on any machine, and if I used the built-in encryption, that means my files are at risk of being essentially gone.

      That’s not to discredit the encryption features in rClone. If you want to use that and are okay with exclusively using rClone indefinitely, you definitely can.

      Like

      • It’s definitely a widespread issue, all indications are that it’s an issue with Amazon’s API. As of this moment, I’ve noticed my directory listings succeeding every few tries, which is an improvement from not at all earlier today.

        Like

  18. Do you know how I would go about making this work in such a way that 2 separate servers could have the same contents of the media folder, accessing the same encrypted shares?

    Like

      • Thanks for the help. I am having two issues.
        1) I setup rClone & encFS on my second server and copied over my encfs file, and used the same mount commands. This did create identical encrypted structures on both machines and on ACD. However, I am also trying to replicate the media folder from server 1 to server 2 automatically. So, to be clear, Server 1 is where the files will reside and be moved to the encrypted ACD, but my Plex is on Server 2, so I need to propagate the media folder on Server 2 from Server 1.

        2) I am having a problem on Server 2, in that
        rclone copy /home/plex/.local acd:Plex
        works perfectly, puts everything on ACD, and shows up perfectly in my media folder. But, when I free the space with:
        rm -Rf “/home/plex/local/TV Shows/”*
        It deletes everything, everywhere (ACD, media folder, etc.). This does not happen on Server 1, and I cannot figure out where I may have gone wrong.

        Like

  19. So, me again. I managed to create a new server and it was running perfectly, but then I rebooted it and it stopped working.

    So I am retracing my steps, and when I try to do rclone mount acd:Plex /home/plex/.acd & I guet the following

    [1] 24239mount helper error: fusermount: failed to access mountpoint /home/plex/.acd: Permission denied
    Fatal error: failed to mount FUSE fs: fusermount: exit status 1
    [1]+ Exit 1

    Any idea what may be causing this?

    Thanks!

    Like

      • So, I used ls -la and for the five directories that we create (local, .local, acd, .acd, media) I got the same sequence: drwxrwxr-x with the user plex and the owner plex. So, unless I’m doing something wrong with the command, rclone mount acd:Plex /home/plex/.acd & should work, right?

        Like

  20. Pingback: Plex – a network error occurred | Helene Klungvik

  21. Hello, thanks for this write up. Having an issue with Plex seeing the directories under the unionfs-fuse mount. I can see everything just fine from the cli but Plex will not see the subdirectories of the media directory. In order to see my media I have to browse with Plex to the /acd/plex path. Thanks Again

    Like

  22. well… now that ACD has booted rclone as well.. and won’t accept anymore third party apps… i guess that party is over : (

    are there any other options?

    luckily (i think?) i never switched to rclone crypt… used encfs.

    what options are there to see/download the files unencrypted.. are there really NONE now for linux servers? if not what are the windows encfs options with acd?

    are those official non ebay gsuite google drive accounts really unlimited with less than 5 users (on the rclone forum everyone seems to be saying they are) and does gdrive really work with plex (i read so many tales of account banning from just updating the library.)?

    Like

    • As it stands, if this issue with ACD is permanent, your only real option is to download files from the web portal and place them manually into your current directories and encfs will decrypt them. However I’d hold off on panicking just yet, because last time there were 429 errors en masse, the web portal was also finicky, and at least for me the web portal has been very slow and temperamental.

      As far as the GDrive accounts go, you actually can just create a business account with one user and get unlimited drive storage for $10 per month. That’s what I’ve been doing. Luckily I moved over to GDrive as my primary storage backend a few months ago, so this ACD outage has been a minor inconvenience. If you’re using GDrive, rClone will definitely get your account banned (24h) all the time. I’ve been using GDrive-ocaml-fuse and it’s actually the best streaming performance I’ve seen of any of these utilities. My streams start in under three seconds, regardless of bitrate. It’s actually faster than Netflix for me, which is great. It also caches your file/directory structure, so it won’t incur any bans when scanning your libraries. In fact, I’ve pointed Radarr and Sonarr at my ocaml-fuse mount as well as Plex and I’ve had zero issues.

      Like

      • I think this is a permanent ban. A lot of rclone users contaced Amazon support that confirm it.
        I am currently transferring ACD Plex encrypted folder to a GDrive folder (with MultiCloud).
        The ACD setup was exactly as in the tutorial here with encryption and fuse.
        Will I be able to to switch to ocaml-fuse and decrypt the data?

        Like

  23. Hi, I want to know if anybody tried this with a Google Drive acount. I have an unlimited acount and I want to do the same as I did with my ACD acount but I´m having some issues with the atuthorization on rclone.
    Anybody did it? Thanks!

    Like

      • What command are you using when mounting gdrive-ocaml-fuse? I’m using –

        /usr/bin/google-drive-ocamlfuse -o allow_other,ro,atomic_o_trunc,large_read,default_permissions “$HOME/.acd-sorted”

        It’s reading the files but when adding the folder to Plex it’s taking forever for things to show up.

        Thanks

        Like

  24. Hello,

    is there a step-by-step on getting my encrypted data off of ACD? I tried this . . .

    “your only real option is to download files from the web portal and place them manually into your current directories and encfs will decrypt them.”

    . . . but not sure what that did. I am not that great with Linux so more details on this would be great.

    Thanks

    Like

    • I’ll have to do some further testing on this, I’m not sure if the directory structure is important when decrypting. If it is, then people will be in for a miserable time because the only way to truly get your files would be to fully download each root directory of TV and Movies. If the structure isn’t necessary for decrypting individual files, things might go a bit smoother. Either way you’d need ample disk space. Let me do some testing for some options.

      Liked by 1 person

      • Thanks, I would appriciate any help you can provide. I will test things too and see what / if I can come up with. LOL, not worried about space as my QNAP is on the way. 🙂

        Like

  25. i was about to post the same questions as Steve…

    this guide was such a help for me (and everything was runing perfect.. especially after the switch to rclone mount) : ) : (

    but i was wondering.. if someone had a step by step (idiot/non linux expert) guide to moving everything to a google drive/suite account…

    someone was nice enough to post this run down on the rclone forums..

    but i’m wondering what commands i need to change in here… assuming my amazon mount is called “acd” and everything is encrypted with encfs on amazon in a remote called acd..

    thanks.

    ——————
    “How to migrate 15TB ACD to GSUITE (with only the GSUITE cost)

    I am transfering at 55-137MB/s! average: ~120MB/s

    First Step: Get G Suite Account
    1. signed up for G Suite trial.
    2. skipped setup assistant.
    3. added my domain via domain verification.
    4. upgraded to G Suite Business (8$ p month) (still on trial though! :slight_smile: )

    Second Step: Get Google Compute Account
    You get 300$ to spend for free on signup!+
    1. Register for free 300$ at Google Compute.
    2. Add billing details.
    3. Add a budget and limit it to 1$, just to be sure!
    4. Upgrade Google Compute to subscription (Is still on trial after uprade) (you have to, otherwise you only have 2TB disk space limit) (this will get you a 10TB limit)
    5. Request a additional +10TB contigent at “region” of your choice via Google Compute interface. I have a german ACD account, thererfore I use “eu-west 1” region. It only took 2min to approve! Bravo Google!
    6. Create Custom Instance VM with 2vCPU + 1.8GB RAM + 20 TB persistent HDD
    7. Wait for it to boot up and login via SSH

    Third Step: Use odrive to sync to VM instance
    1. Register free odrive account
    2. Get linux odrive sync agent
    3. setup a mount
    4. recursive sync all directories
    5. recursive sync all files (with 3 parallel sync jobs)
    Tip: use ‘screen’. otherwise odrive sync jobs stop on connection abort / ssh session end!
    details see here: https://forum.rclone.org/t/acd-429-too-many-requests/1792/525?u=philip_konighofer

    Fourth Step: Use rclone to sync to google
    1. you should know what todo. 😉
    Beware: Since you are syncing the already encrypted files, you have to create a PLAIN remote.

    Fifth Step: Use rclone with your CRYPT remote
    Once finished uploading files files, switch to encrypted remote.

    Sixth Step: Get ACD refund

    Seventh Step: Unsubscribe Google Compute
    1. Stop VM instance
    2. Delete disk image
    3. unsubcribe (while still in trial period)
    4. remove your billing information

    Like

  26. I’m doing the odrive sync down to Google Platform now. All the content that was waiting to be synced to Amazon when it was cut off is now being synced to Google and working on Plex. I set everything up with rClone and to avoid the 24 hour ban found some instructions on setting up your own client_id on the Google API Console.

    1) Log into the Google API Console (https://console.developers.google.com/) with your Google account. It doesn’t matter what Google account you use. (It need not be the same account as the Google Drive you want to access)

    2) Select a project or create a new project.

    3) Under Overview, Google APIs, Google Apps APIs, click “Drive API”, then “Enable”.

    4) Click “Credentials” in the left-side panel (not “Go to credentials”, which opens the wizard), then “Create credentials”, then “OAuth client ID”. It will prompt you to set the OAuth consent screen product name, if you haven’t set one already.

    5) Choose an application type of “other”, and click “Create”. (the default name is fine)

    It will show you a client ID and client secret. Use these values in rclone config to add a new remote or edit an existing remote.

    Like

    • Can you give a short guide on how to copy over those files via odrive from ACD to GD? Do i Have to Download Them to the local Server First?

      Like

      • I have to redo mine as the server ran out of space (I thought the 20tb was available on sda1 but was only 2tb). Basically, my understanding is you add Amazon in odrive (web) and then install odrive agent on Linux, mount and sync down to the local machine.

        You then use rclone to sync over to google.

        Like

      • To install odrive Linux (64 bit)

        Web
        1) create free odrive account
        2) add Amazon Drive

        SSH into VM
        1) Download and install odrive agent

        od=”$HOME/.odrive-agent/bin” && curl -L “http://dl.odrive.com/odrive-py” –create-dirs -o “$od/odrive.py” && curl -L “http://dl.odrive.com/odriveagent-lnx-64” | tar -xvzf- -C “$od/” && curl -L “http://dl.odrive.com/odrivecli-lnx-64” | tar -xvzf- -C “$od/”

        2 ) Run odrive agent

        nohup “$HOME/.odrive-agent/bin/odriveagent” > /dev/null 2>&1 &

        3) Verify agent is running

        python “$HOME/.odrive-agent/bin/odrive.py” status

        4) Authenticate the agent

        python “$HOME/.odrive-agent/bin/odrive.py” authenticate (your auth key can be found in odrive web portal)

        5) Create directory where you are going to sync your files

        mkdir “$HOME/odrive-agent-mount”
        (if you are using a mounted drive you will need to change the path)

        6) Mount odrive

        python “$HOME/.odrive-agent/bin/odrive.py” mount “$HOME/odrive-agent-mount” /
        (if you are using a mounted drive you will need to change the path)

        7) Sync folders to VM

        find ~/odrive-agent-mount/ -type f -name “*.cloudf” -exec python “$HOME/.odrive-agent/bin/odrive.py” sync “{}” \;

        (keep running this command until no further outputs occur)

        8) run 3 screen sessions

        screen -S sync1
        screen -S sync3
        screen -S sync3

        9) Sync folders to VM (run this command in each screen session)

        find ~/odrive-agent-mount/ -type f -name “*.cloud” -exec python “$HOME/.odrive-agent/bin/odrive.py” sync {} \;

        This will sync everything down to your VM. Once this has completed you can sync files to Google (you can also do it whilst files are syncing from Amazon). To do this you will need to install rClone on the VM. Make sure you have created your API auth tokens in Google in order to set up rClone as you will need them when setting up the mount.

        I’m not a Linux expert so have been fumbling my way around this but my content is now starting to appear back in Plex again so I must’ve done something right :). Hope this helps.

        Like

  27. I changed a bit the mount.sh (replaced rclone .acd mount with
    google-drive-ocamlfuse ) and all is fine. But with what do we replace the syncing abilities of rclone now?

    Like

  28. FWIW

    I’ve managed to get half of my Plex library over to Google. It’s up and running and the performance is much better than Amazon. I have 2 Dedicated Servers

    1 x Downloader (Sonarr, Radarr, Sab) with rclone and unionfuse going
    1 x Plex Server

    On the Plex Server I’m mounting google using PlexDrive and haven’t been banned since implementing. I tweaked the script provided by enztv by removing the rclone mount and putting in PlexDrive

    Like

  29. I’m running all the components of my media server (sonarr, radarr, nzbget) in docker containers. rClone is installed on my host. How can I get sonarr post-processing script that perform rClone copy command to run on the host?

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s