UPDATED – 26 January, 2017
This guide previously made use of acd_cli for mounting Amazon Cloud Drive. It’s been updated to use rClone for both mounting and uploading, as this is not only more efficient, but also much more robust. Using rClone for mounting eliminates the need to sync/remount your ACD mount.
If you’ve already used this guide, you can simply head over to the automatic mounting script guide, update your script with the changes in the guide, and execute it, you should be good to go!
The idea of an “unlimited” Plex server is an exciting one, but there are some bases you’ll want to cover before dumping all your media into the cloud. The biggest issue with keeping your media hosted in the cloud is encryption; you don’t want any storage providers snooping around your files, regardless of their source or content. This guide will get you going, with a focus on security.
Also, with the announcement of Plex’s new “Plex Cloud” offering, I’ve seen quite a few people posting on Reddit’s Plex subreddit for help with using Amazon Cloud Drive (ACD) as a storage provider for their Plex media on their standalone Plex server. There have been some guides floating around the Internet, but some of these guides use a confusing approach (in my opinion) to accomplish a task that’s already a bit confusing for some. I’ve been using the setup detailed in this guide for a few years now, even with multiple servers utilizing the same ACD mount, and I’ve had very few issues. I’ve automated this setup with tools like CouchPotato, Sonarr, and custom scripts in conjunction with Plex Requests to the point where I rarely have to do any maintenance on my Plex servers, and anybody can request media and have it available in minutes without any kind of interaction from me.
If you haven’t set up Plex yet, or you’ve got a fresh installation of Ubuntu, you can use my other guide, Getting Started with Plex to get the prerequisite setup finished first.
In this guide, we’ll be making use of several different projects to get this setup working smoothly, so while a guide can help you get set up, troubleshooting any problems can be difficult and a guide won’t always be helpful. However, if everything is set up in a logical way to begin with, the pain of troubleshooting can eased. Let’s get started on that now!
What you’ll need:
- Amazon Cloud Drive account (obviously)
- User account running these tools and Plex (preferably named “Plex”)
- unzip
- Python 3 (installed by default on Ubuntu 14.04+)
What we’ll be installing:
How it Will Work:
Basically, we will be using rClone both to read our Amazon Cloud Drive and to write to back ACD, and EncFS to encrypt it all. The real magic happens with UnionFS, as it will merge our read and write directories into one, so we can essentially forget about those other directories and focus on a single path for all our needs, including other tools like CouchPotato and Sonarr.
I’m going to assume your user account running Plex is not root, and is called “Plex” for the sake of convenience. Substitute usernames as necessary here, and stay logged in as this user throughout the guide. We’ll use “sudo” for anything requiring elevated privileges.
Anytime “~” is specified in a directory, it is assumed to be equal to ‘/home/plex’
Step 1 – Creating Necessary Directories
Let’s create the directories we’ll be using to make all this work. For ease of use, I like to put these directories right inside my Plex user’s home directory. So let’s make sure we’re at the right place (again, you should be logged in as your Plex user, not root):
cd ~
Now that we’re in the home directory, let’s create the five directories we’ll be using:
mkdir .acd mkdir acd mkdir .local mkdir local mkdir media
These directories will be used as follows:
~/.acd – ACD mount containing encrypted media
~/acd – ACD mount decrypted with EncFS
~/.local – local media after being encrypted by EncFS
~/local – where your local/downlaoded media will be copied to
~/media – the UnionFS mount point which will merge ‘~/acd’ and ‘~/local’ directories
Step 2 – Setting up rClone
Install rclone
Start off by making sure you’re in your Downloads directory, then grabbing the rclone package:
cd ~/Downloads wget http://downloads.rclone.org/rclone-current-linux-amd64.zip
Now, unzip the archive and change directories to it:
unzip rclone-current-linux-amd64.zip cd rclone-*-linux-amd64
Copy the binary and give root ownership:
sudo cp rclone /usr/sbin/ sudo chown root:root /usr/sbin/rclone sudo chmod 755 /usr/sbin/rclone
Then install the manpage so you can read all about rclone:
sudo mkdir -p /usr/local/share/man/man1 sudo cp rclone.1 /usr/local/share/man/man1/ sudo mandb
Set Up rclone
Now we’re ready to configure rclone. Once again, this process is different depending on whether you’re using a desktop session or a headless SSH session. If you’re using a terminal in a desktop session (GUI), then simply enter the commands below, otherwise scroll down a tad further to “Using SSH or a headless server”
Using a desktop interface/GUI
In your terminal, simply type the following command to enter the rclone configuration process:
rclone config
This will guide you through a fairly easy process. Let’s go through the steps together:
- Press “n” to add a new remote connection, then “enter” to proceed
- You’ll want to name your instance something simple, like “acd” to make it easier to upload files
- Press “1” for ACD, then “enter” to proceed
- Press “enter” for the next two fields to use defaults
- Press “y” to use the “automatic” mode, which will open a browser window. Enter your ACD credentials, and log in. The config process should continue automatically.
- Press “y” if you can see your token text on the screen
- Press “q” to exit the config process, we’re done for now
Using SSH or a headless server
If you’re managing a headless Plex server, or you don’t have a desktop environment installed, authenticating rClone gets a bit trickier. I’ve found success using the w3m browser. So lets install that:
sudo apt-get install -y w3m
This will install a text-only terminal-based browser. It should do fine for this purpose, but don’t plan on surfing the web with it.
Now we’ll install a utility called Screen (this is included in later versions of Ubuntu). Screen allows you to have multiple terminal sessions open without worrying about them expiring or closing out. This is useful if you need to access your server remotely with Putty or something similar to execute a script or other process, but can’t have the SSH session open until that script/process is complete. Ending your SSH session also ends whatever script/process you were running. Screen is a great way to prevent that from happening, and lets you reconnect to existing sessions from any other SSH session. So let’s install Screen:
sudo apt-get install -y screen
Sometimes Screen is already installed, but either way it should be now for sure. To open a new terminal session with Screen, just type:
screen
Press “enter” to dismiss the message, and now you’re in a second terminal session. You can exit a Screen session while leaving it active at anytime by pressing “CTRL+A” then “D” to disconnect. Now that we have a second session running, let’s start the rclone config process:
rclone config
Move past these steps the same as the GUI users would have, up to the “use auto-config” step:
- Press “n” to add a new remote connection, then “enter” to proceed
- You’ll want to name your instance something simple, like “acd” to make it easier to upload files
- Press “1” for ACD, then “enter” to proceed
- Press “enter” for the next two fields to use defaults
- PAUSE at the auto-config step. Pressing “y” to use the “automatic” mode will attempt to open a browser window. This won’t work, however it will show you a URL to use. Copy this URL, then disconnect this Screen session by pressing “CTRL+A” then “D” which will keep that session active in the background
- Open the w3m browser to the rclone URL by entering:
w3m http://127.0.0.1:53682/auth
- Enter your ACD credentials by moving your cursor to the required fields, pressing “enter” and typing your username/password, then pressing “enter” again
- Place your cursor on “Sign in using our secure server” and press “enter” to sign in. You should see a “success” message
- Press “q” to exit the w3m browser
- Type the following command to reconnect to your previous Screen session:
screen -r
- Press “y” if you can see your token text on the screen
- Press “q” to exit the config process, we’re done for now
- Press “CTRL+A” then “K” to permanently kill this Screen session
Just to make sure everything worked correctly, we can run the following command to list the remote accounts configured with rClone, and it should list “acd” under the “Name” heading and “amazon cloud drive” under the “Type” heading:
rclone listremotes
Mounting ACD with rClone
Now we’re ready to mount our Amazon Cloud Drive instance to make it accessible to the server. This is accomplished with one simple command, however you can automate this process (and the rest of this guide) using the scripts over at our other guide here! The command to mount ACD is as follows (don’t forget the ampersand at the end!):
rclone mount acd:Plex /home/plex/.acd &
This will mount your ACD instance in the ‘.acd’ directory in the plex user’s home directory. The ampersand executes the command in the background so we can continue using our terminal. Now we can move on to the encryption phase!
Step 3 – Setting up EncFS
Install EncFS
Now that we have our ACD instance mounted using rClone, let’s make sure it’s encrypted:
sudo apt-get install -y encfs
Set Up and Test Encryption
Then we’ll go ahead and get our directories encrypted. Using EncFS, the first directory in the command is the one you’ll want encrypted, and the second directory is the one you want decrypted. The nice thing about EncFS is you don’t really have to specify the action, if the first directory already contains encrypted files, the second directory will automatically decrypt them. So let’s encrypt our local media directories:
encfs /home/plex/.local /home/plex/local
You should see a prompt from EncFS regarding encryption settings. Use the default and just press “enter” to move on to password creation. Make sure you remember this password! Type it a second time, and your local directories are now encrypted. You can test this by creating the directories Plex will use in the ‘/home/plex/local’ directory. In this guide, we’ll only worry about movies and TV shows, so let’s create a directory for each:
cd ~/local mkdir Movies mkdir "TV Shows"
Now let’s list the ‘local’ directory to make sure they’re created:
ls ~/local
And you should see the “movie” and “tv” directories listed. Now, let’s list the encrypted directory:
ls ~/.local
You should see two directories with a bunch of random characters. This confirms our files are being encrypted.
Move Settings File
Now let’s move our EncFS settings file to a more convenient location. First we need to unmount our local media directories. This command is also used to unmount other EncFS directories in the future, though you shouldn’t need to use it very often:
fusermount -u /home/plex/local
Next, we’ll move the “encfs6.xml” file and rename it to something simpler:
mv /home/plex/.local/.encfs6.xml /home/plex/encfs.xml
Encrypting Directories Using EncFS Configuration Files
This “encfs.xml” file is your encryption “key,” not to be confused with the password. These work in tandem to keep your files secure, so you’ll want to make sure you keep a copy of this file backed up somewhere safe. Now, when mounting an EncFS directory, you’ll need to specify the location of the config file. Anytime you mount an encrypted directory, the command will always start the same, only the directories will change as needed. Let’s go ahead and mount our local media directory again using the config file:
ENCFS6_CONFIG='/home/plex/encfs.xml' encfs /home/plex/.local /home/plex/local
Okay, now let’s do the same for our ACD directories. Enter the following into your terminal:
ENCFS6_CONFIG='/home/plex/encfs.xml' encfs /home/plex/.acd /home/plex/acd
And that’s it for encryption! It seems like a lot, but underneath it all, everything is pretty simple. Four directories, two of which are encrypted mirrors of the others, and a fifth to merge the two decrypted directories together. Let’s move on to merging those directories with UnionFS!
Step 4 – Merging directories using UnionFS-Fuse
Install UnionFS-Fuse
Once again, let’s get UnionFS-Fuse installed;
sudo apt-get install -y unionfs-fuse
Merge Directories Using UnionFS
There’s no other setup required for this part, we just need to specify which directories we want to merge. Those will be the decrypted directories, local and ACD, so Plex can be refreshed to grab new movies/episodes when they’re downloaded without waiting for them to be uploaded, and so other tools like CouchPotato and Sonarr can move files into this directory and they’ll appear to be in the same location as all the ACD media.
unionfs-fuse -o cow /home/plex/local=RW:/home/plex/acd=RO /home/plex/media/
The “RW” and “RO” arguments in that command specify which directory is writable (local directory) and which is read-only (ACD directory). So now anything you copy/move to ‘/home/plex/media’ will actually be in ‘/home/plex/local’ and thus encrypted in ‘/home/plex/.local’, and anything in your ‘/home/plex/acd’ directory will also show up in the ‘/home/plex/media’ directory. We only need to worry about one directory to point Plex libraries at, or to have CouchPotato/Sonarr monitor. This makes life much, much easier! Now, we’re ready to move on to the final phase of this guide; uploading your downloaded media to ACD.
Step 5 – Adding ACD Directories to Plex
Let’s make sure Plex can see the ACD directories now! Hop over to your Plex web portal, and add a new library (or edit an existing one). When choosing your directory, browse to ‘/home/plex/media’ and you should see the “movie” and “tv” directories we created. Choose the respective directory for your library, and that’s it! Anything you upload to ACD will show up in that directory.
If you don’t see the ‘media’ directory when trying to add it through Plex, it could be a permissions issue. One thing you can try is browsing to the ‘/home/plex/acd’ directory from Plex web and seeing if those directories show up. If they do, this is a permissions issue with the UnionFS mount. Unmount it by typing:
fusermount -u /home/plex/media
And then edit the fuse configuration file:
sudo nano /etc/fuse.conf
Find the line “#user_allow_other” and remove the “#” by pressing the delete key in front of that line. Save the file by pressing “CTRL+X” and “Y” to overwrite. Then, mount the merged UnionFS-Fuse directory by adding the “allow_other” option to the mount command:
unionfs-fuse -o cow,allow_other /home/plex/local=RW:/home/plex/acd=RO /home/plex/media/
Now, you should be able to see the “movie” and “tv” folders inside ‘/home/plex/media’ directory when adding a library in Plex web.
Step 6 – Uploading to ACD with rclone
Now that our directories are set up and encrypted and we have them added to our Plex server, it’s time to start adding media to them.
Upload Local Files/Directories to ACD Using rclone
The syntax for copying files with rclone is “rclone copy sourcedir remote:destinationdir”. In this context, “sourcedir” is your local encrypted directory, “remote” is what you named your ACD instance when setting up rclone (I suggested acd), and “destinationdir” is the directory in your ACD you want to move those files to, which in our case would be inside the “Plex” folder you created in the ACD web portal.
Anyways, on to the meat and potatoes! To upload your local files over to your ACD mount, we’ll be uploading your encrypted local directory to the encrypted ACD mount with the following command:
rclone copy /home/plex/.local acd:Plex
This should copy the files from your local encrypted directory to your encrypted ACD directory, and then you can manually delete them from ‘home/plex/local’ if the copy was successful.
One thing I like to do is upload my movie and TV directories separately by appending those directory names to the ends of the source and destination directories in the rclone move command. You can just list your ‘/home/plex/.local’ directory to get the names of the directories, and you can figure out which is which by using the “touch” command to place a test file in either directory, then listing the encrypted directories to see which has a file in it. Keep note of those directory names then for future reference, and then you can add them into your “rclone move” command rather than using the “rclone copy” command to have these sub-directories automatically deleted after a successful copy. Here’s what my commands look like to upload with rclone (using my encrypted directory names):
rclone copy /home/plex/.local/BZxQ7QeNh8jos9pEzcKXk5vy/ acd:Plex/BZxQ7QeNh8jos9pEzcKXk5vy rclone copy /home/plex/.local/L,ULJSSLPjETMTzk5OhzCioC/ acd:Plex/L,ULJSSLPjETMTzk5OhzCioC
And then to empty out my local storage while maintaining those parent directories:
rm -Rf "/home/plex/local/Movies/"* rm -Rf "/home/plex/local/TV Shows/"*
This will maintain a “Movies” and “TV Shows” directory inside the ‘/home/plex/local’ directory for you.
Step 7 – Use Crontab to Mount rClone, EncFS, and UnionFS Automatically (UPDATE)
I’ve added a new post for handling all your mounts automatically! Check out Automatically Mounting acd_cli, EncFS, and UnionFS Mounts at Boot
Wrap-Up
Well, you made it this far, so hopefully everything is working for you and you’re starting to take advantage of that “unlimited” storage! If this guide garners a decent amount of interest, I’ll write another guide on automating the downloading, renaming, moving, and uploading of all your media using scripts with CouchPotato and Sonarr. You can also fully automate the conversion of your media if you just hate transcoding and want all your media in a universal MP4 format. Leave a comment below if you’d find that useful!
Also, if you’re having any issues, you can either leave a comment below or hit me up on Reddit.
If these guides have been helpful and you’d like to help out with funding my blog, you can make a donation here. It’s greatly appreciated!
Is there a way to organise the content on ACD after uploading?
LikeLike
Found out my own answer;
There is!
Managing ~/acd directory does the job.
LikeLike
yea… but every time i move things around in there it breaks the sync.. io gotta delete the nodes and do an old-sync to get it back up : (
LikeLike
I’m stuck @ Step 5 – Adding ACD Directories to Plex
When I do the #user_allow_other
Basically permissions get randomly broken somewhere
can’t even delete/modify adc folder. I’m stuck, help is much appreciated.
LikeLike
I had to reformat my server but saved the encfs file. How to I use that one to keep everything I have on my acd server? I tried copy and pasting what was in the original one into the new one, but its still creating different encrypted file names for the Movies/TV Show file names.
LikeLike
Copy and pasting the original encfs file now has my server using the same encrypted file names, so I believe I have it working. But it isn’t syncing all of the files I originally uploaded to ACD.
LikeLike
I found the issue. Somehow, the permissions to .acd ended up as:
drwxr-xr-x 1 root root 0 Feb 24 14:48 .acd
All my other folders are correct:
drwxrwxr-x 2 plex plex 4096 Feb 23 01:55 acd
How do I fix this? Using chmod isn’t changing anything.
LikeLike
If I remove the acd mount, I’m able to sudo chown .acd to myself. I’m unable to do so if acd is mounted to it:
drwxrwxr-x 2 plex plex 4096 Feb 24 14:47 .acd
Once I change it, if I run:
rclone mount acd:Plex /home/plex/.acd &
I get:
plex@bbm:~$ rclone mount acd:Plex /home/plex/.acd &
[1] 1635
plex@bbm:~$ 2017/02/24 15:02:02 mount helper error: fusermount: mount failed: Operation not permitted
2017/02/24 15:02:02 Fatal error: failed to mount FUSE fs: fusermount: exit status 1
c
c: command not found
[1]+ Exit 1 rclone mount acd:Plex /home/plex/.acd
[1]+ Exit 1 rclone mount acd:Plex /home/plex/.acd
Once I redo it using sudo, it sucessfully mounts, but changes permissions of .acd back to root:
drwxr-xr-x 1 root root 0 Feb 24 15:02 .acd
How do I fix this?
LikeLike
It is OK for /.acd/ to be root:root
Mine is and everything works smoothly.
Are you still having issues getting this all to work, or were you just worried about that folder flipping to root?
LikeLike
Great how to thank you!!!
Did any one already tested with CryFS? EncFS is working perfect but I don’t like that folder structur and file size isn’t encrypted.
LikeLike
CryFS is probably not the best, many small files throughput will be very low. Maybe we can solve it with a container like VeraCrypt?
LikeLike
Do you have to install acd_cli as well. Everything is working via your guide but I already have 10TB of content on Amazon. So whilst everything is mounted it doesn’t show my existing files already residing on Amazon.
Is there a sync command that I need to run in order for it to see what is on Amazon? on my existing box i’m doing everything via acd_cli commands so i’m new to rclone
LikeLike
Did you encrypt your original media with encfs?
LikeLike
Yes. It’s all encrypted via encfs. I copied the XML file from my old server. If I build a new server with acd_cli and use the encryption key it all shows up but I’m assuming the acd_cli sync does that part.
What’s interesting with this setup is if I create the Movies folder in the local directory it’s identical in name to what’s on my Amazon cloud drive so the encryption key is right and no errors occur when I use the ENCFS command. It’s like it needs to sync or something.
LikeLike
Hey! I have a question
I had set up a whole server and I have all the files on ACD. I have the encfs xml file, and the password. My question would be, if I were to start anew, how would I modify the encfs steps in order to use the old password and xml file? Do I just introduce the same password, and then move the old xml file in lieu of the new one?
Basically, I’m doing the whole process again, but I already have files on ACD, how do I connect to them again (I have password and key file).
Thanks!
LikeLike
been here done this with FOUR different servers.. hahahaha….
all ya need to do is set up the new server with the same exact directory structure (ie the folders inside plex folder.. movies tv etc..)
copy your old xml encfs file to the new server
and its all good : )
just set it up exactly the same.. all the same steps… only difference is the old xml file
fyi… the mount scripts have always been problematic for me…. i just set up MobaXterm macros for all the mounting encfs and password stuff….
my only remaining bugaboo… is how to move and delete files on acd without breaking the sync having to delete nodes and do an old sync etc…. i’ve still yet to figure that out… : (
LikeLike
If you try using rclone for the mount and remove the -ro flag, it’ll be much more reliable and you can delete files from the acd directory as you would normally. They’ll go to the ACD trash.
LikeLike
Are you doing anything special after the mounting? My folders are completely empty when mounted even though the content is on Amazon. I’ve rebuilt my servers numerous times using ACD_CLI and once the XML is copied and acd_cli sync is run everything shows up.
With this method nothing is showing up so trying to determine what i’m doing wrong or missing
LikeLike
Thanks so much for your reply. I was basically following this guide up until where it says “move settings file”, but instead of moving the newly generated settings file I uploaded the old one. That apparently worked, as I can see the old folder names in my /.local folder.
I moved on and followed the guide again. I got to step six “Upload Local Files/Directories to ACD using rclone” and when I try the rclone copy /home/plex/.local/”old encrypted folder name” I get “Failed to copy: failed to make directory: HTTP code 409: “409 conflict”: response body: “{\”code\”:\”NAME_ALREADY_EXISTS\”
For the record, I still can’t see anything on either my encrypted or unencrypted folders when looking at them through filezilla. The files are still in the amazon cloud drive.
Any idea what may be going on?
LikeLike
If you run a mount command, can you see both encfs mounts and the unionfs mount?
LikeLike
This is what I see
encfs on /home/plex/acd type fuse.encfs (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
encfs on /home/plex/local type fuse.encfs (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
unionfs-fuse on /home/plex/media type fuse.unionfs-fuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000,default_permissions)
acd:Plex on /home/plex/.acd type fuse.rclone (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
LikeLike
Alright so when you say your folders are empty, are we talking in the media directory or the acd directory? Can you check both? The acd directory should show unencrypted ACD
LikeLike
Correct. Both the Media and ACD folder is empty. If I create folders with the same name in Local as they are on Amazon they are identical in name so I know the encryption key is right. The only difference with this server to the other ones i’ve built are i’m using acd_cli on the others and use the following folder structure
.local-sorted
local-sorted
.acd-sorted
acd-sorted
sorted (fuse)
I don’t think this is the problem as i’m just changing the name in the ENCFS command. The directories in those folders are the same as your documentation
TV Shows
Movies
I’m going to rebuild it from scratch and see if I have the same issue. Maybe i’m mixing it up somewhere. I’m not that great with knowing the ins and outs of ubuntu \ linux yet
LikeLike
Hi all
Newbie question about this – how much space do you need to have locally? For example, I have 3TB of media on a NAS which is nearing end of life. I have a laptop with a 300gb HD which I would like to use with this ACD setup but does media stream direct from ACD or get pulled down on the fly and then get streamed from local source?
If it’s streamed from ACD, does the stream route via my broadband line to remote friends or does the ACD stream go direct from Amazon to remote friends and thereby cutting my home out of the equation (apart from marking Plex as the item is watched)?
Thanks
LikeLike
The files are streamed from ACD to your server, then out to the clients. 300GB should be plenty as long as you’re uploading media to ACD as it’s added. Serving remote clients will still depend on your upload speed, as your server will still act as the “middle man” to other people streaming.
LikeLike
Excellent. Thanks for the reply. I feel Windows is going to be wipped off the laptop and Ubuntu installed instead.
I would certainly be interested in the guides for automating, renaming etc media with CP and Sonarr.
Thanks
LikeLike
Thanks enztv for the great work with those tutorials and scripts! Any chance to update the tutorial with encfs –reverse? The current setup without the reverse flag cause from cpu load issues to networks bottlenecks.
LikeLike
I am talking about the discussion going on in the comments from this post: https://www.reddit.com/r/DataHoarder/comments/57ygee/encfs_and_reverse_option_explained/
especially between Kysersoze79 and mrafcho001
LikeLike
Hi, I´m triying to do the same on a Debian headless server but I´m not able to finish the rclone config beause when I enter mi amazon cloud credentials I get a message that says that I have to enter again my password and type the caracters from a captcha that I can´t see on the w3m browser.
Is there any way to avoid that captcha?
Thanks!
LikeLike
Hi again.
I solved it using the rclone authorize method ( http://rclone.org/remote_setup/) from mi Ubuntu laptop.
Thanks and sorry for the stupid question!
LikeLike
Hey there,
I followed your guide till “Step 7 – Use Crontab to Mount rClone, EncFS, and UnionFS Automatically (UPDATE)” but after I move my files to ACD I can’t see them in my media folder. Also my /home/plex/acd folder seems to be a little bit weird:
plex@vpsxxxxx:~$ cd acd
plex@vpsxxxxx:~/acd$ ls
ls: cannot open directory ‘.’: No such file or directory
plex@vps386953:~/acd$ ls -la
ls: cannot open directory ‘.’: No such file or directory
plex@vps386953:~/acd$ cd ..
plex@vps386953:~$ ls -la
ls: cannot access ‘acd’: No such file or directory
total 56
d????????? ? ? ? ? ? acd
drwxrwxr-x 1 plex plex 0 Mar 14 12:08 .acd
drwxrwxr-x 4 plex plex 4096 Mar 14 12:09 local
drwxrwxr-x 4 plex plex 4096 Mar 14 12:09 .local
drwxrwxr-x 1 plex plex 4096 Mar 14 12:09 media
LikeLike
What about rclone crypt?
How to rename folder in /home/plex/acd? (Input/Output error)
LikeLike
i followed the guide and i am stucking in the verification…
rclone seems to run fine exept the sync isnt working.
I mean i can upload and download files to acd this works.
But the files and folders will not synced with the acd folder.
has someone a hint for me why?
LikeLike
ok now i found out that the mount basically will work but only an inital time… if i move or create files and folders it will not synced to the acd folder… but if i remount the the drive it sees the new files…
i can too not copy any data to the acd folder…
LikeLike
Hi!
Tank you for your excellent guid. I have short question about encfs, “souldn’t it say “ENCFS6_CONFIG=’/home/plex/encfs.xml’ encfs –reverse /home/plex/loca /home/plex/.locall” instead of “ENCFS6_CONFIG=’/home/plex/encfs.xml’ encfs /home/plex/.local /home/plex/local”?
At least this is how i understand the encfs wiki…
–reverse
Normally EncFS provides a plaintext view of data on demand: it stores enciphered data and displays
plaintext data. With –reverse it takes as source plaintext data and produces enciphered data on-
demand. This can be useful for creating remote encrypted backups, where you do not wish to keep the
local files unencrypted.
LikeLike
Hi!
Thank you for your excellent guide. I have a short question about encfs, souldn’t it say “ENCFS6_CONFIG=’/home/plex/encfs.xml’ encfs –reverse /home/plex/loca /home/plex/.locall” instead of “ENCFS6_CONFIG=’/home/plex/encfs.xml’ encfs /home/plex/.local /home/plex/local”?
At least this is how i understand the encfs wiki…
–reverse
Normally EncFS provides a plaintext view of data on demand: it stores enciphered data and displays
plaintext data. With –reverse it takes as source plaintext data and produces enciphered data on-
demand. This can be useful for creating remote encrypted backups, where you do not wish to keep the
local files unencrypted.
LikeLike
Hi there,
nice guide. Was really helpful. I am facing a problem. Where when I scan the library In plex it starts to work out fine. But after Servers minutes the mountpoint gets disconnected. When I try to acces via ssh. The output is: cannot access. Transport endpoint is not connected. I can clear and reattach to again get access to ACD but the scan didn’t finished by the time. Any ideas on why I get this problem?
LikeLike
If you got issues with unmount ( “fusermount: failed to unmount /home/plex/local: Device or resource busy”) , use it with force command : fusermount -uz /home/plex/local
LikeLike
Have you noticed any performance issues with using this method? Since switching from ADC_CLI I’ve noticed the media streaming getting slower and slower to start. At the end of every stream it stops to buffer with literally 20 – 30 seconds left of play. Worked flawlessly for a number of weeks and then gradually just started going downhill.
I’ve tried amending the mount script with some flags but that doesn’t appear to be solving the issue. Doing some research people are saying to do the following but for some reason every time I run this it won’t mount acd properly
rclone mount –max-read-ahead 200M –transfers 20 –checkers 40 –read-only –allow-other –acd-templink-threshold 0 –stats 1s –buffer-size 1G acd: /home/plex/.acd &
I’m wondering if I go back to using ACD_CLI to mount and use rClone for everything else. Not sure how it will play when it comes to syncing and updating though.
LikeLike
hi there,
how can I setup that “torrents” go to the “/home/plex/local” directory ?
Transmission starts to download, but after 2-5 seconds it stops with “file exists” error message.
Downloading into /home/plex/downloads works normal
But than I have to copy the files over to ~/local which costs me 2 times HDD space
Hardlinks are not working for me because of “Invalid cross-device link” error
😦
LikeLike
I am having a problem. When I issue the command rm -Rf “/home/plex/local/TV Shows/”* it removes the directories from local but it also breaks Emby and Sickrage which I use. I point them both to the media folder like the setup says but the shows are gone after the rm command. Am I pointing my programs to the wrong location?
LikeLike
“/home/plex/local/TV Shows/*” will work better (last two characters reversed). That way it will delete the content of the folder but will leave the TV Shows folder existing.
LikeLike
you can always add a “.protected” file in cruccial folders by root, that they are not deleted
with command “chattr”
https://unix.stackexchange.com/a/286376
LikeLiked by 1 person
Is there a reason why you didn’t just use the built in rclone encryption? Is it not as secure as EncFS?
LikeLike
Really only because when I first started using ACD as a storage backend, rClone wasn’t capable of doing a “live” mount and I was using acd_cli, which didn’t offer encryption. Though, going through that process made me realize that it’s best in some cases NOT to use built-in encryption with these different mounting programs because what happens when (if) a better alternative to rClone comes around? You’d be handicapped to using rClone because of encryption. So I think using a third-party option keeps you flexible down the road. I can mount my drive on any machine and install encfs to access it. I can’t be guaranteed to get rClone working on any machine, and if I used the built-in encryption, that means my files are at risk of being essentially gone.
That’s not to discredit the encryption features in rClone. If you want to use that and are okay with exclusively using rClone indefinitely, you definitely can.
LikeLike
is anybody else getting those error 429 rate limit exceeded errors today when trying to upload?
LikeLike
It’s definitely a widespread issue, all indications are that it’s an issue with Amazon’s API. As of this moment, I’ve noticed my directory listings succeeding every few tries, which is an improvement from not at all earlier today.
LikeLike
Do you know how I would go about making this work in such a way that 2 separate servers could have the same contents of the media folder, accessing the same encrypted shares?
LikeLike
Just set up rClone and encFS on the second server and copy over your encfs.xml file, then use the same mount commands/directories.
LikeLike
Thanks for the help. I am having two issues.
1) I setup rClone & encFS on my second server and copied over my encfs file, and used the same mount commands. This did create identical encrypted structures on both machines and on ACD. However, I am also trying to replicate the media folder from server 1 to server 2 automatically. So, to be clear, Server 1 is where the files will reside and be moved to the encrypted ACD, but my Plex is on Server 2, so I need to propagate the media folder on Server 2 from Server 1.
2) I am having a problem on Server 2, in that
rclone copy /home/plex/.local acd:Plex
works perfectly, puts everything on ACD, and shows up perfectly in my media folder. But, when I free the space with:
rm -Rf “/home/plex/local/TV Shows/”*
It deletes everything, everywhere (ACD, media folder, etc.). This does not happen on Server 1, and I cannot figure out where I may have gone wrong.
LikeLike
So, me again. I managed to create a new server and it was running perfectly, but then I rebooted it and it stopped working.
So I am retracing my steps, and when I try to do rclone mount acd:Plex /home/plex/.acd & I guet the following
[1] 24239mount helper error: fusermount: failed to access mountpoint /home/plex/.acd: Permission denied
Fatal error: failed to mount FUSE fs: fusermount: exit status 1
[1]+ Exit 1
Any idea what may be causing this?
Thanks!
LikeLike
Check the ownership of that directory and make sure it’s owned by the user mounting rClone
LikeLike
So, I used ls -la and for the five directories that we create (local, .local, acd, .acd, media) I got the same sequence: drwxrwxr-x with the user plex and the owner plex. So, unless I’m doing something wrong with the command, rclone mount acd:Plex /home/plex/.acd & should work, right?
LikeLike
Pingback: Plex – a network error occurred | Helene Klungvik
Hello, thanks for this write up. Having an issue with Plex seeing the directories under the unionfs-fuse mount. I can see everything just fine from the cli but Plex will not see the subdirectories of the media directory. In order to see my media I have to browse with Plex to the /acd/plex path. Thanks Again
LikeLike
Did you use the allow_other option?
LikeLike
well… now that ACD has booted rclone as well.. and won’t accept anymore third party apps… i guess that party is over : (
are there any other options?
luckily (i think?) i never switched to rclone crypt… used encfs.
what options are there to see/download the files unencrypted.. are there really NONE now for linux servers? if not what are the windows encfs options with acd?
are those official non ebay gsuite google drive accounts really unlimited with less than 5 users (on the rclone forum everyone seems to be saying they are) and does gdrive really work with plex (i read so many tales of account banning from just updating the library.)?
LikeLike
As it stands, if this issue with ACD is permanent, your only real option is to download files from the web portal and place them manually into your current directories and encfs will decrypt them. However I’d hold off on panicking just yet, because last time there were 429 errors en masse, the web portal was also finicky, and at least for me the web portal has been very slow and temperamental.
As far as the GDrive accounts go, you actually can just create a business account with one user and get unlimited drive storage for $10 per month. That’s what I’ve been doing. Luckily I moved over to GDrive as my primary storage backend a few months ago, so this ACD outage has been a minor inconvenience. If you’re using GDrive, rClone will definitely get your account banned (24h) all the time. I’ve been using GDrive-ocaml-fuse and it’s actually the best streaming performance I’ve seen of any of these utilities. My streams start in under three seconds, regardless of bitrate. It’s actually faster than Netflix for me, which is great. It also caches your file/directory structure, so it won’t incur any bans when scanning your libraries. In fact, I’ve pointed Radarr and Sonarr at my ocaml-fuse mount as well as Plex and I’ve had zero issues.
LikeLike
I think this is a permanent ban. A lot of rclone users contaced Amazon support that confirm it.
I am currently transferring ACD Plex encrypted folder to a GDrive folder (with MultiCloud).
The ACD setup was exactly as in the tutorial here with encryption and fuse.
Will I be able to to switch to ocaml-fuse and decrypt the data?
LikeLike
You’ll have to mount clone crypt over your ocaml-fuse mount, but yes
LikeLike
Hi, I want to know if anybody tried this with a Google Drive acount. I have an unlimited acount and I want to do the same as I did with my ACD acount but I´m having some issues with the atuthorization on rclone.
Anybody did it? Thanks!
LikeLike
It’s definitely possible. Just don’t use rClone to mount, use gdrive-ocaml-fuse instead
LikeLike
What command are you using when mounting gdrive-ocaml-fuse? I’m using –
/usr/bin/google-drive-ocamlfuse -o allow_other,ro,atomic_o_trunc,large_read,default_permissions “$HOME/.acd-sorted”
It’s reading the files but when adding the folder to Plex it’s taking forever for things to show up.
Thanks
LikeLike
Hello,
is there a step-by-step on getting my encrypted data off of ACD? I tried this . . .
“your only real option is to download files from the web portal and place them manually into your current directories and encfs will decrypt them.”
. . . but not sure what that did. I am not that great with Linux so more details on this would be great.
Thanks
LikeLike
I’ll have to do some further testing on this, I’m not sure if the directory structure is important when decrypting. If it is, then people will be in for a miserable time because the only way to truly get your files would be to fully download each root directory of TV and Movies. If the structure isn’t necessary for decrypting individual files, things might go a bit smoother. Either way you’d need ample disk space. Let me do some testing for some options.
LikeLiked by 1 person
Thanks, I would appriciate any help you can provide. I will test things too and see what / if I can come up with. LOL, not worried about space as my QNAP is on the way. 🙂
LikeLike
i was about to post the same questions as Steve…
this guide was such a help for me (and everything was runing perfect.. especially after the switch to rclone mount) : ) : (
but i was wondering.. if someone had a step by step (idiot/non linux expert) guide to moving everything to a google drive/suite account…
someone was nice enough to post this run down on the rclone forums..
but i’m wondering what commands i need to change in here… assuming my amazon mount is called “acd” and everything is encrypted with encfs on amazon in a remote called acd..
thanks.
——————
“How to migrate 15TB ACD to GSUITE (with only the GSUITE cost)
I am transfering at 55-137MB/s! average: ~120MB/s
First Step: Get G Suite Account
1. signed up for G Suite trial.
2. skipped setup assistant.
3. added my domain via domain verification.
4. upgraded to G Suite Business (8$ p month) (still on trial though! :slight_smile: )
Second Step: Get Google Compute Account
You get 300$ to spend for free on signup!+
1. Register for free 300$ at Google Compute.
2. Add billing details.
3. Add a budget and limit it to 1$, just to be sure!
4. Upgrade Google Compute to subscription (Is still on trial after uprade) (you have to, otherwise you only have 2TB disk space limit) (this will get you a 10TB limit)
5. Request a additional +10TB contigent at “region” of your choice via Google Compute interface. I have a german ACD account, thererfore I use “eu-west 1” region. It only took 2min to approve! Bravo Google!
6. Create Custom Instance VM with 2vCPU + 1.8GB RAM + 20 TB persistent HDD
7. Wait for it to boot up and login via SSH
Third Step: Use odrive to sync to VM instance
1. Register free odrive account
2. Get linux odrive sync agent
3. setup a mount
4. recursive sync all directories
5. recursive sync all files (with 3 parallel sync jobs)
Tip: use ‘screen’. otherwise odrive sync jobs stop on connection abort / ssh session end!
details see here: https://forum.rclone.org/t/acd-429-too-many-requests/1792/525?u=philip_konighofer
Fourth Step: Use rclone to sync to google
1. you should know what todo. 😉
Beware: Since you are syncing the already encrypted files, you have to create a PLAIN remote.
Fifth Step: Use rclone with your CRYPT remote
Once finished uploading files files, switch to encrypted remote.
Sixth Step: Get ACD refund
Seventh Step: Unsubscribe Google Compute
1. Stop VM instance
2. Delete disk image
3. unsubcribe (while still in trial period)
4. remove your billing information
LikeLike
I’m doing the odrive sync down to Google Platform now. All the content that was waiting to be synced to Amazon when it was cut off is now being synced to Google and working on Plex. I set everything up with rClone and to avoid the 24 hour ban found some instructions on setting up your own client_id on the Google API Console.
1) Log into the Google API Console (https://console.developers.google.com/) with your Google account. It doesn’t matter what Google account you use. (It need not be the same account as the Google Drive you want to access)
2) Select a project or create a new project.
3) Under Overview, Google APIs, Google Apps APIs, click “Drive API”, then “Enable”.
4) Click “Credentials” in the left-side panel (not “Go to credentials”, which opens the wizard), then “Create credentials”, then “OAuth client ID”. It will prompt you to set the OAuth consent screen product name, if you haven’t set one already.
5) Choose an application type of “other”, and click “Create”. (the default name is fine)
It will show you a client ID and client secret. Use these values in rclone config to add a new remote or edit an existing remote.
LikeLike
Can you give a short guide on how to copy over those files via odrive from ACD to GD? Do i Have to Download Them to the local Server First?
LikeLike
I have to redo mine as the server ran out of space (I thought the 20tb was available on sda1 but was only 2tb). Basically, my understanding is you add Amazon in odrive (web) and then install odrive agent on Linux, mount and sync down to the local machine.
You then use rclone to sync over to google.
LikeLike
To install odrive Linux (64 bit)
Web
1) create free odrive account
2) add Amazon Drive
SSH into VM
1) Download and install odrive agent
od=”$HOME/.odrive-agent/bin” && curl -L “http://dl.odrive.com/odrive-py” –create-dirs -o “$od/odrive.py” && curl -L “http://dl.odrive.com/odriveagent-lnx-64” | tar -xvzf- -C “$od/” && curl -L “http://dl.odrive.com/odrivecli-lnx-64” | tar -xvzf- -C “$od/”
2 ) Run odrive agent
nohup “$HOME/.odrive-agent/bin/odriveagent” > /dev/null 2>&1 &
3) Verify agent is running
python “$HOME/.odrive-agent/bin/odrive.py” status
4) Authenticate the agent
python “$HOME/.odrive-agent/bin/odrive.py” authenticate (your auth key can be found in odrive web portal)
5) Create directory where you are going to sync your files
mkdir “$HOME/odrive-agent-mount”
(if you are using a mounted drive you will need to change the path)
6) Mount odrive
python “$HOME/.odrive-agent/bin/odrive.py” mount “$HOME/odrive-agent-mount” /
(if you are using a mounted drive you will need to change the path)
7) Sync folders to VM
find ~/odrive-agent-mount/ -type f -name “*.cloudf” -exec python “$HOME/.odrive-agent/bin/odrive.py” sync “{}” \;
(keep running this command until no further outputs occur)
8) run 3 screen sessions
screen -S sync1
screen -S sync3
screen -S sync3
9) Sync folders to VM (run this command in each screen session)
find ~/odrive-agent-mount/ -type f -name “*.cloud” -exec python “$HOME/.odrive-agent/bin/odrive.py” sync {} \;
This will sync everything down to your VM. Once this has completed you can sync files to Google (you can also do it whilst files are syncing from Amazon). To do this you will need to install rClone on the VM. Make sure you have created your API auth tokens in Google in order to set up rClone as you will need them when setting up the mount.
I’m not a Linux expert so have been fumbling my way around this but my content is now starting to appear back in Plex again so I must’ve done something right :). Hope this helps.
LikeLike
stick to gdrive-ocaml-fuse. Even with using a generated API I got banned for 24 hours this afternoon 😦
LikeLike
I changed a bit the mount.sh (replaced rclone .acd mount with
google-drive-ocamlfuse ) and all is fine. But with what do we replace the syncing abilities of rclone now?
LikeLike
You can keep using rClone to sync and upload
LikeLike
FWIW
I’ve managed to get half of my Plex library over to Google. It’s up and running and the performance is much better than Amazon. I have 2 Dedicated Servers
1 x Downloader (Sonarr, Radarr, Sab) with rclone and unionfuse going
1 x Plex Server
On the Plex Server I’m mounting google using PlexDrive and haven’t been banned since implementing. I tweaked the script provided by enztv by removing the rclone mount and putting in PlexDrive
LikeLike
when setting up rclone I get
an error occurs when we tried to process your request (amazon web page)
LikeLike
I’m running all the components of my media server (sonarr, radarr, nzbget) in docker containers. rClone is installed on my host. How can I get sonarr post-processing script that perform rClone copy command to run on the host?
LikeLike
Now that Amazon is dead, just wondering what poepl are using for unlimited storage google or other providers?
LikeLike
So I’ve read over this so many times I guess I’m confused. I’m putting things from Sonaar etc into the movie folder then I copy it to local it encrypts and I upload it to Google. Isn’t the stuff I downloaded in media still left behind? Won’t this fill up? I feel like I should point Plex to the acd folder not media.
LikeLike
Yeah so I deleted what was in Media and now it shows .unionfs-fuse in the media folder. This is coming over from the link I mad from acd right but why is it a hidden folder? Plex can’t see this is it suppose to?
LikeLike
Interesting read! Thank you for the well & detailed explanation on Amazon Cloud Drive with Plex Media Server on Ubuntu. Keep posting.
LikeLike
I am glad you came up with this post! I was looking for the same, this has helped me a lot. Expecting new posts from you on other Amazon Cloud aspects.
LikeLike