- cross-posted to:
- donoperinfo@infosec.pub
- selfhosted@lemmy.world
- cross-posted to:
- donoperinfo@infosec.pub
- selfhosted@lemmy.world
The complete guide to building your personal self hosted server for streaming and ad-blocking.
Captain’s note: This OC
was originally posted in reddit but its quality makes me wants to ensure a copy survices in lemmy as well.
We will setup the following applications in this guide:
- Docker
- AdguardHome - Adblocker for all your devices
- Jellyfin/Plex - For watching the content you download
- Qbittorrent - Torrent downloader
- Jackett - Torrent indexers provider
- Flaresolverr - For auto solving captcha in some of the indexers
- Sonarr - *arr service for automatically downloading TV shows
- Radarr - *arr service for movies
- Readarr - *arr service for (audio)books
- lidarr - *arr service for music
- Bazarr - Automatically downloads subtitles for Sonarr and Radarr
- Ombi/Overseer - For requesting movies and tv shows through Sonarr and Radarr
- Heimdall - Dashboard for all the services so you don’t need to remember all the ports
Once you are done, your dashboard will look something like this.
I started building my setup after reading this guide https://www.reddit.com/r/Piracy/comments/ma1hlm/the_complete_guide_to_building_your_own_personal/.
Hardware
You don’t need powerful hardware to set this up. I use a decade old computer, with the following hardware. Raspberry pi works fine.
Operating system
I will be using Ubuntu server in this guide. You can select whatever linux distro you prefer.
Download ubuntu server from https://ubuntu.com/download/server. Create a bootable USB drive using rufus or any other software(I prefer ventoy). Plug the usb on your computer, and select the usb drive from the boot menu and install ubuntu server. Follow the steps to install and configure ubuntu, and make sure to check “Install OpenSSH server”. Don’t install docker during the setup as the snap version is installed.
Once installation finishes you can now reboot and connect to your machine remotely using ssh.
ssh username@server-ip
# username you selected during installation
# Type ip a to find out the ip address of your server. Will be present against device like **enp4s0** prefixed with 192.168.
Create the directories for audiobooks, books, movies, music and tv.
I keep all my media at ~/server/media. If you will be using multiple drives you can look up how to mount drives.
We will be using hardlinks so once the torrents are downloaded they are linked to media directory as well as torrents directory without using double storage space. Read up the trash-guides to have a better understanding.
mkdir ~/server
mkdir ~/server/media # Media directory
mkdir ~/server/torrents # Torrents
# Creating the directories for torrents
cd ~/server/torrents
mkdir audiobooks books incomplete movies music tv
cd ~/server/media
mkdir audiobooks books movies music tv
Installing docker and docker-compose
Docker https://docs.docker.com/engine/install/ubuntu/
# install packages to allow apt to use a repository over HTTPS
sudo apt-get update
sudo apt-get install \
apt-transport-https \
ca-certificates \
curl \
gnupg \
lsb-release
# Add Docker’s official GPG key:
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
# Setup the repository
echo \
"deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
# Install Docker Engine
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io
# Add user to the docker group to run docker commands without requiring root
sudo usermod -aG docker $(whoami)
Sign out by typing exit in the console and then ssh back in
Docker compose https://docs.docker.com/compose/install/
# Download the current stable release of Docker Compose
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
# Apply executable permissions to the binary
sudo chmod +x /usr/local/bin/docker-compose
Creating the compose file for Adguard home
First setup Adguard home in a new compose file.
Docker compose uses a yml file. All of the files contain version and services object.
Create a directory for keeping the compose files.
mkdir ~/server/compose
mkdir ~/server/compose/adguard-home
vi ~/server/compose/adguard-home/docker-compose.yml
Save the following content to the docker-compose.yml file. You can see here what each port does.
version: '3.3'
services:
run:
container_name: adguardhome
restart: unless-stopped
volumes:
- '/home/${USER}/server/configs/adguardhome/workdir:/opt/adguardhome/work'
- '/home/${USER}/server/configs/adguardhome/confdir:/opt/adguardhome/conf'
ports:
- '53:53/tcp'
- '53:53/udp'
- '67:67/udp'
- '68:68/udp'
- '68:68/tcp'
- '80:80/tcp'
- '443:443/tcp'
- '443:443/udp'
- '3000:3000/tcp'
image: adguard/adguardhome
Save the file and start the container using the following command.
docker-compose up -d
Open up the Adguard home setup on YOUR_SERVER_IP:3000
.
Enable the default filter list from filters→DNS blocklist. You can then add custom filters.
Creating the compose file for media-server
Jackett
Jackett is where you define all your torrent indexers. All the *arr apps use the tornzab feed provided by jackett to search torrents.
There is now an *arr app called prowlarr that is meant to be the replacement for jackett. But the flaresolverr(used for auto solving captchas) support was added very recently and doesn’t work that well as compared to jackett, so I am still sticking with jackett for meantime. You can instead use prowlarr if none of your indexers use captcha.
jackett:
container_name: jackett
image: linuxserver/jackett
environment:
- PUID=1000
- PGID=1000
- TZ=Asia/Kolkata
volumes:
- '/home/${USER}/server/configs/jackett:/config'
- '/home/${USER}/server/torrents:/downloads'
ports:
- '9117:9117'
restart: unless-stopped
prowlarr:
container_name: prowlarr
image: 'hotio/prowlarr:testing'
ports:
- '9696:9696'
environment:
- PUID=1000
- PGID=1000
- TZ=Asia/Kolkata
volumes:
- '/home/${USER}/server/configs/prowlarr:/config'
restart: unless-stopped
Sonarr - TV
Sonarr is a TV show scheduling and searching download program. It will take a list of shows you enjoy, search via Jackett, and add them to the qbittorrent downloads queue.
sonarr:
container_name: sonarr
image: linuxserver/sonarr
environment:
- PUID=1000
- PGID=1000
- TZ=Asia/Kolkata
ports:
- '8989:8989'
volumes:
- '/home/${USER}/server/configs/sonarr:/config'
- '/home/${USER}/server:/data'
restart: unless-stopped
Radarr - Movies
Sonarr but for movies.
radarr:
container_name: radarr
image: linuxserver/radarr
environment:
- PUID=1000
- PGID=1000
- TZ=Asia/Kolkata
ports:
- '7878:7878'
volumes:
- '/home/${USER}/server/configs/radarr:/config'
- '/home/${USER}/server:/data'
restart: unless-stopped
Lidarr - Music
lidarr:
container_name: lidarr
image: ghcr.io/linuxserver/lidarr
environment:
- PUID=1000
- PGID=1000
- TZ=Asia/Kolkata
volumes:
- '/home/${USER}/server/configs/liadarr:/config'
- '/home/${USER}/server:/data'
ports:
- '8686:8686'
restart: unless-stopped
Readarr - Books and AudioBooks
# Notice the different port for the audiobook container
readarr:
container_name: readarr
image: 'hotio/readarr:nightly'
ports:
- '8787:8787'
environment:
- PUID=1000
- PGID=1000
- TZ=Asia/Kolkata
volumes:
- '/home/${USER}/server/configs/readarr:/config'
- '/home/${USER}/server:/data'
restart: unless-stopped
readarr-audio-books:
container_name: readarr-audio-books
image: 'hotio/readarr:nightly'
ports:
- '8786:8787'
environment:
- PUID=1000
- PGID=1000
- TZ=Asia/Kolkata
volumes:
- '/home/${USER}/server/configs/readarr-audio-books:/config'
- '/home/${USER}/server:/data'
restart: unless-stopped
Bazarr - Subtitles
bazarr:
container_name: bazarr
image: ghcr.io/linuxserver/bazarr
environment:
- PUID=1000
- PGID=1000
- TZ=Asia/Kolkata
volumes:
- '/home/${USER}/server/configs/bazarr:/config'
- '/home/${USER}/server:/data'
ports:
- '6767:6767'
restart: unless-stopped
Jellyfin
I personally only use jellyfin because it’s completely free. I still have plex installed because overseerr which is used to request movies and tv shows require plex. But that’s the only role plex has in my setup.
I will talk about the devices section later on.
For the media volume you only need to provide access to the /data/media
directory instead of /data
as jellyfin doesn’t need to know about the torrents.
jellyfin:
container_name: jellyfin
image: ghcr.io/linuxserver/jellyfin
environment:
- PUID=1000
- PGID=1000
- TZ=Asia/Kolkata
ports:
- '8096:8096'
devices:
- '/dev/dri/renderD128:/dev/dri/renderD128'
- '/dev/dri/card0:/dev/dri/card0'
volumes:
- '/home/${USER}/server/configs/jellyfin:/config'
- '/home/${USER}/server/media:/data/media'
restart: unless-stopped
plex:
container_name: plex
image: ghcr.io/linuxserver/plex
ports:
- '32400:32400'
environment:
- PUID=1000
- PGID=1000
- TZ=Asia/Kolkata
- VERSION=docker
volumes:
- '/home/${USER}/server/configs/plex:/config'
- '/home/${USER}/server/media:/data/media'
devices:
- '/dev/dri/renderD128:/dev/dri/renderD128'
- '/dev/dri/card0:/dev/dri/card0'
restart: unless-stopped
Overseer/Ombi - Requesting Movies and TV shows
I use both. You can use ombi only if you don’t plan to install plex.
ombi:
container_name: ombi
image: ghcr.io/linuxserver/ombi
environment:
- PUID=1000
- PGID=1000
- TZ=Asia/Kolkata
volumes:
- '/home/${USER}/server/configs/ombi:/config'
ports:
- '3579:3579'
restart: unless-stopped
overseerr:
container_name: overseerr
image: ghcr.io/linuxserver/overseerr
environment:
- PUID=1000
- PGID=1000
- TZ=Asia/Kolkata
volumes:
- '/home/${USER}/server/configs/overseerr:/config'
ports:
- '5055:5055'
restart: unless-stopped
Qbittorrent - Torrent downloader
I use qflood container. Flood provides a nice UI and this image automatically manages the connection between qbittorrent and flood.
Qbittorrent only needs access to torrent directory, and not the complete data directory.
qflood:
container_name: qflood
image: hotio/qflood
ports:
- "8080:8080"
- "3005:3000"
environment:
- PUID=1000
- PGID=1000
- UMASK=002
- TZ=Asia/Kolkata
- FLOOD_AUTH=false
volumes:
- '/home/${USER}/server/configs/qflood:/config'
- '/home/${USER}/server/torrents:/data/torrents'
restart: unless-stopped
Heimdall - Dashboard
There are multiple dashboard applications but I use Heimdall.
heimdall:
container_name: heimdall
image: ghcr.io/linuxserver/heimdall
environment:
- PUID=1000
- PGID=1000
- TZ=Asia/Kolkata
volumes:
- '/home/${USER}/server/configs/heimdall:/config'
ports:
- 8090:80
restart: unless-stopped
Flaresolverr - Solves cloudflare captcha
If your indexers use captcha, you will need flaresolverr for them.
flaresolverr:
container_name: flaresolverr
image: 'ghcr.io/flaresolverr/flaresolverr:latest'
ports:
- '8191:8191'
environment:
- PUID=1000
- PGID=1000
- TZ=Asia/Kolkata
restart: unless-stopped
Transcoding
As I mentioned in the jellyfin section there is a section in the conmpose file as “devices”. It is used for transcoding. If you don’t include that section, whenever transcoding happens it will only use CPU. In order to utilise your gpu the devices must be passed on to the container.
https://jellyfin.org/docs/general/administration/hardware-acceleration.html Read up this guide to setup hardware acceleration for your gpu.
Generally, the devices are same for intel gpu transcoding.
devices:
- '/dev/dri/renderD128:/dev/dri/renderD128'
- '/dev/dri/card0:/dev/dri/card0'
To monitor the gpu usage install intel-gpu-tools
sudo apt install intel-gpu-tools
Now, create a compose file for media server.
mkdir ~/server/compose/media-server
vi ~/server/compose/media-server/docker-compose.yml
And copy all the containers you want to use under services. Remember to add the version string just like adguard home compose file.
Configuring the docker stack
Start the containers using the same command we used to start the adguard home container.
docker-compose up -d
Jackett
Navigate to YOUR_SERVER_IP:9117
Add a few indexers to jackett using the “add indexer” button. You can see the indexers I use in the image below.
Qbittorrent
Navigate to YOUR_SERVER_IP:8080
The default username is admin
and password adminadmin
. You can change the user and password by going to Tools → Options → WebUI
Change “Default Save Path” in WebUI section to /data/torrents/
and “Keep incomplete torrents in” to /data/torrents/incomplete/
Create categories by right clicking on sidebar under category. Type category as TV
and path as tv
. Path needs to be same as the folder you created to store your media. Similarly for movies type Movies
as category and path as movies
. This will enable to automatically move the media to its correct folder.
Sonarr
Navigate to YOUR_SERVER_IP:8989
- Under “Download Clients” add qbittorrent. Enter the host as
YOUR_SERVER_IP
port as**8080
,** and the username and password you used for qbittorrent. In category typeTV
(or whatever you selected as category name(not path) on qbittorent). Test the connection and then save. - Under indexers, for each indexer you added in Jackett
- Click on add button
- Select Torzab
- Copy the tornzab feed for the indexer from jackett
- Copy the api key from jackett
- Select the categories you want
- Test and save
- Under general, define the root folder as
/data/media/tv
Repeat this process for Radarr, Lidarr and readarr.
Use /data/media/movies
as root for Radarr and so on.
The setup for ombi/overseerr is super simple. Just hit the url and follow the on screen instructions.
Bazarr
Navigate to YOUR_SERVER_IP:6767
Go to settings and then sonarr. Enter the host as YOUR_SERVER_IP
port as 8989
. Copy the api key from sonarr settings→general.
Similarly for radarr, enter the host as YOUR_SERVER_IP
port as 7878
. Copy the api key from radarr settings→general.
Jellyfin
Go to YOUR_SERVER_IP:8096
- Add all the libraries by selecting content type and then giving a name for that library. Select the particular library location from
/data/media
. Repeat this for movies, tv, music, books and audiobooks. - Go to dashboard→playback, and enable transcoding by selecting as
VAAPI
and enter the device as/dev/dri/renderD128
Monitor GPU usage while playing content using
sudo intel_gpu_top
Heimdall
Navigate to YOUR_SERVER_IP:8090
Setup all the services you use so you don’t need to remember the ports like I showed in the first screenshot.
Updating docker images
With docker compose updates are very easy.
- Navigate to the compose file directory
~/server/compose/media-server
. - Then
docker-compose pull
to download the latest images. - And finally
docker-compose up -d
to use the latest images. - Remove old images by
docker system prune -a
What’s next
- You can setup VPN if torrents are blocked by your ISP/Country. I wanted to keep this guide simple and I don’t use VPN for my server, so I have left out the VPN part.
- You can read about port forwarding to access your server over the internet.
Nice guide! However, I’ve always wondered if all of these even make sense. Like, can’t you just stream from the internet? I understand having thing on your physical storage device is an extra degree of freedom but it’s very rare for me watching something more then once. Also while you can technically run it off a Raspberry Pi, it’s not really recommended and you would need a separate PC which just adds to the cost. Meanwhile, with a simple app like Cloudstream, you can just get whatever you want whenever you want. The only advantage I see of the *arr +media server approach is not needing to connect to a VPN.
EDIT: After reading the replys just realized I should have specified by streaming sites I mean the shady ones, in my country we use different words and I see how that can confuse some people
I used to be in your camp, but then switched to plex setup etc.
Main reasons:
I’m seeing the trend of media being removed from people and I’m getting sick of it. I want my shit to be mine and available to me at a moments notice.
My collection basically exists if all top movies / shows that I can rotate watching.
It makes it so that my tech illiterate family can enjoy everything too without knowing how anything works.
I could cancel all those greedy corporate assholes splitting everything into a thousand services.
not discrediting you, this is just my point of view. Media being removed is in not really a problem on streaming sites since there’s usually many where you can watch the same thing, and as for point 4 streaming sites are basically the same.
I guess it’s just different usage because I don’t really like rewatching things and my family doesn’t usually watch movies/TV series.
So in the end the only thing I don’t like with how I do it is not being able to physically have the files
EDIT: I just realized I should have specified by streaming sites I mean the shady ones, in my country we use different words
I meant free streaming sites with reuploads, but the other point still stand strong, thanks
The nature of pirating means that specific media/torrents/indexes/domains are frequently down or unavailable. A solution today might be taken down or raided by authorities tomorrow.
It’s just a little more stable/redundant to have media stored locally. Plus, by streaming from something like cloud stream, you’re not contributing to torrent seeding, not to mention that a turnkey solution is a large target for authorities, so it’s possible if not likely that it’ll stop working someday and you’ll have to find something else.
It’s not for everyone certainly, but if you can afford it it’s a nice solution.
Either the content will have a lower bitrate or lower resolution.
Personally I just think it’s easier to pick out the movies and shows I want to watch, and then be sure that they will be there once I sit down to watch them. No uncertainty, no hunting down a good stream or missing episode, everything is just there and ready. The process is very simple once everything is set up, and you can still delete video files after you watch them if you want to.
It’s all about use case. You don’t rewatch shows or movies, so maybe storing media isn’t for you. I’m often rewatching or just having background stuff playing, so it’s nice having it available.
On top of that, I was tired of streaming services removing content. Netflix removing It’s Always Sunny actually got me started, and the subsequent removal of episodes from shows confirmed I made the right choice. I actually have control over my media, in that I can put a playlist of any number of shows together I want.
I have playlists for 70’s-80’s shows like The Brady Bunch, The A-Team, Knight Rider, just hit shuffle and it’s 1,000 episodes of random nostalgia. I can set up programs like DizqueTV and set up my own TV channels on top of this. Why pick and choose a show when TV can pick for me?
In regards to “the hardware” I ran my Plex server on a Pi3 for years. Unless you’re pushing 4k content or certain filetypes, the Pi is more than enough.
In addition to all this, I’m not reliant on my internet. If power goes out partially, I still have access to my hard drives and have always been able to pop on a show or movie while I clean up in the dark. Or sometimes the internet just goes out and it’s really nice being unaffected.
I think it’s been 7 or 8 years since I started in college, I’ve spent about $600 total on hard drives that I’m still using today? The money I’ve spent is invested into my server, rather than paying some service for something I can do myself. A service that has to submit to the will of the government, I was curious of the price range of Cloudstream and saw that they took the site and code down, so it’s just another streaming situation that’s no different, except the chance of payment being sent to the actual people who worked on the show is now completely gone. Even just $30/month after 5 years is $1,800.
I pirate content because I can’t trust Netflix/Hulu/Disney to not fuck with their content. So why would I pay another 3rd party to do the same thing? Moreover, when I subscribe to these streaming services I can contribute to the metrics to say, “Hey, I want more It’s Always Sunny after S14!”.
Finally - it’s a hobby as well. I like computers. Linux another the shit out of me but I’ve enjoyed setting up a server used for more than just media. On the Pi I would just search for what I wanted and add it as I see fit. Obviously, there’s the *arrs as well which can get it all automated for you. That’s a bit of setup on its own, but it’s fairly straightforward.
Thanks for the reply! First of all, I’m sorry for not having specified that by streaming sites I mean the shady ones where stuff gets reuploaded for free, in my country we use different words and I got confused.
I’m happy to hear a pi can be enough, I’ve seen many times people saying you would need a mini PC for media servers.
As for cloudstream, I don’t know what you found, but I was referring to https://github.com/recloudstream/cloudstream , a free, open source android app with a very nice ui that basically acts asan client for many of the above mentioned shady sites, offering a nice, ad-free, ux
As for the hobby part, I’m with you but I really can’t justify having another thing running if I won’t use it at all
You’re all good :) I figured Cloudstream was on the freer side since I saw it on git when I looked, but it’s not as popular compared to Streamio or Real Debrid which get suggested quite often and are paid, so I always think it’s worth thinking about the 5 year cost and whether that price is worth your efforts doing a hobby (and for something like a media server, can be super hands off).
People often suggest stronger hardware than the pi either because of hardware or use case and experiencs. If you have a 4k or higher definition TV, the Pi definitely isn’t ideal, unless you’re okay with 1080p media playing on it, just as an example of someone who would feel that the Pi isn’t enough. Another example might be someone who doesn’t want to try and work around any potential limitations of the Pi hardware, so they install a lightweight OS to maximize what the server can use, which I’ve done but found it to be, well, it didn’t really matter.
But for anyone who just wants their favorites readily available in one place without a monthly fee, a Pi with external hard drives is very reliable and is something that is accumulative, the hard drives will last. I have the same pair that I bought in college, just a couple 1TB HDDs. It’s pretty much 5 steps, set up your media server and torrent client to the external hard drives, set up and add jackett or Prowlarr to see the torrent client, then add Radarr/sonarr to see Jacket/Prowlarr. If you need a VPN then run the torrent client through that. Basically OP’s guide ;) For the Pi and peace of mind, once you did this then you would copy that SD card image to another SD card to back it up, that way you don’t have to go through that whole process again.
It’s when you start adding more and more self-hosted services is when the Pi is no longer enough, which people in this hobby are likely to start doing. It’s also somewhat the nature of the Pi, they are very often Project PC’s - kiosks, magic mirrors, mostly single service devices - so that’s another reason. If you want a media server, you may inevitably want to use that server for other things, “at which point just invest in a full build!” Lol.
I personally moved away from the Pi only because I moved and never set it back up, then upgraded my gaming PC and had enough parts left over to build a server PC. It’s a b450m with an R5 3600, a 1660, and 16GB of RAM. I have 18 services running and I’ve never gone over 3.5gb of RAM in use, I don’t think the CPU has ever gone over 20%, and the GPU only does transcoding for Plex when it’s not direct playing (which would usually be 4k or file related, so rare). So I do have a little trouble seeing how I would ever need anything more than this now, but then I get into home lab and suddenly 128gb of RAM is looking really thin… It’s alllll about use case :) whenever people suggest stuff, sometimes I’ll look and see what type of stuff they’re into, because if it doesn’t align then I appreciate the input but it’s not something that I’ll ever be doing.
On the Pi 3 I was running the same set up that I have for media now, which I think is 10 services, and if I recall correctly under load I don’t think it ever hit over 75% usage overall. Only transcoding 4k content would make it buffer, otherwise subtitles on 1080p was fine and I could have 3 streams running in my house no problem.
On my current server I’ve added some personal things not media related, like personal photo backup sync and viewer to get away from Google photos, mind maps, and a homepage. Although frankly I think the Pi 4 would be able to handle even these on top of it, Synching and Photoprism are quite light, mind maps just runs an apache server which is nothing and the homepage I’m using does have some overhead occasionally, but I would be surprised if it wasn’t enough. Like I said, all this under load I’m sitting on 12.3gb of RAM free.
Anyway sorry for the lengthy reply about my setup, I got very little sleep lol. Good luck on your endeavors, whatever they may be :)
No worries, I like reading!
I’m already at the lightweight os step since I’m running some services on it at once, but since it’s a rpi4 it doesn’t really use up much of the cpu. Apart from the previously mentioned reasons, another thing holding me back is the storage drive cost, since i can’t really buy many things.
Oh another thing, more of a curiosity, how do you have your HDDs attached to your pi? external dock i would assume?
In the end, as you said, it’s all about usecase -Another sleep deprived person
For the Pi hard drives I went the USB route because it’s slightly less power intensive, and it seemed easier on the external HDDs to be mounted, technically there can be a USB bandwidth issue though I never ran into that luckily.
I would also keep off-site hard drives stored, which are standard sized ones that plug in with SATA. For those I would use an enclosure that I have, however a dock would definitely be the way to go for that.
Cost of entry is definitely understandable. It’s one of the reasons I went the route I did, as I was able to get 1TB hard drives for cheap and use those until I was able to get my first 4TB USB one which lasted me until I retired it just this year (cold storage, still technically “in use” lol).