What are you folks doing for self-hosted storage? Recommendations wanted.

[uacnt83982803]

I am not sure if this is the best place to post this, so pardon if I landed this in the wrong forum.

I have been struggling trying to settle on a solution for self-hosted backups and media storage. Currently, I have a cloud provider (secure) that I trust but capacity is limited (I was an early adopter so got a free account with decent storage, but not enough for everything). If I have to pay for a cloud service, that's fine but then the data is still on someone else's computer. I have photos and data on my laptop, some on my existing cloud storage, and a bunch of photos on A*****.

Ideally, I'd like to have enough capacity that I can start saving all of my videos and movies to whatever this solution will be, so that I can access them within my LAN using my Ubuntu and Windows PC's and Android phones and tablets. I'd estimate my current data volume at about 200GB but that's without any movies, etc.

From what I've read, a NAS with Openmediavault would probably be a good idea. Then the question becomes, what NAS? I don't want something that has to tie to the internet (such as WD's Mycloud). I want it to be standalone (within my LAN, that is) and if I have to access the data from outside the LAN I understand the best way is a VPN into the LAN or SSH.

I toyed with the idea of building a NAS using a Raspberry Pi 4. Looks good and the price is right, but then that's something else I have to manage. And, whatever NAS I use, I'll still need to be backing it up (it's a given, no problem with doing it).

There's always the simpler option of external hard drives, tied to one of my PCs (probably Ubuntu). I could share it, but then speeds will be slow and it won't be any good for media viewing other than opening pictures.

So, I'm really interested in what some of you are doing along these lines. Sorry for the long post but I wanted to give an idea of what my needs are. Thanks in advance for any feedback.

[SeijiSensei]

I back up my local machines and remote servers at Linode to a 4 TB external USB drive overnight. I don't know why you're worried about speed from the backup device; presumably you would be using the original copies of the files not the backups.

Also, as a test, I just watched a few minutes of a 720p Blu-ray rip over the network from the external drive. Had no issues with performance.

[uacnt83982803]

Thanks for your input. Yes, speed of backup functions isn't important. It's speed for viewing media that is important to me.

[TheFu]

Viewing media doesn't usually stress any computer with GigE networking hardware. It isn't like you'll be using USB2 connections, so pretty much any media can be played fine over a wired network using any spinning disks today. Even USB2 connections are fast enough for a single stream of 4K video, provided the networking is solid.

[CelticWarrior]

Viewing media doesn't usually stress any computer with GigE networking hardware. It isn't like you'll be using USB2 connections, so pretty much any media can be played fine over a wired network using any spinning disks today. Even USB2 connections are fast enough for a single stream of 4K video, provided the networking is solid.

+1

Exactly, tried and tested successfully.

[TheFu]

My home-built NAS is a $100 CPU+MB combo from 2014, a dual core G3258 w/ 8G RAM. It has about 32TB connected, runs Plex, mpd, Calibre, and NFS services. It also runs a kvm virtual machine with Nextcloud, ZNC, and Wallabag. Nextcloud is handy sometimes, but most of the NFS storage is provided to it and plex for read-only access. They can't change or delete any content.

$ inxi -Fz
System:    Host: istar Kernel: 5.4.0-64-generic x86_64 bits: 64 Desktop: N/A
           Distro: Ubuntu 18.04.5 LTS
Machine:   Device: desktop Mobo: Gigabyte model: H81M-HD3 v: x.x serial: N/A
           BIOS: American Megatrends v: FA date: 12/01/2014
CPU:       Dual core Intel Pentium G3258 (-MCP-) cache: 3072 KB
           clock speeds: max: 3200 MHz 1: 1421 MHz 2: 1269 MHz
Graphics:  Card: Intel Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller
           Display Server: X.Org 1.20.8
           drivers: modesetting (unloaded: fbdev,vesa)
           Resolution: 1920x1200@59.95hz
Network:   Card-1: Realtek RTL8111/8168/8411 PCIE Gigabit Ethernet Controller
           driver: r8168
           IF: eth0 state: down mac: <filter>
           Card-2: Marvell 88E8001 Gigabit Ethernet Controller driver: skge
           IF: eth1 state: up speed: 1000 Mbps duplex: full mac: <filter>
RAID:      No RAID devices: /proc/mdstat, md_mod kernel module present
Info:      Processes: 230 Uptime: 7 days Memory: 1931.1/7413.7MB
           Client: Shell (bash) inxi: 2.3.56

check
It is very stable, though the Realtek RTL8111 is crap, drops packets, so I replaced it with a $10 (in 2005) cheapo GigE NIC. I really need to swap in an Intel PRO/1000 NIC that is laying around here, but I seldom reboot that box. I don't use RAID on it, but about 50% of the storage is a periodic mirror. OS backups are network based to a different system.

It is a server, not a desktop, not a playback system either. For playback, I use raspberry pi v2 or v3 systems or Android tablets for video. For audio, I use mpd (mostly), but sometimes I'll pull audio files to a tablet using nextcloud and play them using local apps and storage. Music and audiobooks need completely different playback apps.

This system doesn't run any VPN, but my VPN has complete access into the LAN - or it used to a year ago - which was the last time I checked. We've been hold up here almost a year now - with just 5 grocery trips into the world. Anyways, I don't know if the VPN still works. Don't have any way to really validate it from home.

I'm not a fan of the all-in-one storage server idea. To me, they are too constraining, inflexible. But I can understand why someone would seek out a total solution like FreeNAS (or whatever the name is today). They see a list of features that appear to be 1-click installs, and hear lots of zealots who love their version of it. Guess I'm a zealot too - just for using the flexibility of Linux, which has no limits. But it is harder since the skills to make good choices on an unbounded solution take time to learn.

BTW, I'm looking at replacing Plex Server with Jellyfin. Going by DNS query counts, Plex Server is the 3rd most privacy sucking device on my LANs. Jellyfin appears to do all that Plex does, but it is 100% F/LOSS.

[C.S.Cameron]

I have a 1TB internal SSD and couple of Seagate 4TB external drives for data, TV, Movies and OTR, they are getting full, so I recently added a 4TB 3.5" with external enclosure for overflow, I could not find another Seagate locally here in Southern Provence. Have four old 1TB drives for back up of backup. I have quite a few drives back in Canada with old operating systems and other backup on them. Internet is expensive here at ~ 1500 RS, ($12) for 100GB/Mo. I normally copy the movies and TV to a USB flash drive for viewing and my Old Time Radio and Texted to Speech e-books to MP3 player for listening.

[CatKiller]

I'm not a fan of the all-in-one storage server idea. To me, they are too constraining, inflexible. But I can understand why someone would seek out a total solution like FreeNAS (or whatever the name is today). They see a list of features that appear to be 1-click installs, and hear lots of zealots who love their version of it. Guess I'm a zealot too - just for using the flexibility of Linux, which has no limits. But it is harder since the skills to make good choices on an unbounded solution take time to learn.

Similarly, I appreciate that there are reasons why people would like the flexibility of rolling their own and the opportunity to develop their skills, particularly if they're already coming from a systems administration or network administration background, but I wouldn't want that for myself. Luckily, it's a diverse field with a spectrum of options.

Picking some parts out of the OP:

Ideally, I'd like to have enough capacity that I can start saving all of my videos and movies to whatever this solution will be, so that I can access them within my LAN using my Ubuntu and Windows PC's and Android phones and tablets. I'd estimate my current data volume at about 200GB but that's without any movies, etc.

I want it to be standalone (within my LAN, that is) and if I have to access the data from outside the LAN I understand the best way is a VPN into the LAN or SSH.

I toyed with the idea of building a NAS using a Raspberry Pi 4. Looks good and the price is right, but then that's something else I have to manage. And, whatever NAS I use, I'll still need to be backing it up (it's a given, no problem with doing it).

The solution I went for, which I think would tick all the OP's boxes, is a straightforward appliance. Synology in my case.

You get centralised storage accessible over the LAN using a variety of protocols, including DLNA. Backing up the data to cloud storage or a local cold copy is a one click affair. Maintenance can be done through a web browser if you don't want to use an SSH terminal (although that's an option). Security updates and drive monitoring happen automatically out of the box, with status lights. Services like media transcoding or BitTorrent are just checkboxes.

One could replicate all of that functionality themselves in a less-good case, at higher cost and with sufficient knowledge. If the learning and the control are the reasons for undertaking the project then an appliance wouldn't be an attractive option. If it's the end result that's the important thing, then rolling your own is a less attractive option. Swings and roundabouts.

[uacnt83982803]

I agree with the ease of use by using a packaged NAS solution. I'm just not crazy about routing access from the internet, through Company X, into my network. For one thing, it relies on them staying in business, for another thing it relies on me having an internet connection. If I'm going to do that, I'll just use a hosted cloud provider. Don't get me wrong, that's a great solution for many people, it's just not my preferred strategy.

I decided to take a 2-pronged approach:
1. Obtain a large, USB 3.1-connected external drive. On its way here now. I'll encrypt it with LUKS to protect it. To this drive I'll copy all of my critical data, photos, etc. including pulling down all of my photos on A*****. That solves the immediate, and more pressing, issue of owning my own storage for critical data.
2. Next, I'll work on a NAS solution that I can keep 100% inside my network which will allow me to handle media, etc. and sharing between PCs. Since #1 above will have been accomplished, there's less time pressure. I'm considering building a NAS using the Raspberry Pi platform.

[CatKiller]

I agree with the ease of use by using a packaged NAS solution. I'm just not crazy about routing access from the internet, through Company X, into my network.

Getting security updates from Synology is no different to getting security updates from Canonical. The rest of the traffic stays on your LAN.