Samba + ZFS Very Slow

[jpaytoncfd]

I have been working with ubuntu very hard for about 2 weeks now. I have a upcycled AMD single core 2gb ram and 4 HDD.

3 of the drives are a ZFS Stripe making a 2.1TB volume labeled Data
The other drive is just a IDE 30gb I used as the OS drive.

Anyway when I first got Ubuntu up and running with ZFS and Samba everything seemed to work well. Main use is a NAS/Internet PVR for XBMC. Today while trying to watch a 1080P TV show it would stop randomly. And seeking took forever. 720 and lower not bad but 1080 will not work. So I have been all over google looking for answers. If you search samba, zfs, or both slow I have tried it. Through some testing I made a few conclusions.

1 transfers between zfs and ubuntu system drive (ext2?) are pretty quick. (No bench-marking I could find)

2 transfers from windows to windows through wireless 10-15Mb/s

3 Transfer from ZFS on ubuntu through samba to wireless "n-300" windows client 2-4Mb/s

4 Transfer from ?ext2? to windows on samba 2-4Mb/s

5 Transfer from ZFS through sftp to windows 2-4Mb/s

6 Windows to ZFS or ?ext2? 2-4Mb/s

So from that I figured it must be networking on ubuntu.
Desktop has Nvidia on-board 10/100 and realtek 10/100 PCI.
Tried both, same outcome. Transfers inside the server even between file systems seem pretty fast. I couldn't figure out how to get a speed with the "dd" command. But that would suggest its not a file system problem. and sftp is as slow as samba so I think it leaves networking but on both adapters? The Server and the XBMC host are both on 10/100 running through a 10/100/1000 router with ddwrt. DDWRT stays under .1 load when transferring. This is the first problem that google has not solved for me. (209 tabs open at one point)

Hope someone can help. I am sure there are some logs I can get that would be helpful but I dont know what so let me know and I will get them.

Thanks,
Joe

[rubylaser]

Hello. You are in a tough situation. ZFS is an awesome filesystem, but it has some pretty high hardware needs if you want good performance. 2GB of RAM and an old single core AMD aren't going to cut it. Next, Samba isn't a fast filesharing protocol (it is faster in Solaris or Openindiana because if it's native CIFS implementation, but it would still be slow on your hardware).

I would like to know what your read and write speeds are to the array locally.

dd if=/dev/zero of=/your/array/testfile.out bs=1M count=10000
sleep 30
dd if=/your/array/testfile.out of=/dev/null bs=1M
rm /your/array/testfile.out

Replace /your/array with the path to your array, and please post the results back here. Also, the overhead of networking + Samba is going to make these numbers even worse. Can you try an FTP client or NFS to see what your performance looks like?

My final advise is if you just want storage for XBMC, I'd be using SnapRAID on your hardware. It doesn't have the hardware requirements of ZFS, is easy to manage, and you can still run a check on your array to prevent bitrot.

snapraid check

[TheFu]

Which ZFS are you using? Kernel module or usermode? Usermode will be slow - painfully slow.

It is unclear how your disks are connected - all SATA?

It is unclear if you are using wifi or wired ethernet for every transfer. Please avoid all wifi transfers.

If there are any USB2 or slower HDDs involved, those will always have slow throughput.

Be very clear on Mbps and MBps. Most networks are 100Mbps, so about 10MBps is the top speed available.

I use Samba between Windows and Linux and routinely see 70+MBps (650-ish Mbps) throughput on the GigE network here - no router involved because an unmanaged GigE switch is the connection.

There are some tuning parameters for both the ethernet and disk drives that can make a difference, but these are usually not a great idea if the system isn't a pure file server. Be extremely careful if the buffers become too large - this isn't a good thing always.

Laptop disk controllers are notoriously slow when compared to standard desktop components. My testing shows 7200 rpm disks in both to be 2.5x slower in a laptop than a desktop. You didn't say anything about laptops being used, but I figured that info could be helpful.

http://www.smallnetbuilder.com/nas/n...er-nas?start=3 might be helpful too, though he doesn't use ZFS.

[jpaytoncfd]

Hello. You are in a tough situation. ZFS is an awesome filesystem, but it has some pretty high hardware needs if you want good performance. 2GB of RAM and an old single core AMD aren't going to cut it. Next, Samba isn't a fast filesharing protocol (it is faster in Solaris or Openindiana because if it's native CIFS implementation, but it would still be slow on your hardware).

I would like to know what your read and write speeds are to the array locally.

dd if=/dev/zero of=/your/array/testfile.out bs=1M count=10000
sleep 30
dd if=/your/array/testfile.out of=/dev/null bs=1M
rm /your/array/testfile.out

Replace /your/array with the path to your array, and please post the results back here. Also, the overhead of networking + Samba is going to make these numbers even worse. Can you try an FTP client or NFS to see what your performance looks like?

My final advise is if you just want storage for XBMC, I'd be using SnapRAID on your hardware. It doesn't have the hardware requirements of ZFS, is easy to manage, and you can still run a check on your array to prevent bitrot.

snapraid check

Ran the codes you told me:

10000+0 records in
10000+0 records out
10485760000 bytes (10 GB) copied, 101.003 s, 104 MB/s
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB) copied, 100.83 s, 104 MB/s

I do like the ZFS but I really am only using it because I moved from FreeNAS. The only raid feature I am really interested in is making 4-6 drives show as one volume. I also like the ability for one drive to fail and not loose the whole dataset.

TheFU:

ZFS is ubuntu-zfs, I followed this guide, http://www.servercobra.com/freenas-t...etup-with-zfs/

All of the ZFS drives (3) are SATA. The system drive (1) is IDE all of the 5400RPM

The most common transfers are through 100Mb LAN. I am in the process of ordering gbit cards. Any suggestions or brand or chipset?
Also some WIFI backup from laptops. Small ammount of data on 300Mb WLAN-N. The faster the better but not important.

No USB Drives

Curious about using an unmanaged switch. Although with 2 laptops and the server the load on my router never goes over .2

This serer runs as a file server, runs Sabnzbd-Couchpotato-SickBeard (2.5MB average download speed), Webserver, DLNA Media server (Serviio), and at some point Motion for security. Maybe also a home DNS.

I am working on getting new hardware but power consumption is a consern. We are working on going to solar so I really want to get it under 100w. Currently 110-130w but I need more power. I have a prolient server w/ dual, dual core xenon @ 3.2 only has scsi controler and wont boot from anything else.

[TheFu]

ZFS is ubuntu-zfs, I followed this guide, http://www.servercobra.com/freenas-t...etup-with-zfs/

All of the ZFS drives (3) are SATA. The system drive (1) is IDE all of the 5400RPM

The most common transfers are through 100Mb LAN. I am in the process of ordering gbit cards. Any suggestions or brand or chipset?
Also some WIFI backup from laptops. Small ammount of data on 300Mb WLAN-N. The faster the better but not important.

No USB Drives

Curious about using an unmanaged switch. Although with 2 laptops and the server the load on my router never goes over .2

This serer runs as a file server, runs Sabnzbd-Couchpotato-SickBeard (2.5MB average download speed), Webserver, DLNA Media server (Serviio), and at some point Motion for security. Maybe also a home DNS.

I am working on getting new hardware but power consumption is a consern. We are working on going to solar so I really want to get it under 100w. Currently 110-130w but I need more power. I have a prolient server w/ dual, dual core xenon @ 3.2 only has scsi controler and wont boot from anything else.

I love the Intel PRO/1000 line of NICs. Newegg usually sells them for $25-ish. They ARE the GigE standard and cheap enough that I don't try to save $6 on an off-brand that might suck.

Being below 100W is easy if there isn't a powerful GPU involved. I have an AMD E350 for xbmc that uses about 25W - best $100 I've spent. Of course storage is on the LAN, not local.

I've used GigE switches for years with slower 100base-tx routers. I needed the extra ports and run storage on a different network than everything else. "Server" hardware, especially old server hardware, sucks power that isn't needed anymore. A Core i5 will do phenomenal things for less power if you don't need the SERVER class capabilities for hot-swap and direct storage access through a virtual machine.

I run the DNS on the router, myself.

I was pissed to discover that my laptop WiFi-N card was only 75Mbps capable. The onboard NIC is GigE on the same machine. What vendor does that crap? Dell. Of course, they "offered" a $20 upgrade to a better wifi-N chip, but without clearly saying why, I didn't bother. Plus we all know that wifi sucks, regardless. Most people don't get anywhere near the theory for wifi bandwidth. During a 1200 location wifi deployment, I learned the dirty truth about wifi. Every powered on device halves the bandwidth. If the connection is marginal, it is worse. A 300Mbps N connection with 1 device, on average, will push 150Mbps. Add another device and both drop to 75Mbps (assuming both are 300 capable). Whatever the slowest device on the wifi-LAN supports, that will be the new max **before** you start halfing everything.

I don't have time to read that ZFS link. Usermode or kernel mode is all that matters.

[jpaytoncfd]

I love the Intel PRO/1000 line of NICs. Newegg usually sells them for $25-ish. They ARE the GigE standard and cheap enough that I don't try to save $6 on an off-brand that might suck.

Being below 100W is easy if there isn't a powerful GPU involved. I have an AMD E350 for xbmc that uses about 25W - best $100 I've spent. Of course storage is on the LAN, not local.

I've used GigE switches for years with slower 100base-tx routers. I needed the extra ports and run storage on a different network than everything else. "Server" hardware, especially old server hardware, sucks power that isn't needed anymore. A Core i5 will do phenomenal things for less power if you don't need the SERVER class capabilities for hot-swap and direct storage access through a virtual machine.

I run the DNS on the router, myself.

I was pissed to discover that my laptop WiFi-N card was only 75Mbps capable. The onboard NIC is GigE on the same machine. What vendor does that crap? Dell. Of course, they "offered" a $20 upgrade to a better wifi-N chip, but without clearly saying why, I didn't bother. Plus we all know that wifi sucks, regardless. Most people don't get anywhere near the theory for wifi bandwidth. During a 1200 location wifi deployment, I learned the dirty truth about wifi. Every powered on device halves the bandwidth. If the connection is marginal, it is worse. A 300Mbps N connection with 1 device, on average, will push 150Mbps. Add another device and both drop to 75Mbps (assuming both are 300 capable). Whatever the slowest device on the wifi-LAN supports, that will be the new max **before** you start halfing everything.

I don't have time to read that ZFS link. Usermode or kernel mode is all that matters.

I'm not sure what you mean by usermode or kernelmode I couldent find any deffination. I think from context I am running in usermode as I have not updated or changed the kernel.

[rubylaser]

If you are using ubuntu-zfs, you are using the kernel module. The other, older method uses FUSE which is terribly slow. Those are decent read/write speeds on the machine. Have you tried using another transfer protocol other than Samba as a test?

[TheFu]

I'm not sure what you mean by usermode or kernelmode I couldent find any deffination. I think from context I am running in usermode as I have not updated or changed the kernel.

https://wiki.ubuntu.com/ZFS - I was using "usermode" really to mean "not kernel mode." A better way would have been to say FUSE or kernel?

Your testing returned 100MBps, which is what slower HDDs like yours should produce. I do not think you are using FUSE. FUSE file systems have a place, but not when performance is a concern. As an example, I use sshfs for convenience from time to time, but it is very slow.

Sorry, I'm at a loss for an answer to the ZFS performance issue. Here's another thread about ZFS you might find interesting: http://ubuntuforums.org/showthread.php?t=2061495 Compression and deduplication have performance impacts, if enabled.

[tgalati4]

Put your mount in /etc/fstab and rerun your video streaming tests. Kernel-space are processes that are started and managed by the system--typically at startup. User-space are those processes initiated by the user in their own account. Opening up nautilus and "network" places to perform a file transfer is user-space. Starting a mount using /etc/fstab is kernel-space and tends to be faster.

[TheFu]

Put your mount in /etc/fstab and rerun your video streaming tests. Kernel-space are processes that are started and managed by the system--typically at startup. User-space are those processes initiated by the user in their own account. Opening up nautilus and "network" places to perform a file transfer is user-space. Starting a mount using /etc/fstab is kernel-space and tends to be faster.

This doesn't sound correct, if I'm reading it right.

Kernel modules are closer to the hardware, more trusted, and reduce the extra layers involved for moving data around. There are fewer permissions gates.

User-space, or FUSE file systems, run at the same level of the OS as any other end-user process. This means it has more permissions to request to accomplish the same tasks. A FUSE file system can easily be mounted from the /etc/fstab, but it will still be slower, sometimes significantly slower than the same file system mounted using a kernel module.

https://en.wikipedia.org/wiki/Filesystem_in_Userspace explains more.

FUSE has been reworked to significantly improve performance, but it usually does not compare well to kernel module versions of the same file system. CPU utilization is almost always much, much higher for FUSE. Nothing shows this like benchmarks - http://www.phoronix.com/scan.php?pag...formance&num=1

OTOH, I could be completely wrong.