large NFS copy locks up/hang client with large files (again) (lucid)

[Lepy]

I also get great read speeds from NFS shares (on my nas, check my first post).
Local file copy is fine though.

I setup a CIFS share on the nas and proceeded to copy a 1.2 gig file over CIFS and NFS (to different folders, though they reside in the same ZFS pool).
The NFS transfer completed but would lock up for a bit then burst write.
The CIFS transfer went fine. It did not lock up, the percentage bar steadily moved forward, and for the first time, I feel like I had an accurate write speed displayed through nautilus.
The server showed 30% memory usage and 30% load average (dual core proc) during both transfers.
I've attached another network graph. The CIFS transfer is first and looks much less erratic than the NFS transfer.

I am fairly sure this problem occurred in 9.10 with less frequency. I transferred a terabyte or two to the nas in only a few separate transfers under 9.10. I feel like it would have taken many more transfers under 10.04.
I have tried messing with mtu as well as rsize and wsize. Doesn't help.
The only other export options I can think to change are sync and async, but I have not been able to test this as FreeNAS does not seem to allow this change.
This is all I have for now.

[ScarletPea]

RIIIIGHT!
Some (only some) PROGRESS!
ok Ime yelling, but finally getting somewhere.

It turs out we have two (maybe unrelated) Problems, with potential solutions, so please please try these and let me know your milage:

1. Client Freezing
   
   • Apply the following patch to your (client) kernel
     https://bugzilla.kernel.org/attachment.cgi?id=26815
   • I think you do this as follows (ish)
     

     sudo apt-get install linux-source
     cd /usr/src/
     wget "https://bugzilla.kernel.org/attachment.cgi?id=26815"
     tar -xjf linux-source-2.6.33.tar.bz2
     cd linux-source-2.6.33
     patch -P1 < ../
     fakeroot make-kpkg --initrd --append-to-version=-nfs-fix kernel-image
     sudo dpkg -i ../liux-image-2.6.33-nfs-fix...
     sudo mkinitramfs-kpkg -o /boot/initrd.img-2.6.33-nfs-fix 2.6.33-nfs-fix
     reboot, choose new kernel w/grub

     note the mkinitrd-kpkg, the initrd hook needs to be setup properly, and I havnt figured it out quite right yet... maybee you have 8) Everything above is approximate as I havnt done it quite like that... follow _some_ of: https://help.ubuntu.com/community/Kernel/Compile, ime old fashioned I guess
2. Intermittant Transfer
   Aparently this could be the vm stuff on the server, try adjusting the following:
   

   echo 40 >  /proc/sys/vm/dirty_expire_centisecs; \
   echo 1 >  /proc/sys/vm/dirty_background_ratio; \
   echo 20 >  /proc/sys/vm/dirty_writeback_centisecs

   These numbers didnt quite work for me so check before you do what things are set to. Would be interesting to know what you had before, and what fixed them problem, please report... so check with:
   

   cat /proc/sys/vm/dirty_expire_centisecs;\
   cat /proc/sys/vm/dirty_background_ratio; \
   cat /proc/sys/vm/dirty_writeback_centisecs

Let me know please, the more feedback the better!

[ggerri]

Having the same problem for a while now. First time that Win7 (my office notebook) is doing better here than Ubuntu

ok, this is how I just solved it for myself:

1. in NAS nfs settings: disabled sync mode
2. changed nfs entry in fstab from

192.168.1.2:/media     /media/nas/media      nfs  rsize=8192,wsize=8192,timeo=14,intr 

to

192.168.1.2:/media     /media/nas/media      nfs  rsize=32768,wsize=32768,timeo=14,intr 

Hope that works for you as well

[ScarletPea]

Thanks ggerri,
What problem were you experiencing please,
client lockups? intermittant bursting transfers, both...?

Ille try what you suggested anyway and see what milage I get!

cheers.

Jasper

[ScarletPea]

192.168.1.2:/media     /media/nas/media      nfs  rsize=32768,wsize=32768,timeo=14,intr 

Hope that works for you as well

naaa didndt help with the lockups or intermittant transfers here.

So far best results with patched kernel and chnaging vm_dirty etc.

Any other takers?

[ScarletPea]

The fix by patch and modifying vm\etc* mostly worked for me.
Theres be no real comments to the contraty, Ille mark this solved.

Thanks all.

[leespa]

SUCCESS for me as well!

In my case, however, it was the 'sync' option causing all the grief. I followed a couple NFSv4 guides and decided to use the 'sync' versus 'async' since it sounds safer...

Changed /etc/exports to:

/exports 192.168.x.y(rw,fsid=0,insecure,async,no_subtree_check)
/exports/share0 192.168.x.y(rw,nohide,insecure,async,no_subtree_check)
/exports/temp0 192.168.x.y(rw,nohide,insecure,async,no_subtree_check)

AND my NFSv4 WRITE speeds went from 7 MB/s to 80MB/s - quite an improvement! Did md5sum on xfers and all is well.

I also tried the /proc/sys/vm/* changes suggested above. Got 80 MB/s WITH the changed values, but also got same 80 MB/s after reboot and /proc/sys/vm/* values reverted back to orignal speed. Therefore, aysnc made all the diff for me...

time dd if=hd1c_8x4GB.mpg of=/mnt/nt_tmp/hd1c_8x4GB.mpg
17676766+1 records in
17676766+1 records out
9050504616 bytes (9.1 GB) copied, 113.261 s, 79.9 MB/s
real 1m53.372s

[ganjaman2009]

I'm using a DLINK-323 as my NFS server with firmware 1.08. I'm experiencing the same graphical freezes tho' my ssh sessions are unaffected if I remote in.

My desktop is 2.6.32-24-generic #38-Ubuntu SMP and the /etc/fstab entries are:

nas:/mnt/HD_a2/ganja /media/NAS nfs rw,nolock,rsize=32768,wsize=32768,timeo=14,intr,ha rd,tcp,async 0 0

Any ideas?

[leespa]

ganjaman2009:

All I am using for mount command on the client is:

mount -t nfs4 192.168.x.y:/temp0 /mnt/nt_tmp/

NFSv4 will automagically figure out the best wsize,rsize. I manually tried tweaking this to no avail.

What fixed things for me is to use the 'async' option in /etc/exports as indicated in post #27. See if you can tweak on the DLink-323 - I know its Linux based too...

Also, to see current nfs mount config run this on the client:

nfsstat -mounts

/mnt/nt_tmp from 192.168.x.y:/temp0
Flags: rw,vers=4,rsize=524288,wsize=524288,namlen=255,har d,nointr,proto=tcp,timeo=600,retrans=2,sec=sys,cli entaddr=192.168.x.y,addr=192.168.x.y

Hopw this helps a bit.

[Lepy]

I was able to change my FreeNAS mounts to async by adding vfs.nfsrv.async 1 to its sysctl.conf. This seems to alleviate my problems client freezing problems.

Allowing NFSv4 to set the r/wsizw seemed to make my transfer graph a little less spiky.

I have not tried the patch, but if I encounter any more problems, I will report back.