Page 2 of 3 FirstFirst 123 LastLast
Results 11 to 20 of 23

Thread: Script to tune mdadm raid5 (others too, maybe)

  1. #11
    Join Date
    Jul 2005
    Beans
    42

    Re: Script to tune mdadm raid5 (others too, maybe)

    Quote Originally Posted by jchung View Post
    Just wanted to revive this thread... I recently started using a software raid on my Ubuntu server and came across this thread to help tune an mdadm raid array.
    I had to tweak your script a bit to get it working on Oneiric (didn't test on natty)

    mdadm needed --verbose or it didn't print the raid level.
    it also returns a symlink to the md device (in the format /dev/md/1), so I did

    ARRAY=`readlink -f $x2`
    ARRAY=`basename $ARRAY`

    to get the old md1 name

    your script sets a much larger readahead than i used. id be interested in seeing benchmarks for random access, as when I previous tested, I decided to use lower values for improved random access.
    Last edited by exobuzz; October 12th, 2011 at 07:07 AM.

  2. #12
    Join Date
    Oct 2011
    Beans
    4

    Re: Script to tune mdadm raid5 (others too, maybe)

    Quote Originally Posted by exobuzz View Post
    I had to tweak your script a bit to get it working on Oneiric (didn't test on natty)

    mdadm needed --verbose or it didn't print the raid level.
    it also returns a symlink to the md device (in the format /dev/md/1), so I did

    ARRAY=`readlink -f $x2`
    ARRAY=`basename $ARRAY`

    to get the old md1 name

    your script sets a much larger readahead than i used. id be interested in seeing benchmarks for random access, as when I previous tested, I decided to use lower values for improved random access.
    Would you be able to upload your version, i am using Oneiric with raid5 and would like to see if i can improve the raid performance

    Cheers

  3. #13
    Join Date
    Oct 2011
    Beans
    4

    Re: Script to tune mdadm raid5 (others too, maybe)

    Quote Originally Posted by exobuzz View Post
    I had to tweak your script a bit to get it working on Oneiric (didn't test on natty)

    mdadm needed --verbose or it didn't print the raid level.
    it also returns a symlink to the md device (in the format /dev/md/1), so I did

    ARRAY=`readlink -f $x2`
    ARRAY=`basename $ARRAY`

    to get the old md1 name

    your script sets a much larger readahead than i used. id be interested in seeing benchmarks for random access, as when I previous tested, I decided to use lower values for improved random access.
    i have tried editing the script to match the above changes but cant seem to get it to see the device as /dev/md0

    any ideas??

  4. #14
    Join Date
    Aug 2005
    Location
    Manchester, NH, USA
    Beans
    17
    Distro
    Ubuntu

    Re: Script to tune mdadm raid5 (others too, maybe)

    I rewrote the script a bit to be more consistent, and use one source of data (mdadm). Currently, if you operate against a RAID10 array, you will get an error when the script attempts to modify the stripe_cache_size.

    I also cleaned up the code to detect the number of disks in an array. There's no need to have a counter variable since mdadm will tell you the number of disks already.

    Finally, I call mdadm once at the beginning of the script to make sure the data I operate on is the same throughout the execution of the script file itself. This is simply in case someone is modifying the array while we're running the script. It won't prevent anything bad from happening, it's just for the script's sanity.

    Hope this helps!

    BTW: MDSPEEDLIMIT should be modified based on the number of devices you have in your array and how much bandwidth you want a rebuild process to take. 200000 KB/s is basically 2/3 of a single SATA-II drive's maximum bus bandwidth. With large RAID10 pools you can easily hit 1000000 KB/s. Just something to keep in mind.


    Code:
    #!/bin/bash
    
    # NEED FOLLOWING UTILS
    # -- hdparm
    # -- lvm
    
    # Add VM tuning stuff?
    #vm.swappiness = 1		# set low to limit swapping
    #vm.vfs_cache_pressure = 50	# set lower to cache more inodes / dir entries
    #vm.dirty_background_ratio = 5	# set low on systems with lots of memory
    				# Too HIGH on systems with lots of memory 
    				# means huge page flushes which will hurt IO performance
    #vm.dirty_ratio = 10		# set low on systems with lots of memory
    
    # DEFAULTS
    BLOCKSIZE=4		# of filesystem in KB (should I determine?)
    FORCECHUNKSIZE=true	# force max sectors KB to chunk size > 512
    TUNEFS=false		# run tune2fs on filesystem if ext[3|4]
    SCHEDULER=deadline	# cfq / noop / anticipatory / deadline
    NR_REQUESTS=64		# NR REQUESTS
    NCQDEPTH=31		# NCQ DEPTH
    MDSPEEDLIMIT=200000	# Array speed_limit_max in KB/s
    
    
    # ----------------------------------------------------------------------
    #
    # BODY
    #
    # ----------------------------------------------------------------------
    
    # determine list of arrays
    mdadm -Es | grep ARRAY | while read x1 x2 x3 x4 x5; do
    	# INIT VARIABLES
    	RAIDLEVEL=0
    	NDEVICES=0
    	CHUNKSIZE=0
    	ARRAYSTATUS=0
    	DISKS=""
    	SPARES=""
    	NUMPARITY=0
    	NCQ=0
    
    	RASIZE=0
    	MDRASIZE=0
    	STRIPECACHESIZE=0
    	MINMAXHWSECKB=999999999
    
    	STRIDE=0
    	STRIPEWIDTH=0
    
    	# GET DETAILS OF ARRAY
    	if [ -L $x2 ]; then
    		# Array name is a new-style name. Get the symlink target.
    		LINKNAME=`readlink $x2`
    		ARRAY="`basename $LINKNAME`"
    	else
    		# Array name is an old-style name. Use it directly.
    		ARRAY="`basename $x2`"
    	fi
    	MDADMOUTPUT=`mdadm --detail /dev/$ARRAY`
    	RAIDLEVEL=`echo "$MDADMOUTPUT" | grep "Raid Level" | cut -d ':' -f 2 | sed 's/\s//g'`
    	RAIDDEVICES=`echo "$MDADMOUTPUT" | grep "Raid Devices" | cut -d ':' -f 2`
    
    	case $RAIDLEVEL in
    		"raid10")
    			NUMPARITY=$(($RAIDDEVICES/2)) ;;
    		"raid6")
    			NUMPARITY=2 ;;
    		"raid5")
    			NUMPARITY=1 ;;
    		"raid4")
    			NUMPARITY=1 ;;
    		"raid3")
    			NUMPARITY=1 ;;
    		"raid1")
    			NUMPARITY=1 ;;
    		"raid0")
    			NUMPARITY=0 ;;
    		*)
    			echo "Unknown RAID level"
    			exit 5 ;;
    	esac
    
    	echo ""
    	echo "======================================================================"
    	echo "FOUND ARRAY - $ARRAY / $RAIDLEVEL / Parity: $NUMPARITY"
    	CHUNKSIZE=`echo "$MDADMOUTPUT" | grep 'Chunk Size' | tr -d A-Za-z':'[:blank:]`
    	echo "-- Chunk Size = $CHUNKSIZE KB"
    
    	# DETERMINE SETTINGS
    	RASIZE=$(($RAIDDEVICES*($RAIDDEVICES-$NUMPARITY)*2*$CHUNKSIZE))	# Disk read ahead in 512byte blocks
    	MDRASIZE=$(($RASIZE*$RAIDDEVICES))				# Array read ahead in 512byte blocks
    	STRIPECACHESIZE=$(($RASIZE*2/8))				# in pages per device
    
    	# GET LIST OF DISKS IN ARRAY
    	echo ""
    	echo "Getting active and spare device list"
    	while read x1 x2 x3; do
    		DSTATUS="$x1"
    		DACT="$x2"
    		DNAME="$x3"
    
    		if [ "$DSTATUS" = "active" ]; then
    			DCURNAME=`basename $DNAME`
    			DISKS="$DISKS $DCURNAME"
    			echo "-- Active - $DCURNAME"
    		elif [ "$DSTATUS" = "spare" ]; then
    			if [ "$DNAME" = "" ]; then
    				DCURNAME=`basename $DACT`
    				SPARES="$SPARES $DCURNAME"
    				echo "-- Spare - $DCURNAME"
    			else
    				DCURNAME=`basename $DNAME`
    				DISKS="$DISKS $DCURNAME"
    				echo "-- $DACT - $DCURNAME"
    			fi
    		else
    			echo "-- $DSTATUS: $DACT - `basename $DNAME`"
    		fi
    	done < <(echo "$MDADMOUTPUT" | grep -E "active|spare" | tr -s ' ' | cut -d ' ' -f 6-)
    	echo ""
    	echo "Active Disks: $DISKS"
    	echo "Spares Disks: $SPARES"
    
    	for DISK in $DISKS $SPARES; do
    		# check max_hw_sectors_kb
    		echo "Checking max_hw_sectors_kb for $DISK"
    		FOO1=`cat /sys/block/$DISK/queue/max_hw_sectors_kb | awk '{print $1}'`
    		if [ $FOO1 -lt $MINMAXHWSECKB ]; then
    			MINMAXHWSECKB=$FOO1
    		fi
    	done
    
    	if [ $CHUNKSIZE -le $MINMAXHWSECKB ]; then
    		MINMAXHWSECKB=$CHUNKSIZE
    	fi
    
    	echo ""
    	echo "TUNED SETTINGS"
    	echo "-- DISK READ AHEAD  = $RASIZE blocks"
    	echo "-- ARRAY READ AHEAD = $MDRASIZE blocks"
    	echo "-- STRIPE CACHE     = $STRIPECACHESIZE pages"
    	echo "-- MAX SECTORS KB   = $MINMAXHWSECKB KB"
    	echo "-- NCQ DEPTH        = $NCQDEPTH"
    
    	# TUNE ARRAY
    	echo ""
    	echo "TUNING ARRAY"
    	blockdev --setra $MDRASIZE /dev/$ARRAY
    	echo "-- $ARRAY read ahead set to $MDRASIZE blocks"
    
    	echo "$STRIPECACHESIZE" > /sys/block/$ARRAY/md/stripe_cache_size
    	echo "-- $ARRAY stripe_cache_size set to $STRIPECACHESIZE pages"
    
    	echo $MDSPEEDLIMIT > /proc/sys/dev/raid/speed_limit_max
    	echo "-- $ARRAY speed_limit_max set to $MDSPEEDLIMIT"
    
    	# TUNE DISKS
    	echo ""
    	echo "TUNING DISKS"
    	echo "Settings : "
    	echo "  read ahead     = $RASIZE blocks"
    	echo "  max_sectors_kb = $MINMAXHWSECKB KB"
    	echo "  scheduler      = $SCHEDULER"
    	echo "  nr_requests    = $NR_REQUESTS"
    	echo "  queue_depth    = $NCQDEPTH"
    
    	for DISK in $DISKS $SPARES; do
    		echo "-- Tuning $DISK"
    		blockdev --setra $RASIZE /dev/$DISK
    		echo $MINMAXHWSECKB > /sys/block/$DISK/queue/max_sectors_kb
    		echo $SCHEDULER > /sys/block/$DISK/queue/scheduler
    		echo $NR_REQUESTS > /sys/block/$DISK/queue/nr_requests
    		echo $NCQDEPTH > /sys/block/$DISK/device/queue_depth
    	done
    
    	# TUNE ext3/exti4 FILESYSTEMS
    	STRIDE=$(($CHUNKSIZE/$BLOCKSIZE))
    	STRIPEWIDTH=$(($CHUNKSIZE/$BLOCKSIZE*($NUMDISKS-$NUMPARITY)))
    	echo ""
    	echo "TUNING FILESYSTEMS"
    	echo "For each filesystem on this array, run the following command:"
    	echo "  tune2fs -E stride=$STRIDE,stripe-width=$STRIPEWIDTH <filesystem>"
    	echo ""
    done
    Last edited by drdabbles; October 28th, 2011 at 11:24 PM.

  5. #15
    Join Date
    Oct 2011
    Beans
    4

    Re: Script to tune mdadm raid5 (others too, maybe)

    Quote Originally Posted by drdabbles View Post
    I rewrote the script a bit to be more consistent, and use one source of data (mdadm). Currently, if you operate against a RAID10 array, you will get an error when the script attempts to modify the stripe_cache_size.

    I also cleaned up the code to detect the number of disks in an array. There's no need to have a counter variable since mdadm will tell you the number of disks already.

    Finally, I call mdadm once at the beginning of the script to make sure the data I operate on is the same throughout the execution of the script file itself. This is simply in case someone is modifying the array while we're running the script. It won't prevent anything bad from happening, it's just for the script's sanity.

    Hope this helps!

    BTW: MDSPEEDLIMIT should be modified based on the number of devices you have in your array and how much bandwidth you want a rebuild process to take. 200000 KB/s is basically 2/3 of a single SATA-II drive's maximum bus bandwidth. With large RAID10 pools you can easily hit 1000000 KB/s. Just something to keep in mind.


    Code:
    #!/bin/bash
    
    # NEED FOLLOWING UTILS
    # -- hdparm
    # -- lvm
    
    # Add VM tuning stuff?
    #vm.swappiness = 1		# set low to limit swapping
    #vm.vfs_cache_pressure = 50	# set lower to cache more inodes / dir entries
    #vm.dirty_background_ratio = 5	# set low on systems with lots of memory
    				# Too HIGH on systems with lots of memory 
    				# means huge page flushes which will hurt IO performance
    #vm.dirty_ratio = 10		# set low on systems with lots of memory
    
    # DEFAULTS
    BLOCKSIZE=4		# of filesystem in KB (should I determine?)
    FORCECHUNKSIZE=true	# force max sectors KB to chunk size > 512
    TUNEFS=false		# run tune2fs on filesystem if ext[3|4]
    SCHEDULER=deadline	# cfq / noop / anticipatory / deadline
    NR_REQUESTS=64		# NR REQUESTS
    NCQDEPTH=31		# NCQ DEPTH
    MDSPEEDLIMIT=200000	# Array speed_limit_max in KB/s
    
    
    # ----------------------------------------------------------------------
    #
    # BODY
    #
    # ----------------------------------------------------------------------
    
    # determine list of arrays
    mdadm -Es | grep ARRAY | while read x1 x2 x3 x4 x5; do
    	# INIT VARIABLES
    	RAIDLEVEL=0
    	NDEVICES=0
    	CHUNKSIZE=0
    	ARRAYSTATUS=0
    	DISKS=""
    	SPARES=""
    	NUMPARITY=0
    	NCQ=0
    
    	RASIZE=0
    	MDRASIZE=0
    	STRIPECACHESIZE=0
    	MINMAXHWSECKB=999999999
    
    	STRIDE=0
    	STRIPEWIDTH=0
    
    	# GET DETAILS OF ARRAY
    	if [ -L $x2 ]; then
    		# Array name is a new-style name. Get the symlink target.
    		LINKNAME=`readlink $x2`
    		ARRAY="`basename $LINKNAME`"
    	else
    		# Array name is an old-style name. Use it directly.
    		ARRAY="`basename $x2`"
    	fi
    	MDADMOUTPUT=`mdadm --detail /dev/$ARRAY`
    	RAIDLEVEL=`echo "$MDADMOUTPUT" | grep "Raid Level" | cut -d ':' -f 2 | sed 's/\s//g'`
    	RAIDDEVICES=`echo "$MDADMOUTPUT" | grep "Raid Devices" | cut -d ':' -f 2`
    
    	case $RAIDLEVEL in
    		"raid10")
    			NUMPARITY=$(($RAIDDEVICES/2)) ;;
    		"raid6")
    			NUMPARITY=2 ;;
    		"raid5")
    			NUMPARITY=1 ;;
    		"raid4")
    			NUMPARITY=1 ;;
    		"raid3")
    			NUMPARITY=1 ;;
    		"raid1")
    			NUMPARITY=1 ;;
    		"raid0")
    			NUMPARITY=0 ;;
    		*)
    			echo "Unknown RAID level"
    			exit 5 ;;
    	esac
    
    	echo ""
    	echo "======================================================================"
    	echo "FOUND ARRAY - $ARRAY / $RAIDLEVEL / Parity: $NUMPARITY"
    	CHUNKSIZE=`echo "$MDADMOUTPUT" | grep 'Chunk Size' | tr -d A-Za-z':'[:blank:]`
    	echo "-- Chunk Size = $CHUNKSIZE KB"
    
    	# DETERMINE SETTINGS
    	RASIZE=$(($RAIDDEVICES*($RAIDDEVICES-$NUMPARITY)*2*$CHUNKSIZE))	# Disk read ahead in 512byte blocks
    	MDRASIZE=$(($RASIZE*$RAIDDEVICES))				# Array read ahead in 512byte blocks
    	STRIPECACHESIZE=$(($RASIZE*2/8))				# in pages per device
    
    	# GET LIST OF DISKS IN ARRAY
    	echo ""
    	echo "Getting active and spare device list"
    	while read x1 x2 x3; do
    		DSTATUS="$x1"
    		DACT="$x2"
    		DNAME="$x3"
    
    		if [ "$DSTATUS" = "active" ]; then
    			DCURNAME=`basename $DNAME`
    			DISKS="$DISKS $DCURNAME"
    			echo "-- Active - $DCURNAME"
    		elif [ "$DSTATUS" = "spare" ]; then
    			if [ "$DNAME" = "" ]; then
    				DCURNAME=`basename $DACT`
    				SPARES="$SPARES $DCURNAME"
    				echo "-- Spare - $DCURNAME"
    			else
    				DCURNAME=`basename $DNAME`
    				DISKS="$DISKS $DCURNAME"
    				echo "-- $DACT - $DCURNAME"
    			fi
    		else
    			echo "-- $DSTATUS: $DACT - `basename $DNAME`"
    		fi
    	done < <(echo "$MDADMOUTPUT" | grep -E "active|spare" | tr -s ' ' | cut -d ' ' -f 6-)
    	echo ""
    	echo "Active Disks: $DISKS"
    	echo "Spares Disks: $SPARES"
    
    	for DISK in $DISKS $SPARES; do
    		# check max_hw_sectors_kb
    		echo "Checking max_hw_sectors_kb for $DISK"
    		FOO1=`cat /sys/block/$DISK/queue/max_hw_sectors_kb | awk '{print $1}'`
    		if [ $FOO1 -lt $MINMAXHWSECKB ]; then
    			MINMAXHWSECKB=$FOO1
    		fi
    	done
    
    	if [ $CHUNKSIZE -le $MINMAXHWSECKB ]; then
    		MINMAXHWSECKB=$CHUNKSIZE
    	fi
    
    	echo ""
    	echo "TUNED SETTINGS"
    	echo "-- DISK READ AHEAD  = $RASIZE blocks"
    	echo "-- ARRAY READ AHEAD = $MDRASIZE blocks"
    	echo "-- STRIPE CACHE     = $STRIPECACHESIZE pages"
    	echo "-- MAX SECTORS KB   = $MINMAXHWSECKB KB"
    	echo "-- NCQ DEPTH        = $NCQDEPTH"
    
    	# TUNE ARRAY
    	echo ""
    	echo "TUNING ARRAY"
    	blockdev --setra $MDRASIZE /dev/$ARRAY
    	echo "-- $ARRAY read ahead set to $MDRASIZE blocks"
    
    	echo "$STRIPECACHESIZE" > /sys/block/$ARRAY/md/stripe_cache_size
    	echo "-- $ARRAY stripe_cache_size set to $STRIPECACHESIZE pages"
    
    	echo $MDSPEEDLIMIT > /proc/sys/dev/raid/speed_limit_max
    	echo "-- $ARRAY speed_limit_max set to $MDSPEEDLIMIT"
    
    	# TUNE DISKS
    	echo ""
    	echo "TUNING DISKS"
    	echo "Settings : "
    	echo "  read ahead     = $RASIZE blocks"
    	echo "  max_sectors_kb = $MINMAXHWSECKB KB"
    	echo "  scheduler      = $SCHEDULER"
    	echo "  nr_requests    = $NR_REQUESTS"
    	echo "  queue_depth    = $NCQDEPTH"
    
    	for DISK in $DISKS $SPARES; do
    		echo "-- Tuning $DISK"
    		blockdev --setra $RASIZE /dev/$DISK
    		echo $MINMAXHWSECKB > /sys/block/$DISK/queue/max_sectors_kb
    		echo $SCHEDULER > /sys/block/$DISK/queue/scheduler
    		echo $NR_REQUESTS > /sys/block/$DISK/queue/nr_requests
    		echo $NCQDEPTH > /sys/block/$DISK/device/queue_depth
    	done
    
    	# TUNE ext3/exti4 FILESYSTEMS
    	STRIDE=$(($CHUNKSIZE/$BLOCKSIZE))
    	STRIPEWIDTH=$(($CHUNKSIZE/$BLOCKSIZE*($NUMDISKS-$NUMPARITY)))
    	echo ""
    	echo "TUNING FILESYSTEMS"
    	echo "For each filesystem on this array, run the following command:"
    	echo "  tune2fs -E stride=$STRIDE,stripe-width=$STRIPEWIDTH <filesystem>"
    	echo ""
    done
    Tested the above on Oneiric

    mdadm: cannot open /dev/0: No such file or directory
    Unknown RAID level

  6. #16
    Join Date
    Apr 2010
    Beans
    11
    Distro
    Ubuntu 9.10 Karmic Koala

    Re: Script to tune mdadm raid5 (others too, maybe)

    wow, I stumbled on my old post here purely by accident and I'm stunned to see what has happend since!

    I guess by now you know more about the script then I do (had totally forgotten it, still works away in my lucid server untouched since week one)

    Sorry I never replied to any questions, after the first edit I kind of abbandoned it and I'm all but a regular viewer of this forums



    I'll try to help out with some stuff I still remember:

    Chunk size and block size of array
    Afair I only knew those values because I specified them when creating the array. I guess there is a way to get them from the array, but idkh. jchung posted a rewritten variant with
    4. auto detect chunk size of array
    RASIZE, MDRASIZE, omminous 512b blocks comment
    The goal when calculating this numbers was to read ahead the same amount of chunks on all disks in the array. With this formula the following happens:
    - Something/one wants to read sth from the array
    - The stuff to be read is distributed among the disks, each disk holding consecutive pieces of data of size CHUNKSIZE. The first NUMDEVS-NUMPARITY disks hold chunks containing actual data, the next NUMPARITY disks hold chunks containing parity data (well, that's a simplification, but I hope it's clear enough so you hear what I'm trying to say)
    - if I want to read the same ammount from all disks at the same time I have to read N*number of disks*CHUNKSIZE kbyte
    - since I'm not sure if parity chunks are skipped when reading, I multiply NUMDEVS with NUMDEVS-NUMPARITY for my number of disks, so the formula works for both cases. Side effect: generally higher read ahead size which should translate into higher performance, and it's even somewhat deterministic
    - factor of two is in there because now I have kbyte and in "blockdev --setra N" the N is in 512 byte sectors according to it's man page, I guess that is independent from the actual logical, actual physical and reported physical block size of the drive (mine are 4k physical, 512b logical sector drives reporting as 512b physical, 512b logical)

    stripe cache size
    Afair the higher the stripe cache size the more RAM is needed to cache. But idk if I don't mix up stuff I read about read ahead and stripe cache size right now, it's been a while after all. More cache should always be faster.

    force max sectors kb
    I think I read that something bad can happen if this is set above 512kb while scouring mailing lists back then. But for the life of me, that's all I can remember.



    by the way, when I asked for help to create the code to get the neccessary info out of the mdadm output, geirha (who helped me a lot. I just realized I never propperly credited him. Sorry geirha!) suggested to me that I should try to improve on my bash skills, since the script uses some old sh-style stuff that is not as good as the equivalents available in bash. If you should continue to improve on the script, maybe you would want to use his advice (http://ubuntuforums.org/showthread.php?t=1510176). He later also suggested http://mywiki.wooledge.org/BashPitfalls to me as a good starting point.

    Well, that's it I guess, hope I could help! I'll try to remember to check back in over the next few days, but I've got a lot of university stuff going on right now, so I can't promise anything unfortunately.

    regards,
    alex

  7. #17
    Join Date
    Dec 2008
    Beans
    17

    Re: Script to tune mdadm raid5 (others too, maybe)

    I just tried this script by drdabbles and had the same failure as MikeBuzz. I was able to get the first part to work by manually making it look at /dev/md0 instead of dev/md/0 (which doesn't exist).

    Also, my array is on /dev/sdb1, /dev/sdc1, and /dev/sdd1 . I had to manually change DISKS to sdb,sdc, and sdd as the original script was looking at sdb1,sdc1,sdd1 and the path doesnt exist.

    For example, this path in the script
    /sys/block/$DISK/queue/max_hw_sectors_kb needs "sdb" not "sdb1"

    I was able to make it run by doing the above, however, it didn't really improve my read performance. ~120 mb/sec now it is ~114 mb/sec. My write speed did improve from ~70mb/sec to ~107mb/sec.

    Thats my feedback for now. I'm pretty new to running scripts, so I'm not sure a more elegant way to fix this...

  8. #18
    Join Date
    Aug 2007
    Beans
    53

    Re: Script to tune mdadm raid5 (others too, maybe) (update)

    I had used the original script, not this newer one. I am using CentOS 5.5, but the script worked fine aside from modifying it to use sdx3 instead of sdx1.

    For an update, I added 3 more 2tb drives into my array (same brand, same specs, but the "green" label drives instead). I basically followed this guide:
    http://allintech.info/2008/06/mdadm-...-array-ubuntu/

    Basically you create new partitions on the drive the same size as the others, mark them as type raid, then add them into the array, grow the filesystem, then finally resize the filesystem to use the new space.

    I did not have to unmount the filesystem, resize2fs now works with online drives. It took about 50 hours to grow the filesystem (due to its large size), and apparently about 16 hours to resize2fs. Now I have 6 2tb drives in total, for a total usable space of 9.3TB. resize2fs did see them as 4K block sizes.

    When it was done, read speeds were still ~160MB/s as before, but write speeds were abysmally slow (12.5MB/s). I edited the tune_raid.sh script to turn off apm on these new drives, but it looks like they don't support this command (they give an error only on the new drives). To get proper write speeds all I had to do was reboot the box, and then read speeds also went up. Just re-running the tune_raid.sh didn't seem to help for some reason.

    Now with 6 drives in raid5 I get:
    Read speed: 330MB/s
    Write speed: 125MB/s

  9. #19
    Join Date
    Oct 2004
    Beans
    160

    Re: Script to tune mdadm raid5 (others too, maybe) (update)

    Quote Originally Posted by nosebreaker View Post
    I had used the original script, not this newer one. I am using CentOS 5.5, but the script worked fine aside from modifying it to use sdx3 instead of sdx1.

    For an update, I added 3 more 2tb drives into my array (same brand, same specs, but the "green" label drives instead). I basically followed this guide:
    http://allintech.info/2008/06/mdadm-...-array-ubuntu/

    Basically you create new partitions on the drive the same size as the others, mark them as type raid, then add them into the array, grow the filesystem, then finally resize the filesystem to use the new space.

    I did not have to unmount the filesystem, resize2fs now works with online drives. It took about 50 hours to grow the filesystem (due to its large size), and apparently about 16 hours to resize2fs. Now I have 6 2tb drives in total, for a total usable space of 9.3TB. resize2fs did see them as 4K block sizes.

    When it was done, read speeds were still ~160MB/s as before, but write speeds were abysmally slow (12.5MB/s). I edited the tune_raid.sh script to turn off apm on these new drives, but it looks like they don't support this command (they give an error only on the new drives). To get proper write speeds all I had to do was reboot the box, and then read speeds also went up. Just re-running the tune_raid.sh didn't seem to help for some reason.

    Now with 6 drives in raid5 I get:
    Read speed: 330MB/s
    Write speed: 125MB/s
    I'm guessing the writes are low because of the CPU, what processor is in that system?
    Archlinux / Ubuntu.

  10. #20
    Join Date
    Aug 2005
    Location
    Manchester, NH, USA
    Beans
    17
    Distro
    Ubuntu

    Re: Script to tune mdadm raid5 (others too, maybe)

    Sorry everyone! The modifications I made to the script read the device names from the mdadm tool, so I'm not too sure why it would read bad device names for some of you. I'd be interested in seeing the output of mdadm when listing devices.

    I also modified the script to parse the raw disk device names without the partition numbers, but my NAS's boot device just died on me a few days ago. Once I get it running again, I'll post the new version here. Again, sorry everyone!

Page 2 of 3 FirstFirst 123 LastLast

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •