Originally Posted by
apeganz
edit: new version of the script, you now have to do less adjustments. should be pretty save to run on almost any system. regards, alex
I just wrote a script to improve my mdadm-raid's performance and felt like sharing. Maybe someone else can use it or get some inspiration from it.
Happy tuning,
Alex
PS: do NOT just run it, depending on your configuration it might seriously screw up your system if you don't alter it accordingly!
Hi,
I did a bit of modification to the original script to make it a bit more automatic and more usable for newbies like me.
there is still some stuff that I don't understand but I think it's a bit more readable now.
Use it at your own risk!
Code:
#!/bin/bash
###############################################################################
# simple script to set some parameters to increase performance on a mdadm
# raid5 or raid6. Ajust the ## parameters ##-section to your system!
#
# WARNING: depending on stripesize and the number of devices the array might
# use QUITE a lot of memory after optimization!
#
# 27may2010 by Alexander Peganz
# 18/01/2012 Alfonso made the script more verbose
###############################################################################
# use bash -x scriptname.sh to debug
## parameters ##
LOGFILE=/tmp/tune_raid.log # just a log file
BLOCKSIZE=4 # of file system in kb
# this is a parameter of mkfs when you created your filesystem
# e.g. mkfs.ext4 -b 4096 -E stride=16,stripe-width=32 /dev/md0
# if you don't know your block size, run dumpe2fs -h /dev/md0 | grep "Block size"
NCQ=disable # disable, enable. ath. else keeps current setting
# to be honest I couldn't find any clear and short article
# on NCQ but I saw all my disks are already disabled
# so I'll keep this option disable
NCQDEPTH=31 # 31 should work for almost anyone
# you only have to care about this if you want to enable NCQ
FORCECHUNKSIZE=true # force max sectors kb to chunk size > 512
# I tried to figure out what this is... but I don't know...
DOTUNEFS=false # run tune2fs, ONLY SET TO true IF YOU USE EXT[34]
EXECUTE=false # if "true", run actual commands. Otherwise only inform you
echo check $LOGFILE for messages in case of error.
## code ##
# test for priviledges
if [ "$(whoami)" != 'root' ]
then
echo $(date): Need to be root >> $LOGFILE
exit 1
fi
if [ $EXECUTE == "true" ]
then
echo
echo "************************************************"
echo "You are about to make real changes to the system"
echo "************************************************"
echo
read -p "Press [Enter] to continue or Ctrl-C to abort"
fi
# find out which one is your md
# note that the script only works for one md. If you have more than one
# just uncomment the line below and type something like MDDEV=md0
MDDEV="`cat /proc/mdstat | grep md | head -1 | awk '{print $1}'`"
# MDDEV=md0
if [ -z "$MDDEV" ]
then
echo $(date): Something wrong, I can\'t find any md >> $LOGFILE
exit 1
fi
#
# find out which RAID level
#
RAIDLEVEL="`mdadm --detail /dev/$MDDEV | grep raid | tr " " "\n" | grep raid`"
if [ -z "$RAIDLEVEL" ]
then
echo $(date): Something wrong, I can\'t find which raidlevel you are using >> $LOGFILE
exit 1
fi
if [ $RAIDLEVEL != "raid5" ] && [ $RAIDLEVEL != "raid6" ]
then
echo $(date): Something wrong, this script only works for raid5 and raid6 >> $LOGFILE
exit 1
fi
#
# find out your chunk size
#
# this expression takes the output of /proc/mdstat
# then takes the line with chunk
# then cuts it in many lines
# then takes the line with chunk
# then prints the first word
# then removes all the letters and keeps the number
# CAREFUL! CHECK IF YOUR /proc/mdstat REPORT CHUNK IN A DIFFERENT UNIT (e.g. mega instead of kilo)
# this will BREAK the script
CHUNKSIZE="`cat /proc/mdstat | grep chunk | tr "," "\n" | grep chunk | awk '{print $1}' | tr -d [a-z]`"
# set number of parity devices
NUMPARITY=1
if [[ $RAIDLEVEL == "raid6" ]]
then
NUMPARITY=2
fi
#
# get the letter of all NON spare devices from cat /proc/mdstat
#
# this expression takes the output of /proc/mdstat
# then takes the line of our md
# then changes spaces into new lines
# then takes only lines starting with sd
# then takes the lines without (S) at the end of the string (which are the NON spare disks)
# then take the 3rd character (a for sda1, etc)
# and then remove new lines to make a single string
DEVS="`cat /proc/mdstat | grep $MDDEV | tr " " "\n" | grep '^sd' | grep -v \(S\)$ | awk '{print substr($0,3,1)}' | tr -d "\n"`"
#
# get the letter of all spare devices from cat /proc/mdstat
#
# this expression takes the output of /proc/mdstat
# then takes the line of our md
# then changes spaces into new lines
# then takes only lines starting with sd
# then takes the lines with (S) at the end of the string (which are the spare disks)
# then take the 3rd character (a for sda1, etc)
# and then remove new lines to make a single string
SPAREDEVS="`cat /proc/mdstat | grep $MDDEV | tr " " "\n" | grep '^sd' | grep \(S\)$ | awk '{print substr($0,3,1)}' | tr -d "\n"`"
NUMDEVS=${#DEVS}
NUMSPAREDEVS=${#SPAREDEVS}
# test if number of devices makes sense
if [ ${#DEVS} -lt $[1+$NUMPARITY] ]
then
echo $(date): Need more devices >> $LOGFILE
exit 1
fi
# set read ahead
RASIZE=$[$NUMDEVS*($NUMDEVS-$NUMPARITY)*2*$CHUNKSIZE] # in 512b blocks
echo suggested read ahead size per device: $RASIZE blocks \($[$RASIZE/2]kb\)
MDRASIZE=$[$RASIZE*$NUMDEVS]
echo suggested read ahead size of array: $MDRASIZE blocks \($[$MDRASIZE/2]kb\)
echo RUN blockdev --setra $RASIZE /dev/sd[$DEVS]
echo your current value for readahead is `blockdev --getra /dev/sd[$DEVS]`
if [ $EXECUTE == "true" ]
then
blockdev --setra $RASIZE /dev/sd[$DEVS]
fi
if [ $NUMSPAREDEVS -gt 0 ]
then
echo RUN blockdev --setra $RASIZE /dev/sd[$SPAREDEVS]
echo your current value for readahead is `blockdev --getra /dev/sd[$SPAREDEVS]`
if [ $EXECUTE == "true" ]
then
blockdev --setra $RASIZE /dev/sd[$SPAREDEVS]
fi
fi
echo RUN blockdev --setra $MDRASIZE /dev/$MDDEV
echo your current value for readahead is `blockdev --getra /dev/$MDDEV`
if [ $EXECUTE == "true" ]
then
blockdev --setra $MDRASIZE /dev/$MDDEV
fi
# set stripe cache size
STRCACHESIZE=$[$RASIZE/8] # in pages per device
echo suggested stripe cache size of devices: $STRCACHESIZE pages \($[$STRCACHESIZE*4]kb\)
echo RUN echo $STRCACHESIZE \> /sys/block/$MDDEV/md/stripe_cache_size
echo current value of /sys/block/$MDDEV/md/stripe_cache_size is `cat /sys/block/$MDDEV/md/stripe_cache_size`
if [ $EXECUTE == "true" ]
then
echo $STRCACHESIZE > /sys/block/$MDDEV/md/stripe_cache_size
fi
# set max sectors kb
DEVINDEX=0
MINMAXHWSECKB=$(cat /sys/block/sd${DEVS:0:1}/queue/max_hw_sectors_kb)
until [ $DEVINDEX -ge $NUMDEVS ]
do
DEVLETTER=${DEVS:$DEVINDEX:1}
MAXHWSECKB=$(cat /sys/block/sd$DEVLETTER/queue/max_hw_sectors_kb)
if [ $MAXHWSECKB -lt $MINMAXHWSECKB ]
then
MINMAXHWSECKB=$MAXHWSECKB
fi
DEVINDEX=$[$DEVINDEX+1]
done
if [ $CHUNKSIZE -le $MINMAXHWSECKB ] &&
( [ $CHUNKSIZE -le 512 ] || [[ $FORCECHUNKSIZE == "true" ]] )
then
echo setting max sectors kb to match chunk size
DEVINDEX=0
until [ $DEVINDEX -ge $NUMDEVS ]
do
DEVLETTER=${DEVS:$DEVINDEX:1}
echo RUN echo $CHUNKSIZE \> /sys/block/sd$DEVLETTER/queue/max_sectors_kb
echo current value of /sys/block/sd$DEVLETTER/queue/max_sectors_kb is `cat /sys/block/sd$DEVLETTER/queue/max_sectors_kb`
if [ $EXECUTE == "true" ]
then
echo $CHUNKSIZE > /sys/block/sd$DEVLETTER/queue/max_sectors_kb
fi
DEVINDEX=$[$DEVINDEX+1]
done
DEVINDEX=0
until [ $DEVINDEX -ge $NUMSPAREDEVS ]
do
DEVLETTER=${SPAREDEVS:$DEVINDEX:1}
echo RUN echo $CHUNKSIZE \> /sys/block/sd$DEVLETTER/queue/max_sectors_kb
echo current value of /sys/block/sd$DEVLETTER/queue/max_sectors_kb is `cat /sys/block/sd$DEVLETTER/queue/max_sectors_kb`
if [ $EXECUTE == "true" ]
then
echo $CHUNKSIZE > /sys/block/sd$DEVLETTER/queue/max_sectors_kb
fi
DEVINDEX=$[$DEVINDEX+1]
done
fi
# enable/disable NCQ
DEVINDEX=0
if [[ $NCQ == "enable" ]] || [[ $NCQ == "disable" ]]
then
if [[ $NCQ == "disable" ]]
then
NCQDEPTH=1
fi
echo setting NCQ queue depth to $NCQDEPTH
until [ $DEVINDEX -ge $NUMDEVS ]
do
DEVLETTER=${DEVS:$DEVINDEX:1}
echo RUN echo $NCQDEPTH \> /sys/block/sd$DEVLETTER/device/queue_depth
echo current value of /sys/block/sd$DEVLETTER/device/queue_depth is `cat /sys/block/sd$DEVLETTER/device/queue_depth`
if [ $EXECUTE == "true" ]
then
echo $NCQDEPTH > /sys/block/sd$DEVLETTER/device/queue_depth
fi
DEVINDEX=$[$DEVINDEX+1]
done
DEVINDEX=0
until [ $DEVINDEX -ge $NUMSPAREDEVS ]
do
DEVLETTER=${SPAREDEVS:$DEVINDEX:1}
echo RUN echo $NCQDEPTH \> /sys/block/sd$DEVLETTER/device/queue_depth
echo current value of /sys/block/sd$DEVLETTER/device/queue_depth is `cat /sys/block/sd$DEVLETTER/device/queue_depth`
if [ $EXECUTE == "true" ]
then
echo $NCQDEPTH > /sys/block/sd$DEVLETTER/device/queue_depth
fi
DEVINDEX=$[$DEVINDEX+1]
done
fi
# tune2fs
if [[ $DOTUNEFS == "true" ]]
then
STRIDE=$[$CHUNKSIZE/$BLOCKSIZE]
STRWIDTH=$[$CHUNKSIZE/$BLOCKSIZE*($NUMDEVS-$NUMPARITY)]
echo setting stride to $STRIDE blocks \($CHUNKSIZE kb\)
echo setting stripe-width to $STRWIDTH blocks \($[$STRWIDTH*$BLOCKSIZE] kb\)
echo RUN tune2fs -E stride=$STRIDE,stripe-width=$STRWIDTH /dev/$MDDEV
echo PLEASE NOTE: RUN tune2fs ONLY if you use ext3 or ext4
if [ $EXECUTE == "true" ]
then
read -p "Press [Enter] to execute tune2fs or Ctrl-C to abort"
tune2fs -E stride=$STRIDE,stripe-width=$STRWIDTH /dev/$MDDEV
fi
fi
if [ $EXECUTE != "true" ]
then
echo
echo PLEASE NOTE: this script did NOTHING! It simply informed you of your current parameters and suggested some changes.
echo YOU HAVE TO run the commands suggested if you want to execute them or run again this script setting EXECUTE to true.
fi
# exit
echo $(date): Success >> $LOGFILE
exit 0
Use it at your own risk!
EDIT: based on further tests, this tuning seem to degrade performances for my system.
So finally I wrote my own script to tune settings.
First Cup of Ubuntu
Adv Reply
Bookmarks