Sevoma
February 20th, 2006, 08:19 AM
Well, this isn't really a defrag. It sorts files on the file system by size, and it does make the fragmentation go down by a lot. Someone who had a reiserfs partition for a year with 30%(!) fragmentation had it go down to 5%.
Anyway, here are the tools. Please- don't give me credit for them. I did not make them.
This is the script for checking fragmentation:
#!/usr/bin/perl -w
#this script search for frag on a fs
use strict;
#number of files
my $files = 0;
#number of fragment
my $fragments = 0;
#number of fragmented files
my $fragfiles = 0;
#search fs for all file
open (FILES, "find " . $ARGV[0] . " -xdev -type f |");
while (defined (my $file = <FILES>)) {
#quote some chars in filename
$file =~ s/!/\\!/g;
$file =~ s/#/\\#/g;
$file =~ s/&/\\&/g;
$file =~ s/>/\\>/g;
$file =~ s/</\\</g;
$file =~ s/\$/\\\$/g;
$file =~ s/\(/\\\(/g;
$file =~ s/\)/\\\)/g;
$file =~ s/\|/\\\|/g;
$file =~ s/'/\\'/g;
$file =~ s/ /\\ /g;
#nb of fragment for the file
open (FRAG, "filefrag $file |");
my $res = <FRAG>;
if ($res =~ m/.*:\s+(\d+) extents? found/) {
my $fragment = $1;
$fragments+=$fragment;
if ($fragment > 1) {
$fragfiles++;
}
$files++;
} else {
print ("$res : not understand for $file.\n");
}
close (FRAG);
}
close (FILES);
print ( $fragfiles / $files * 100 . "% non contiguous files, " . $fragments / $files . " average fragments.\n");
And this is the script for defraging:
#!/bin/sh
# defrag v0.06 by Con Kolivas <kernel@kolivas.org
# Braindead fs-agnostic defrag to rewrite files in order largest to smallest
# Run this in the directory you want all the files and subdirectories to be
# reordered. It will only affect one partition.
cd /home
trap 'abort' 1 2 15
renice 19 $$ > /dev/null
abort()
{
echo -e "\nAborting"
rm -f tmpfile dirlist
exit 1
}
fail()
{
echo -e "\nFailed"
abort
}
declare -i filesize=0
declare -i numfiles=0
#The maximum size of a file we can easily cache in ram
declare -i maxsize=`awk '/MemTotal/ {print $2}' /proc/meminfo`
(( maxsize-= `awk '/Mapped/ {print $2}' /proc/meminfo` ))
(( maxsize/= 2))
if [[ -a tmpfile || -a dirlist ]] ; then
echo dirlist or tmpfile exists
exit 1
fi
# Sort in the following order:
# 1) Depth of directory
# 2) Size of directory descending
# 3) Filesize descending
echo "Creating list of files..."
#stupid script to find max directory depth
find -xdev -type d -printf "%d\n" | sort -n | uniq > dirlist
#sort directories in descending size order
cat dirlist | while read d;
do
find -xdev -type d -mindepth $d -maxdepth $d -printf "\"%p\"\n" | \
xargs du -bS --max-depth=0 | \
sort -k 1,1nr -k 2 |\
cut -f2 >> tmpfile
if (( $? )) ; then
fail
fi
done
rm -f dirlist
#sort files in descending size order
cat tmpfile | while read d;
do
find "$d" -xdev -type f -maxdepth 1 -printf "%s\t%p\n" | \
sort -k 1,1nr | \
cut -f2 >> dirlist
if (( $? )) ; then
fail
fi
done
rm -f tmpfile
numfiles=`wc -l dirlist | awk '{print $1}'`
echo -e "$numfiles files will be reordered\n"
#copy to temp file, check the file hasn't changed and then overwrite original
cat dirlist | while read i;
do
(( --numfiles ))
if [[ ! -f $i ]]; then
continue
fi
#We could be this paranoid but it would slow it down 1000 times
#if [[ `lsof -f -- "$i"` ]]; then
# echo -e "\n File $i open! Skipping"
# continue
#fi
filesize=`find "$i" -printf "%s"`
# read the file first to cache it in ram if possible
if (( filesize < maxsize ))
then
echo -e "\r $numfiles files left \c"
cat "$i" > /dev/null
else
echo -e "\r $numfiles files left - Reordering large file sized $filesize ... \c"
fi
datestamp=`find "$i" -printf "%s"`
cp -a -f "$i" tmpfile
if (( $? )) ; then
fail
fi
# check the file hasn't been altered since we copied it
if [[ `find "$i" -printf "%s"` != $datestamp ]] ; then
continue
fi
mv -f tmpfile "$i"
if (( $? )) ; then
fail
fi
done
echo -e "\nSucceeded"
rm -f dirlist
Save it to your home directory or /usr/bin as fragchk and defrag.
Chmod +x fragchk defrag
They MUST be run as root.
sudo fragchk
sudo defrag
Use this at your own risk, as it can be dangerous on your filesystem!
Known bugs:
Will not work with files with ( or ) in the name. I think it just will skip them.
*NOTE: It only makes a real difference when your fragmentation is greater than 3 percent.
Enjoy, Sevoma
Anyway, here are the tools. Please- don't give me credit for them. I did not make them.
This is the script for checking fragmentation:
#!/usr/bin/perl -w
#this script search for frag on a fs
use strict;
#number of files
my $files = 0;
#number of fragment
my $fragments = 0;
#number of fragmented files
my $fragfiles = 0;
#search fs for all file
open (FILES, "find " . $ARGV[0] . " -xdev -type f |");
while (defined (my $file = <FILES>)) {
#quote some chars in filename
$file =~ s/!/\\!/g;
$file =~ s/#/\\#/g;
$file =~ s/&/\\&/g;
$file =~ s/>/\\>/g;
$file =~ s/</\\</g;
$file =~ s/\$/\\\$/g;
$file =~ s/\(/\\\(/g;
$file =~ s/\)/\\\)/g;
$file =~ s/\|/\\\|/g;
$file =~ s/'/\\'/g;
$file =~ s/ /\\ /g;
#nb of fragment for the file
open (FRAG, "filefrag $file |");
my $res = <FRAG>;
if ($res =~ m/.*:\s+(\d+) extents? found/) {
my $fragment = $1;
$fragments+=$fragment;
if ($fragment > 1) {
$fragfiles++;
}
$files++;
} else {
print ("$res : not understand for $file.\n");
}
close (FRAG);
}
close (FILES);
print ( $fragfiles / $files * 100 . "% non contiguous files, " . $fragments / $files . " average fragments.\n");
And this is the script for defraging:
#!/bin/sh
# defrag v0.06 by Con Kolivas <kernel@kolivas.org
# Braindead fs-agnostic defrag to rewrite files in order largest to smallest
# Run this in the directory you want all the files and subdirectories to be
# reordered. It will only affect one partition.
cd /home
trap 'abort' 1 2 15
renice 19 $$ > /dev/null
abort()
{
echo -e "\nAborting"
rm -f tmpfile dirlist
exit 1
}
fail()
{
echo -e "\nFailed"
abort
}
declare -i filesize=0
declare -i numfiles=0
#The maximum size of a file we can easily cache in ram
declare -i maxsize=`awk '/MemTotal/ {print $2}' /proc/meminfo`
(( maxsize-= `awk '/Mapped/ {print $2}' /proc/meminfo` ))
(( maxsize/= 2))
if [[ -a tmpfile || -a dirlist ]] ; then
echo dirlist or tmpfile exists
exit 1
fi
# Sort in the following order:
# 1) Depth of directory
# 2) Size of directory descending
# 3) Filesize descending
echo "Creating list of files..."
#stupid script to find max directory depth
find -xdev -type d -printf "%d\n" | sort -n | uniq > dirlist
#sort directories in descending size order
cat dirlist | while read d;
do
find -xdev -type d -mindepth $d -maxdepth $d -printf "\"%p\"\n" | \
xargs du -bS --max-depth=0 | \
sort -k 1,1nr -k 2 |\
cut -f2 >> tmpfile
if (( $? )) ; then
fail
fi
done
rm -f dirlist
#sort files in descending size order
cat tmpfile | while read d;
do
find "$d" -xdev -type f -maxdepth 1 -printf "%s\t%p\n" | \
sort -k 1,1nr | \
cut -f2 >> dirlist
if (( $? )) ; then
fail
fi
done
rm -f tmpfile
numfiles=`wc -l dirlist | awk '{print $1}'`
echo -e "$numfiles files will be reordered\n"
#copy to temp file, check the file hasn't changed and then overwrite original
cat dirlist | while read i;
do
(( --numfiles ))
if [[ ! -f $i ]]; then
continue
fi
#We could be this paranoid but it would slow it down 1000 times
#if [[ `lsof -f -- "$i"` ]]; then
# echo -e "\n File $i open! Skipping"
# continue
#fi
filesize=`find "$i" -printf "%s"`
# read the file first to cache it in ram if possible
if (( filesize < maxsize ))
then
echo -e "\r $numfiles files left \c"
cat "$i" > /dev/null
else
echo -e "\r $numfiles files left - Reordering large file sized $filesize ... \c"
fi
datestamp=`find "$i" -printf "%s"`
cp -a -f "$i" tmpfile
if (( $? )) ; then
fail
fi
# check the file hasn't been altered since we copied it
if [[ `find "$i" -printf "%s"` != $datestamp ]] ; then
continue
fi
mv -f tmpfile "$i"
if (( $? )) ; then
fail
fi
done
echo -e "\nSucceeded"
rm -f dirlist
Save it to your home directory or /usr/bin as fragchk and defrag.
Chmod +x fragchk defrag
They MUST be run as root.
sudo fragchk
sudo defrag
Use this at your own risk, as it can be dangerous on your filesystem!
Known bugs:
Will not work with files with ( or ) in the name. I think it just will skip them.
*NOTE: It only makes a real difference when your fragmentation is greater than 3 percent.
Enjoy, Sevoma