pyfragtools: Defragmenter for Linux

[webbertiger]

For those hitting the problem of ZeroDivisionError, my solution is to change line 541:

    return frags/(os.path.getsize(file)/1024.0/1024.0)

to the following code block:

    filesize=os.path.getsize(file)
    if filesize == 0:
        return 0
    else:
        return frags/(filesize/1024.0/1024.0)

So files with 0 size can be skipped and the program will continue to run. See attached file.

[mamamia88]

any real point to this?

[Chame_Wizard]

I never ever needed one.

[Xbehave]

Fortunately, pyfragtools' defrag does work; however, the analyzing process takes an insane amount of time. mount.ntfs jumps up to 100% CPU usage, and trying to analyze 132 GBs of data takes about 20 hours on a 2.2 GHz AMD CPU!

It would be nice if I could address whatever is causing this CPU bottleneck.

If you use something like htop you can break down cpu usage by type, i'd guess a lot of the bottleneck will be in io-waits and I'd be fairly sure that the bottleneck is in the ntfs drivers as the python doesn't do any real "work" it just asks other programs to (filefrag, find and rsync), these programs themselves just make calls to the filesystem so are unlikely to use much CPU (well apart from rsync that could be the CPU hog if only the move is slow).

Finally you should note that pyfragtools doesn't understand ntfs but windows XP does so, the windows XP defrager will but the files in a better location on the disk (except for system files as it can't move them), so my advice would be
1) run windows XP defrag to do what it can
2) run pyfrag tools to move the systemfiles

[ShadowTek]

Finally you should note that pyfragtools doesn't understand ntfs but windows XP does so, the windows XP defrager will but the files in a better location on the disk (except for system files as it can't move them), so my advice would be
1) run windows XP defrag to do what it can
2) run pyfrag tools to move the systemfiles

I *did* do things in that order, but, as I said before, XP wouldn't defrag the vast majority of the files (which weren't system files), in spite of the fact that most of the filesystem was empty space.

The whole reason that I had to use pyfrag in the fist place is because XP's defragmenter wouldn't do an acceptable job.

[johnviglis]

Actually there might be a point to all this.

I checked my fragmentation level on a clean installation of 10.10 and the result was fragmentation 0%
The only fragmented files were some files in ~./mozilla and about 10 log files.
The result was:

========= COMPLETE ===========
Remaining Fragmented Files:
8.91 /var/log/kern.log
16.55 /var/log/messages
22.57 /var/log/syslog
48.23 /var/log/debug
50.40 /var/log/daemon.log
177.71 /var/log/Xorg.0.log
196.73 /var/log/gdm/:0.log
261.48 /var/log/auth.log
262.56 /home/user/.mozilla/firefox/8kvrt15u.default/cookies.sqlite
434.37 /var/log/user.log
Frags/unit Before: 1479.52
Frags/unit After: 1479.52
Improvement: 0.0 %
===============================

This was the second time I ran the script so no improvement was made, "Frags/unit Before" were about 5000 in the first execution. All the files that remained fragmented were scipped because they were opened for write.
Of cource there was no effect on my pc's performance, and the whole process lasted about 10 minutes.


On the contrary , I checked the fragmentation on a 10.10 installation, which was a dist upgrade of 10.04, which was a dist upgrade of 9.10, which was a clean installation a few months ago... The hdd capacity is 60GB with 46GB free space.

The sound of the hard working hdd and the increase in boot time brought me to this thread. After 40 minutes of analyzing the whole partition (containing both system and user locations), the result was fragmentation 36.8%

Analyze finished. 36.8 % fragmentation (55473 files), 192.4 average frags
Fragmented files:
511.88 /usr/src/linux-headers-2.6.35-24/include/sound/emu8000.h
511.88 /usr/src/linux-headers-2.6.35-24/arch/blackfin/include/asm/dma-mapping.h
511.88 /usr/share/icons/oxygen/48x48/apps/oxygen.png
511.88 /usr/share/icons/hicolor/48x48/apps/oxygen.png
511.88 /usr/share/icons/Humanity/actions/48/media-eject.svg
511.88 /usr/lib/pymodules/python2.6/orca/scripts/apps/packagemanager/speech_generator.pyc
511.75 /usr/src/linux-headers-2.6.35-24/include/linux/delayacct.h
511.75 /usr/share/kde4/apps/katepart/script/lilypond.js
511.75 /usr/share/icons/Humanity/actions/16/media-seek-backward.svg
511.75 /usr/share/gnome/help/gnomine/C/legal.xml
...
How many passes to run? [10]

Tomorrow I'll know if this script is effective in such a messy hdd, as well as if the system's performance was increased.

[iX9]

Hi!

I dont believe pydefrag is working on ext4:

I have 8 GiB ext4 system partition with kubuntu 11.04. It has ~60% used. Then I create 2GiB swapfile there. Big file created on high use of disk - it must be fragmented! So I ran pydefrag:

$ sudo defrag /Swap
Building list of files to analyze... done!

Analyze finished.
/Swap is not fragmented. Go home.
$

No fragments?! Hmm.

$ sudo filefrag -v /Swap
Filesystem type is: ef53
File size of /Swap is 2147483648 (524288 blocks, blocksize 4096)
ext logical physical expected length flags
0 0 364544 2048
1 2048 380928 366591 2048
2 4096 409600 382975 2048
3 6144 423936 411647 2048
4 8192 430080 425983 2048
5 10240 509952 432127 14336
6 24576 716800 524287 4096
7 28672 722944 720895 6144
8 34816 733184 729087 2048
9 36864 737280 735231 2048
10 38912 743424 739327 4096
11 43008 749568 747519 8192
12 51200 798720 757759 2048
13 53248 804864 800767 2048
14 55296 811008 806911 8192
15 63488 835584 819199 2048
16 65536 831488 837631 4096
17 69632 903168 835583 14336
18 83968 919552 917503 2048
19 86016 962560 921599 32768
20 118784 995328 12288
21 131072 1009664 1007615 32768
22 163840 1042432 6144
23 169984 1220608 1048575 2048
24 172032 1269760 1222655 32768
25 204800 1302528 32768
26 237568 1335296 32768
27 270336 1368064 32768
28 303104 1400832 32768
29 335872 1433600 32768
30 368640 1466368 32768
31 401408 1499136 32768
32 434176 1531904 32768
33 466944 1564672 8192
34 475136 1695744 1572863 32768
35 507904 1728512 16384 eof
/Swap: 24 extents found
$

By filefrag the file have I thing 24 fragments!
Or am I wrong?

[jerrrys]

i do not have pydefrag installed, its not in my synaptic. but i think you can take it at face value. meaning that pydefrag will not see it as fragemented with just 24 frags. i have used this command in the past to check mine

sudo fsck -vn /dev/sda1

and it always reads clean

[TheFerminator]

I know this is an old thread, but I need to get some .iso images to 0 frags so grub will let me boot them. I ran pydefrag, but they are stil fragmented enough that grub wont load them. I get ERROR 60: ...