Page 3 of 3 FirstFirst 123
Results 21 to 29 of 29

Thread: [HOW TO]Removing Duplicate Files(batch proccess)

  1. #21
    Join Date
    Jul 2006
    Beans
    36

    Re: [HOW TO]Removing Duplicate Files(batch proccess)

    Hi,

    I just ran fslint on my system, it used filename definitely and not md5 hash to discover duplicates, as it has selected 10 files with the same name, all of them photos which are different pictures and sizes.

    Is there a way to force the use of md5?

    Thanks,
    Steve

  2. #22
    Join Date
    Feb 2006
    Location
    Bordeaux, FRANCE
    Beans
    16
    Distro
    Ubuntu 7.04 Feisty Fawn

    Re: [HOW TO]Removing Duplicate Files(batch proccess)

    Hello.

    Many thnxX rabid9797 ... fdupes saved my life !

    @+...

  3. #23
    Join Date
    Nov 2006
    Beans
    Hidden!

    Re: [HOW TO]Removing Duplicate Files(batch proccess)

    Quote Originally Posted by Endolith View Post
    FSlint would be great if it had a useful interface. This finddups Windows program does it better
    The fslint version in ubuntu is quite old unfortunately.
    I've been trying without luck to get it upgraded.
    Fedora and debian always have the latest version available,
    and you can download the latest ubuntu package atwww.pixelbeat.org/fslint/

    Quote Originally Posted by Endolith View Post
    It lets you select an entire directory, and all duplicates in that directory will be removed, recursively, and any empty directories it creates will be deleted.

    You can also have it "auto mark" all but one of each file. It would be best if you could choose the criteria, though, like "all but the oldest".
    FSlint does all of the above since version 2.18.
    Well all except auto remove empty directories.
    That is a separate function within FSlint, but
    probably would be a good idea to incorporate
    into the duplicate removal functionality. Thanks.

  4. #24
    Join Date
    Feb 2007
    Location
    New York
    Beans
    894
    Distro
    Ubuntu 9.10 Karmic Koala

    Re: [HOW TO]Removing Duplicate Files(batch proccess)

    FSlint does all of the above since version 2.18.
    Really? I just installed 2.25 and I still don't see any way to auto-mark all files in a directory.

    I just now noticed the "mark within groups" in the right-click menu, though. That takes care of the "auto-mark" functionality, and goes a step above by letting you choose how it selects.

    Where is the documentation? All I see on the website is a compilation FAQ.

    Also I don't know how to find an "About.." box to be sure I'm running the current version.

    Well all except auto remove empty directories.
    That is a separate function within FSlint, but
    probably would be a good idea to incorporate
    into the duplicate removal functionality. Thanks.
    Ah. Well sometimes you might only want to delete the empty directories it leaves behind, and not ones that were already there.

    Other things the Windows program ha...



    Oh wait! You did see my previous requests, and implemented some of them!

    http://groups.google.com/group/pixel...184b15047cc4ec
    http://codebrowse.launchpad.net/~vcs...k/revision/111

    Let me try some things...

  5. #25
    Join Date
    Feb 2007
    Location
    New York
    Beans
    894
    Distro
    Ubuntu 9.10 Karmic Koala

    Re: [HOW TO]Removing Duplicate Files(batch proccess)

    Things that are done:
    • Already could select all but one in each group from the right-click menu, including the ability to select oldest or newest for keeping.
    • Delete with Delete key.
    • After you've deleted all but one instance of a particular file, it's removed from the list to let you focus on the others


    Maybe done:
    • Apparently selecting by directory is supported by wildcards, but I can't figure out how to use wildcards. I right-click, "Select using wildcard", type part of the path or filename, and nothing happens. In the program I am used to, you can delete all the dupes in a certain directory at once with a directory chooser dialog, which lets you filter out files which have changed between two copies of a directory, merge directories that have a lot of the same files, but aren't the same, etc. Wildcards would probably be better, but I can't get it to work.


    Things that could still be done:
    • "Groups" are more obvious because they're separated by color. Even just making the "4 x 351,863 (1,069,056)" group headers a different color or adding a dark line to separate groups would work.
    • Progress bar as it's scanning, with results displayed as it finds them, so you can stop partially through if you've found a particular file you were looking for. I think it scans by biggest files, first, too, so if you are just trying to clear some space you can stop after it finds two or three giant files.
    • It warns you if you (accidentally) mark every copy of a file for deletion, which is not the same as warning before all deletions. With wildcards this might be easier to do accidentally, too. Could show this by coloring the selection red perhaps? And popping up an "are you sure" box when you try to delete an entire group?
    • Can delete directories that become empty during the removal process, as above.
    • Can right-click on a directory in Windows Explorer and click "Find duplicates" and it opens with that directory as the default. Would be great if it could be integrated into Nautilus/etc this way.
    • Should be able to "fold up" or remove groups that you don't want to look at, like when it finds 500+ 1-byte files in my Google Earth directory, and I want to skip past them easily.
    • Column headers stretch to fit filenames and directories, even when they're super long. I can resize them, but after deleting one thing, they pop back to the super-wide widths.
    • Just cuts off the ends of paths and filenames when columns are too narrow. The best way to show a long path in a confined space is to cut out the middle: "/home/username/Pictu...ation/Filename.jpg"


    I was also expecting it to be able to access things from my ssh:// and smb:// bookmarks, and to move things to the trash instead of deleting them, but I guess those are really GNOME things, and this isn't a GNOME app?

    Thanks for this program!

  6. #26
    Join Date
    Nov 2006
    Beans
    Hidden!

    Re: [HOW TO]Removing Duplicate Files(batch proccess)

    Quote Originally Posted by Endolith View Post
    Things that are done:
    • Already could select all but one in each group from the right-click menu, including the ability to select oldest or newest for keeping.
    • Delete with Delete key.
    • After you've deleted all but one instance of a particular file, it's removed from the list to let you focus on the others
    Thanks for testing all this!

    Quote Originally Posted by Endolith View Post
    Maybe done:
    • Apparently selecting by directory is supported by wildcards, but I can't figure out how to use wildcards. I right-click, "Select using wildcard", type part of the path or filename, and nothing happens. In the program I am used to, you can delete all the dupes in a certain directory at once with a directory chooser dialog, which lets you filter out files which have changed between two copies of a directory, merge directories that have a lot of the same files, but aren't the same, etc. Wildcards would probably be better, but I can't get it to work.
    To select a directory use: /full/path/*
    To select a file type use: *.tmp
    To select common sub dirs use: */tmp/*

    Quote Originally Posted by Endolith View Post
    Things that could still be done:[*]"Groups" are more obvious because they're separated by color. Even just making the "4 x 351,863 (1,069,056)" group headers a different color or adding a dark line to separate groups would work.
    The group headers should be a different colour (specifically the same
    colour as the app backgound). Perhaps if you have a non standard theme
    this is a problem. I'll look into making it more obvious.

    Quote Originally Posted by Endolith View Post
    [*]Progress bar as it's scanning, with results displayed as it finds them, so you can stop partially through if you've found a particular file you were looking for.
    If you know what your looking for you should have more specific paths to scan. Adding results incrementally would require a complete rewrite of
    the logic underneath.

    Quote Originally Posted by Endolith View Post
    I think it scans by biggest files, first, too, so if you are just trying to clear some space you can stop after it finds two or three giant files.
    It actually tries to scan the disk in the order files are laid out on the disk,
    to minimise disk head seeking. This is a big performance win in general.

    Quote Originally Posted by Endolith View Post
    [*] It warns you if you (accidentally) mark every copy of a file for deletion, which is not the same as warning before all deletions. With wildcards this might be easier to do accidentally, too. Could show this by coloring the selection red perhaps? And popping up an "are you sure" box when you try to delete an entire group?
    It does warn for the first deletion, or for "all in group" deletions.
    I think this is OK.

    Quote Originally Posted by Endolith View Post
    [*]Can delete directories that become empty during the removal process, as above.[*]Can right-click on a directory in Windows Explorer and click "Find duplicates" and it opens with that directory as the default. Would be great if it could be integrated into Nautilus/etc this way.[*]Should be able to "fold up" or remove groups that you don't want to look at, like when it finds 500+ 1-byte files in my Google Earth directory, and I want to skip past them easily.[*]Column headers stretch to fit filenames and directories, even when they're super long. I can resize them, but after deleting one thing, they pop back to the super-wide widths.[*]Just cuts off the ends of paths and filenames when columns are too narrow. The best way to show a long path in a confined space is to cut out the middle: "/home/username/Pictu...ation/Filename.jpg"
    All very good points which I will try and incorporate in the next version.

    Quote Originally Posted by Endolith View Post
    I was also expecting it to be able to access things from my ssh:// and smb:// bookmarks, and to move things to the trash instead of deleting them, but I guess those are really GNOME things, and this isn't a GNOME app?
    Yes it's not a gnome app. Also there may be logical/performance issues
    for doing this over the network. I will keep this in mind though.

    thank you!

  7. #27
    Join Date
    Feb 2007
    Location
    New York
    Beans
    894
    Distro
    Ubuntu 9.10 Karmic Koala

    Re: [HOW TO]Removing Duplicate Files(batch proccess)

    Quote Originally Posted by pixelbeat View Post
    To select a directory use: /full/path/*
    To select a file type use: *.tmp
    To select common sub dirs use: */tmp/*
    OH! That works. I didn't know it was expecting asterisks.

    To save time, if you right-click a file, and select "Select using wildcard", it should pre-fill the text entry box with the filename you clicked on, so you can delete the parts you don't want and replace them with stars.

    The group headers should be a different colour (specifically the same
    colour as the app backgound). Perhaps if you have a non standard theme
    this is a problem. I'll look into making it more obvious.
    I just switched between all the GNOME themes and the colors didn't change. Both the filenames and group headers are black on white. Maybe I'm missing something.

    Adding results incrementally would require a complete rewrite of
    the logic underneath.

    It actually tries to scan the disk in the order files are laid out on the disk,
    to minimise disk head seeking. This is a big performance win in general.
    Ok. Just comparing to the Windows program. Neither is a big deal.

    It does warn for the first deletion, or for "all in group" deletions.
    I think this is OK.
    Well, the warning is really needed when the program might do something unexpected, like if you accidentally select all of one file and don't leave any copies of it on the drive, or if you mean to select three things with a wildcard and accidentally select two others as well and delete something you weren't aware of. If you just warn before every deletion, the person is just going to click Yes mindlessly every time.

  8. #28
    Join Date
    Jan 2012
    Beans
    1

    deleting duplicate files

    I am also facing the same issue of duplicate files. i got the solution when i used a utility http://duplicatefilesdeleter.com/ and it worked for me.

  9. #29
    Join Date
    Mar 2013
    Beans
    1

    Re: [HOW TO]Removing Duplicate Files(batch proccess)

    I've recently found a software "Duplicate Files Deleter", and it can help you with this. Check their web site . There is a video to explain the process. Pretty nice solution to delete duplicates safely.

Page 3 of 3 FirstFirst 123

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •