Page 1 of 3 123 LastLast
Results 1 to 10 of 27

Thread: Confused over MD5 checksums (Windows vs Linux)

  1. #1
    Join Date
    Mar 2012
    Location
    Pittsburgh, PA [USA]
    Beans
    46
    Distro
    Ubuntu 12.04 Precise Pangolin

    Confused over MD5 checksums (Windows vs Linux)

    I'm confused about the whole MD5 checksum thing.

    Allow me to explain what it is I am confused about.

    I have MD5 text files that I've created in Windows and now that I'm using Linux (in this case Ubuntu 12.04 Beta 1) I am finding that I cannot check them without being told that there is a formatting error.

    So I use md5sum (in terminal) to create a new MD5 file. When I compare it to the other MD5 file (that was made in Windows) I see that the numbers (the MD5 checksums) match up and that the formatting is ALMOST the same but not exact. I'll give you an example.

    Say I have a folder with two files in it. We'll call them filename01.mkv and filename02.jpg for this example. The MD5 file made in Windows looks like Example A below and the MD5 file made in Linux looks like Example B below:

    Code:
    Example A - Windows MD5
    253343b4d39e9e8b337a7db300d5e7dd *filename01.mkv
    cd103030ad0dd35ecf0af1d6cc327417 *filename02.jpg
    
    Example B - Linux MD5
    253343b4d39e9e8b337a7db300d5e7dd  filename01.mkv
    cd103030ad0dd35ecf0af1d6cc327417  filename02.jpg
    As you can see the two are almost exactly the same other than the ASTERISK in front of the file names (in the Windows created MD5 file). The Linux created MD5 file doesn't have the ASTERISKS. Instead there is a space where the ASTERISK would be.

    So as I said when I try to use md5sum in Linux to check a MD5 made in Windows I'm given an error (wrong formatting). When I try to check a MD5 made in Linux using Windows I also get an error (again due to the difference in formatting).

    Is there a way around this formatting difference? Right now the only solution I see is to manually edit the MD5 file to remove all the of the ASTERISKS (since most of my MD5 files are Windows created and yet I'm now trying to convert myself full time to Linux via Ubuntu which means I need to check these Windows created MD5 files in Linux).

    All help appreciated!

  2. #2
    Join Date
    Dec 2007
    Location
    California
    Beans
    4,912
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: Confused over MD5 checksums (Windows vs Linux)

    Based on this http://thestarman.narod.ru/DOS/MD5progs.html

    it shouldn't really matter, using the asterisk is supposdly an archaic way to format md5sum files, on my system md5sum doesn't complain abou asterisks being there. Perhaps you simply need a more up-to-date program for your md5suming in windows?
    "You can't expect to hold supreme executive power just because some watery tart lobbed a sword at you"

    "Don't let your mind wander -- it's too little to be let out alone."

  3. #3
    Join Date
    Mar 2012
    Location
    Pittsburgh, PA [USA]
    Beans
    46
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: Confused over MD5 checksums (Windows vs Linux)

    Been playing with this more since I posted the above and now I'm even more confused LOL

    As I said in the post above I can't seem to use the Linux 'md5sum' terminal command to check the MD5 files made in MS Windows without getting an error. I pointed out the difference in formatting between my MS Windows made MD5 files and the MD5 files that I create under Linux (again using Ubuntu 12.04 Beta 1 right now).

    Well here is how I've been making my Linux MD5 files:
    md5sum *.* > md5.txt

    However I was reading about the -b binary option so I tried this:
    md5sum *.* -b > md5.txt

    The MD5 without the -b option looks like my "Example B" in the above post. However the MD5 using the -b option looks like my "Example A" in the above post.

    In other words if I create a MD5 in Linux with the md5sum -b option then it looks EXACTLY like the MD5 made in Windows.

    Now as you can imagine I can use md5sum to verify the Linux made MD5 and it works regardless of how the MD5 was made (be it with the -b option or without). That makes sense but what doesn't make sense is that the Windows made MD5 (which looks EXACTLY like the Linux made MD5 with the -b option) gives me an error when trying to verify with md5sum. What the hell?

    And if that isn't confusing enough ... check this out:
    I went back into MS Windows and from there I can successfully use the Linux made MD5 (the one made with the -b option) and it works.

    I hope this isn't confusing. Again to be clear: The Windows made MD5 has the ASTERISKS but md5sum in Linux gives me errors when I try to check it (I get no such errors checking the same Windows made MD5 in Windows). However when I make a MD5 in Linux using the -b command I get the ASTERISKS and as far as I can tell it looks EXACTLY like the Windows made MD5 and md5sum will successfully check it (and the same file even works in Windows).

    So why can't md5sum check the Windows made MD5 files?

    I don't understand what the problem is here <<< rips out hair >>>

    *** EDIT ***
    I just wanted to add that on MS Windows I use a program called "Trader's Little Helper" in order to make and verify MD5 checksum files. I use this program because it was the only simple MD5 GUI type program I could find that seemed to work (again the MD5 files never gave me trouble in MS Windows and I even tried using another different MD5 GUI program on Windows to double check the MD5's made with Trader's Little Helper and it had no problems either with the files. I stopped using that other program though because it was old and would crash a lot ... it was made back when WinXP was current and never updated ... whereas Trader's Little Helper was updated as recently as 2011)
    Last edited by FulciLives; March 16th, 2012 at 05:26 AM.

  4. #4
    Join Date
    Dec 2007
    Location
    California
    Beans
    4,912
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: Confused over MD5 checksums (Windows vs Linux)

    I wonder if it's the line endings, Windows uses diferent line endings than Unix like OS's.

    If you install the program dos2unix, and run it with the -b option, it will convert your file to unix line endings and make a backup of the original. Check it's man page for more info. ie

    Code:
    dos2unix -b /pita/md5file
    Again, just taking stabs in the dark.
    "You can't expect to hold supreme executive power just because some watery tart lobbed a sword at you"

    "Don't let your mind wander -- it's too little to be let out alone."

  5. #5
    Join Date
    Mar 2012
    Location
    Pittsburgh, PA [USA]
    Beans
    46
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: Confused over MD5 checksums (Windows vs Linux)

    Quote Originally Posted by jerome1232 View Post
    I wonder if it's the line endings, Windows uses diferent line endings than Unix like OS's.

    If you install the program dos2unix, and run it with the -b option, it will convert your file to unix line endings and make a backup of the original. Check it's man page for more info. ie

    Code:
    dos2unix -b /pita/md5file
    Again, just taking stabs in the dark.
    Interesting.

    I didn't actually try this (what you suggested) but it gave me an idea ...

    I opened the Windows made MD5 with a text editor (while in Ubuntu) and copied it and pasted it into a new txt file. Just a simple COPY & PASTE is all. Guess what? The new file works now with md5sum. Son of a bitch, huh?

    Apparently it is some sort of Windows vs Linux file/text formatting issue.

    Please note that after I tried this (and saw that it worked) I decided if maybe I couldn't make it even simpler. So I tried to open the Windows made MD5 in Ubuntu with a text editor and then simply use the "SAVE AS" feature to save it again (under the same name). Unfortunately that didn't work. It seems I need to COPY AND PASTE it into a new Linux made txt document for it to work.
    Last edited by FulciLives; March 16th, 2012 at 05:53 AM.

  6. #6
    Join Date
    Dec 2007
    Location
    California
    Beans
    4,912
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: Confused over MD5 checksums (Windows vs Linux)

    What text editor are you using for Linux, Gedit has an option to specify your line endings, (hit save as and change the line endings lower right corner) and for Windows I believe Notepad+ (or is it notepad++? can't remember but it's a great text editor) has the option to specify your line endings as well.
    Last edited by jerome1232; March 16th, 2012 at 06:02 AM.
    "You can't expect to hold supreme executive power just because some watery tart lobbed a sword at you"

    "Don't let your mind wander -- it's too little to be let out alone."

  7. #7
    Join Date
    Mar 2012
    Location
    Pittsburgh, PA [USA]
    Beans
    46
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: Confused over MD5 checksums (Windows vs Linux)

    Quote Originally Posted by jerome1232 View Post
    What text editor are you using for Linux, Gedit has an option to specify your line endings, (hit save as and change the line endings lower right corner) and for Windows I believe Notepad+ (or is it notepad++? can't remember but it's a great text editor) has the option to specify your line endings as well.
    I could kiss you LOL

    When I right click on my MD5 files I get an option to open into a text editor and I guess by default Ubuntu uses gedit. So yes I tried what you said and it works.

    So basically I just have to open the file into gedit (very easy to do via the GUI) and hit SAVE AS and change the "Line Ending" to "Unix/Linux" and that does the trick!

    Jeez what a pain in the *** that was but THANK YOU for helping me to figure it all out.

    I greatly appreciate it!

    HOWEVER there is still one more issue I need to tackle in regards to all this (ha ha and you thought this was finally solved).

    Some of my Windows made MD5 checksum files have sub folders in them.

    For instance I have a lot of music files. Every album is in its own folder (or dir). All the files for that album are in that folder but then I have a folder in that folder called "Artwork" that has the CD scans.

    The MD5 file is located in the ALBUM folder but also references the files in the ARTWORK sub folder.

    Here is an actual MD5 file to show you exactly what I mean:

    Code:
    210af50eaf926e6b7b20bbff14435437 *00. Manfred Hubler & Sigfried Schwab - 3 Films by Jess Franco.m3u
    17e2157b86c1414e897189eb10150c19 *01. The Lions and the Cucumber.flac
    3bf44f702be425b0b211a56c8fb6a08a *02. Psycho Contact - Part One.flac
    e4cb3cce799aea2f3c8ac303497de378 *03. There is No Satisfaction.flac
    b05c05a5c0f6eecc521097b714b2bb1b *04. Psycho Contract - Part Two.flac
    c0380de9cdc70a00e7b14b5aa9ffe551 *05. The Message.flac
    4a41ca7499e6c5933d6b25c2c32c5361 *06. Psycho Contract - Part Three.flac
    a655eb87af3fed10493c411f261c7710 *07. Ghost or Good and Bad Onions.flac
    a1ba312bb9c3adb7778bb8ad700ee261 *08. Psycho Contract - Part Four.flac
    311ba7cb146f829a9ad0ab2d0a372878 *09. Countdown to Nowhere.flac
    a41726f960c83228bfde81cf6e230c97 *10. Psycho Contract - Part 5.flac
    2be7f5d2490b4cfdbb4209d21fa52fc8 *11. Droge CX 9.flac
    7117d5f6729f7b870ffec4598ebe8f18 *12. Dedicated to Love.flac
    786735d660c44af213a15ad76112a5b3 *13. People's Playground Version A.flac
    e64e4729e34e0d321b3c6ddcb0779233 *14. We Dong Care.flac
    749ef42462c0dc7943742c6ac6e9ec08 *15. People's Playground Version B.flac
    f6fb69cfd4ce1b51c6544630a9e51dd2 *16. The Ballad of a Fair Singer.flac
    73a21858a2b7ccb1a02dd66129935c0c *17. Peoples Plaground Version C.flac
    b8d740fc9e17424a4c32d7755704a952 *18. Necronomania.flac
    e1c5acce5377f3d4fd397335bde5fb50 *19. Kama Sutra.flac
    e031b7e1a5fd1913717fec91031a120c *20. People's Playground Version D.flac
    28a188dbd6d9de413a2d4ae6a510d4c3 *21. Shindai Lovers.flac
    b4ecd964390ac0c7864f17fcbd532346 *22. Konkubination.flac
    90efc07a5905e50dc848f046c6f2e21a *23. People's Playground Version E.flac
    e4cbdbd14b76d6ece5b79305df700494 *24. The Six Wisdoms of Aspasia.flac
    75293a26165c2351474fc0e13e523747 *CUE Sheet File - 3 Films by Jess Franco (flac).cue
    66ed65564b21fdaf805816572c8cefd4 *CUE Sheet File - 3 Films by Jess Franco (wav).cue
    55db3167f43399418a1e9c189cfea9cd *CUETools LOG File - 3 Films by Jess Franco.accurip
    f21a19ff4ce11e78b6a8aac66dc212dd *EAC LOG File - 3 Films by Jess Franco.log
    c77962db9d5df2286fe83b3de39c6b9d *folder.jpg
    c77962db9d5df2286fe83b3de39c6b9d *Artwork\3 Films by Jess Franco - folder image.jpg
    b18cc3be93c94d4f1880e440e8381d65 *Artwork\3 Films by Jess Franco back booklet Large.jpg
    f4cf0a0ff8fb82a54066ca647e3f6d45 *Artwork\3 Films by Jess Franco back cvr.jpg
    4c3c4070e261e77d57cbf0e6ee3a2c4f *Artwork\3 Films by Jess Franco book 1.jpg
    e796f318f2672487463f5bb56eed82d2 *Artwork\3 Films by Jess Franco book 2.jpg
    5ffdc9d961345d3acd88697e03c7bd60 *Artwork\3 Films by Jess Franco book 3.jpg
    a28b5e74bc382e5a730ab2f0d6e4580e *Artwork\3 Films by Jess Franco book 4.jpg
    f24972555be578d4dca5a211822ae360 *Artwork\3 Films by Jess Franco front cvr Large.jpg
    c072c5de4ab54b811048a4ac34cdf0e5 *Artwork\CD Info.jpg
    Now when I use md5sum to check this it checks all of the files except for those in the ARTWORK sub directory. It simply says that it cannot find them or that they are missing or some such thing.

    Currently I'm using the following Terminal command: md5sum -c checksum.md5
    Note "checksum.md5" is the name of the txt file with the MD5 checksum totals (as seen in the example above for the Jess Franco CD)

    So is there some way to check those files in the sub directory using md5sum?

    I did try to google this and I came up with something called "md5deep" which is like md5sum except it can also check sub directories. Here is a link to it: CLICK HERE

    Now I was able to install this "md5deep" but despite the examples given on that website I can't figure out how to use it. The examples seem to be geared more towards creation of MD5 files rather than checking existing MD5 files. I find all this command line syntax confusing without examples for my exact situation. Help please?

  8. #8
    Join Date
    Dec 2007
    Location
    California
    Beans
    4,912
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: Confused over MD5 checksums (Windows vs Linux)

    Quote Originally Posted by FulciLives View Post
    So basically I just have to open the file into gedit (very easy to do via the GUI) and hit SAVE AS and change the "Line Ending" to "Unix/Linux" and that does the trick!
    Kind of why I mentioned dos2unix, you could batch convert every .txt file in a folder as easily as

    Code:
    dos2unix *.txt

    Quote Originally Posted by FulciLives View Post
    Code:
    210af50eaf926e6b7b20bbff14435437 *00. Manfred Hubler & Sigfried Schwab - 3 Films by Jess Franco.m3u
    17e2157b86c1414e897189eb10150c19 *01. The Lions and the Cucumber.flac
    3bf44f702be425b0b211a56c8fb6a08a *02. Psycho Contact - Part One.flac
    e4cb3cce799aea2f3c8ac303497de378 *03. There is No Satisfaction.flac
    b05c05a5c0f6eecc521097b714b2bb1b *04. Psycho Contract - Part Two.flac
    c0380de9cdc70a00e7b14b5aa9ffe551 *05. The Message.flac
    4a41ca7499e6c5933d6b25c2c32c5361 *06. Psycho Contract - Part Three.flac
    a655eb87af3fed10493c411f261c7710 *07. Ghost or Good and Bad Onions.flac
    a1ba312bb9c3adb7778bb8ad700ee261 *08. Psycho Contract - Part Four.flac
    311ba7cb146f829a9ad0ab2d0a372878 *09. Countdown to Nowhere.flac
    a41726f960c83228bfde81cf6e230c97 *10. Psycho Contract - Part 5.flac
    2be7f5d2490b4cfdbb4209d21fa52fc8 *11. Droge CX 9.flac
    7117d5f6729f7b870ffec4598ebe8f18 *12. Dedicated to Love.flac
    786735d660c44af213a15ad76112a5b3 *13. People's Playground Version A.flac
    e64e4729e34e0d321b3c6ddcb0779233 *14. We Dong Care.flac
    749ef42462c0dc7943742c6ac6e9ec08 *15. People's Playground Version B.flac
    f6fb69cfd4ce1b51c6544630a9e51dd2 *16. The Ballad of a Fair Singer.flac
    73a21858a2b7ccb1a02dd66129935c0c *17. Peoples Plaground Version C.flac
    b8d740fc9e17424a4c32d7755704a952 *18. Necronomania.flac
    e1c5acce5377f3d4fd397335bde5fb50 *19. Kama Sutra.flac
    e031b7e1a5fd1913717fec91031a120c *20. People's Playground Version D.flac
    28a188dbd6d9de413a2d4ae6a510d4c3 *21. Shindai Lovers.flac
    b4ecd964390ac0c7864f17fcbd532346 *22. Konkubination.flac
    90efc07a5905e50dc848f046c6f2e21a *23. People's Playground Version E.flac
    e4cbdbd14b76d6ece5b79305df700494 *24. The Six Wisdoms of Aspasia.flac
    75293a26165c2351474fc0e13e523747 *CUE Sheet File - 3 Films by Jess Franco (flac).cue
    66ed65564b21fdaf805816572c8cefd4 *CUE Sheet File - 3 Films by Jess Franco (wav).cue
    55db3167f43399418a1e9c189cfea9cd *CUETools LOG File - 3 Films by Jess Franco.accurip
    f21a19ff4ce11e78b6a8aac66dc212dd *EAC LOG File - 3 Films by Jess Franco.log
    c77962db9d5df2286fe83b3de39c6b9d *folder.jpg
    c77962db9d5df2286fe83b3de39c6b9d *Artwork\3 Films by Jess Franco - folder image.jpg
    b18cc3be93c94d4f1880e440e8381d65 *Artwork\3 Films by Jess Franco back booklet Large.jpg
    f4cf0a0ff8fb82a54066ca647e3f6d45 *Artwork\3 Films by Jess Franco back cvr.jpg
    4c3c4070e261e77d57cbf0e6ee3a2c4f *Artwork\3 Films by Jess Franco book 1.jpg
    e796f318f2672487463f5bb56eed82d2 *Artwork\3 Films by Jess Franco book 2.jpg
    5ffdc9d961345d3acd88697e03c7bd60 *Artwork\3 Films by Jess Franco book 3.jpg
    a28b5e74bc382e5a730ab2f0d6e4580e *Artwork\3 Films by Jess Franco book 4.jpg
    f24972555be578d4dca5a211822ae360 *Artwork\3 Films by Jess Franco front cvr Large.jpg
    c072c5de4ab54b811048a4ac34cdf0e5 *Artwork\CD Info.jpg
    Now when I use md5sum to check this it checks all of the files except for those in the ARTWORK sub directory. It simply says that it cannot find them or that they are missing or some such thing.

    Currently I'm using the following Terminal command: md5sum -c checksum.md5
    Note "checksum.md5" is the name of the txt file with the MD5 checksum totals (as seen in the example above for the Jess Franco CD)

    So is there some way to check those files in the sub directory using md5sum?

    I did try to google this and I came up with something called "md5deep" which is like md5sum except it can also check sub directories. Here is a link to it: CLICK HERE

    Now I was able to install this "md5deep" but despite the examples given on that website I can't figure out how to use it. The examples seem to be geared more towards creation of MD5 files rather than checking existing MD5 files. I find all this command line syntax confusing without examples for my exact situation. Help please?
    I'd have to look into it, I'll get back to you on it.
    "You can't expect to hold supreme executive power just because some watery tart lobbed a sword at you"

    "Don't let your mind wander -- it's too little to be let out alone."

  9. #9
    Join Date
    Dec 2007
    Location
    California
    Beans
    4,912
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: Confused over MD5 checksums (Windows vs Linux)

    Okay, take a look at md5deep man page.

    Code:
    man md5deep
    Based on what it says (give it a read, it's got a lot of good info in there)

    I would think on of these options is what you want

    Code:
    md5sum -k -m /path/to/md5file
    ###
    ### or
    ###
    md5sum -k -M /path/to/md5file
    ###
    ### maybe it will need the -r option?
    ###
    md5sum -rk -m /path/to/md5file
    #### maybe this one?
    md5sum -nk -m /path/to/md5file
    Anyways just look at the man page and try various options out, read what they do, you have an understanding of what you want to see.
    Last edited by jerome1232; March 16th, 2012 at 07:41 AM. Reason: Forgot an option
    "You can't expect to hold supreme executive power just because some watery tart lobbed a sword at you"

    "Don't let your mind wander -- it's too little to be let out alone."

  10. #10
    Join Date
    Mar 2012
    Location
    Pittsburgh, PA [USA]
    Beans
    46
    Distro
    Ubuntu 12.04 Precise Pangolin

    Re: Confused over MD5 checksums (Windows vs Linux)

    I assume jerome1232 that whenever you said "md5sum" in your last post that you actually meant "md5deep"

    Anyway I tried your suggestions and some others and despite 'playing around' with it some I just can't figure it out.

    Nothing seems to be working as it should with this "md5deep"

Page 1 of 3 123 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •