Is there a Linux application that can compare two binary files?

[Wybiral]

I don't think so... However you do bring up a very interesting feature that could be added to a binary differing tool... Especially with assembly... That would probably be very helpful!

Good idea.

This inspires me to work on a small toolkit for dealing with binary files...

jblebrun, do you mind if I steal your idea?

[jblebrun]

I don't think so... However you do bring up a very interesting feature that could be added to a binary differing tool... Especially with assembly... That would probably be very helpful!

Good idea.

This inspires me to work on a small toolkit for dealing with binary files...

jblebrun, do you mind if I steal your idea?

Go for it! I stole it from the way text diffing works, anyway. If you do a diff on a text file, it can detect inserted or removed lines... I just transfered that concept to individual bytes. I never come up with my own ideas... I just shuffle other people's ideas around

[Lux Perpetua]

Ok, so I have a question. I haven't really used any of these binary differencing tools/libraries. Consider a case like this:

File 1:
01 45 6a 89 12 46 78 99

File 2:
45 6a 89 12 45 ff 78 99

Do the differencing tools have a method for reporting a "missing" or "added" byte?

Showing how file 2 is different from file 1 would be:
1 -01
5 -46 +45
6 +ff

That problem has a name: "edit distance." The edit distance between two strings is the minimum number of character insertions, deletions, and mutations required to transform one into the other. IIRC, computational biologists make good use of this to compare DNA sequences.

edit: As a workaround, you can use a combination of diff and hexdump to detect inserted and deleted bytes:

hexdump -e "1 1 \"%02x\n\"" file1 > temp1
hexdump -e "1 1 \"%02x\n\"" file2 > temp2
diff temp1 temp2

[jblebrun]

That problem has a name: "edit distance." The edit distance between two strings is the minimum number of character insertions, deletions, and mutations required to transform one into the other. IIRC, computational biologists make good use of this to compare DNA sequences.

Thanks for giving me the official name for that. Defining a metric like edit distance makes the problem much more clear!

I'm gonna guess that you recalled correctly: if you ask me, it sounds like a problem that's 100% applicable to DNA.

[Scott Rose]

The answer sort of depends what you want to know about the files, and how large they are.

If all you care about is whether they differ at all, then use the md5sum program to compute a hash of each file, and compare the hashes.

diff will compare binary files, but like the md5sum approach only tells you if they differ.

cmp will- if called with -l- tell you where and how the files differ.

[wangrui]

I need to compare very large files. Which need several hour. diff don't print any progress. I wrote a very simple java program to do the thing.

import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileInputStream;

public class Test {

	public static void main(String[] args) throws Exception {
		String name = "backup.part1.rar";
		File file1 = new File("/home/bak/1/" + name);
		File file2 = new File("/mnt/" + name);
		java.io.BufferedInputStream fi1 = new BufferedInputStream(
				new FileInputStream(file1));
		java.io.BufferedInputStream fi2 = new BufferedInputStream(
				new FileInputStream(file2));
		byte[] bs1 = new byte[1024 * 1024];
		byte[] bs2 = new byte[1024 * 1024];
		long pos = 0;
		while (true) {
			System.out.format("%,d\n", pos);
			int len1 = fi1.read(bs1);
			int len2 = fi2.read(bs2);
			if (len1 != len2)
				throw new Error();
			if (len1 < 0)
				break;
			for (int i = 0; i < bs1.length; i++) {
				if (bs1[i] != bs2[i]) {
					System.out.println(pos + i);
					throw new Error();
				}
			}
			pos += len1;
		}
		fi1.close();
		fi2.close();
	}
}

[cb951303]

I recently needed to compare binary files to reverse engineer a homebrew file format. What I needed was a program to find big blocks that are same in each file. For binary, a line by line comparison like diff doesn't mean much. But then again, it may be what you need

[Endolith]

Edit and compare giant binary files with lfhex

http://www.linux.com/feature/135690

(Edit: After installing 80 MB of dependencies, it still won't install. I give up.)


My application is two almost-identical .wav files that I created a long time ago and stored in different places. They are not binary-equal, though, so some errors must have occurred in transmission or storage. I want to see if I can figure out which is the original.

cmp -l Untitled.wav Untitled2.wav 
  4973185 125 135
 93124225 155 145
 93212289 361 371
100802177 274 264

So they only differ in four places, by single bits?

[fie]

VBINDIFF(1) Christopher J. Madsen VBINDIFF(1)

NAME
vbindiff - hexadecimal file display and comparison

SYNOPSIS
vbindiff file1 [ file2 ]

DESCRIPTION
Visual Binary Diff (VBinDiff) displays files in hexadecimal and ASCII (or EBCDIC). It can also display two files at once, and highlight the differences between them. Unlike diff, it works well with large files (up to 4 GB).

[Krupski]

What I need is a hex editor type application that can take two binary files and show the difference between the two files. Does such an application exist for Linux?

Hey... look at this thread. It *might* be what you need.

http://ubuntuforums.org/showthread.php?t=1075459

-- Roger