Output a Unix style diff report from a failed Unit Test

With unit tests, it is useful to know WHEN they fail, but also WHY they are failing.

For unit tests which validate the output from a program, it can be handy to have a simple 'known good output', in a text file. The unit tests can then use this text file to validate the current output.

If the current output is different from the 'known good output', then the unit test fails.

This is very simple - however, when refactoring and maintaining, chances are that the program output will change, and so the unit test will fail.

So here is a simple way to get a Unix style diff output, so that you can see exactly which parts of the output have changed.

You then know whether the new behaviour is:

* as desired -> so you need to update the 'known good output'
* NOT as desired -> so you have a bug. When you have made the fix, you can verify the fix by just running your unit tests

Unix style diff from a unit test

* Download the Unix utilities for Windows:

http://unxutils.sourceforge.net/

* Unzip the utilities to some temporary location such as %TEMP%\unixutils.

* Copy the file diff.exe to be located where ever you normally store 3rd party binaries in your source control. For example: {root}\{module}\3rdParty\unixutils\diff.exe

* In your Unit Test in C#, you can then perform a diff on the actual output 'plainOut' versus the known good output 'sampleOut':

[Test]
public void myTest()
{
  ...
  ...
  string plainOut = ...; //the current text output from your test

  //now verify the current output, against the known good output:
  string pathToKnownGoodOutputFile = "../test_data/myTest_goodOutput.txt";
  verifyMyTestOutput("myTest", plainOut, pathToKnownGoodOutputFile);
}

public void verifyMyTestOutput(string caller, string plainOut, string pathToKnownGoodOutputFile)
{
  StringBuilder sbSampleOutput = readTextFile(pathToKnownGoodOutputFile);
  string sampleOut = sbSampleOutput.ToString().Trim();

  if (plainOut != sampleOut)
  {
    StringBuilder sbDiffs = getDifferences(plainOut, sampleOut);
    Assert.Fail(caller + " - Output is not as expected!\n" + sbDiffs.ToString());
  }
}

private StringBuilder getDifferences(string plainOut, string sampleOut)
{
  StringBuilder sbDiff = new StringBuilder();

  string pathToDiff = @"..\3rdParty\unixutils\diff.exe";

  Process proc = new Process();
  string tempFile1 = writeToTempFile(plainOut);
  string tempFile2 = writeToTempFile(sampleOut);
  proc.StartInfo = new ProcessStartInfo(pathToDiff, "\"" + tempFile1 + "\" \"" + tempFile2 + "\"");
  proc.StartInfo.UseShellExecute = false;
  proc.StartInfo.RedirectStandardOutput = true;
  proc.Start();

  StreamReader reader = proc.StandardOutput;
  proc.WaitForExit();

  string text;
  while (!string.IsNullOrEmpty(text = reader.ReadLine()))
  {
    sbDiff.Append(text + "\n");
  }

  return sbDiff;
}

protected static StringBuilder readTextFile(string filePath)
{
  StringBuilder textBlockIn = new StringBuilder();
  using (StreamReader re = File.OpenText(filePath))
  {
    string input = null;
    while ((input = re.ReadLine()) != null)
    {
      textBlockIn.AppendLine(input);
    }
  }
  return textBlockIn;
}

  private string writeToTempFile(string text)
  {
   string tempFilePath = Path.GetTempFileName();

   using (TextWriter tw = new StreamWriter(tempFilePath))
   {
    tw.Write(text);
    tw.Flush();
    tw.Close();
   }
   return tempFilePath;
  }