A quick look at Smart Unit Tests

Looking at the latest Visual Studio 2015 CTP release I noticed the new Smart Unit Tests feature. This is an evolution of Microsoft Research’s Pex project and to quote Microsoft:

Smart Unit Tests explores your .NET code to generate test data and a suite of unit tests

That sounds pretty amazing, so I fired VS 2015 to have a look. I grabbed a piece of sample code that I wrote a while back for an interview, you can find it on GitHub if you want to follow along:

public class RomanNumeralGenerator : IRomanNumeralGenerator 
{ 
  static Dictionary<int, string> numerals = new Dictionary<int, string> 
  { 
    { 1000, "M" }, { 900, "CM" }, { 500, "D" }, { 400, "CD" }, 
    { 100, "C" }, { 90, "XC" }, { 50, "L" }, { 40, "XL" }, { 10, "X" }, 
    { 9, "IX" }, { 5, "V" }, { 4, "IV" }, { 1, "I" } 
  }; 
  public string Generate(int number) 
  {
    Validate(number); 
    StringBuilder numeralBuilder = new StringBuilder();
    foreach (var numeral in numerals) 
    { 
      while (number >= numeral.Key) 
      { 
        numeralBuilder.Append(numeral.Value); 
        number -= numeral.Key; 
      } 
    } 
    return numeralBuilder.ToString(); 
  } 
  void Validate(int number) 
  { 
    if (number < 1 || number > 3999) 
    { 
      throw new ArgumentOutOfRangeException(); 
    } 
  } 
}

RomanNumeralGenerator generates the roman numeral string equivalent of an integer, it throws an exception if the integer is above 3999 or below 1. To generate the test for this code I just right click inside the class file and select Smart Unit Tests.

SmartUnitTestsScreenshot1


This fires up the Smart Unit Tests Exploration generates a set of tests and lets you view details about them. ‌

SmartUnitTestsScreenshot2


This is actually really neat; tests have been created that trigger the exceptions as well as two tests that confirm correct strings are returned. Technically this class now has 100% code coverage, although really we would need to write a much more in depth test suite to cover all the potential bugs, and if I select one of the tests I can see what test code was generated.

SmartUnitTestsScreenshot3


I can also explore the events that generated the tests, modify how those tests are generated and finally select any or all the tests and save them. ‌

SmartUnitTestsScreenshot4


This generates a new unit test project, MSTest only currently, and adds it to your solution.

My first impressions

This seems like a great way to backport tests onto existing code but I hope that people don’t think that now there’s no longer any need to write tests because Visual Studio will write them for you. Smart Unit Tests are going to be a great tool for refactoring currently untested code, creating baseline tests so you can change with more confidence. I’m looking forward to trying out using them on a more complicated code base next.