BgRva

software, data, designs, …

NUnit Tricks for Parameterized Tests

Having spent a fair amount of time using unit test frameworks, you inevitable come up with a few tricks to make things more efficient. This post simply describes some of those tricks that I’ve picked up while testing code. The examples will be done with NUnit versions 2.6, but I’ve applied these same techniques to MBunit, XUnit, and am currently using them with the NUnit beta 3.0.

Test Ids to Increase Readability

Often, some units under test have a wide variation of input parameters and require many test cases to effectively beat on the code. Take the following example. Though it is contrived, it demonstrates situations I’ve had to handle in almost every project I’ve been involved with. The class below has a single method to verify some input string based on a defined rule set.

Verifyer.cs class
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
public class Verifier
{
    /// <summary>
    /// Returns true if input contains the substring of criteria.ToUpper()
    /// or criteria.ToLower() and the qualifier occurs after the criteria.
    /// </summary>
    /// <param name="input">The input string being validated</param>
    /// <param name="criteria">The criteria for validating the input string</param>
    /// <returns>bool</returns>
    public bool Verify(string input, string criteria, string qualifier)
    {
        var criteriaIndex = input.IndexOf(criteria.ToLower());
        if(criteriaIndex < 0)
        {
            criteriaIndex = input.IndexOf(criteria.ToUpper());
        }

        if (criteriaIndex >= 0 && input.IndexOf(qualifier, criteriaIndex) > criteriaIndex)
            return true;

        return false;
    }
}

Effective unit tests for the code above would need to test for cases returning true, false, and error cases, (I would normally generate the tests before the method, but for simplicity I will dispense with preaching TDD). For the true cases, here are several approaches using NUnit, (for brevity’s sake, I have not included all possible variations of input parameters to test for true).

VerifierFixture.cs class
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
[TestFixture]
public class VerifierFixture
{
    [TestCase(0, "blahA blah", "blah", "A")]
    [TestCase(1, "blah Ablah", "blah", "A")]
    [TestCase(2, "blah blahA", "blah", "A")]
    [TestCase(3, "AblahA blah", "blah", "A")]
    [TestCase(4, "Ablah Ablah", "blah", "A")]
    [TestCase(5, "Ablah blahA", "blah", "A")]
    // more ...
    public void IsValid_Returns_True_By_TestCase(int id, string input, string criteria, string qualifier)
    {
        //Arrange
        var validator = new Verifier();

        //Act
        var result = validator.Verify(input, criteria, qualifier);

        //Assert
        Assert.True(result);
    }

    [Test, TestCaseSource("VerifyCasesThatAreTrue")]
    public void IsValid_Returns_True_By_TestCaseSource(int id, string input, string criteria, string qualifier)
    {
        //Arrange
        var validator = new Verifier();

        //Act
        var result = validator.Verify(input, criteria, qualifier);

        //Assert
        Assert.True(result);
    }

    static object[] VerifyCasesThatAreTrue =
    {
        new object[] {0, "blahA blah", "blah", "A"},
        new object[] {1, "blah Ablah", "blah", "A"},
        new object[] {2, "blah blahA", "blah", "A"},
        new object[] {3, "AblahA blah", "blah", "A"},
        new object[] {4, "Ablah Ablah", "blah", "A"},
        new object[] {5, "Ablah blahA", "blah", "A"},
        // more ...
    };
}

Both test methods in the fixture above perform the same tests, but each handles the multiple test inputs using different features of NUnit. You will notice a unique id associated with each test input, this is one of those ‘tricks’ I picked up. By assigning a unique id for test methods with multiple inputs, you can quickly find a specific test case. For example, here is an image of the NUnit runner output with ids and it is easy to identify in the code the specific test cases which fail.

When should you use unique ids with your parameterized test cases? This is completely up to your style of test kung-fu. If it helps your more efficiently beat on your code then by all means do it.

Overriding ToString() in Complex Parameters

While NUnit recognizes the individual test parameters and consequently each individual test cast, it does not effectively present this information to the user. For example, lets say there is an additional Verify() method that takes in a custom object called UserItem as a parameter. UserItem is a stand in for any POCO that you may need to validate.

Verifier.cs and UserItem.cs
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
public class Verifier
{
    public bool Verify(string input, string criteria, string qualifier)
    {
        var criteriaIndex = input.IndexOf(criteria.ToLower());
        if(criteriaIndex < 0)
        {
            criteriaIndex = input.IndexOf(criteria.ToUpper());
        }

        if (criteriaIndex > 0 && input.IndexOf(qualifier, criteriaIndex) > criteriaIndex)
            return true;

        return false;
    }

    //New Method
    public bool Verify(UserItem input, string criteria, string qualifier)
    {
        return Verify(input.Name, criteria, qualifier);
    }
}

public class UserItem
{
    public DateTime TimeStamp { get; set; }
    public string Name { get; set; }
    public string SomeThing { get; set; }
}

In order to set up multiple test cases for this new method, we are unable to use the TestCaseAttribute out of the box since we cannot embed a UserItem constructor in an attribute. But, we can still use the TestCaseSourceAttribute like so:

VerifyByUserItemFixture.cs
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
[TestFixture]
public class VerifyByUserItemFixture
{
    [Test, TestCaseSource("VerifyCasesThatAreTrue")]
    public void IsValid_Returns_True_By_TestCaseSource(int id, UserItem input, string criteria, string qualifier)
    {
        //Arrange
        var validator = new Verifier();

        //Act
        var result = validator.Verify(input, criteria, qualifier);

        //Assert
        Assert.True(result);
    }

    static object[] VerifyCasesThatAreTrue =
    {
        new object[] {0, new UserItem() { Name = "blahA blah" }, "blah", "A"},
        new object[] {1,  new UserItem() { Name= "blah Ablah" }, "blah", "A"},
        new object[] {2,  new UserItem() { Name= "blah blahA" }, "blah", "A"},
        new object[] {3,  new UserItem() { Name= "AblahA blah" }, "blah", "A"},
        new object[] {4,  new UserItem() { Name= "Ablah Ablah" }, "blah", "A"},
        new object[] {5,  new UserItem() { Name= "Ablah blahA" }, "blah", "A"},
        // more ...
    };
}

When you look at the output you will notice that although NUnit identifies the individual test cases, it displays each UserItem by the default ToString() implementation:

To improve the readability of each test, you can override the ToString() method:

UserItem.cs with ToString() override
1
2
3
4
5
6
7
8
9
10
public class UserItem
{
    public DateTime TimeStamp { get; set; }
    public string Name { get; set; }
    public string SomeThing { get; set; }
    public override string ToString()
    {
        return string.Format("{0};{1};{2}", Name, TimeStamp, SomeThing);
    }
}

NUnit uses the parameters to differentiate individual test cases. If you do override ToString(), to preclude NUnit from skipping test cases it considers duplicates then you need to ensure the override of ToString() returns unique values for each object, or use the separate id parameter technique mentioned earlier. Using the id will guarantee each input results in an individual test even if the ToString() override does not provide unique strings for each object.

Just a few tricks that may help you in your code writing endeavors. Test early & test often: the more you beat on your code the stronger is the force in you.

Comments