Annotating Your Code with Simple Tests

Another idea: what if you annotated your functions with notes that say exactly what the most interesting edge cases are for that function.

Basically at the top of each method you could specify a few test cases. And only where you really need them. It's kind of a minimalists approach to test driven development.

Here's an example function -- leap year. With a few deliberate bugs in it.

continues...

    /// <summary>

    /// Is Year a leap year?

    /// </summary>

    /// <param name="Year">The year in question</param>

    /// <returns>

    /// true if year is a leap year. false otherwise

    /// </returns>

    private bool IsALeapYear(int Year)

    {

        return (Year % 4 == 0);

    }

Now the weird thing about 'leap years' is that every hundred years we skip a leap year. And every four hundred years we skip skipping a leap year.(I think that's it?) Anyway -- this means that a few simple tests will pick up the most common mistakes.

    /// <summary>

    /// Is Year a leap year?

    /// </summary>

    /// <param name="Year">The year in question</param>

    /// <returns>

    /// true if year is a leap year. false otherwise

    /// </returns>

    /// <testCase Year="2000" return="true"/> (a standalone test case)

    /// <testCase Year="1900" return="false"/> (another standalone test case)

    /// <testCase Year="1996" return="true"/> (a few more normal cases)

    /// <testCase Year="2003" return="false"/> (a few more normal cases)

    /// <testCase Year="1973" return="false"/> (and so on...)

    private bool IsALeapYear(int Year)

    {

        return (Year % 4 == 0);

    }

So in the sample above we've used XML Comments to indicate some very simple tests that would find the most common mistakes in such a function.

We then run some nifty little tool that inspects our code comments, generates the test code, runs it, and tells us how we went. "Oh we failed for this value? Better check out my code..."

(Note, i've used XML Comments rather than 'code attributes' [i.e. the code decorations you wrap in square brackets for C#, or angle brackets for VB.net] so that the units tests wouldn't need to be shipped with your assembly.)

I often take these ideas too far. So I want to pretty much stop there.

Part of me is screaming though, saying "That's so obvious!! Why can't we do that today!! That should be built into all languages!!" And I'm certain that other, smarter devs, have described this exact same system before. There's probably a university course on it somewhere. A lecture circuit ad so on)

(Quick shout out -- I'm using the excellent GhostDoc here (as I do everyday). And using Copy Source as Html. What a tool!)

Setup/Tear Down

Two aspects (intentional pun ;-) of Nunit are the setup and tear down for the test and test-fixture.

Maybe you could specify a particular setup for a test, like so:

    /// <summary>

    /// Gets the full name.

    /// </summary>

    /// <value>The full name.</value>

    /// <testCase return="Fred Smith" setup='PreparePerson("Fred","Smith");' />

    public string FullName

    {

        get { return String.Concat(m_FistName, " ", m_LastName);}       

    }

The idea of the snippet above is that the 'PreparePerson' function is called, and the test is then performed on the output of that function -- or something like that. In the same scope as that function, maybe. It's a bit of a stretch and requires a little bit of magic.

Tear down (i.e. a function that gets called after a test has run) would be specified in the same way.

Moving past that -- what if you have a bunch of tests that all require one function to be run first (this is more like a test-fixture setup in Nunit). Maybe you could do something like this:

    /// <summary>

    /// If person was born on birthdate, what's their age, in whole years at atDate?

    /// </summary>

    /// <param name="birthDate">The birth date.</param>

    /// <param name="atDate">At date.</param>

    /// <returns>Age in years (floored)</returns>

    /// <tests>

    ///  <setup>DateTime d = new DateTime(2006,09, 26)</setup>

    ///  <testCase birthDate="new DateTime(2006, 09, 25)" atDate="d" return="0" />

    ///  <testCase birthDate="new DateTime(2005, 09, 26)" atDate="d" return="1" />

    ///  <testCase birthDate="new DateTime(2005, 09, 26)" atDate="d" return="1" />

    ///  <testCase birthDate="new DateTime(2003, 09, 23)" atDate="d" return="3" />

    /// </tests>

    public int AgeAt(DateTime birthDate, DateTime atDate)

    {

        return atDate.AddYears(-1 * birthDate.Year).Year;

    }

More Complex Cases...

What if you wanted to write bigger test fixtures that wrap around more than one method?

In that case, i guess you've moved beyond the type of testing that this approach could suit. You could use nunit i guess, or mbUnit or some other integration suite style test.

Here's something else... say if you started with some simple tests as shown above, and then needed to do some more complex test, using NUnit. Well, there's no technical reason you couldn't have a right click option that refactors the tests above into their own Nunit test fixtures, say.

Here's a dodgy mockup...

Any comments, insults, pointers to further info... please respond.

'Simon' on Wed, 27 Sep 2006 06:49:24 GMT, sez:

Another good idea. I like the idea of having the test with the method - it improves readability no end.

Two slight concerns (and they are slight): firstly, when the tests become more complicated, there may be more code needed for the parameters. As this code is in the comment, it won't have intellisense or syntax colouring, and it won't be compiled as it is. I know that this is sorted out when you hit "Extract tests into NUnit", which may sort it out, but that leads to slight concern number two.

The "Extract ..." is basically code generation (which I know you're good at from the WSCG), and one of the main problems with code generation is the source and the generated code getting out of sync. In this case, the tests will be generated, and if they fail, or don't compile, the temptation is always there to fix the generated code in place, which leaves the comments in the source being different, and hence worth less than if they were not there at all. Clearly this just requires discipline on the part of the programmer, and if they write unit tests, then they're likely to be more disciplined than average anyway.

Anyway, this is becoming a ramble. They are only slight concerns, and I think it is a good idea worth thinking about some more.

'Marcos' on Wed, 27 Sep 2006 11:42:39 GMT, sez:

You rock Leon =)

Is an excellent idea, nothing can't be more readable.

Another idea can be to use a file with the input to the method like:

/// <testCase inputFile="TestAgeAt.txt" />

and in the file:

birthDate,atDate => result
2003-09-23,2006-10-23 => 3
2000-02-23,2006-10-23 => 6

and so on, of course that you need a standard DateTime format.

I know, I know, I pass too much with the FileHelpers :P and I´m trying to use it everywhere, hahaha.

This only work for basic types =( but anyway is easy to the QA team to add tests without recompile the app =)

In fact is easy to generate some random values to put inside the file and you only need to add the return values, but this has a reduced scope because the hard to test members need complex info or build complex object to pass in to the method.

Just my 2 cents
Keep the good work
Marcos

'Emperor XLII' on Wed, 27 Sep 2006 11:57:02 GMT, sez:

I could see it working as a build step that generates N/MBUnit code and places it in a separate project, with an area for adding your own "manual" test cases (along the lines of the winforms designer separating Form.cs and Form.Designer.cs).

'Einars <elfz>' on Wed, 27 Sep 2006 18:38:59 GMT, sez:

I very much like your idea - the approach that the tests should be physically close to the functions themselves.

Thought their area wouldn't be complex procedures with external dependencies, they could appear extremely convenient for everyday functions, as indeed tesing procedure sometimes is unusually burdened with need to write much enough code, usually even in different files.

So, all four thumbs up for your idea. Whether the tests are xml-style, or some other, preferrably easier type-able notation - that would be less important.

'Mike Gale' on Wed, 27 Sep 2006 18:48:13 GMT, sez:

Barring the generation I hope a lot of code comments already list some tests.

I'd be happy with an editor that linked test code and "actual" code. It could maybe list the tests in-line if I chose.

Great post.

'Esad Hajdarevic' on Wed, 27 Sep 2006 20:32:13 GMT, sez:

It would be even better if tests could be collapsed/hidden by the IDE, similar to #region in VS

'lb' on Wed, 27 Sep 2006 21:13:19 GMT, sez:

>main problems with code generation is the >source and the generated code getting out >of sync

so very true. how about this:
when you extract a test into it's own testfixture, it removes the test from the xml comments **BUT** replaces it with a link to the test fixture!


i was washing the dishes last night and thinking about this idea some more (as one does). i thought i could write to roland weigelt and see if he can work out some extensions to ghostdoc to enable the generation of test elements for a method. and then see if we can get some kind of Nunit integration happening. maybe get Mr Osherove to do some magic. Or James Avery. Anyway, it's one of those ideas that is outside my expertise, and which i don't have time to implement. but if it was implemented by someone else i know i'd use it. ANd this coming from someone who just can't seem to get started with test driven development.

>if tests could be collapsed/hidden by
>the IDE

well comments can already be collapsed by the ide, much like a region -- except that the entire comment region would be collapsed (not just the tests). But maybe the code collapse feature in VS could be smarter and give proper xml collapse features to XML code comments. This is a nice idea for a feature. Who do we hassle in the VS team for this??

'Hermann Klinke' on Thu, 28 Sep 2006 11:41:07 GMT, sez:

I think you can do something like the first think with MbUnit's RowTest attribute.

'Marcos' on Thu, 28 Sep 2006 13:31:55 GMT, sez:

Hermann:

The problem of using something like MbUnit Row is that you need to add a reference in your main app to the dll's of the testing framework.

You can use something like [Conditional] to make it only compile in debug mode, but anyway you need to reference =(

Cheers

'NinjaCross' on Fri, 29 Sep 2006 22:42:32 GMT, sez:

Amazing idea, I like so much this kind of straight-forward approach.
Unfortunately no complex test could be performed, but anyway it's a good start step :)

'Mars' on Thu, 12 Apr 2007 17:42:35 GMT, sez:

You might want to look at the Python Doctest package for some ideas. It does what you describe, but in more depth.

A Python doctest is a simple series of tests embedded in a Python function's documentation string. Each doc string tests the function it describes.

Doctests may also be embedded in a stand-alone text file, often a README for a module or package. The text file serves as a tutorial for using the module, with the tests doing double-duty as examples. Very slick!

'Frederic Torres' on Sat, 12 May 2007 14:47:31 GMT, sez:

I like it so much I almost want to try to write something for unit testing classes.

I just implemented parameterized test for our web functional testing tool InCisif.net and that is very close.

www.InCisif.net
Web Testing with C# or VB.NET

'Frederic Torres' on Sat, 23 Jun 2007 13:53:03 GMT, sez:

It is done and you can try it at www.quickunit.net.

/// <summary>
/// Is Year a leap year?
/// </summary>
/// <param name="Year">The year in question</param>
/// <returns>
/// true if year is a leap year. false otherwise
/// </returns>
[
QuickUnit.Net.TestMethod(true, new object[] { 2000 }),
QuickUnit.Net.TestMethod(false, new object[] { 1900 }),
QuickUnit.Net.TestMethod(true, new object[] { 1996 }),
QuickUnit.Net.TestMethod(false, new object[] { 2003 }),
]
public static bool IsALeapYear(int year) {

return (((year % 4) == 0) && ((year % 100) != 0)) || ((year % 400) == 0);
}

(Comments closed due to spam. Sorry.)