Negative Testing

Today I was asked to clarify the scope of unit testing. The question was around the idea of negative tests. A fundamental axiom of unit testing is that you perform both positive and negative tests against your method or function. This brought about a good conversation at work. I wanted to capture it for later in life, when I am too thick to think on my feet.

CONCEPT ONE ATOMICITY:
The scope of unit tests is constrained by the scope of the method or function under test. I am going to use just method here after, as it is annoying to refer to both methods and functions. That means certain test scenarios are inappropriate for unit testing. An example test I would not worry about in a unit test would be a data type mismatch test. The method will have a signature that should prevent a float value (a decimal value) from being passed into a member declared as an int (accepts only whole numbers). Only test that which is in the scope of the unit under test. Integration testing should cover issues where a data type mismatch would occur. Another note on atomicity, the smaller the test scope for unit test (focused just on the unit under test) the more granular your test reporting and the faster you can successfully debug and find the root cause of an issue.

CONCEPT TWO NON-DEPENDENCY:
Mock the data used for testing rather than leveraging an external data source, when you can. I have talked around this issue muchas, and I have to say that the more layers of test dependency you inject, the more time you fiddle with your test infrastructure. I am sure there are other reasons (like a greater risk of test inaccuracy) but the time spent managing test resources is my big issue. It has been argued that a 'when you cannot" scenario would be testing a stored procedure with a complex signature requiring configuration data and transactional inputs needs a database to store the inputs and expected values. I would suggest that, if you need this include it as a resource in your database build project and do not reuse the schema from your product. Tying the two schema together also increases time spent managing test infrastructure.

CONCEPT THREE NEGATIVE TEST:
The division method is a classic unit test example. The positive test is to pass in 4/2 and assert that the result is 2. The negative test is to pass in 4/0 and assert that the method under test elegantly raises a divide by zero error. One of the great values of doing unit testing is to engage the developer in the what-if analysis of their code. You can prevent bugs by simply adding comments for each negative test cases prior to writing your method. Given the business requirement "given a comma separated list of last name, first name, and zero or more certifications, store it in the user table. Only store records with first and last names." The positive tests would be (a)parse a single complete record with MD as the credential (b)parse a single record with no credential (c)parse a single complete record with MD. PHD. as the credentials. The negative tests would be (a) pass in a record with a last name and a credential (b)pass in a record with a first name and a credential (c) pass in a record with a last name (d)pass in a record with a first name (e) pass in a record with just a credential. As you see we have more negative scenarios than positive scenarios.