March 4, 2008

Smart Driven Development

At the beginning it was very easy: you just write your test first, ahead of the application code and you’re done. Of course it turned out not to be so easy. First of all, what is a unit? It’s a class, it’s a package or what? The easy answer is: a unit is a class, so when you start test first on something, you name the unit test class as the class you intend to test. But then there are other questions you need to ansewer before coding:

  • Do you want a top-down or a bottom-up approach?
  • If I start from the top and test are failing, can I commit to the continous integration?
  • Do I need to mock objects so that I can  test also interactions? How far the interaction should be described in the test?
  • How far I have to go wih testing, should I try for 100% coverage?

The answer once again is flexibility. There is not a common answer for all situations, there is just a smart approach based on testing principles. I want to start from this interview with “Uncle” Bob Martin. Here are his 3 definitions of test driven development:

  • A test driven developer does not write a line of production code until he has written a failing unit test.
  • A test driven developer doesn’t write more of a unit test that is sufficient to fail and waits until red bars go green before writing another one.
  • A test driven developers doesn’t write more production code that is sufficient to pass a failing unit test.

But these principles are just a starting point. Only experience can tell what else you need in your daily development.

Bottom-Up or Top-Down?

I prefer the Top-Down approach whenever possible. But I’m gradually lerning that the best place to start is where you have the best understanding of the problem.  So I’ll try to start from the top, but if it makes more sense to start from the bottom because I know what to do without too much thinking, I will create higher level testing later. In this way you can be productive while you increase your understanding of the business logic. Other approach like trying to understand the problem without coding are not as effective. There is also the risk sometimes that in order to start from the acceptance test you need to solve technical problems like how to mock external services or how the result should be produced by the system. Starting from the middle/bottom layer can bring the solution to the acceptance technical problem as a side effect.

Failing Tests

It’s really great when acceptance criteria has been specified for you. Even better if those are already executable and not plain english. But also in the case the customer can only produce plain english requirements it’s not bad at all. The reason is that acceptances are those “forcing constraints” driving you in the process of writing the first tests. Acceptances run as part of the continous build, exactly like unit tests. The problem is that an acceptance will fail until you have written all the necessary unit tests and production code to make it pass. You also want to integrate in the repository with the work of the team as fast as you can. So you don’t really want a situation where you wait until the end of a user story to commit your work. Well, the reason of the acceptances is to drive the design discovery process and the writing of the unit test for the lower layers. Of course it’s also a regression test for the business requirements you’re writing. But in this situation I don’t see a problem commenting out the acceptance test from the continous integration. The same for functional/integration testing. Sure you need to remember to activate them as soon as possible. Acceptance framework or unit test framework supporting testing “categories” are a great help in this. You just need to label the test as “functional” and exclude that category from the continous integration scripts. You will also see in the report from the build something like “X test skipped” that will remind you about the exclusions.

Mocking

For the same reason an acceptance will fail at the beginning, also testing lower layers can produce the same problem. There is a big difference though. While acceptances (written by the customer) or functional/integration tests are supposed to test the real application exactly like it will work in the production environment, you don’t have the same requirement for unit tests. In other words, you can mock interactions with other component at the interface level, leaving the implementation details for the next layer of unit testing. Yes, it’s a cycle through the upper layers of the application to the bottom. In every cycle you write a suite of tests for the current layer, mocking what has not been implemented yet. So you need to mock objects or stub results, especially your approach is top-down. The risk can be to over specify. I usually specify just types for arguments to mock method calls and sometimes I just ignore arguments. I also skip verification of mock calls when the mock is not strictly related with the subject of the unit test. You should try not to create fragile unit tests. An unit test is fragile when even small changes in the production code force you to change a lot of unit tests. This situation should happen only if you touch core business values of the application where it’s worth spending more time. But if your mocks are over specified then you aren’t only testing the core business proposition of the business logic, but also all the other structural details.

Coverage

100% coverage is not possible or really difficult to achieve. The choice you make should be once again strategic: aim at higher coverage numbers for core business functionalities and leave the rest at an acceptable level. The coverage numbers are also very related to the tool you use. Only experience can say what is the coverage number and metrics that you should try to have in your project. In my experience there is no an universal metric or number you should see. There is a range you can expect (for example I consider  good a coverage between 60-80%) and then wait iteration after iteration to see how hard is to improve. You reach your optimal coverage when after putting a consistent effort on it, the metrics hardly increase.

Conclusions

I think flexibility and the capacity to contestualize experience to the surrounding environment are key factor in the success of a project. There is no silver bullet or universal rule that can be applied. There are general principles you can apply to different situation to maximize your productivity and be more effective in the less time possible.