At my old job, I and another developer embarked on writing a .NET assembly that offered low level functions into the system that our company developed. One of our goals was to unit test the heck out of the assembly since it was so low level. That assembly had to be as bug free as possible and when a change was made, we needed a way to try to ensure, without a huge effort on QA's side, that nothing would break.
We therefore needed to make sure that we had as much code coverage as possible. Code coverage is simply the lines of code actually executed during the unit tests. Now 100% code coverage, while it means that every single line of code has been executed, does not by any means mean your code is bug-free. Code coverage does not take into account code you didn't write. For example if I have a function Add(int a, int b) that returns an integer and my method just does 'return a+b;', I've got a huge bug if a+b > Int.MaxValue. I could have had 100% code coverage and still missed that bug.
So while having 100% code coverage does not mean a great deal towards the quality of your unit testing, having 10% code coverage is a horrible level of code coverage. In the past, we had unit tested some projects and eventually fallen behind on testing certain classes, or didn't employ test-driven development (TDD), etc. and we found we had about 20-30% coverage. If you do not employ TDD, 20-30% is probably the best you're going to do (unless you are an amazing developer and can write completely testable code without even thinking about it). With this project though, we did employ TDD and we achieved about 80% code coverage. Most of that remaining 20% was untestable code because it interacted with the database through SQL command objects or with system APIs.
Let me clarify my definition of "untestable". When I write unit tests, anything that interacts with another system or the environment is considered untestable. Unit tests should really be environment independent. Anything that requires a specific setup on the environment or a specific system could mean that the next developer is not able to run the unit tests on their system. For example, if code interacts with SQL Server, what happens when the next developer does not have access to that server or the server is down? Then the tests fail. If possible, we will substitute one dependency for an equivalent, move versatile dependency. When testing our nHibernate mapping files, we use SQLite because nothing needs to be installed and this unit test can run on any environment.
With a project I am working on right now, web services that our customers use to interact with our system, we need a heavy amount of unit testing as well. We do not want a change on our side to force our customers to change on their side as well. This requires them to spend money and until they change their code, there is a potential for their application to either break or cause invalid in our system (depending on the change and how it is implemented).
So I've been working hard on writing the library assembly for these web services using TDD and employing any techniques I can to streamline development, increase testability, leave the code easy to read and develop, etc. I am using Spring.NET to help me do this. I am using their Dependency Injection (DI) functionality of their Inversion of Control (IoC) container as well as their Aspect-Oriented Programming (AOP) components. Lastly, I used a Design-By-Contract to assist in validating parameter arguments.
Using the DI, my classes already are stripped of their hard dependencies making them easier to test. Using the AOP, I've simplified the code in my services classes by removing things such as authentication, logging and exception handling. Each one of these aspects can be tested independently of the service class. By not having them mixed in with the actual service execution code, the unit tests for the service code have less paths to test so less unit tests that need to be written. In addition to this, as I was developing the library assembly, I stayed mindful of what code would be untestable and coded around that pulling those pieces of code out into their own classes.
The result? I have about 75% code coverage already (the remaining 25% is a combination of untestable code, code that will eventually be removed as it is no longer needed and method stubs that haven't been developed yet and for now just return generic data for UI testing). The interesting effect of my development process I noticed is that every single class had either 0% code coverage or 100% code coverage.. So the 0% tested classes were classes that were not developed yet or were untestable code. A quick glance at those classes and I noticed that all of the untestable classes were very minimalistic (about 3 - 10 lines of actual code per class). These classes can easily be checked by two developers to determine if there are any bug potentials in them and most likely will be fully tested in a simple, manual QA test.
I have to admit, I thought that was really cool. I used to think just having high code coverage was exciting. But then seeing that every single class has either 0% or 100% code coverage, that definitely tops the cake (at least until I can achieve 100% code coverage on an entire assembly).
1 comments:
Good job, 100% test coverage rocks!
Post a Comment