I recently wrote a post about the level of code coverage that I achieved on a project that I was working on. Then I started seeing articles about how 100% code coverage is a fallacy, or how code coverage tools are "evil". I disagree.
Ok, before everyone kills me, let me explain. I do agree to the following:
- Code coverage numbers can be abused (just like a defect/bug count)
- Code coverage is NOT and indicator of how well tested your code is
- Code coverage number should never be used by themselves to draw any sort of conclusion about the code
Argument 1: It's Waste of Time
The first argument that I saw was that testing simple pieces of code such as simple getter/setter properties is a waste of time that can better used writing a unit test for a more complex piece of code. So more bang for your buck.
You don't have to waste your time writing tests for simple getter/setter properties. A good testing framework can take care of this for you with little or no work on your part. NewtonSoft Utilities.NET contains a class that will dynamically test all of your properties. Using this utility, you can write a single unit test that tests all of your properties on a class. And this unit test can copied and pasted with one or two minor modifications to test the next class. Elapsed time: 30 seconds to create class, write function and run test.
In addition, no unit test should ever be considered a waste of time. A single unit test is meant to test a single scenario and that unit test will also ensure that chunk of code will always behave as expected in that scenario. What starts as a simple getter/setter property may evolve to a more complex one and the unit tests written for that property will ensure that the evolution of that property does not sacrifice functionality without you know about it.
Argument 2: Trust Your Programming Language/Framework
Another argument against testing things such as simple getter/setter properties is that why would you not trust your programming language/framework/environment to perform a simple get or set operation?
Why would you trust your programming language/framework/environment? Programming languages are software and thus can contain bugs too. Now the likelihood that you will ever encounter such a bug is low, but why take the risk? As long as you have a good testing framework as I stated above and unit testing your properties takes little or no time, then it is far better to catch the bug in unit tests than in production code.
More importantly, unit testing the simple pieces of code is not about not trusting your programming language but rather not trusting yourself. I, and I'm sure most of you, have at one time or another fat-fingered something in code. There are numerous ways to screw up on a simple property. How about having the following? It is very easy to do the following, especially with code-completion. This is not really a bug you want making it into production code, especially if a single unit test would have exposed this. And if the problem doesn't immediately stand out at you, then you've proved my point. And now imagine this property inside of a class with other code and documentation. Suddenly this bug because camouflaged and less easy to see with the naked eye.
public int Width
{
get { return Width; }
set { _width = value;}
}Argument 3: Code Coverage Is Not An Indicator Of Code Quality
This argument is half true. The true reason that code coverage can not be used as an indicator of how well the code is written is because code coverage only indicates that the individual lines were executed. It does not indicate what combination of individual lines were executed. In other words, it says nothing about code execution paths.
However, code coverage can help indicate how modularized the code is. What?! I said it helps indicate that. If you have complex code, the quicker you are able to achieve 100% code coverage (or a high level of code coverage), the more modularized your code possibly is. Very modular code means that the individual pieces of your code are smaller and less dependent on other pieces of code. This in turn means that your unit tests will be smaller, more focused and require less setup. Given all of that, the unit tests should be quicker to write.
In my previous post about code coverage, I stated that I employed IoC/DI and AOP and was able to achieve a high level of code coverage very quickly and every class was either 100% covered or 0% covered and the 0% covered classes were very small and contained "untestable" code. The application had its complexities, yet I was able to achieve this code coverage scenario relatively quickly. Analyzing the code showed that it was very modular. TDD yields the same results. Modular code with high code coverage.
Having modular code means that you should be able to write more unit tests quicker and more effective unit tests because they are more focused on smaller pieces of code. This could lead to more quality code. It is important to note that highly tested modular code does not guarantee quality. But the chances of it are higher.
Conclusion
While it does not give you the world, aiming for 100% code coverage does help you write better software depending on the steps you take to achieve this high code coverage. Code coverage in and of itself is not evil and the code coverage percentage does provide some useful insight when used in conjunction with other pieces of data about your code.
Just as with everything else, the quality of your code depends on the methodologies, techniques and technology that you use coupled with your development skill level. Aiming for 100% code coverage is just another technique.
Aim high. Never let your grasp outreach your goals.
6 comments:
I posted a followup here: Unit tests are for functionality, not code
For the most part I will agree with everything you said.
But I will add one more reason not to have 100% code coverage: some sections of code are more trouble than they are worth to write tests for. More spesifically, WebForms and WinForms UI code. You have to have the code there (you are manipulating the UI, responding to click events, etc), but testing it can be a royal pain in the backside.
You can test this code with selinium, mercury, WinFormUnit, etc. But if your UI is still in flux then the test will break -- and how. Suddenly, all that time you spent writing those tests to gain code coverage is now seen as wasted and it all has to be redone.
Now, if the UI is stabilized, then go for it. But it still isn't easy.
Final caveat to these statements: all code that isn't testable should be isolated from the rest of your code to minimize the amount of untestable code. That should be a given.
I'm with chris. Testing the view is a pain, and I believe not worth the effort.
Maybe if reliability/safety/performance were qualities you needed in the view it's worth it, but for functionality i leave view's off the unit test sheet.
Our views change too quickly because customers don't know what they want things to look like.
I may change my mind in the future though.
If you're having to write tests explicitly for your property setter/getters, then you're doing something wrong.
You're properties are there to influence *behavior*, so as you're testing the desired behaivor, you'd necessarily be executing the property.
I totally agree with Chris Brandsma.
I'm quite happy with about 90%.
I agree with Mark. Something stinks if you're writing code for the sole purpose of getting code coverage. I'd steer clear of a utility that writes tests for your getters/setters as this will only mask areas of the code that aren't actually used in the normal course of functionality.
I wrote a post a while back on how an organization can get value from code-coverage, but the focus isn't on establishing 100% coverage. The focus should be on change deltas in coverage.
On testing the UI, while a language-pattern can help mitigate the nature of volatile changes in the UI, fine grained tests can be difficult to establish and maintain. Besides, we gotta give our QA guys something to do.
Post a Comment