Importance of Code Coverage Tools
September 06, 2019
Blog
Test coverage is an important indicator of software quality and an essential part of software maintenance.
The world has witnessed some catastrophic events merely because of errors in software. One such incident happened in 2008 where glitches in test coverage practices of baggage handling system software caused the cancellation of 500+ flights.
Test coverage is an important indicator of software quality and an essential part of software maintenance. It helps in evaluating the effectiveness of testing by providing data on different coverage items. This insight allows the improvement of testing by defining new test cases for the untested code, thereafter, improving your code quality, which ultimately increases code coverage and reduces costs.
Why is Code Coverage required? - Purpose & Benefits
- Defect prevention at early stages of project life cycle
- Creates additional test cases to increase coverage
- Better ROI can be achieved by reduction in UAT defects and production defects
- It helps in finding areas of a program not exercised by a set of test cases
- Testing will become smoother by managing the risk-based testing approach
- Helps in determining a quantitative measure of code coverage, which indirectly measures the quality of the application or product
How to Choose a Code Coverage Tool
The real challenge is choosing which code coverage tools to use for application testing. An efficient tool can help measure the effectiveness of the testing efforts There are several options to create code coverage reports depending on the language(s) used for coding. Some of the popular tools with respect to coding languages are listed below:
Some tools like istanbul (a JavaScript test coverage tool) will place the results straight into your terminal while others can generate a full HTML report that lets you explore which part of the code you are lacking coverage.
Also, there are other coverage testing tools available for all the major languages and some of them are relevant not only for unit testing but also for the overall tests performed (unit, integration, system, manual and more). Below is the list of some trending code coverage tools conducive for running testing procedures.
1. Default Visual Studio Code Coverage
Visual Studio Enterprise has Code Coverage collection as an inbuilt feature. It can be applied to both managed (CLI) and unmanaged (native) code. Code coverage is provided as an option while trying to run test methods using Test Explorer. You can select all of your tests or subsets of them. It helps you visualize the coverage while you look at the code. The below table of results shows the percentage of the code that was run in each assembly, class and method. In addition, the source editor shows which code has been tested.
Report can be obtained in blocks or in lines. It is also easy to manage the code coverage results as well as merge results from different runs.
2. Cobertura
Cobertura is one of the most used and widely known code coverage tools. It is popular for being a free Java tool that calculates the percentage of code accessed by various tests performed. It can be used to identify which parts of your Java program are lacking test coverage. It is based on jcoverage.
Execution can be done via Ant, Command Line, Eclipse plugin or Maven plugin. It is easy to use and can measure coverage without having the source code. It provides reports in HTML or XML format and it has the capacity to test lines and branches of class and method.
3. Coverage.py
Coverage.py is a code coverage tool for Python. Main phases of using Coverage.py includes execution, analysis and reporting. It monitors the Python programs, notes which parts of the code have been executed, and then analyzes the source to identify code that could have been executed but was not.
Coverage.py is available with Tidelift to deliver one enterprise subscription. Other ways include execution via command line, APIs as well as Plug-ins for Django and Mako.
4. Gcov-Lcov
Gcov is a test coverage program used with GCC. It analyzes the programs to help create a more efficient, faster running code, to discover untested parts of the code.
It acts as a profiling tool, which helps in analyzing some basic performance statistics, such as:
- number of times each line of code executes
- what lines of code are actually executed
- how much computing time each section of code uses
Lcov is a graphical front-end for Gcov. It collects gcov data for multiple source files and creates HTML pages, which contains the source code annotated with coverage information. It also adds overview pages for easy navigation within the file structure. Lcov supports statement, function and branch coverage measurement.
5. Testwell CTC ++
Testwell CTC++ is a powerful instrumentation-based code coverage and dynamic analysis tool for C and C++ code. It can be used on C#, Java and Objective-C code with particular add-on components. It can also be used to analyze code basically at any embedded target machines.
CTC++ provides Line Coverage, Statement Coverage, Function Coverage, Decision Coverage, Multicondition Coverage, Modified Condition/Decision Coverage (MC/DC), Condition Coverage.
As a dynamic analysis tool, CTC++ shows the execution counters (how many times executed) in the code. It can also be used to measure function execution costs (normally time) and to enable function entry/exit tracing at test time.
This tool is user-friendly and fast. Report generation can be in straight text, HTML, XML, JSON (JavaScript Object Notation) and Excel input form. On some environments, e.g. Microsoft Visual Studio, CTC++ use is possible directly from the compiler IDE.
6. NCover
NCover is a code coverage tool for .NET programs and applications. NCover has been designed to integrate with the build process and has extensive command line support. The code coverage process is simple and can be automated. The NCover application suite is made up of three tools: NCover.Explorer.exe, NCover.Console.exe and NCover.Reporting.exe.
NCover.Explorer.exe is a GUI program that allows you to generate and view detailed code coverage information.
It supports statement coverage, branch coverage and cyclomatic complexity. It gives provision to select Low, Medium, High and Complete levels of code coverage, separately. It is very easy to use and fast, also available on open source as well as on commercial license.
Conclusion
For better product quality, depending on the specific needs of the project and the level of code coverage required, it is easy to select a code coverage tool. All the available tools for the code coverage suffice the required capabilities of code coverage based on independent needs of code language and structure.
To see how we have helped our esteemed clients ensure quality of their product, read our case studies.
Hetal Chauhan is associated with VOLANSYS Technologies as a Sr. Engineer with around seven years’ experience in quality assurance for various domains.
For more about development tools, check out:
Imagination Technologies Releases IMG Edge Services Platform
