Friday, October 9, 2015

Making Sense of Sonar Qube Stats like Unit Test Coverage, Line Coverage and Condition Coverage


In Sonar Cube, the equations for "Coverage", "Line Coverage" and "Condition Coverage" are given as:

"Coverage"
coverage = ( (conditions_to_cover - uncovered_conditions) + (lines_to_cover - uncovered_lines) ) / ( conditions_to_cover + lines_to_cover )

"Line Coverage"
line_coverage = (lines_to_cover - uncovered_lines) / (lines_to_cover)

"Condition Coverage"
branch_coverage = (conditions_to_cover - uncovered_conditions) / (conditions_to_cover)

These equations are written using the metric values, so that you can query sonar to get the values for a given project. References include docs.sonarqube.org and www.slideshare.net.

AKA
Several of these terms have synonyms, so keep in mind:
  • 2B = conditions_to_cover
  • CT + CF = conditions_to_cover - uncovered_conditions
  • Condition Coverage = Branch Coverage
  • Condition Coverage = (CT + CF) / (2*B)
  • Coverage = (CT + CF + LC) / (2*B + EL)
  • Line Coverage = LC /EL
The 7 parameters above all have to do with Unit Test Coverage.  The 7 parameters for Integration Test Coverage and Overall Test Coverage are listed here.

"Unit Test Coverage"
  • conditions_to_cover
  • uncovered_conditions
  • lines_to_cover
  • uncovered_lines
  • coverage
  • line_coverage
  • branch_coverage
"Integration Test Coverage"
  • it_conditions_to_cover
  • it_uncovered_conditions
  • it_lines_to_cover
  • it_uncovered_lines
  • it_coverage
  • it_line_coverage
  • it_branch_coverage
"Overall Test Coverage"
  • overall_conditions_to_cover
  • overall_uncovered_conditions
  • overall_lines_to_cover
  • overall_uncovered_lines
  • overall_coverage
  • overall_line_coverage
  • overall_branch_coverage
Use Drilldown HTML Page
You can use the drilldown to find values for these specific metrics.  Just substitute your sonar domain (replace my.sonar.com) and project key (replace My-Project-Name) and the metric you are interested in (replace line_coverage).

http://my.sonar.com/drilldown/measures/My-Project-Name?metric=line_coverage 

Query using Sonar REST API
The format for the query is:

http://{sonarDomain}/api/resources?resource={keyOrID}&metrics={metrics}

http://my.sonar.com/api/resources?resource=55555&metrics=ncloc,conditions_to_cover,uncovered_conditions,lines_to_cover,uncovered_lines,coverage,line_coverage,branch_coverage,it_conditions_to_cover,it_uncovered_conditions,it_lines_to_cover,it_uncovered_lines,it_coverage,it_line_coverage,it_branch_coverage,overall_conditions_to_cover,overall_uncovered_conditions,overall_lines_to_cover,overall_uncovered_lines,overall_coverage,overall_line_coverage,overall_branch_coverage

Combining Projects
One nice thing about using these variables, rather than using the percentages is that you can combine any number of projects and re-calculate coverage.  For instance, if you have 3 projects and you would like to calculate "Unit Test Coverage", you could use an equation like the following to sum the projects values to get a total.

  • conditions_to_cover_total = conditions_to_cover_1 + conditions_to_cover_2 + conditions_to_cover_3
  • uncovered_conditions_total uncovered_conditions_1 uncovered_conditions_2 uncovered_conditions_3
  • lines_to_cover_total lines_to_cover_1 lines_to_cover_2 lines_to_cover_3
  • uncovered_lines_total uncovered_lines_1 uncovered_lines_2 uncovered_lines_3
Then, you can use the formula to calculate an overall coverage:
  • coverage_total = ( (conditions_to_cover_total - uncovered_conditions_total) + (lines_to_cover_total - uncovered_lines_total) ) / ( conditions_to_cover_total + lines_to_cover_total )