In this tutorial I’ll show how to use Jacoco with Maven/Tycho to create a code coverage report of multiple Eclipse plug-in projects.
The code of the example is available here: https://github.com/LorenzoBettini/tycho-multiproject-jacoco-report-example.
This is the structure of the projects:
Each project’s code is tested in a specific .tests project. The code consists of simple Java classes doing nothing interesting, and tests just call that code.
The Maven parent pom file is written such that Jacoco is enabled only when the profile “jacoco” is explicitly activated:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
<profile> <id>jacoco</id> <activation> <activeByDefault>false</activeByDefault> </activation> <build> <plugins> <plugin> <groupId>org.jacoco</groupId> <artifactId>jacoco-maven-plugin</artifactId> <version>${jacoco-version}</version> <configuration> <excludes> <exclude>**/plugin1/Main.class</exclude> </excludes> </configuration> <executions> <execution> <goals> <goal>prepare-agent</goal> </goals> </execution> </executions> </plugin> </plugins> </build> </profile> |
This is just an example of configuration; you might want to tweak it as you see fit for your own projects (in this example I also created a Main.java with a main method that I exclude from the coverage). By default, the jacoco-maven-plugin will “prepare” the Jacoco agent in the property tycho.testArgLine (since our test projects are Maven projects with packaging eclipse-plugin-test); since tycho.testArgLine is automatically used by the tycho-surefire-plugin and since we have no special test configuration, the pom.xml of our test projects is just as simple as this:
|
1 2 3 4 5 6 7 8 9 10 11 12 |
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>example</groupId> <artifactId>example.parent</artifactId> <version>0.0.1-SNAPSHOT</version> </parent> <artifactId>example.plugin1.tests</artifactId> <packaging>eclipse-test-plugin</packaging> </project> |
if you need a custom configuration, then you have to explicitly mention ${tycho.testArgLine} in the <argLine>.
Now we want to create an aggregate Jacoco report for the classes in plugin1 and plugin2 projects (tested by plugin1.tests and plugin2.tests, respectively); each test project will generate a jacoco.exec file with coverage information. Before Jacoco 0.7.7, creating an aggregate report wasn’t that easy and required to store all coverage data in a single an .exec file and then use an ant task (with a manual configuration specifying all the source file and class file paths). In 0.7.7, the jacoco:report-aggregate has been added, which makes creating a report really easy!
Here’s an excerpt of the documentation:
Creates a structured code coverage report (HTML, XML, and CSV) from multiple projects within reactor. The report is created from all modules this project depends on. From those projects class and source files as well as JaCoCo execution data files will be collected. […] This also allows to create coverage reports when tests are in separate projects than the code under test. […]
Using the dependency scope allows to distinguish projects which contribute execution data but should not become part of the report:
compile: Project source and execution data is included in the report.test: Only execution data is considered for the report.
So it’s just a matter of creating a separate project (I called that example.tests.report) where we:
- configure the report-aggregate goal (in this example I bind that to the verify phase)
- add as dependencies with scope compile the projects containing the actual code and with scope test the projects containing the tests and .exec data
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 |
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>example</groupId> <artifactId>example.parent</artifactId> <version>0.0.1-SNAPSHOT</version> </parent> <artifactId>example.tests.report</artifactId> <packaging>pom</packaging> <profiles> <profile> <id>jacoco</id> <activation> <activeByDefault>false</activeByDefault> </activation> <build> <plugins> <plugin> <groupId>org.jacoco</groupId> <artifactId>jacoco-maven-plugin</artifactId> <version>${jacoco-version}</version> <executions> <execution> <phase>verify</phase> <goals> <goal>report-aggregate</goal> </goals> </execution> </executions> </plugin> </plugins> </build> </profile> </profiles> <dependencies> <dependency> <groupId>example</groupId> <artifactId>example.plugin1</artifactId> <version>${project.version}</version> <scope>compile</scope> </dependency> <dependency> <groupId>example</groupId> <artifactId>example.plugin2</artifactId> <version>${project.version}</version> <scope>compile</scope> </dependency> <dependency> <groupId>example</groupId> <artifactId>example.plugin1.tests</artifactId> <version>${project.version}</version> <scope>test</scope> </dependency> <dependency> <groupId>example</groupId> <artifactId>example.plugin2.tests</artifactId> <version>${project.version}</version> <scope>test</scope> </dependency> </dependencies> </project> |
That’s all!
Now run this command in the example.parent project:
|
1 |
mvn clean verify -Pjacoco |
and when the build terminates, you’ll find the HTML code coverage report for all your projects in the directory (again, you can configure jacoco with a different output path, that’s just the default):
/example.tests.report/target/site/jacoco-aggregate
Since, besides the HTML report, jacoco will create an XML report, you can use any tool that keeps track of code coverage, like the online free solution Coveralls (https://coveralls.io/). Coveralls is automatically accessible from Travis (I assume that you know how to connect your github projects to Travis and Coveralls). So we just need to configure the coveralls-maven-plugin with the path of the Jacoco xml report (I’m doing this in the parent pom, in the pluginManagement section in the jacoco profile):
|
1 2 3 4 5 6 7 8 9 10 |
<plugin> <groupId>org.eluder.coveralls</groupId> <artifactId>coveralls-maven-plugin</artifactId> <version>4.3.0</version> <configuration> <jacocoReports> <jacocoReport>${project.basedir}/example.tests.report/target/site/jacoco-aggregate/jacoco.xml</jacocoReport> </jacocoReports> </configuration> </plugin> |
And here’s the Travis file:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
sudo: false language: java jdk: oraclejdk8 cache: directories: - $HOME/.m2 install: true script: - mvn -f example.parent/pom.xml clean verify -Pjacoco coveralls:report |
This is the coveralls page for the example project https://coveralls.io/github/LorenzoBettini/tycho-multiproject-jacoco-report-example. And an example of coverage information:
That’s all!
Happy coverage! 🙂
Thank you so much for putting this together, Lorenzo!
Detailed and extremely helpful – Thank you!
What if you have multiple projects as described but only one project with tests. Do you still need a separate project for reporting?
Yes, you still need an aggregated report goal since you must specify both the projects with test and, most of all, the projects with sources.
Hm, I’m able to generate a report now following your instructions but the coverage is all 0.
that’s strange… I can’t tell what’s wrong without further information, though…
Sure. Please find my repo at https://git.io/vwpaK. I’ve added the aggregate report in https://git.io/vdFim.
If you run it locally does the generated HTML report shows 0 coverage? Or is it only on coveralls? I can’t test it myself right now…
Yes, if I run it locally, the HTML report shows 0 coverage.
I experience the issue locally.
I’ll try to have a look at this ASAP
Thanks, Lorenzo, but I was able to fix it with the help of th JaCoCo team. See https://git.io/vbqPQ.
Right! Indeed jacoco automatically sets tycho.testArgLine if that’s an eclipse test plugin project and you don’t need to set anything else, UNLESS you need to configure the argline yourself with other arguments 🙂
Took really long time to land on your article, which gave me the prefect solution for my merge requirement. Tried a whole lot of approaches before and nothing really worked. Thanks for this article Lorenzo !!!
Lorenzo, could you please give me advise how can I get code coverage in this example for one exact test-plugin (e.g. example.plugin2.tests)? In my case to get code coverage for all tests is very time consuming. I need possibility to run one exact plugin and get coverage for all my dependent plugins.
Thank you Lorenzo for this helpful post.
Could you please advice how can I get code coverage for one exact test plugin (e.g. example.plugin2.tests) including all its dependencies?
In my case run all test is very time consuming and not necessary.
In the ideal case I’d like to create separate test launcher with code coverage for each test plugin.
You probably have to play a bit with Maven profiles or use ‘mvn install’ and then run maven on that very single project (see Maven documentation for that); in any case, since we’re talking about Eclipse plugins, you may want to execute code coverage for only a limited set of tests directly from Eclipse, using EclEmma, independently from Maven.