I’d like to share with everyone a system we’ve set up to help projects get faster TCK results, starting of course with Apache TomEE.
We’ve had this system for a while, but were unable to open access due to the legal restrictions around access to the TCK and results. Now that all of this is public we can finally make it available.
This installation is dedicated to open source projects and per each TCK run will spin up 100 m3.medium spot instances in AWS to chew through 95% of the TCK in about 2 hours. A handful of tests take much longer. Note it still looks and feels like an internal tool and currently only supports TomEE, but we hope to make improvements over the coming months and expand it to cover more Apache projects incorporated in TomEE.
Enabling The Community
We’re sharing this to hopefully enable the community to realistically help with TCK certification. Running individual tests on your own machine is very realistic and something anyone can do. However, running the whole TCK on your machine is not realistic — unless you don’t need your machine for a week as that’s how long it would take to complete one box of almost any size.
The goal is the community can use this to see what work needs to be done (which tests are failing) and then dig in by running that test on your own machine using the official Apache TomEE TCK setup (https://github.com/apache/tomee-tck).
With that in mind here are some of the more critical features to help you move the needle on our TCK progress.
Main Page
The main page https://tck.work/tomee/projects shows currently 3 projects and it’s specific status, JVM, version and container.
- webprofile-jakartaee-8: TCK runs targeting Jakarta EE 8 compliance on TomEE 8.x
- custom-jakartaee-8: One-off or partial runs of the Jakarta EE 8 TCK
- webprofile-jakartaee-9: TCK runs targeting Jakarta EE 9 compliance on TomEE 9.x
More flavors of projects/builds will likely be added as the project decides to target other JVMs, profiles, etc.
Analyze Issues
At the bottom of most TCK results page there will be a link ‘analyze issues’ that will bring you to a page like this:
- https://tck.work/tomee/issues?build=1606688456935
- https://tck.work/tomee/issues?build=1606688456935&status=FAILED
- https://tck.work/tomee/issues?path=com.sun.ts.tests.jpa&issue=&build=1606688456935&status=FAILED
This page is created by looking at the stack traces printed by each failing test and counting and ranking them. For example, one of the exceptions we’re seeing is a NoClassDefFoundError affecting these 54 tests:
Filtering by pass/fail status
At the top of most TCK results pages there are “PASSED” and “FAILED” links that will allow you to filter tests by status. This is important for the above feature as many passing tests do throw exceptions. You’ll want to make sure to click “FAILED” after or before you navigate to “analyze issues.”
Filtering by package
In the upper right of most test results pages you’ll see “Subsections” showing a list of java packages under the current test package. Some test sections have many subsections such as EJB lite:
View output
The “Tests” table on every test results page has a “Package”, “Test” and “Result” columns.
- Click “Test” to see TCK output for that exact test. This is the log output of the TCK itself and should be your first thing to examine.
- Click “Package” to see any *.log files created and uploaded during the testing of that section. This is where you’ll find maven output, server log files, etc.
Running and Fixing a test
This system does not help you run an individual test. That is done on your machine via the project’s official TCK harness:
This system also uses the above harness and simply does a checkout and build of that just like any of us would. The best way to make progress fixing a test is on your machine.
You Can Make A Difference
You might be thinking, “can I realistically help with something as complicated as passing a TCK? Isn’t that just for full-time developers?” Here’s a fact you may not know:
Apache TomEE was built and reached Java EE 6 Web Profile certification without even a single full-time developer.
When Apache TomEE 1.0 milestone 1 was announced in 2011 there was no Tomitribe and all of us worked in our spare time, including me. In the very first Tomitribe blog ever, “Who Are The Real Heroes Of Open Source?” we tell a story about many of us in Apache taking a vacation to travel to Tours, France, and hack on Apache TomEE and the TCK together for a week. It was the first time most of us had never met in person and to this day it is still one of the project’s most cherished memories.
Now that the TCKs are open source, this adventure is open to everyone.
With great confidence, I can say my entire career was made possible by contributing to open source. Famous open source hackers are open source hackers first. That opportunity is open to everyone and it is entirely what you make of it.
We hope this enables a new generation of open source hackers to dig in, forge their way and become tomorrow’s leading experts in all things Jakarta EE. More than that, we hope this enables true collaboration and the kind of personal and community bond only possible from making a brave jump and achieving the impossible together.
With that said, let’s do the impossible and we’ll see you on the Apache TomEE dev list!
