THESE ARE JUST SOME NOTES FROM STEFAN RIGHT NOW. NOT A FULL DOCUMENTATION
This page should help to explain our ongoing effort to improve the quality assurance within the Enlightenment project. Most of the implementation details can be found at Continuous Integration Setup. On this page we describe the goals of this effort.
- We compile our code for different architectures (x86, x86_64 and x32).
- Cross builds for Windows with mingw64 are disabled right now due to some problems.
- For Linux we compile with gcc as well as with clang
- Our two main libraries (EFL and Elementary) contain a test suite with various unit tests.
- We are also tracking the code coverage we reach with our unit tests.
- New APIs should come with test cases
Besides some side effects when running the unit tests we do no real runtime testing yet.
Given the nature of a graphical toolkit unit tests for APIs can only go that far. We developed a tool, called exactness, to test various scenarios against a set of known to be fine screenshots.
We have a tool, called expedite, to measure performance on various aspects of our libraries. So far this is only used manually by some people. Work is in progress to automate this testing to keep track of potential performance regressions.
We use the Coverity Scan service to run static analysers over our code. Builds are submitted every night. Additionally we used the clang static analyser before but are not actively looking at these reports anymore.
In the stabilization phase before our release we are also running API/ABI checkers to verify newly added API and check for potential API/ABI regressions.
To keep the project members informed and motivated we send out a weekly QA newsletter summarising last weeks highlights and giving statistics.
- Enable Windows builds again
- Add MacOSX builds
- Add ARM builds
- Bring graphical testing with exactness into a working state
- Bring performance testing with expedite into a working state
- Integrate address and thread sanitizers into testing
- Measure energy consumption and look for regressions