Page MenuHomePhabricator

Quality Assurance
Updated 1,215 Days AgoPublic

THESE ARE JUST SOME NOTES FROM STEFAN RIGHT NOW. NOT A FULL DOCUMENTATION

Quality Assurance

This page should help to explain our ongoing effort to improve the quality assurance within the Enlightenment project. Most of the implementation details can be found at Continuous Integration Setup. On this page we describe the goals of this effort.

Compile testing

  • We compile our code for different architectures (x86, x86_64 and x32).
  • Cross builds for Windows with mingw64 are disabled right now due to some problems.
  • For Linux we compile with gcc as well as with clang

Unit testing

  • Our two main libraries (EFL and Elementary) contain a test suite with various unit tests.
  • We are also tracking the code coverage we reach with our unit tests.
  • New APIs should come with test cases

Runtime testing

Besides some side effects when running the unit tests we do no real runtime testing yet.

Graphical testing

Given the nature of a graphical toolkit unit tests for APIs can only go that far. We developed a tool, called exactness, to test various scenarios against a set of known to be fine screenshots.

Performance testing

We have a tool, called expedite, to measure performance on various aspects of our libraries. So far this is only used manually by some people. Work is in progress to automate this testing to keep track of potential performance regressions.

Static analysis

We use the Coverity Scan service to run static analysers over our code. Builds are submitted every night. Additionally we used the clang static analyser before but are not actively looking at these reports anymore.

API/ABI compliance checks

In the stabilization phase before our release we are also running API/ABI checkers to verify newly added API and check for potential API/ABI regressions.

Weekly QA newsletter

To keep the project members informed and motivated we send out a weekly QA newsletter summarising last weeks highlights and giving statistics.

Future plans

  • Enable Windows builds again
  • Add MacOSX builds
  • Add ARM builds
  • Bring graphical testing with exactness into a working state
  • Bring performance testing with expedite into a working state
  • Integrate address and thread sanitizers into testing
  • Measure energy consumption and look for regressions
Last Author
stefan_schmidt
Last Edited
Nov 25 2015, 3:01 AM
Projects
None
Subscribers
None