- D5968 / rEFLbbbd05148ad6: tests: disable ecore system modules in elm_suite
D5967 / rEFL54ca34c15d2b: tests: disable efreetd for elm tests
D5965 / rEFL0cf63649c70a: tests: use a global win object in fork mode when using buffer engine
D5959 / rEFLab430e3ee555: tests: preload elm csd theme in elm_suite startup
D5958 / rEFL9bfa85fdf010: tests: skip second ecore_shutdown call in elm_suite when forking
D5915 / rEFL67705589ac75: tests: preload elm_init and default theme groups in elm_suite
D5941 / rEFL9dcf6f71d066: ecore: further reduce pipe wait time during shutdown.
@stefan_schmidt I've discovered that the biggest bottleneck in test runtime, by FAR, is building with coverage enabled. I think we should be running coverage maybe once or twice a week at most, and all other test runs should be without coverage. The speed difference is actually unbelievable.
@zmike Not sure I follow you here. You mean make check vs make lcov-check runtime or do you mean the runtime of make check when configured with coverage vs regular?
The increased test time from make check vs make lcov-check is indeed huge (61s vs 152s) but to be expected.
I see only a marginal difference in running make check configured with coverage vs. regular (61s vs 59s)
For the normal test flow I would expect developers and CI tools to run only make check (and we can have it configured as regular as well). The coverage run could easily be done during a nightly or weekly build.
Actually I do mean running make check --with-tests=coverage vs --with-tests=regular. The difference here is absolutely staggering. As an example, elm_suite takes ~6-7s to run with coverage enabled, but only 1s to run without it; fork() calls with coverage enabled take 0.03s, vs 0.0s without coverage. While this may seem like a trivial number, consider that when running parallelized, each test case has a fork() call, and then each test in the test case also has a fork--meaning that the total number of fork calls in a test suite is $num_test_cases * (1 + $num_tests). In elm_suite, this results in ~5s spent doing nothing but executing the fork calls. As coverage increases and the number of tests increases, this number will also continue to increase.