Coroutines
Needs RevisionPublic
Actions

Authored by barbieri on Aug 27 2017, 8:40 AM.

Details

Reviewers

cedric
raster
q66
iscaro
felipealmeida
zmike

Summary

Coroutines in EFL

Since users do not get callbacks, let's allow them to write
"synchronous" code that cooperates with the main loop.

This allows eina_coro_yield() to cooperatively give back control to
the main loop and ask to be rescheduled later, a common pattern we use
with a bunch of functions -- one to schedule, one for callback, one to
save context, other to free context.

And it is possible to await on a future of a promise to be resolved
with eina_coro_await() (or eina_future_await() helper). These will
allow for even simpler code and efficient scheduling -- the coroutine
is put to sleep until the future resolves.

Since there are no platform independent ways to save and restore a
function context (makecontext()/swapcontext() doesn't work on all
platforms) and I'm not willing to waste time with manual assembly for
each platform the initial implementation uses
Eina_Thred/Eina_Lock/Eina_Condition to do it. However the API allows
to change to single-thread coroutines -- code is also simple to
"ifdef" shall we come to an agreement: _eina_coro_signal() and
_eina_coro_wait().

This is not about efficiency, more on ease of use and targeted at end
users, such as application developers. See _await() function in
ecore_test_promise2.c, it does the traditional "sleep()" without
blocking the main thread and is as simple to use -- if this becomes a
common pattern we can introduce efl_loop_coro_sleep(loop, coro,
seconds) to make it a single-line.

NOTE: this is on top of devs/barbieri/future, which contains fixes for devs/iscaro/future as mentioned in D5131.

Diff Detail

Repository

rEFL core/efl

Branch

devs/barbieri/coroutines

Lint

No Linters Available

Unit

No Unit Test Coverage

Build Status

Buildable 4368
Build 4433: arc lint + arc unit

barbieri created this revision.Aug 27 2017, 8:40 AM

Herald added a subscriber: jpeg. · View Herald TranscriptAug 27 2017, 8:40 AM

This could help T5301

also note that eina_thread_create() doesn't allow us to provide a stack size, so on Linux it's 2MB per thread -- quite a lot. If we create a new eina primitive, even if an internal primitive to set attributes then we could easily reduce the overhead... most coroutines should run under 16-32kb.

I must say that it doesn't seems so much better to use, but well, this is C. I am wondering how much of this is usable by bindings for implementing async/await in other language. I think @felipealmeida should have look at this.

@cedric maybe you overlooked it based on the _await() which is very minimalist. Take the complex_chain from Future & Promise, it would look much simpler as you can see:

single function versus 4 functions. If number of "async" stuff you had to wait grows, the function remains a single one...
no need to create and manage "context" data
looks plain C, "eina_future_await()" looks like a traditional function
in C++, "eina_future_await()" could throw exceptions and unpack value automatically, reducing number of lines.

complex_chain_with_coro


// we could add some helpers to make C easier
#define eina_future_await_value_or_goto(future, coro, success_type, value, storage, label) \
   do { \
      value = eina_future_await(future, coro, success_type); \
      if (value.type == EINA_VALUE_TYPE_ERROR) goto label; \
      eina_value_get(&value, storage); \
    } while (0)

#define eina_future_await_or_goto(future, coro, label) \
   do { \
      Eina_Value __value = eina_future_await(future, coro, NULL); \
      Eina_Bool __is_error = __value.type == EINA_VALUE_TYPE_ERROR; \
      eina_value_flush(&__value); \
      if (__is_error) goto label; \
    } while (0)

static Eina_Value
async_do(void *data, Eina_Coro *coro, Eo *loop)
{
    const char *fname = data;
    Eina_Future *f;
    Eina_Value r;
    Eo *img;

    f = img_load(loop, fname); // for multi main loop, we must carry "loop"...
    eina_future_await_value_or_goto(f, coro, EINA_VALUE_TYPE_OBJECT, &img, error);

    efl_ref(img);
    eina_value_flush(&r); // unrefs object

    f = img_async_operation(img); // img is a loop user, no need to explicitly give "loop"
    eina_future_await_goto(f, coro, error);

    f = img_save(img, fname); // img is a loop user, no need to explicitly give "loop"
    r = eina_future_await_or_goto(f, coro, error;

    printf("done async operation!\n");
    
error:
    efl_del(img);
    return r;
}


int main(int argc, char *argv[]) {

    efl_loop_coro(ecore_main_loop_get(), EFL_LOOP_CORO_PRIO_HIGH,
        argv[2] /* data = path */,
        async_do /* coroutine */,
        NULL /* free function */);

    // return is a promise, we could "chain/then" to know when it's finished...

    ecore_main_loop_begin();
    return 0;
}

as for "await" in other languages, if they are all based on a future (as in JavaScript), then yes. The logic is pretty, pretty simple:

coroutine "thens" the future, save the pointer as "awaiting" and yield;
main routine returns from eina_coro_run() with p_awaiting set to the internal future. Caller will "then" on that future instead of scheduling using idler/timer/job...
promise resolves in the main thread, call future chain:
- gets the promise result and store in coroutine context
- scheduler calls eina_coro_run(), this unlocks the coroutine
coroutine checks for value, it's there, so keeps running (if called without a value, would "yield" again)

tell me if i'm wrong... yield() here literally has to stop and run a nested loop inside of it before it returns... right? for this to work.... or this can only be used in thread with no loops... ?

@raster, it's not what you thought... it's much more like what's done with ecore_main_loop_thread_safe_call_sync(), however it's simpler as you can see below:

eina_coro_run(): main thread signals turn == COROUTINE and then sleeps;
coroutine runs, main thread is sleeping;
coroutine eina_coro_yield() or eina_coro_await(): signals turn == MAIN and then sleeps;
main thread awakes from sleep, coroutine is sleeping... main thread runs normally (no nested main loop!). It will schedule a new call to eina_coro_run(), either using a job/timer/idler or will eina_future_then() on any future that the coroutine is waiting with await.
scheduled callback triggers, calls eina_coro_run()... back to step 1

The loop is finalized when coroutine exits, then eina_coro_run() retuns false and step 4 will not reschedule it, but dispatch the returned value using eina_promise_resolve() on an internal promise, which the future was returned by efl_loop_coro().

+-----------+-----------------------+----------------------+-------------------------+
| Turn      | Method                | Main Routine         | Coroutine               |
+===========+=======================+======================+=========================+
| MAIN      | `eina_coro_new()`     | set `turn = MAIN`,   | thread starts and       |
|           | or `efl_loop_new()`   | create thread,       | sleeps waiting          |
|           |                       | schedules "run"      | condition               |
|           |                       | (timer/job/idler...) | `turn == COROUTINE`     |
+-----------+-----------------------+----------------------+-------------------------+
| MAIN      | scheduled callback    | signals              | _sleeping..._           |
|           | calls                 | `turn = COROUTINE`,  |                         |
|           | `eina_coro_run()`     | sleeps waiting       |                         |
|           |                       | condition            |                         |
|           |                       | `turn == MAIN`       |                         |
+-----------+-----------------------+----------------------+-------------------------+
| COROUTINE |                       | _sleeping..._        | awakes signaled,        |
|           |                       |                      | notices                 |
|           |                       |                      | `turn == COROUTINE`     |
|           |                       |                      | call **user function ** |
+-----------+-----------------------+----------------------+-------------------------+
| COROUTINE | `eina_coro_yield()`   | _sleeping..._        | signals                 |
|           | or                    |                      | `turn = MAIN`,          |
|           | `eina_coro_await()`   |                      | sleeps waiting          |
|           | or                    |                      | condition               |
|           | `eina_future_await()` |                      | `turn == COROUTINE`     |
+-----------+-----------------------+----------------------+-------------------------+
| MAIN      |                       | awakes signaled,     | _sleeping..._           |
|           |                       | notices              |                         |
|           |                       | `turn == MAIN`,      |                         |
|           |                       | schedule based on    |                         |
|           |                       | `p_awaiting` or      |                         |
|           |                       | timer/job/idler...   |                         |
|           |                       | **Goes to row 2 **   |                         |
+-----------+-----------------------+----------------------+-------------------------+

@barbieri Oh, I wasn't clear on what I find "problematic" here. My issue is that you have to create and run the coroutine function in a specific way. You can't have the JS/C# freedom that the language give. I am really fine with this, it is still an improvement. I just want to be sure that the same facility is available in language where it is really powerful. Other than that I am fine with this proposal.

This patch needs rebasing.

This revision now requires changes to proceed.May 2 2018, 3:58 PM

Revision Contents
Changeset List

		Path
M		src/Makefile_Eina.am (9 lines)
M		src/Makefile_Eo.am (1 line)
M		src/lib/ecore/ecore_main.c (176 lines)
M		src/lib/ecore/efl_loop.eo (64 lines)
M		src/lib/eina/Eina.h (1 line)
A	M	src/lib/eina/eina_coro.h (554 lines)
A	M	src/lib/eina/eina_coro.c (642 lines)
M		src/lib/eina/eina_main.c (3 lines)
M		src/lib/eina/eina_promise.h (62 lines)
M		src/lib/eina/eina_promise.c (48 lines)
M		src/lib/eo/eina_types.eot (2 lines)
M		src/lib/eo/eo.c (126 lines)
M		src/tests/ecore/ecore_test_promise2.c (99 lines)
M		src/tests/eina/eina_suite.h (1 line)
M		src/tests/eina/eina_suite.c (1 line)
A	M	src/tests/eina/eina_test_coro.c (562 lines)
M		src/tests/eo/suite/eo_suite.h (1 line)
M		src/tests/eo/suite/eo_suite.c (1 line)
A	M	src/tests/eo/suite/eo_test_coro.c (52 lines)

Commit	Tree	Parents	Author	Summary	Date
306ec6937bb1	141883e823fa	109bf1b387f2	Gustavo Sverzut Barbieri	Efl.Loop.coro: easy to use coroutines attached to a main loop. (Show More…)	Aug 26 2017, 8:42 PM
109bf1b387f2	2313df5ff1f8	d10a35628cc2	Gustavo Sverzut Barbieri	Efl_Object: make main domain accessible in Eina_Coro. (Show More…)	Aug 26 2017, 1:26 PM
d10a35628cc2	feb0fa20fab1	4d7661b1746d	Gustavo Sverzut Barbieri	eina: add Eina_Coro - coroutine support. (Show More…)	Aug 25 2017, 11:54 AM

Diff	ID	Base	Description	Created	Lint	Unit
Base			Base
Diff 1	12101	4d7661b		Aug 27 2017, 8:40 AM	★	★

CoroutinesNeeds RevisionPublicActions

Details

Diff Detail

Revision ContentsChangeset List

Diff 12101

src/Makefile_Eina.am

src/Makefile_Eo.am

src/lib/ecore/ecore_main.c

src/lib/ecore/efl_loop.eo

src/lib/eina/Eina.h

src/lib/eina/eina_coro.h

src/lib/eina/eina_coro.c

src/lib/eina/eina_main.c

src/lib/eina/eina_promise.h

src/lib/eina/eina_promise.c

src/lib/eo/eina_types.eot

src/lib/eo/eo.c

src/tests/ecore/ecore_test_promise2.c

src/tests/eina/eina_suite.h

src/tests/eina/eina_suite.c

src/tests/eina/eina_test_coro.c

src/tests/eo/suite/eo_suite.h

src/tests/eo/suite/eo_suite.c

src/tests/eo/suite/eo_test_coro.c

Coroutines
Needs RevisionPublic
Actions

Revision Contents
Changeset List