Page MenuHomePhabricator

Enlightenment For Mobile
Updated 1,223 Days AgoPublic

This page describe possible changes to make Enlightenment and EFL better for mobile users.

Application Lifecycle Management

On mobile systems we have often single-instance fullscreen applications. As resources are scarce we need to manage their lifecycle.

Common Use Case

  • launch image gallery. If it was not running start it;
  • may show a placeholder window/content while launching;
  • ask it to open some folder or picture. As it may be already running, it can't go through argc/argv;
  • User changed to another application (multi-task), hide gallery window and possibly drop resources such as window, image pixels... keeping the necessary information (path);
  • User came back from another application (multi-task), recreate/reload resources and show the window;
  • System is low on memory or battery, drop caches and possibly resources (if memory). If no windows were visible, save state and exit the process;
  • When application is re-activated load its last state as if it was still running;
  • Screen may be rotated to portrait/landscape;
  • System time may be changed, providing a time skew for clocks and other timers (eg: "picture taken 1 hour ago");
  • Locale/language may be changed;
  • Virtual Keyboard may be required and cover parts of the screen;
  • System-wide indicator (usually top bar with battery, clock and network status). Due performance this must be embedded into the application's window so that window can remain fullscreen and do not need the compositor;

Work Areas

  • Enlightenment could load extra key from .desktop file describing placeholder image to show until "startupnotification" protocol (already supported) says the window is ready. Alternatively we can wait the first window of that application to be mapped to hide the placeholder. This may be extended to the case where application pid has no windows (deleted all of them) and may take some time to recreate those. [Tizen/iOS/Android/Desktop]
    • (raster) I don't like this solution. What would be better is that the WM can "screenshot" the app once it has launched and sized correctly. The app must indicate (maybe via a client message) that it is ready for a screenshot (ie initial sizing/layout is done). the WM can store position/size etc. at this point and the screenshot if it doesn't already have one. next time we launch the app we bring up its screenshot(s) in the same position/size UNTIL the matching windows appear and we then "replace" our screenshots with the windows, but only once the app indicates the same "i am ready for screenshot" message. this should mean we are at the same point in startup. note we can do this for more than 1 window if an app opens up 2 to begin with. an app needs to indicate a window is a primary window somehow so elm can send off the "i'm ready for a screenshot" message when that window is shown/sized/rendered initially. one problem might be to figure out when the screnshot needs a real update. my thoughts here are to sample a small maybe 16x16 region (in a random location in the window) and then compare the pixels to the screenshot. if more than N pixels are changed, re-screenshot and save. this means the screenshot will update over time as theme, font sizes, or app updates make it change. the screenshot "db" is really just a cache for making it look like apps start faster than they do. :)
    • (cedric) I don't like that either. Most application should start fast enough (EFL initialization can take around 150ms, giving ample time for the application to start). For those that really can't this feature should be an opt-in. Maybe with some information in the .desktop file. Displaying an image will slow down the application startup time , so lets avoid it as much as we can by pushing people to have fast to first frame application. As a reference point, elemine first frame take 800ms on RaspberryPi to show.
    • (k-s) I don't think it's an option to not have it Cedric. I like Raster's idea, but it should be the default but we must support the other alternative, even if it's for the initial case. Also bear in mind that some applications take time not even to do something, just linking to heavy libraries such as webkit. However I'd split this into 2 points, my suggested implementation should take few hours, Raster's is more work and will take much longer to do.
    • (raster) After much thought on this, my complex solution is a lot more work, so maybe a .desktop file field will do for now. We can ignore it on desktop, OR just place the image centered on the screen as a "splash box" until window is up, and mobile can display fullscreen. We can make an automagic method later on.
  • Enlightenment and applications could negotiate a DBus method to generically start/open a resource Enlightenment App Spec Proposal. For instance we could define org.enlightenment.Application interface with an Open(dict) method. The parameters given are freely defined in the dict, with a name and typed value. Elementary could expose a callback that exposes an Eina_Value of type struct (set of key=value), that applications can handle.[Tizen/iOS/Android/Desktop]
    • (raster) This sounds like the reset event... but much more specialized?
    • (k-s) Yes, the same with more specialized intent.
    • (raster) So this is the proposal (or part of the proposal) over at Enlightenment App Spec Proposal?
  • Enlightenment Illume could hide windows that are unused, show them when they are requested again. The applications could listen to hide/show event and act accordingly to drop resources -- which is useful for every system, including desktop. Enlightenment already lower the prio of unfocused windows, Illume could augment this behavior by lowering ioprio/class and even setting oomadj. [Tizen/iOS/Android]
    • (raster) e already does this - but a bit differently. it can send a dump event to iconified apps. dump messages are already handled by dumping out all images/fonts/textures (and allowing page-in to re-load them). the problem is they are still composited (consume at least a pixmap buffer). adding an extra layer of "stop compositing these things" would help on the wm/compositor end, but protocol-wise we have this done already.
    • (k-s) but we miss oomadj and ioprio, right? These should be easy to add given that we already handle the application priority.
    • (raster) sure - just shovel in some ioprio changes and oomadj logic next to the nice code. :) it falls into the same bucket.
  • Enlightenment Illume could listen to screen orientation changes and notify applications. Elementary could then listen to this and elm_win would callback "orientation,changed" which could trigger a "rotate_and_resize" manually or by means of a call similar to elm_win_autodel_set(). [Tizen/iOS/Android]
  • DONE: Enlightenment Illume could notify applications of virtual keyboard presence and size. It is already done and even acts in 2 ways, if the window is "conformant" it will handle the resize internally, doesn't need to be resized by WM. [Tizen/iOS/Android]
  • Enlightenment Illume could provide a default indicator as elm_plug to be used by applications. Alternatively it should check if window doesn't provide internal indicator and fallback to show the indicator itself as an external window (this part is lower priority, as applications should always cooperate). [Tizen/iOS/Android]
  • Elementary should listen to language changes and inform application, also reloading its own language/locale aware components which includes theme. Ecore now provides an event for that and Elementary have a virtual and callback for language changes -- needs hooking. [Tizen/iOS/Android]
  • Elementary should listen to time changes and inform application, eventually reloading theme to have timers/clock fixed. Ecore now provides an event for that and Elementary may call its "theme changed" under that circumstances (??? should be better per component, like clock/datetime/...). [Tizen/iOS/Android]
  • Elementary should monitor low battery and performance to drop caches and increase animator frametime, possibly trading performance over quality (smooth=0). Ecore now provides an event for those and Elementary may listen and act accordingly. Applications can listen the same event -- needs hooking. [Tizen/iOS/Android]
    • (raster) Already doing a lot of this. the auto-throttle mode kicks in if ALL windows are withdrawn (or optionally iconified). we could indeed add other modes like lowering animator framerate (won't help if animator is timed off the vsync interrupt though). In fact we have a design flaw. animators should be tied to a window... fix that and then the window itself can control the animator mode based on its state/location etc. :)
    • (cedric) That one should be easy to fix with Eo as we could just define the parent of the animator to be the window and have special logic for that case. Then all animator creation in edje just require to be linked with the window the object is created on.
    • (k-s) Now we have the event in ecore to notify system really needs this, so it can complement the existing auto-throttle. Needs input on exactly what to do when the system is in the need of resources.
    • (raster) This is now no longer in the world of "mechanism" but now we're down to defining policies and config to tune the policies... so I think we're good.
  • Elementary should have a new policy to help battery and performance it could "request,delete" windows that are hidden after a threshold and system is under low-memory/battery state. This would complement the existing policy of quit application after last window is closed. Applications could reject delete request, so it is safe. Need a way to reload the resources, for instance if window X is auto-deleted and application exits, WM should be able to request that exact window to be recreated -- could we use name/class, eventually as a call to Open(name=X, class=Y) as mentioned above? If so Enlightenment task switcher would have to track applications and windows even if they are not running. [Tizen/iOS/Android]
    • (raster) this one is tricky and for now i suggest we try and push the "drop compositing", "dump textures/images/fonts" stuff as it is less complex/problematic in general. come back to this once the others are doing as much as they can.
    • (k-s) all of this is nice and should exist, but I do think that helping application to auto-exit when unneeded is pretty common in mobile, particularly if you have things that are started because of others, not just because user explicitly requested. For instance I don't close windows explicitly everyday, I know my parents never do it, for months... Having applications to auto-exit would help the work of a system memory manager to go ask them to exit. Which problems do you foresee with such approach? It can be an elm_policy, as well as the one we have to quit if all windows were closed. The major bucket of work is to implement in Enlightenment a winlist module that remembers opened windows even if they were auto-closed.
  • (raster) As a counter-example, i recently had to explain to a non-geek why maybe they lose the blog entry they were editing every time they switch over to take a photo with their camera to then attach to the entry.. that they have to re-write now... and they are very unhappy about it. Auto-delete shouldn't be something we turn on or do lightly... rather play games of minimizing memory sage but keeping all your state as much as possible first. Reality is the kernel oom killer can do this already for us in dire emergencies... so we're implementing a non-emergency version of this now.
  • Ecore should provide more of the upower features. Upower (and also systemd) allow you to do things like inhibit suspend and hibernate, request a suspend or hibernate as well as provide events like "sleeping" and "resuming", if you can hibernate or suspend at all, as well as even lid state events or if it is present. you also have the "about to sleep" request (prepare for it). we probably should support these. We'll need them for tizen too.
    • (k-s) I like this, but preventing suspend and hibernate is less common and eventually we need something more explicit about it. It could come in ecore, but should be an explicit call that would refcount the need, not just inferred from other information.
    • (raster) I agree that it should be a refcount based interface - even at the ecore level. Explicitly asking for a suspend inhibit is one path and needs to exist, but having things AUTOMATICALLY do this for certain tasks/actions/activities without the programmer needing to explicitly do it is best. The question is... what are these cases? Should this be a method/property on an object - eg an emotion object. when you set this, suspend/hibernate inhibit is on.. until u disable the property of that obj (refcount-- then), but this nicely handles object deletion which refcount--'s the inhibit for the programmer automatically? it could be tied to a window or an entry box, a web widget, hell even an ecore-con-url download connection (need to make these full blown eo objects)? etc. too.

General WM

Common Requirements

  • Edge events to task switch, show notification panel, etc.
  • Different visual transitions depending on the need. If application starts it should use one animation (ie: fade), while if it's being requested as a screen from another application (ie: want to show image picker for my app) it may use another (ie: slide, should match elm_naviframe default theme).
    • (raster) This should be done semantically and shoulnt allow the app to specify the exact animation, just the intent.
  • Lockscreen plugin/gadget, to allow different kinds of unlock (pattern, code, bio...)
  • Above-lockscreen policy, needed for things like "incoming call" or "camera"
    • (raster) OR should this be an "unlock me" request from an app? Danger - it has security implications, but so does "above lockscreen" if you want to place certain apps there. the "unlock now" request is simpler though...
    • (k-s) reality proved that all ways have security issues, iOS had buggy versions for years. I think that above-lockscreen is easier to identify (window must be explicitly allowed), but still the window contents may be sensible to lockscreen and in that case we need application cooperation. Say you open the camera, you don't want gallery to be visible -- one window, different content.
    • (raster) Good point. Then this needs to be a window property. for wayland it'd be a protocol request for that surface to enable "abovelock" etc.
  • Screen Locked signal/state, needed for the above-lockscreen to identify the level of disclosure should be allowed. One doesn't want the camera to allow access to gallery if screen was locked.
    • (cedric) I may be wrong, but I think that systemd/logind already define something like that.
    • (k-s) I don't think they provide this, but they could hold this state. However we'd need to interact with that in Enlightenment.
  • Virtual Keyboard plugin/gadget to allow different keyboards while being fast and responsive
  • Screen auto-brightness control.
  • Screen orientation monitoring and signaling.
  • System notification area (similar to gadman top layer).
  • Screen brightness control based on application (application can set desired LCD brightness through API and Window Manager maintains the brightness based on application preference)
Last Author
sreedeep.m
Last Edited
Jan 14 2016, 1:37 AM
Projects
None
Subscribers
None