Add order for Obj_Info struct.
Needs ReviewPublic

Authored by Deepwarrior on Aug 30 2017, 6:21 AM.

Details

Summary

Objects might be identified with key, that will be same for different
launches of one app.

Record user actions in connected app from objects introspective extension.

Integration of Automatization Toolkit into clouseau. When record starts
add callbacks on Ecore_Events in app, this callbacks send data to clouseau
in format: pointer to object, type of event, Ecore_Event struct. When clouseau
receives this data, it writes python script into std output like it was in
Automatization Toolkit recorder.
@feature

Save script to file.

Prepare for use recording for scripts.

Diff Detail

Repository
rCLS tools/clouseau
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 4407
Build 4472: arc lint + arc unit
Deepwarrior created this revision.Aug 30 2017, 6:21 AM

Hi fellows,

If I understand well, you are trying to record scenario from Clouseau by sending the mouse events from the app to the debugger. Then you save them in some python script.

What do you intend to do with that script then? When you click on the play button, you launch the app and apply the scenario?

What is the goal of this feature?

Hi @JackDanielZ
The history of this feature is: a few guys from Kiev needed some tool for testing UI behaviour of the EFL applications. All what to need it is record user actions, replay them and compare with ethalon results. Pretty similar to thing what exactness project do. But exactness was created for pixelperfect comparsion. In case if ethalon screenshots was created with one render (lets assume OpenGL) and tests ran on the hosts with another render (software render for example) the result will show that those screenshots are not the same.
The main idea of needed tool was to compare how widget tree changes during user input. Based on this was created PoC of automation toolkit that could provide functionality for access to the any widget inside EFL application from human-readable script. Independly how widget looks and where it located.
Simple usecase:

  1. record user actions from EFL applications to the python script (actually it could be any language or whatever);
  2. when need to test application, recorded script replay user actions back to the EFL application.
  3. For check result - the widget tree status should be checked. Or do the same what exactness do - compare screenshots.

The result of development PoC could be find here: https://git.enlightenment.org/devs/nikawhite/automation-toolkit.git/log/?h=devs/nikawhite/develop . This toolkit could record and replay user actions to/from python scenarios. There is a task tracking for this project: Automatization Toolkit

Few weeks ago I was in Korea and had conversation with @raster about future of those toolkit for EFL and Tizen. Summarizing all what I heard - one more standalone tool that do mostly the same things as exactness doesn't needed. Better way will be to merge feautres from exactness and automation toolkit together. And the place where all this could be located is Clouseau. My point of view that Clouseau could be an excelent center for debugging/profiling of the EFL applications.

Exactly this patch is just for discussion with @Deepwarrior , that why you wasn't added as reviewer. :)

Hi,

Thank you for the explanation.
I have a few comments:

  • Why do you need to use the objects introspection extension for that. Except the widgets tree, I don't see why it cannot be in a separate extension. Maybe it should be out of the Clouseau repo, until it is fully stable.
  • You expect keys (I suppose names) to remain unchanged over iterations of the application. I am not sure this is true, especially for widgets such as genlist, where content refreshing seems (from user scope) out of control. From my experience, a lots of items objects are created, destroyed and recreated just at the beginning. I am not sure you can be sure that you will have the same amount of objects at a given time, especially when EFL behavior can change internally. For example, some weird genlist optimization makes that some item content may be used for another item. I know, this seems weird, ackward, crazy! But edje does that to optimize internally.

JackDanielZ

  • Why do you need to use the objects introspection extension for that. Except the widgets tree, I don't see why it cannot be in a separate extension. Maybe it should be out of the Clouseau repo, until it is fully stable.

Hi. I`m new with eina_debug, so objects introspection was used as a base for connecting to app with eina_debug. Also it contain widgets tree so I don`t require to duplicate functionality for collecting widgets.
When integration into clouseau will be complete, it will be easy to separate toolkit into new extension. And while I'm doing this I want to integrate few changes that can be useful for introspection extension like indexes for widgets and collecting widget tree in realtime.

  • You expect keys (I suppose names) to remain unchanged over iterations of the application. I am not sure this is true, especially for widgets such as genlist, where content refreshing seems (from user scope) out of control. From my experience, a lots of items objects are created, destroyed and recreated just at the beginning. I am not sure you can be sure that you will have the same amount of objects at a given time, especially when EFL behavior can change internally. For example, some weird genlist optimization makes that some item content may be used for another item. I know, this seems weird, ackward, crazy! But edje does that to optimize internally.

    JackDanielZ

You are absolutely right about the genlist/gengrid items content.
The idea of that kind automation is to representing object by text description. Let say we are have a window, with genlist inside and at current time genlist contai 100 items with button as content. The button from the item with order number 39 could be described as Efl.Ui.Win[0]:Elm.Genlist[0]:Elm.Genlist.Item[39]:Elm.Button[0]. Walkin through the tree of elementary widgets it is possible to get object pointer of Efl.Ui.Win[0]:Elm.Genlist[0]. After that walking through genlist items by elm_genlist_first_item_get()/elm_genlist_next_item_get(). it is possible to calculate geometry of received Elm_Widget_Item object. And all what left is just find a widget inside geomtry area of the items. This is the mostly difficult point for me right now.

zmike added a project: Restricted Project.May 2 2018, 3:59 PM