Page MenuHomePhabricator

Elementary Atspi
Updated 1,353 Days AgoPublic

Introduction

Elm_Atspi is a implementation of AT-SPI2 standard (http://accessibility.linuxfoundation.org/a11yspecs/) for Elementary library. Purpose of this module is to provide unified d-bus interfaces for Accessibility Technology Provides like Orca Screen reader.

Design

Elm_Atspi uses new Eo object model available in EFL libraries since version 1.10. Core components of AT-SPI2 implementation are:

  • elm_atspi_bridge.c - module managing d-bus connection and translating d-bus methods call into elm_interface_atspi_* function calls.
  • elm_interface_atspi_* - set of interfaces providing AT-SPI2-like accessibility features.

elm_interface_aspi_* interface implementation is done directly in elementary widgets.

Implementation per widget status

Interfaces support legend
[ x ] -> implemented
[ P ] -> planned to support but currently not implemented
[ - ] -> no plan to implement

Interfaces
WidgetRoleAccessibleComponentValueSelectionImageTextEditable TextActionWindowComments
elm_photoxxx-------
elm_boxxxx-------
elm_tablexxx-------
elm_thumbxxx-------
elm_menuxxx-------
elm_photocamxxx-----x-
elm_winxxx-----x-
elm_imagexxx--x--x-
elm_iconxxx--x--x-
elm_prefsxxx-------
elm_mapxxx--P--x-
elm_glviewxxx-------
elm_webxxx-------
elm_toolbarxxx-------
elm_gridxxx-------
elm_diskselectorxxx-P---x-
elm_notifyxxx-------
elm_mapbufxxx-------
elm_flipxxx-------
elm_layoutxxx-------
elm_sliderxxxx----x-
elm_listxxx-x---x-
elm_colorselectorxxx-------
elm_ctxpopupxxx-----x-
elm_separatorxxx-------
elm_entryxxx---xx--
elm_calendarxxxP----x-
elm_inwinxxx-------
elm_gengridxxx-P---x-
elm_radioxxx-----x-
elm_scrollerxxx-----x-
elm_framexxx-------
elm_datetimexxxP------
elm_playerxxx-----x-
elm_bgxxx--P----
elm_videoxxx-------
elm_segmentcontrolxxx-P-----
elm_progressbarxxx-------
elm_checkxxx-----x-
elm_panelxxx-----x-
elm_indexxxx-P-----
elm_flipselectorxxx-P---x-
elm_fileselectorxxx-------
elm_clockxxxP------
elm_fileselectorentryxxx---P-x-
elm_labelxxx-------
elm_spinnerxxxx------
elm_buttonxxx-----x-
elm_hoverselxxx-----x-
elm_fileselector_buttonxxx---P---
elm_bubblexxx---P---
elm_dayselectorxxx-------
elm_panesxxxP------
elm_popupxxx---P---
elm_genlistxxx-P---x-
elm_slideshowxxx-P---x-
elm_actionsliderxxxP------
elm_hoverxxx-------
elm_multibuttonentryxxx-------
elm_conformantxxx-------
elm_naviframexxx-----x-
Interfaces
WidgetRoleAccessibleComponentValueSelectionImageTextEditable TextActionWindowComments
elm_GenlistItemxxx---P-P-
elm_GengridItemxxx---P-P-
elm_IndexItemxxx---P-P-
elm_MenuItemxxx---P-P-
elm_ListItemxxx---P-P-
elm_ToolbarItemxxx---P-P-

Merging atspi object trees

Accessible applications should expose well-defined, complete object tree on AT-SPI2 dbus bus. This step is essential in order to make your application usable to any Assistive Technology Providers like Orca screen reader. A complete AT-SPI tree means all GUI elements (viewable in one window from end user's perspective) should be exposed as one atspi tree on AT-SPI bus. This task is especially demanding when parts of window UI are rendered in a different process that main application UI. Such example is indicator service combined with elm_conformant widget, or more generally elm_win of type ELM_WIN_SOCKET_IMAGE combined with elm_plug widget.
Happily, Elementary library provides default implementation for elm_win and elm_plug widgets allowing to connect accessibility object from two processes and expose them as one on AT-SPI2 bus. This feature is possible by using elm_atspi_proxy objects.

problem

Consider an application having embeded content (eg. one using indicator and elm_conformant in case of Elementary) provided by different process. From end user perspective it doesn'y really matter if the content comes from different process or not - they are viewable and interactable like any other GUI element. Blue color depicts main applications UI, red embeded content provided by different process.


From Assistive Technology Provider perspective situation looks a bit different, if no elm_atspi_proxy mechanism or similar one is used accessibility trees are viewed by screen reader as a separate application instances. This may cause serious troubles to Assisitive Technology Provider application logic making it for example unable to read a whole content of the window.

Above situation was fixed by attaching embeded process root note into embeding process container node. From now on Assistive Technology Provider application and User perspective do not differ.

using elm_atspi_proxy

elm_atspi_proxy is a special purpose object representing a proxy connection to accessibility object in different process. Elm_win and elm_plug widgets create and manage lifecycle of proxy objects by themselves. It is generally advisable to use elm_socket and elm_plug in order to provide embeded content in Elementary applications, however manual creation of proxy objects is be needed in following situation:
Embeded content is provided by different toolkit which exposes AT-SPI tree hierarchy
If the content is rendered in separate process and these process is using Elementary library please use elm_win and elm_plug mechanism, which have build-in elm_atspi_proxy support.


Above diagram depicts usage of elm_atspi_proxy objects (yellow). Each proxy object refers to exactly one dbus object registered on a11y bus (blue ones). Proxy object by itself are not exposed on a11y bus. There are two types of proxy object:

  1. socket type - is used in a process whose content is embedded
  2. plug type - is used in process which is embedding other process content

creating elm_atspi_proxy

Atspi is still beta API in Elementary library. To use it please include Elementary header as below:

#define EFL_EO_API_SUPPORT
#define EFL_BETA_API_SUPPORT
#include <Elementary.h>

Socket

Socket objects should be created only by a process which content will be embedded. To create an proxy object of type ELM_ATSPI_PROXY_TYPE_SOCKET we use customized eo constructor:

Eo *socket = eo_add(ELM_ATSPI_PROXY_CLASS, parent, elm_obj_atspi_proxy_constructor(ELM_ATSPI_PROXY_TYPE_SOCKET));

As a parent a embedded widget (eg. elm_win) should be passed. Please consider that parent object will keep reference to proxy object and proxy object will be destroyed along with parent object, Parent object will be used internally as a child object in AT-SPI tree hierarchy.
Additionally a parent object must implement ELM_INTERFACE_ATSPI_ACCESSIBLE_MIXIN interface to ensure that it will be registered on dbus by elm_atspi.

Plug

Plug objects should be created only by a process which is embedding external content. To create an proxy object of type ELM_ATSPI_PROXY_TYPE_PLUG we use customized eo constructor:

Eo *plug = eo_add(ELM_ATSPI_PROXY_CLASS, parent, elm_obj_atspi_proxy_constructor(ELM_ATSPI_PROXY_TYPE_PLUG));

As a parent a parent widget (eg. elm_plug) should be passed. Please consider that parent object will keep reference to proxy object and proxy object will be destroyed along with parent object,
Additionally a parent object must be implemening ELM_INTERFACE_ATSPI_ACCESSIBLE_MIXIN interface to ensure that it will be registered on dbus by elm_atspi.

set proper proxy address

After creation every proxy object must point to valid bus object. It is up to developer to obtain proper bus and path strings and keep them synchronized between processes.

eo_do(proxy, elm_obj_atspi_proxy_address_set(bus, path));

generating children changed events for PLUG type

After creation of plug proxy and setting bus and path address, we need to generate an event to atspi_bridge informing that new atspi children is available:

Eo *parent;
eo_do(plug, parent = elm_interface_atspi_accessible_parent_get());
elm_interface_atspi_accessible_children_changed_added_signal_emit(parent, plug);

intergrating widget tree with elm_atspi_proxy

When implementing a custom elm_atspi_proxy support we must ensure integrity of atspi object tree in a framework. To do so we must ensure that every elementary object implementing ELM_INTERFACE_ATTSPI_ACCESSIBLE_MIXIN will return elm_atspi_proxy as its parent (in case of proxy type SOCKET), and as its child (in case of proxy type PLUG)

proxy socket child widget [Embeded process]

In order to ensure integrity of atspi tree by overloading default eo implementation. To do so we add following section to default eolian file:

implements {
    Elm_Interface_Atspi_Accessible.parent.get;
}

A possible implementation should return previously constructed elm_atspi_proxy object of type SOCKET

EOLIAN Eo *_my_widget_elm_interface_atspi_accessible_parent_get(Eo *obj EINA_UNUSED, My_Widget_Data *_pd)
{
     return _pd->socket;
}

plug parent widget [Embeding process]

In order to ensure integrity of atspi tree by overloading default eo implementation. To do so we add following section to default eolian file:

implements {
    Elm_Interface_Atspi_Accessible.chidlren.get;$
}

A possible implementation should return previously constructed elm_atspi_proxy object of type PLUG

EOLIAN Eina_List  *_my_widget_elm_interface_atspi_accessible_children_get(Eo *obj EINA_UNUSED, My_Widget_Data *_pd)
{
     return eina_list_append(NULL, _pd->plug);
}
Last Author
stanluk
Last Edited
May 8 2015, 6:59 AM
Projects
None
Subscribers
subodh6129