Page MenuHomePhabricator

py-efl compiler issues out of memory errors -and fails
Open, HighPublic

Description

June 2017
python3 setup.py build
fails consistently with or without cython
from git sources or from pip install.
The (any) build reaches :

elm_slider_range_set(pyx_v_self->pyx_base.pyx_base.pyx_base.pyx_base.pyx_base.obj, pyx_t_5, pyx_t_6);

then aborts with:
cc1: out of memory allocating 65536 bytes after a total of 1924927488 bytes
error: command 'i686-linux-gnu-gcc' failed with exit status 1

Attached is the pip log .
I'm probably not seeing an elephant or something ...

Yep this is a bug.
(learning phabricator)

kuuko claimed this task.Jul 17 2017, 5:28 PM
kuuko triaged this task as High priority.
kuuko added a subscriber: DaveMDS.
kuuko added a subscriber: simotek.

Yep, thats the same as the one i've seen, its hitting the process memory limit for i586/i686 our machines still had plenty more room available.

I'm also hitting this one on aarch64 full log: https://build.opensuse.org/public/build/X11:Enlightenment:Testing/openSUSE_Leap_42.2_Ports/aarch64/python3-efl/_log

[ 4493s] efl/elementary/__init__.c: In function 'PyInit_elementary':
[ 4494s] efl/elementary/__init__.c:438417:16: note: variable tracking size limit exceeded with -fvar-tracking-assignments, retrying without
[ 4494s]  PyMODINIT_FUNC PyInit_elementary(void)
[ 4494s]                 ^
[ 4891s] /tmp/ccuZYbjf.s: Assembler messages:
[ 4891s] /tmp/ccuZYbjf.s:1391633: Error: conditional branch out of range
[ 4891s] /tmp/ccuZYbjf.s:1391670: Error: conditional branch out of range

possibly we should try and split that file up, a 438417 line file seems rather massive.

Indeed the problem is the elementary module that is huge,
the only idea on my mind atm is to split again the module as it was before the merge, but It's quite a huge task to do :(

kuuko added a comment.Jul 18 2017, 5:38 AM

It's that, or we make releases with docstrings disabled, which should help with this issue.

kuuko added a comment.Jul 18 2017, 5:41 AM

Can't think of any other build options that could be disabled other than that one.

kuuko added a comment.Jul 18 2017, 5:48 AM

To disable docstrings, you can patch line 260 of setup.py:

Cython.Compiler.Options.docstrings = True

to

Cython.Compiler.Options.docstrings = False

This needs to be done with Cython build enabled (DISABLE_CYTHON=0) as it affects the generated C files.

Not that I see so much difference with doc strings disabled,
here the generated C files goes from 511628 lines to 508292

I don't know how to test the out-of-memory issue, but I bet this will not resolve it :(

(as a side note I can successfully build python-efl on a RaspberryPI3 with 760MB of ram and 2GB of swap)

Not that I see so much difference with doc strings disabled,
here the generated C files goes from 511628 lines to 508292

I don't know how to test the out-of-memory issue, but I bet this will not resolve it :(

(as a side note I can successfully build python-efl on a RaspberryPI3 with 760MB of ram and 2GB of swap)

RaspberryPI3 is a 64 bit arch where as the original i586 error is related to cc1 hitting the maximum memory limit for a process on a 32bit processor. Although a RPI3 build should be pretty similar to the second issue I was hitting.

ok, I got the same error while building on the rpi3 with arch linux installed.

I made it works by rebuilding the sources using an older cython (23.5 in particular)

I also created the new release tarballs using old cython, thus the latest release should
also work for you:
https://download.enlightenment.org/pre-releases/python-efl-1.20.0.tar.gz

please give this tarball a try

adrien added a subscriber: adrien.Aug 19 2017, 3:49 AM

You should try to disable compiler optimizations: they're completely useless for bindings. If that helps, you can try to enable some again.

@DaveMDS I feel this problem isn't going to go away, so we should find a solution. We can divide the elm modules again and import their contents at efl.elementary, but I'm not sure if we can keep things backwards compatible this way.

News on this issue:

the workaround we was using (generating with cython 23.5) is no more an option, because that version is not compatible with python 3.7

I am forced to generate the next tarball with the latest cython release, that probaly have the issue reported in this ticket.

Please report to me any issue you may encounter with the next release