Page MenuHomePhabricator

docker images MUST have packages locally cached
Open, TODOPublic

Description

After checking out a recent build to see what was taking so long (https://travis-ci.org/Enlightenment/efl/jobs/398321124) I came across line 443, which runs a script that downloads all the dependencies and then installs them.

This script takes ~975 seconds to run, or over 90% of the entire build time.

Docker images must have all these dependencies cached; it's insane that we waste almost 20 minutes on every build doing nothing.

zmike created this task.Jun 29 2018, 10:09 AM
zmike triaged this task as Showstopper Issues priority.

I am not sure what you are seeing here. If I look at the log from the link and at line 443 I see the following:

443 $ if "$TRAVIS_OS_NAME" == "linux" && "$DISTRO" != ""; then
444 docker pull stefanschmidt1/ci-support-files:$DISTRO
445 fi

Listed as taking 32.79s.

The behaviour you are describing sounds more like the execution of the Dockerfile and not the ready made image we use on Travis.
(Basically what you are asking for is already happening. We have a Dockerfile which has all the efl deps listed and these Dockerfiles are being used on dockerhub to generate the images with all deps installed. On Travis we only use docker pull to update the latest image and run the build through it)

If you have a log that still shows the behaviour you are seeing I would be happy to have a log. I could only imagine this to be branch with a very out of date rebase from master.

zmike lowered the priority of this task from Showstopper Issues to TODO.Jul 4 2018, 8:09 AM

Argh I cited the wrong line number. It's L482 from that log, which seems like it includes the entire if block, ie. the command inside the conditional.

Now that I read it again with more sleep, it looks like the total time for the script includes both running the dockerfile to update the image and executing the entire build. Is there any way to split out the commands to have more precise timing for each item?

stefan_schmidt closed this task as Invalid.Jul 6 2018, 12:03 AM

For the entire build time this number sadly makes sense.

I would need to check the travis docs if we can get better timings for the different blocks of the build.

Closing this bug as the the initial problem reported is already done the way it was requested.

zmike reopened this task as Open.Jul 11 2018, 3:39 PM

It turns out I was right, the OSX build takes forever with this step: see the conditional at L41 for this build: https://travis-ci.org/Enlightenment/efl/jobs/402830050

I think in addition to D6610 we can start using a conditional HOMEBREW_NO_AUTO_UPDATE=1 in the osx deps script if the deps aren't being forced to update; this will avoid triggering the sync and save some time off the install...