[Buildroot] [PATCH 00/51] TLP/PPS: intro, package cleanup.
Gustavo Zacarias
gustavo.zacarias at free-electrons.com
Mon Oct 17 16:02:13 UTC 2016
On 2016-10-14 12:48, Arnout Vandecappelle wrote:
> Hi Gustavo,
>
> On 14-10-16 00:13, gustavo.zacarias at free-electrons.com wrote:
>> From: Gustavo Zacarias <gustavo.zacarias at free-electrons.com>
>>
>> This is the first step in picking up Fabio Porcedda's work
>> ( http://lists.busybox.net/pipermail/buildroot/2015-June/131051.html )
>> regarding TLP (Top-Level Parallel) support and moving it forward for
>> mainstream buildroot inclusion.
>>
>> PPS (Per-Package Staging) is the central piece in enabling the
>> configure/build/install of packages simultaneously. Right now packages
>> pick up libraries/includes from a shared (the staging) directory.
>> Once TLP is enabled this is problematic because despite the
>> developers/contributors best efforts not all dependencies are
>> accounted
>> for properly.
>> When building and installing in parallel a package 'A' can be
>> installing development files (headers/libraries) while package 'B'
>> is configuring, and potentially probing for its dependencies.
>> If package 'B' can use unaccounted dependencies from package 'A' this
>> can be a problem since the feature probe might not be atomic, hence
>> it might find erroneous package 'A' features since the install to the
>> staging area might not be complete when it's doing this.
>> To avoid this the proposal is to limit the staging directory scope
>> of packages only to the required and requested dependencies.
>
> Note that it's not enough to do it for staging, host needs it as well.
Hi Arnout.
Indeed, i'll be sending a new series that handles more target package
fixes, addressed the ones from the previous series that needed fixes and
does so for host packages as well.
For host packages i call this PPH (Per-Package Host directory/area), i'm
working on PPS first and then i'll move on to PPH.
And then there's PPL (Per-Package Logging) since looking at the output
when building TLP makes debugging very hard, so having a per-package
configure/build/install/etc log will make life easier (with a decent
playback when things go bad).
>> For this to happen packages need a special staging area that fulfills
>> its dependencies, hence overriding its sysroot, which must happen via
>> environment variable overrides (there are a couple of ways to
>> accomplish this, but they all require overrides via env variables).
>>
>> To accomplish this some packages need a small cleanup, which is
>> passing the $(TARGET_MAKE_ENV) variable which will contain this setup
>> for their build and install steps, to be able to override the default
>> build rules.
>
> Actually, I think it's better/safer/easier to just do a global export
> from the
> Makefile:
>
> export BR2_STAGING_DIR = $($(PKG)_STAGING_DIR)
>
> and perhaps also
>
> _STAGING_DIR = $(STAGING_DIR)
>
> to allow it to be used outside package build rules as well.
I'm toying with the idea of a sandbox as well that will just restrict
the view of the full staging area per-package to the dependencies it
needs (LD_PRELOADed).
This would alleviate the overhead in copying/linking all those files,
but it's a bit more complex.
Regards.
More information about the buildroot
mailing list