[Buildroot] [PATCH 00/15] Overwritten file detection and fixes, one more step to TLP build
Arnout Vandecappelle
arnout at mind.be
Mon Jun 21 20:42:18 UTC 2021
Hi Hervé,
Great that someone is picking thuis up!
On 21/06/2021 16:11, Herve Codina wrote:
> Hi,
>
> This series fixes some issues in the TLP build feature.
> The starting point was the work done by Thomas Petazzoni on overwrites
> detection (PATCH 1).
I've marked [1] as Changes Requested.
However, it seems the tests [2][3][4] are missing from this series. Could you
include them next time you post it?
> Based on this work, this series fixes some overwrites, introduces and uses
> <PKG>_PER_PACKAGE_TWEAK_HOOKS. This variable is used to collect specific
> per-package path tweaks needed by per-package build.
> With this tweaks collected in this variable, they can be re-applied
> when they are needed (<pkg>_{reconfigure,rebuild,reinstall}).
That's excellent!
> It also reworks the final {HOST,TARGET}_DIR rsync.
> For each packages only files generated by the package are rsynced (PATCH 13).
> This is done using an exclusion list computed in (PATCH 12).
> In order to avoid modifications performed in target-finalize step to be seen
> as overwrites on next build, the final {HOST,TARGET}_DIR do not contain any
> hardlink anymore and are a full copy of expected files and directories (PATCH 14).
Hm, that will have quite the impact on disk size, no? In non-PPS builds,
HOST_DIR is typically about 1/3 of the total size (the other 2/3 is BUILD_DIR,
the rest is negligible). By doing a non-hardlinked copy, the size of HOST_DIR is
basically going to double, right?
I still think it's acceptable, but it's something to take into account...
> The last patch (PATCH 15) fixes <pkg>_{reconfigure,rebuild,reinstall} recreating
> per-package {HOST,TARGET}_DIR from scratch.
> Indeed, for a given package, re-generating the same file is detected as an overwrite.
> The simplest way to avoid this is to fully discard previous per-package {HOST,TARGET}_DIR
> directory and recreate them.
+1 to that!
> To have a TLP build fully fonctionnal, several things are still missing:
> - Be sure that all buildroot packages do not perform any overwrites.
I guess that's easy: increase the chance of PER_PACKAGE enabled in
genrandconfig, and see what happens to the autobuilders. Or is there anything
else missing?
> - Undo tweak stuff that have be done for per-package build (fixup-libtool-files,
> fixup-python-files, <PKG>_PER_PACKAGE_TWEAK_HOOKS) after collecting files
> in final {HOST,STAGING}_DIR.
> This mainly means reverting .../per-package/xxxxx/... path to point to
> the correct ones in final {HOST,STAGING}_DIR.
>
> I hope this series will help in TLP build feature.
Regards,
Arnout
[1]
https://patchwork.ozlabs.org/project/buildroot/patch/20200430095249.782597-9-thomas.petazzoni@bootlin.com/
[2]
https://patchwork.ozlabs.org/project/buildroot/patch/20200430095249.782597-10-thomas.petazzoni@bootlin.com/
[3]
https://patchwork.ozlabs.org/project/buildroot/patch/20200430095249.782597-11-thomas.petazzoni@bootlin.com/
[4]
https://patchwork.ozlabs.org/project/buildroot/patch/20200430095249.782597-12-thomas.petazzoni@bootlin.com/
>
> Best regards,
> Hervé Codina
>
> Herve Codina (14):
> package/e2fsprogs: fix fsck overwrite in HOST_DIR
> package/pkg-generic.mk: Remove Info documents dir entry
> package/pkg-generic.mk: Fix .la files overwrite detection
> package/pkg-generic.mk: Perform .la files fixup in per-package
> HOST_DIR
> package/pkg-generic.mk: Introduce <PKG>_PER_PACKAGE_TWEAK_HOOKS
> package/apr-util: Use <PKG>_PER_PACKAGE_TWEAK_HOOKS
> package/apache: Move APACHE_FIXUP_APR_LIBTOOL to
> <PKG>_PER_PACKAGE_TWEAK_HOOKS
> package/pkg-python: Remove _sysconfigdata*.pyc files when
> _sysconfigdata*.py are changed
> package/pkg-generic.mk: Move python fixup to generic package
> infrastructure
> package/owfs: Remove Python sysconfigdata fixup
> package/pkg-generic.mk: Generate final rsync exclude file list
> Makefile: Rsync global {TARGET,HOST}_DIR using exclusion file list
> Makefile: Breaks hardlinks in global {TARGET,HOST}_DIR on per-package
> build
> package/pkg-generic.mk: Fix per-package
> <pkg>-{reconfigure,rebuild,reinstall}
>
> Thomas Petazzoni (1):
> package/pkg-generic.mk: detect files overwritten in TARGET_DIR and
> HOST_DIR
>
> Makefile | 24 +++++++-
> package/apache/apache.mk | 2 +-
> package/apr-util/apr-util.mk | 12 ++--
> package/e2fsprogs/e2fsprogs.mk | 1 +
> package/owfs/owfs.mk | 9 ---
> package/pkg-generic.mk | 103 ++++++++++++++++++++++++++++++++-
> package/pkg-python.mk | 10 ----
> 7 files changed, 134 insertions(+), 27 deletions(-)
>
More information about the buildroot
mailing list