[Buildroot] Build reproducibility

Thomas Petazzoni thomas.petazzoni at free-electrons.com
Mon Sep 2 08:53:37 UTC 2013


Dear Thomas De Schampheleire,

On Mon, 2 Sep 2013 10:44:17 +0200, Thomas De Schampheleire wrote:

> > I agree with this. Thomas, I'm not sure that what you say what a
> > conclusion of a developer day. I believe we always said that it is
> > hardly possible to guarantee that a package .mk file will contain *all*
> > the possible dependencies of the package. Whenever someone bumps a
> > package, we rarely look in detail at whether the software has gained
> > some optional dependencies, and make sure they are handled in the
> > corresponding package .mk file.
> >
> > Having the packages always built in the same order guarantees that, in
> > the absence of expressed dependencies, the build result will be
> > reproducible.
> 
> I may be wrong about the conclusion.
> Regardless: it's true that it's hard to guarantee that all
> dependencies are expressed properly in the .mk files. However, by
> 'fixing' the wildcard behavior into a sorted one such as with previous
> versions of make, we just hide the problem. We will never notice that
> some dependencies are missing, as long as the dependency comes before
> the dependant (or whichever word) in alphabetical order.
> With the random behavior of wildcards in newer versions of make, we
> would still see issues in autobuilds, and get the opportunity to fix
> them. It may take time, and may be a never-ending story as packages
> are bumped and new packages are added, but at least there can be
> improvement.
> So, my viewpoint is to keep the current behavior and instead focus on
> fixing any missing dependency when we spot it.

I obviously disagree, because in the mean time, our users are having
non-reproducible builds. An user within a company uses Buildroot to
create a system, adds some packages, creates a configuration for his
project. Then he passes this Buildroot to another colleague: the
date/times of the various Buildroot files will be different, maybe
affecting the order in which the wildcards are resolved by make 3.82.
This colleague will attempt the build, and maybe get a failure, or a
different build result than the first colleague who has created the 
Buildroot configuration. This is really damaging for Buildroot's
reputation and the user experience. We clearly to not want this to
happen.

Of course, if within the Buildroot project we are interested in fixing
such missing dependencies, then we can find a way of adding randomness
into the build order in our autobuilders. But clearly, we do want to
expose this randomness to our users.

In fact, I had already thought about adding such randomness in the
autobuilders. But I've refrained from doing so because it also means
that the builds that the autobuilders are doing cannot be reproduced.
So when you'll get an autobuilder failure that you can't reproduce
locally, you'll never know if it's due to the random order of package
building, or due to some difference in the build environment.

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com



More information about the buildroot mailing list