[Buildroot] Allowing to build from Git branches

Michael Nazzareno Trimarchi michael at amarulasolutions.com
Tue Jan 14 21:14:24 UTC 2020


Hi Thomas

On Tue, Jan 14, 2020 at 10:10 PM Vincent Fazio <vfazio at xes-inc.com> wrote:
>
> I can see the argument for this. I used to use a hacked up BR branch
> that added support for checking if the reference to the git branch has
> changed since the last download but this was only while images  were in
> development. I always switched to fixed hashes after the iterations were
> completed.
>
> On 1/14/20 3:04 PM, Thomas Petazzoni wrote:
> > Hello,
> >
> > I know some people are going to yell at me for bringing up this topic,
> > but let's do it anyway.
> >
> > Currently, Buildroot says that using a Git branch as the VERSION for a
> > package is not supported. There are some patches that have been
> > proposed to explicitly prevent using branches, but in the current
> > Buildroot it "works" except that a cached tarball is preserved in
> > DL_DIR, which means that the branch is never fetched again unless the
> > tarball is manually deleted from DL_DIR. We have generally resisted
> > supporting Git branches with the (very valid) argument that it breaks
> > reproducibility.
> >
> > However, over time, I have seen a number of users/companies complaining
> > about this, both on the mailing list and privately with customers or
> > while discussing with users at conferences.
> >
> > The typical use case that they describe is the following: a company has
> > a large number of packages in Buildroot to build in-house software
> > components. Both for their CI runs, and for the developers actively
> > working on the system, they would like for those packages to point to a
> > branch, and always fetch the latest version. Yes, it is not
> > reproducible, but reproducibility is not what is desired at this point
> > of the project: what is wanted is precisely to fetch the latest and
> > greatest for those packages that point to a branch, to make sure the
> > most recent code gets tested and used by developers internally.
> >
> > The only solution we offer for this today is <pkg>_OVERRIDE_SRCDIR or
> > <pkg>_SITE_METHOD = local, but neither of those are very practical: you
> > need to have fetched each of your software components outside of
> > Buildroot, and point Buildroot at them. It's a bit silly when Buildroot
> > is already perfectly capable of doing this.
> >

We use the SRC_OVERRIDE and repo tools. I think that buildroot change
is not required. We have couple of scripts to tag version easily and organize
project that are in continue development outside buildroot.

Michael

> > Overall, I think it makes sense to realize that Buildroot is used in
> > different use-cases:
> >
> >   - During active development, where we want to offer a lot of
> >     flexibility, possibly at the expense of reproducibility.
> >
> >   - During release/generation of the final firmware that will be used on
> >     actual devices, where we want an absolute reproducibility, to the
> >     point where we're working on binary-identical results.
> >
> > Buildroot has its roots in solving the second use-case. However, we
> > have already added some features, like OVERRIDE_SRCDIR and SITE_METHOD
> > = local, to provide the additional flexibility needed for the first use
> > case. In some sense, those features already kill reproducibility: what
> > you're building is some random source code from some random local
> > folder. Completely non-reproducible. Possibility worse that a Git
> > branch, where you can potentially at least trace back which commit was
> > built, and re-do that same build.
> >
> > So I'd like to start the debate on whether it might makes sense to
> > relax our requirements on this, and be more pragmatic, and support
> > fetching from Git branches. Of course, this very likely needs to be
> > guarded by a Config.in option, so that we clearly set the boundary
> > between "you're in the safe zone where your build is reproducible" and
> > "you're in the zone where your build is non-reproducible".
> >
> > And now, let's the flames start! :-)
> >
> > Thomas
>
> --
> Vincent Fazio
> Embedded Software Engineer - Linux
> Extreme Engineering Solutions, Inc
> http://www.xes-inc.com
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot



-- 
| Michael Nazzareno Trimarchi                     Amarula Solutions BV |
| COO  -  Founder                                      Cruquiuskade 47 |
| +31(0)851119172                                 Amsterdam 1018 AM NL |
|                  [`as] http://www.amarulasolutions.com               |



More information about the buildroot mailing list