[Buildroot] [PATCH 09/34] reproducibility/libglib2: allow removing codegen

[Gilles Chanteperdrix]

On Thu, May 12, 2016 at 10:05:36PM +0200, Arnout Vandecappelle wrote:
> On 05/10/16 21:42, Gilles Chanteperdrix wrote:
> > On Tue, May 10, 2016 at 01:40:30AM +0200, Arnout Vandecappelle wrote:
> >> On 05/08/16 22:25, Gilles Chanteperdrix wrote:
> >>> On Sat, May 07, 2016 at 11:04:25PM +0200, Arnout Vandecappelle wrote:
> >>>> On 04/30/16 09:49, Gilles Chanteperdrix wrote:
> >>>>> But this is not sufficient, compiling the python bytecode with the
> >>>>> same interpreter in different environments yields different binaries,
> >>>>
> >>>>   Er, this is worrisome... You are saying that we don't have a chance in hell of
> >>>> generating reproducible python bytecode?
> >>>
> >>> I am not a python specialist, but it seems to me four bytes in the
> >>> python generated bytecode are the build timestamp, so unless there
> >>> is a way to override it with SOURCE_DATE_EPOCH, I do not see that
> >>> possible.
> >>
> >>   I've checked the docs. What is saved is the timestamp of the .py file, not the
> >> build time.
> >
> > Mmmm I don't remember. I would run a compilation manually, twice at
> > a different time, to make sure that the problem is only the file
> > date.
> 
>   I did that - admittedly with just a few seconds difference. Both in 
> python2.7.11 and python3.5, they were identical when compiling a second time, 
> and different after touch'ing.
> 
> >
> >>
> >>   I think it would make sense to run 'touch -d @$(SOURCE_DATE_EPOCH)' on all
> >> files after patching to capture this aspect. This was also proposed for
> >> Fedora[1] (though there it was only for the .py files). Not sure what happened
> >> with that proposal in the end.
> >
> > I think I tried running "touch" before compiling, unfortunately
> > playing with file dates before running "make" is a bit like playing
> > with fire. For instance with autotools based projects for which
> > autoreconf is not run, the project may use versions of the autotools
> > not installed on the user machine, and because of file dates may
> > want to rerun autoconf or autmake and whine because the right
> > versions are not available. Doing this only for .py files is much
> > more reasonable.
> 
>   I tested this as well, with make-3.81 and make-4.04. Both of them do _not_ 
> rebuild if the timestamps are identical. And since the idea is to use touch -d 
> @$(SOURCE_DATE_EPOCH), all timestamps will be identical.

You checked both the pyo and the pyc?

> 
>   But it does indeed mean that if a package has a generated file with an earlier 
> date than the source files, it will now suddenly no longer be
> rebuilt.

Yes, that is another problem. But I tried it, and this is not the
one I had, the one I had was the contrary: the dates made make want
to rebuild some files (the autotools/automake files), whereas the
right versions of autoconf and automake were not installed, this was
a package that did not run autoreconf. The "make" tool is completely
based on file dates, so again, I think messing with file dates
before running make is a bad idea.

-- 
					    Gilles.
https://click-hack.org