[Buildroot] [RFC PATCH] host-localedef: Compile against glibc
Samuel Mendoza-Jonas
sam at mendozajonas.com
Wed Aug 1 01:31:16 UTC 2018
On Tue, 2018-07-31 at 22:55 +0200, Thomas Petazzoni wrote:
> Hello Samuel,
>
> Thanks for taking a look into this!
>
> On Mon, 30 Jul 2018 15:39:36 +1000, Samuel Mendoza-Jonas wrote:
> > In glibc 2.27 the following change occured:
> > "Statically compiled applications attempting to load locales compiled
> > for the GNU C Library version 2.27 will fail and fall back to the
> > builtin C/POSIX locale."
>
> I don't really understand this explanation. Isn't the problem that
> glibc 2.27 doesn't read properly locale files generated by localedef
> from glibc 2.24 ?
>
> I kind of fail to parse the explanation given here :-/
I think it's a bit of both; 2.27 introduces new locale descriptions which
localedef can't handle, eg:
Generating locale ru_RU.UTF-8
/scratch/jenkins-workspace/op-build-upstream/output/host/powerpc64le-buildroot-linux-gnu/sysroot/usr/share/i18n/locales/ru_RU:125: LC_TIME:syntax error
/scratch/jenkins-workspace/op-build-upstream/output/host/powerpc64le-buildroot-linux-gnu/sysroot/usr/share/i18n/locales/ru_RU:149: LC_TIME:syntax error
And we fall back to the C locale at runtime (at least in my environment).
I'll stare at the glibc release notes / bug report a bit more and see if
I can make a more legible explanation.
>
> > This impacts us since upstream buildroot uses a localdef built against
> > an older eglibc release [0].
> > Update the host-localedef package to build against glibc 2.27.
> > Unfortunately we can't compile just the localedef utility, but even so
> > this does not increase the build time too significantly. eg:
> >
> > With glibc localedef:
> > real 33m11.278s
> > user 171m51.684s
> > sys 19m46.528s
> >
> > With eglibc localedef:
> > real 31m12.715s
> > user 155m13.616s
> > sys 16m23.504s
>
> Meh, two more minutes just to build some not so complicated user-space
> program :-/
>
> > -HOST_LOCALEDEF_CONF_ENV = CFLAGS="$(HOST_CFLAGS) -fgnu89-inline"
> > +# glibc is part of the toolchain so disable the toolchain dependency
> > +HOST_LOCALEDEF_ADD_TOOLCHAIN_DEPENDENCY = NO
>
> Not needed for host-localedef.
>
Thanks for noting this and the others below; I was sure a lot of this was
target-only but I figured someone would be able to notice a few of them
better than me :)
> > -# The makefile does not implement an install target
> > +HOST_LOCALEDEF_SUBDIR = build
> > +
> > +# Thumb build is broken, build in ARM mode
> > +ifeq ($(BR2_ARM_INSTRUCTIONS_THUMB),y)
>
> This is a target related option...
>
> > +HOST_LOCALEDEF_EXTRA_CFLAGS += -marm
>
> ... so it is not relevant to observe it when building a host program.
>
> > +endif
> > +
> > +# MIPS64 defaults to n32 so pass the correct -mabi if
> > +# we are using a different ABI. OABI32 is also used
> > +# in MIPS so we pass -mabi=32 in this case as well
> > +# even though it's not strictly necessary.
> > +ifeq ($(BR2_MIPS_NABI64),y)
> > +HOST_LOCALEDEF_EXTRA_CFLAGS += -mabi=64
> > +else ifeq ($(BR2_MIPS_OABI32),y)
> > +HOST_LOCALEDEF_EXTRA_CFLAGS += -mabi=32
> > +endif
>
> Same comment.
>
> > +ifeq ($(BR2_ENABLE_DEBUG),y)
> > +HOST_LOCALEDEF_EXTRA_CFLAGS += -g
> > +endif
>
> Same comment.
>
> > +
> > +# The stubs.h header is not installed by install-headers, but is
> > +# needed for the gcc build. An empty stubs.h will work, as explained
> > +# in http://gcc.gnu.org/ml/gcc/2002-01/msg00900.html. The same trick
> > +# is used by Crosstool-NG.
> > +ifeq ($(BR2_TOOLCHAIN_BUILDROOT_HOST_LOCALEDEF),y)
> > +define HOST_LOCALEDEF_ADD_MISSING_STUB_H
> > + mkdir -p $(STAGING_DIR)/usr/include/gnu
> > + touch $(STAGING_DIR)/usr/include/gnu/stubs.h
>
> STAGING_DIR is observed when building target packages, not host
> packages, so this seems useless.
>
> > +endef
> > +endif
> > +
> > +# Even though we use the autotools-package infrastructure, we have to
> > +# override the default configure commands for several reasons:
> > +#
> > +# 1. We have to build out-of-tree, but we can't use the same
> > +# 'symbolic link to configure' used with the gcc packages.
> > +#
> > +# 2. We have to execute the configure script with bash and not sh.
> > +#
> > +# Note that as mentionned in
> > +# http://patches.openembedded.org/patch/38849/, glibc must be
> > +# built with -O2, so we pass our own CFLAGS and CXXFLAGS below.
> > +define HOST_LOCALEDEF_CONFIGURE_CMDS
> > + mkdir -p $(@D)/build
> > + # Do the configuration
> > + (cd $(@D)/build; \
> > + $(HOST_CONFIGURE_OPTS) \
> > + CFLAGS="-O2 $(HOST_LOCALEDEF_EXTRA_CFLAGS)" CPPFLAGS="" \
> > + CXXFLAGS="-O2 $(HOST_LOCALEDEF_EXTRA_CFLAGS)" \
> > + $(SHELL) $(@D)/configure \
> > + ac_cv_path_BASH_SHELL=/bin/bash \
> > + libc_cv_forced_unwind=yes \
> > + libc_cv_ssp=no \
> > + --target=$(GNU_HOST_NAME) \
> > + --host=$(GNU_HOST_NAME) \
> > + --build=$(GNU_HOST_NAME) \
> > + --prefix=/usr \
>
> Should be --prefix=$(HOST_DIR)
Oh yes, will fix.
>
> > + $(if $(BR2_SOFT_FLOAT),--without-fp,--with-fp) \
> > + $(if $(BR2_x86_64),--enable-lock-elision) \
>
> Both BR2_SOFT_FLOAT and BR2_x86_64 are target options, so observing
> them when building a host package is almost always wrong.
>
> > + --with-pkgversion="Buildroot" \
> > + --without-cvs \
> > + --disable-profile \
> > + --without-gd \
> > + --enable-obsolete-rpc)
> > + $(HOST_LOCALEDEF_ADD_MISSING_STUB_H)
> > +endef
>
> With all this, aren't we building a full glibc for the host ?
>
> Can you use the locale/others and locale/install-others make targets
> instead ?
>
> See how they are doing it in PTXdist:
> https://git.pengutronix.de/cgit/ptxdist/tree/rules/host-localedef.make.
> Most likely using the locale/others make target will help cut the build
> time significantly.
Yes this is the contentious part, we're building a whole glibc. Unfortunately
if we want to build just the locale/others target like PTXdist we also need
their bonus patch to glibc:
https://git.pengutronix.de/cgit/ptxdist/tree/patches/localedef-glibc-2.27/0001-HACK-only-build-and-install-localedef.patch
Otherwise the build fails with, for example:
make[4]: *** No rule to make target '/scratch/builds/locales/build/host-localedef-glibc-2.27-57-g6c99e37f6fb640a50a3113b2dbee5d5389843c1e/build/elf/soinit.os', needed by '/scratch/builds/locales/build/host-localedef-glibc-2.27-57-g6c99e37f6fb640a50a3113b2dbee5d5389843c1e/build/libc.so'. Stop.
Makefile:215: recipe for target 'locale/others' failed
make[3]: *** [locale/others] Error 2
Makefile:9: recipe for target 'locale/others' failed
make[2]: *** [locale/others] Error 2
So the tradeoff is adding a few minutes of extra build time versus carrying
a bit of a hack to enable just the localedef/others target.
>
> Thanks!
>
> Thomas
More information about the buildroot
mailing list