[Buildroot] [git commit branch/2021.05.x] support/testing: don't fail on tests emitting invalid utf-8 sequences
Peter Korsgaard
peter at korsgaard.com
Wed Oct 6 14:56:48 UTC 2021
commit: https://git.buildroot.net/buildroot/commit/?id=541dc3fafdd421c09a38a78bf3b4ab270e4d5c09
branch: https://git.buildroot.net/buildroot/commit/?id=refs/heads/2021.05.x
When booting under EFI, grub2 will output a nice and shiny boot menu,
using extended ASCII characters (in the [0x80..0xFF] range), namely
CP437 [0], on the assumption that the VGA BIOS is a real one and has the
corresponding (and only!) font, as is the case on real hardware.
However, when run in our runtime test infrastructure, this triggers the
infamous python UnicodeDecodeError exception:
Traceback (most recent call last):
[...]
emulator.login()
File "[...]/buildroot/support/testing/infra/emulator.py", line 89, in login
index = self.qemu.expect(["buildroot login:", pexpect.TIMEOUT],
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 340, in expect
return self.expect_list(compiled_pattern_list,
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 369, in expect_list
return exp.expect_loop(timeout)
File "/usr/lib/python3/dist-packages/pexpect/expect.py", line 111, in expect_loop
incoming = spawn.read_nonblocking(spawn.maxread, timeout)
File "/usr/lib/python3/dist-packages/pexpect/pty_spawn.py", line 485, in read_nonblocking
return super(spawn, self).read_nonblocking(size)
File "/usr/lib/python3/dist-packages/pexpect/spawnbase.py", line 178, in read_nonblocking
s = self._decoder.decode(s, final=False)
File "/usr/lib/python3.8/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xda in position 0: invalid continuation byte
Grub2 is not wrong in emitting those chars, and basically we should not
expect the packages we test to always emit correct UTF-8 sequences; at
the very least, this should not cause the test infra to fail.
We fix that by telling pexpect.spawn to "fix" such invalid sequences by
replacing them with the suitable Unicode character, U+FFFD REPLACEMENT
CHARACTER.
[0] https://en.wikipedia.org/wiki/Code_page_437
[1] https://docs.python.org/3/library/codecs.html#error-handlers
Signed-off-by: Kory Maincent <kory.maincent at bootlin.com>
[yann.morin.1998 at free.fr:
- don't change encoding, use codec_errors
- rewrite commit log accordingly
]
Signed-off-by: Yann E. MORIN <yann.morin.1998 at free.fr>
(cherry picked from commit d6d7cbb8e047f86bdf626fec90f10e8590b7adf7)
Signed-off-by: Peter Korsgaard <peter at korsgaard.com>
---
support/testing/infra/emulator.py | 1 +
1 file changed, 1 insertion(+)
diff --git a/support/testing/infra/emulator.py b/support/testing/infra/emulator.py
index 5611ec96e8..2412fe1cc9 100644
--- a/support/testing/infra/emulator.py
+++ b/support/testing/infra/emulator.py
@@ -77,6 +77,7 @@ class Emulator(object):
self.qemu = pexpect.spawn(qemu_cmd[0], qemu_cmd[1:],
timeout=5 * self.timeout_multiplier,
encoding='utf-8',
+ codec_errors='replace',
env={"QEMU_AUDIO_DRV": "none"})
# We want only stdout into the log to avoid double echo
self.qemu.logfile_read = self.logfile
More information about the buildroot
mailing list