[Buildroot] [PATCH] support/scripts/cve.py: use fast ijson backend if available on old ijson versions
Peter Korsgaard
peter at korsgaard.com
Fri Apr 9 11:01:12 UTC 2021
ijson < 2.5 (as available in Debian 10) use the slow python backend by
default instead of the most efficient one available like modern ijson
versions, significantly slowing down cve checking. E.G.:
time ./support/scripts/pkg-stats --nvd-path ~/.nvd -p avahi --html foobar.html
Goes from
174,44s user 2,11s system 99% cpu 2:58,04 total
To
93,53s user 2,00s system 98% cpu 1:36,65 total
E.G. almost 2x as fast.
As a workaround, detect when the python backend is used and try to use a
more efficient one instead. Use the yajl2_cffi backend as recommended by
upstream, as it is most likely to work, and print a warning (and continue)
if we fail to load it.
The detection is slightly complicated by the fact that ijson.backends used
to be a reference to a backend module, but is nowadays a string (without the
ijson.backends prefix).
Signed-off-by: Peter Korsgaard <peter at korsgaard.com>
---
support/scripts/cve.py | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/support/scripts/cve.py b/support/scripts/cve.py
index 6e97ea193f..965fc2a466 100755
--- a/support/scripts/cve.py
+++ b/support/scripts/cve.py
@@ -28,6 +28,12 @@ import operator
try:
import ijson
+ # backend is a module in < 2.5, a string in >= 2.5
+ if 'python' in getattr(ijson.backend, '__name__', ijson.backend):
+ try:
+ import ijson.backends.yajl2_cffi as ijson
+ except ImportError:
+ sys.stderr.write('Warning: Using slow ijson python backend\n')
except ImportError:
sys.stderr.write("You need ijson to parse NVD for CVE check\n")
exit(1)
--
2.20.1
More information about the buildroot
mailing list