[Buildroot] [PATCHv6 0/4] Generate package size statistics
Thomas Petazzoni
thomas.petazzoni at free-electrons.com
Sat Oct 17 13:33:41 UTC 2015
Hello,
Here is the sixth iteration of the patches adding a graph-size
feature, which mainly allows to generate a pie chart of the size
contribution of each package to the overall root filesystem size.
This code is also available at:
http://git.free-electrons.com/users/thomas-petazzoni/buildroot/log/?h=size-stats-v6
Changes since v5:
- Rebased on top of the latest master.
- Added Reviewed-by/Tested-by from Yann on "pkg-generic: add
step_pkg_size global instrumentation hook".
- Added Tested-by from Yann on "support/scripts: add size-stats
script."
- Rename the "size-stats" target back to "graph-size" so that it
matches graph-depends and graph-build, even if graph-size also
generates some CSV files (which are more
by-products). Consequently, I've dropped the Tested-by/Reviewed-by
tags from Yann and Ryan.
- As suggested by Yann and Ryan, generate the CSV files in
$(O)/graphs/ rather than $(O)/build, since they are really products
of the "gaph-size" targets, and people are more likely to find them
here.
- Use the $(GRAPHS_DIR) variable instead of open-coding
$(BASE_DIR)/graphs/.
- Fix the wording of the manual regarding what graph-size is doing,
and fix the path to the generated PDF file. Also, take into account
the change of the Makefile target name, and the location of the
generated CSV files. All suggested by Yann E. Morin.
Changes since v4:
- Rebased on top of the latest master.
- Now that the skeleton logic has been moved to a regular Buildroot
package, remove all the skeleton specific handling in the
size-stats script, as well as the passing of the --skeleton-path
from the main Makefile to the size-stats script.
- Added Reviewed-by/Tested-by from Ryan Barnett on "pkg-generic: add
step_pkg_size global instrumentation hook".
- Rework the commit log of "pkg-generic: add step_pkg_size global
instrumentation hook" to not mention the Config.in option that no
longer exists, and instead explain why we are doing the data
collection unconditionally. Reported by Romain Naour.
- Display the total filesystem on the generated graph. Suggested by
Ryan Barnett.
- Call matplotlib.use('Agg') to make sure matplotlib doesn't try to
connect to a X server. Suggested by Claython Shotwell.
- Added Reviewed-by/Tested-by from Ryan Barnett on "Makefile:
implement a size-stats target".
- Minor tweaks in the manual, suggested by Ryan Barnett. Also added
Reviewed-by/Tested-by from Ryan on the manual patch.
Changes since v3:
- Remove the BR2_COLLECT_FILE_SIZE_STATS option, and collect the
necessary data unconditionally. This was suggested by Yann
E. Morin, who did some testing of the performance impact, and found
that it was so insignificant that it is negligible. See Yann's
analysis at
http://lists.busybox.net/pipermail/buildroot/2015-February/119431.html.
- Store the temporary files listing the files before/after inside
each package build directory, rather than in $(BUILD_DIR)
directly. This was suggested by Yann E. Morin.
They are now named .br_filelist_before and .br_filelist_after. They
are also no longer removed after the step_pkg_size_end hook, as
they don't consume that much space and could be useful for
analysis. If anyone finds that annoying, I'll be happy to change
this back to remove them.
- As suggested by Arnout, make a number of improvements to the
size-stats Python script:
* Use collections.defaultdict()
* Use a dict of tuples (package name, file size) for the filesdict,
instead of a dict of dict containing just two entries.
* Factorize code between build_skeleton_dict() and
build_package_dict() into a add_file() function.
* Use [2:] instead of .replace("./", "") to remove the leading "./"
in relative file paths returned by find.
* Many other small things: one digit for percentages, use sum()
when possible, rename pkgdict to filesdict where it makes sense,
etc.
- In the main Makefile:
* Use mkdir -p instead of install -d, as suggested by Arnout.
* Use $(Q) instead of @, as suggested by Arnout
* Do not cd into $(CONFIG_DIR), as it would break out of tree
build, since the path to the skeleton is passed as a relative
path.
- Add a new patch updating the manual with a new section describing
this feature.
Changes since v2:
- Use print0 in find and -0 in xargs to support spaces in filenames,
just in case. Suggested by Jérôme Pouiller.
- Support custom skeleton locations, instead of assuming the skeleton
comes from system/skeleton.
- Fix the size-stats target to actually call the size-stats script,
and not graph-size, which was the old name of the script in
previous versions of the series.
- Fix usage of the size-stats target in out of tree build
situations. For consistency reasons, we chose to mimic what
graph-depends already does, even if size-stats does not necessarily
need to be executed from within CONFIG_DIR.
- Add one patch to make sure that 'make clean' removes the graphs/
subdirectory in the output directory, which wasn't done today. This
issue isn't specific to size-stats, and already existed with
graph-build and graph-depends.
Changes since v1:
- Improve the logic to properly take into account packages that
overwrite files installed by other packages.
- Make the new pkg_step hook optional.
- Extend the facility to not only generate a graph, but also CSV
files. As a consequence, everything was renamed from 'graph-size'
to 'size-stats'.
Best regards,
Thomas
Thomas Petazzoni (4):
pkg-generic: add step_pkg_size global instrumentation hook
support/scripts: add size-stats script
Makefile: implement a graph-size target
docs/manual: add section about size graphing
Makefile | 8 ++
docs/manual/common-usage.txt | 41 ++++++++
package/pkg-generic.mk | 32 +++++++
support/scripts/size-stats | 217 +++++++++++++++++++++++++++++++++++++++++++
4 files changed, 298 insertions(+)
create mode 100755 support/scripts/size-stats
--
2.6.2
More information about the buildroot
mailing list