[Buildroot] [PATCH] manual: add section about dealing efficiently with big image files

Thomas De Schampheleire patrickdepinguin at gmail.com
Mon Dec 16 07:32:08 UTC 2013


Hi Yann,

On Mon, Dec 16, 2013 at 12:05 AM, Yann E. MORIN <yann.morin.1998 at free.fr> wrote:
> From: "Yann E. MORIN" <yann.morin.1998 at free.fr>
>
> As reported by Ryan, it is not well-known that most tools can deal
> efficiently with big sparse files.
>
> Add a section in the manual about this, with tar and cp used as
> examples, and a hinting to the man pages for the others.
>
> Reported-by: Ryan Barnett <rjbarnet at rockwellcollins.com>
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998 at free.fr>
> Cc: Ryan Barnett <rjbarnet at rockwellcollins.com>
> Cc: Peter Korsgaard <jacmet at uclibc.org>
>
> ---
> changes v1 -> v2:
>   - remove 'dd' since it can be dangerous  (Peter)
>   - add a sentence that sparse files are to be used only on the build
>     machine, not while transferring to the target device  (Peter)
> ---
>  docs/manual/common-usage.txt | 38 ++++++++++++++++++++++++++++++++++++++
>  1 file changed, 38 insertions(+)
>
> diff --git a/docs/manual/common-usage.txt b/docs/manual/common-usage.txt
> index 1290dfc..736ff57 100644
> --- a/docs/manual/common-usage.txt
> +++ b/docs/manual/common-usage.txt
> @@ -100,3 +100,41 @@ or +g+++ for building helper-binaries on your host, then do
>  --------------------
>   $ make HOSTCXX=g++-4.3-HEAD HOSTCC=gcc-4.3-HEAD
>  --------------------
> +
> +Dealing efficiently with filesystem images
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +Filesystem images can get pretty big, depending on the filesystem you choose,
> +the number of packages, whether you provisionned free space... Yet, some

provisioned

> +locations in the filesystems images may just be _empty_ (eg. a long run of
> +'zeroes'); such a file is called a _sparse_ file.
> +
> +Most tools can handle sparse files efficiently, and will only store or write
> +those parts of a sparse file that are not empty.
> +
> +For example:
> +
> +* +tar+ accepts the +-S+ option to tell it to only store non-zero blocks
> +  of sparse files:
> +** +tar cf archive.tar -S [files...]+ will efficiently store sparse files
> +   in a tarball
> +** +tar xf archive.tar -S+ will efficiently store sparse files extracted
> +   from a tarball
> +
> +* +cp+ accepts the +--sparse=WHEN+ option (+WHEN+ is one of +auto+,
> +  +never+ or +always+):
> +** +cp --sparse=always source.file dest.file+ will make +dest.file+ a
> +   sparse file if +source.file+ has long runs of zeroes
> +
> +Other tools may have similar options. Please consult their own man pages.

I find the usage of 'own' strange here. Suggestions:
Please consult their man pages.
Please consult their respective man pages.


> +
> +You can use sparse files if you need to store the filesystem images (eg.
> +to transfer from one machine to another), of if you need to send them (eg.
> +to the Q&A team).
> +
> +Note however that flashing a filesystem image to a device while using the
> +sparse mode of +dd+ may result in a broken filesystem (eg. the block bitmap
> +of an ext2 filesystem may be corrupted; or, if you have sparse files in
> +your filesystem, those parts may not be all-zeroes when read back). You
> +should only use sparse files when handling files on the build machine, not
> +when transferring them to an actual device that will be used on the target.
> --


Best regards,
Thomas



More information about the buildroot mailing list