You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
209 lines
6.7 KiB
209 lines
6.7 KiB
erofs-utils
|
|
===========
|
|
|
|
erofs-utils includes user-space tools for erofs filesystem.
|
|
Currently mkfs.erofs and erofsfuse (experimental) are available.
|
|
|
|
Dependencies & build
|
|
--------------------
|
|
|
|
lz4 1.8.0+ for lz4 enabled [2], lz4 1.9.3+ highly recommended [4][5].
|
|
|
|
libfuse 2.6+ for erofsfuse enabled as a plus.
|
|
|
|
How to build for lz4-1.9.0 or above
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
To build, you can run the following commands in order:
|
|
|
|
::
|
|
|
|
$ ./autogen.sh
|
|
$ ./configure
|
|
$ make
|
|
|
|
mkfs.erofs binary will be generated under mkfs folder.
|
|
|
|
* For lz4 < 1.9.2, there are some stability issues about
|
|
LZ4_compress_destSize(). (lz4hc isn't impacted) [3].
|
|
|
|
** For lz4 = 1.9.2, there is a noticeable regression about
|
|
LZ4_decompress_safe_partial() [5], which impacts erofsfuse
|
|
functionality for legacy images (without 0PADDING).
|
|
|
|
How to build for lz4-1.8.0~1.8.3
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
For these old lz4 versions, lz4hc algorithm cannot be supported
|
|
without lz4-static installed due to LZ4_compress_HC_destSize()
|
|
unstable api usage, which means lz4 will only be available if
|
|
lz4-static isn't found.
|
|
|
|
On Fedora, lz4-static can be installed by using:
|
|
|
|
yum install lz4-static.x86_64
|
|
|
|
However, it's still not recommended using those versions directly
|
|
since there are serious bugs in these compressors, see [2] [3] [4]
|
|
as well.
|
|
|
|
mkfs.erofs
|
|
----------
|
|
|
|
two main kinds of erofs images can be generated: (un)compressed.
|
|
|
|
- For uncompressed images, there will be none of compression
|
|
files in these images. However, it can decide whether the tail
|
|
block of a file should be inlined or not properly [1].
|
|
|
|
- For compressed images, it will try to use lz4(hc) algorithm
|
|
first for each regular file and see if storage space can be
|
|
saved with compression. If not, fallback to an uncompressed
|
|
file.
|
|
|
|
How to generate erofs images
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Currently lz4 and lz4hc are available for compression, e.g.
|
|
$ mkfs.erofs -zlz4hc foo.erofs.img foo/
|
|
|
|
Or leave all files uncompressed as an option:
|
|
$ mkfs.erofs foo.erofs.img foo/
|
|
|
|
How to generate legacy erofs images
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Decompression inplace and compacted indexes have been introduced in
|
|
linux-5.3, which are not forward-compatible with older kernels.
|
|
|
|
In order to generate _legacy_ erofs images for old kernels,
|
|
consider adding "-E legacy-compress" to the command line, e.g.
|
|
|
|
$ mkfs.erofs -E legacy-compress -zlz4hc foo.erofs.img foo/
|
|
|
|
Obsoleted erofs.mkfs
|
|
~~~~~~~~~~~~~~~~~~~~
|
|
|
|
There is an original erofs.mkfs version developed by Li Guifu,
|
|
which was replaced by the new erofs-utils implementation.
|
|
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git -b obsoleted_mkfs
|
|
|
|
PLEASE NOTE: This version is highly _NOT recommended_ now.
|
|
|
|
erofsfuse (experimental, unstable)
|
|
----------------------------------
|
|
|
|
erofsfuse is introduced to support erofs format for various platforms
|
|
(including older linux kernels) and new on-disk features iteration.
|
|
It can also be used as an unpacking tool for unprivileged users.
|
|
|
|
It supports fixed-sized output decompression *without* any in-place
|
|
I/O or in-place decompression optimization. Also like the other FUSE
|
|
implementations, it suffers from most common performance issues (e.g.
|
|
significant I/O overhead, double caching, etc.)
|
|
|
|
Therefore, NEVER use it if performance is the top concern.
|
|
|
|
Note that xattr & ACL aren't implemented yet due to the current Android
|
|
use-case vs limited time. If you have some interest, contribution is,
|
|
as always, welcome.
|
|
|
|
How to build erofsfuse
|
|
~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
It's disabled by default as an experimental feature for now, to enable
|
|
and build it manually:
|
|
|
|
$ ./configure --enable-fuse
|
|
$ make
|
|
|
|
erofsfuse binary will be generated under fuse folder.
|
|
|
|
How to mount an erofs image with erofsfuse
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
As the other FUSE implementations, it's quite simple to mount with
|
|
erofsfuse, e.g.:
|
|
$ erofsfuse foo.erofs.img foo/
|
|
|
|
Alternatively, to make it run in foreground (with debugging level 3):
|
|
$ erofsfuse -f --dbglevel=3 foo.erofs.img foo/
|
|
|
|
To debug erofsfuse (also automatically run in foreground):
|
|
$ erofsfuse -d foo.erofs.img foo/
|
|
|
|
To unmount an erofsfuse mountpoint as a non-root user:
|
|
$ fusermount -u foo/
|
|
|
|
Contribution
|
|
------------
|
|
|
|
erofs-utils is under GPLv2+ as a part of erofs project,
|
|
feel free to send patches or feedback to us.
|
|
|
|
To:
|
|
linux-erofs mailing list <linux-erofs@lists.ozlabs.org>
|
|
Li Guifu <bluce.liguifu@huawei.com>
|
|
Miao Xie <miaoxie@huawei.com>
|
|
Fang Wei <fangwei1@huawei.com>
|
|
|
|
Cc:
|
|
Gao Xiang <xiang@kernel.org>
|
|
Chao Yu <yuchao0@huawei.com>
|
|
|
|
Comments
|
|
--------
|
|
|
|
[1] According to the erofs on-disk format, the tail block of files
|
|
could be inlined aggressively with its metadata in order to reduce
|
|
the I/O overhead and save the storage space (called tail-packing).
|
|
|
|
[2] There was a bug until lz4-1.8.3, which can crash erofs-utils
|
|
randomly. Fortunately bugfix by our colleague Qiuyang Sun was
|
|
merged in lz4-1.9.0.
|
|
|
|
For more details, please refer to
|
|
https://github.com/lz4/lz4/commit/660d21272e4c8a0f49db5fc1e6853f08713dff82
|
|
|
|
[3] There were many bugfixes merged into lz4-1.9.2 for
|
|
LZ4_compress_destSize(), and I once ran into some crashs due to
|
|
those issues. * Again lz4hc is not affected. *
|
|
|
|
[LZ4_compress_destSize] Allow 2 more bytes of match length
|
|
https://github.com/lz4/lz4/commit/690009e2c2f9e5dcb0d40e7c0c40610ce6006eda
|
|
|
|
[LZ4_compress_destSize] Fix rare data corruption bug
|
|
https://github.com/lz4/lz4/commit/6bc6f836a18d1f8fd05c8fc2b42f1d800bc25de1
|
|
|
|
[LZ4_compress_destSize] Fix overflow condition
|
|
https://github.com/lz4/lz4/commit/13a2d9e34ffc4170720ce417c73e396d0ac1471a
|
|
|
|
[LZ4_compress_destSize] Fix off-by-one error in fix
|
|
https://github.com/lz4/lz4/commit/7c32101c655d93b61fc212dcd512b87119dd7333
|
|
|
|
[LZ4_compress_destSize] Fix off-by-one error
|
|
https://github.com/lz4/lz4/commit/d7cad81093cd805110291f84d64d385557d0ffba
|
|
|
|
since upstream lz4 doesn't have stable branch for old versions, it's
|
|
preferred to use latest upstream lz4 library (although some regressions
|
|
could happen since new features are also introduced to latest upstream
|
|
version as well) or backport all stable bugfixes to old stable versions,
|
|
e.g. our unofficial lz4 fork: https://github.com/erofs/lz4
|
|
|
|
[4] LZ4HC didn't compress long zeroed buffer properly with
|
|
LZ4_compress_HC_destSize()
|
|
https://github.com/lz4/lz4/issues/784
|
|
|
|
which has been resolved in
|
|
https://github.com/lz4/lz4/commit/e7fe105ac6ed02019d34731d2ba3aceb11b51bb1
|
|
|
|
and already included in lz4-1.9.3, see:
|
|
https://github.com/lz4/lz4/releases/tag/v1.9.3
|
|
|
|
[5] LZ4_decompress_safe_partial is broken in 1.9.2
|
|
https://github.com/lz4/lz4/issues/783
|
|
|
|
which is also resolved in lz4-1.9.3.
|
|
|