Commit Graph

18 Commits

Author SHA1 Message Date
Linus Torvalds
01cfb7937a squashfs: be more careful about metadata corruption
Anatoly Trosinenko reports that a corrupted squashfs image can cause a
kernel oops.  It turns out that squashfs can end up being confused about
negative fragment lengths.

The regular squashfs_read_data() does check for negative lengths, but
squashfs_read_metadata() did not, and the fragment size code just
blindly trusted the on-disk value.  Fix both the fragment parsing and
the metadata reading code.

Reported-by: Anatoly Trosinenko <anatoly.trosinenko@gmail.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Phillip Lougher <phillip@squashfs.org.uk>
Cc: stable@kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-07-29 12:44:46 -07:00
Sean Purcell
87bf54bb43 squashfs: Add zstd support
Add zstd compression and decompression support to SquashFS. zstd is a
great fit for SquashFS because it can compress at ratios approaching xz,
while decompressing twice as fast as zlib. For SquashFS in particular,
it can decompress as fast as lzo and lz4. It also has the flexibility
to turn down the compression ratio for faster compression times.

The compression benchmark is run on the file tree from the SquashFS archive
found in ubuntu-16.10-desktop-amd64.iso [1]. It uses `mksquashfs` with the
default block size (128 KB) and and various compression algorithms/levels.
xz and zstd are also benchmarked with 256 KB blocks. The decompression
benchmark times how long it takes to `tar` the file tree into `/dev/null`.
See the benchmark file in the upstream zstd source repository located under
`contrib/linux-kernel/squashfs-benchmark.sh` [2] for details.

I ran the benchmarks on a Ubuntu 14.04 VM with 2 cores and 4 GiB of RAM.
The VM is running on a MacBook Pro with a 3.1 GHz Intel Core i7 processor,
16 GB of RAM, and a SSD.

| Method         | Ratio | Compression MB/s | Decompression MB/s |
|----------------|-------|------------------|--------------------|
| gzip           |  2.92 |               15 |                128 |
| lzo            |  2.64 |              9.5 |                217 |
| lz4            |  2.12 |               94 |                218 |
| xz             |  3.43 |              5.5 |                 35 |
| xz 256 KB      |  3.53 |              5.4 |                 40 |
| zstd 1         |  2.71 |               96 |                210 |
| zstd 5         |  2.93 |               69 |                198 |
| zstd 10        |  3.01 |               41 |                225 |
| zstd 15        |  3.13 |             11.4 |                224 |
| zstd 16 256 KB |  3.24 |              8.1 |                210 |

This patch was written by Sean Purcell <me@seanp.xyz>, but I will be
taking over the submission process.

[1] http://releases.ubuntu.com/16.10/
[2] https://github.com/facebook/zstd/blob/dev/contrib/linux-kernel/squashfs-benchmark.sh

zstd source repository: https://github.com/facebook/zstd

Signed-off-by: Sean Purcell <me@seanp.xyz>
Signed-off-by: Nick Terrell <terrelln@fb.com>
Signed-off-by: Chris Mason <clm@fb.com>
Acked-by: Phillip Lougher <phillip@squashfs.org.uk>
2017-09-08 19:33:25 -07:00
Phillip Lougher
9c06a46f15 Squashfs: add LZ4 compression support
Add support for reading file systems compressed with the
LZ4 compression algorithm.

This patch adds the LZ4 decompressor wrapper code.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
2014-11-27 07:44:11 +00:00
Phillip Lougher
9e01242386 Squashfs: add corruption check for type in squashfs_readdir()
We read the type field from disk.  This value should be sanity
checked for correctness to avoid an out of bounds access when
reading the squashfs_filetype_table array.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
2013-09-06 04:57:54 +01:00
Phillip Lougher
47f4396e51 Squashfs: get rid of obsolete definitions in header file
Most of these were never used by the kernel code, but belong to
the time when the header file was used by both the kernel code
and the user space tools.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
2012-03-10 03:01:01 +00:00
Ajeet Yadav
4826d83d9c Squashfs: use define instead of constant
Its better to use defined name instead of constant

Signed-off-by: Ajeet Yadav <ajeet.yadav.77@gmail.com>
Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
2012-03-10 03:00:54 +00:00
Phillip Lougher
7657cacf47 Squashfs: Add an option to set dev block size to 4K
This commit adds an option to set the device block size used to 4K.

By default Squashfs sets the device block size (sb_min_blocksize) to 1K
or the smallest block size supported by the block device (if larger).
This, because blocks are packed together and unaligned in Squashfs,
should reduce latency.

This, however, gives poor performance on MTD NAND devices where
the optimal I/O size is 4K (even though the devices can support
smaller block sizes).

Using a 4K device block size may also improve overall I/O
performance for some file access patterns (e.g. sequential
accesses of files in filesystem order) on all media.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
2011-11-02 17:25:50 +00:00
Phillip Lougher
d7f2ff6718 Squashfs: update email address
My existing email address may stop working in a month or two, so update
email to one that will continue working.

Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
2011-05-26 10:49:11 +01:00
Phillip Lougher
b7fc0ff09d Squashfs: extend decompressor framework to handle compression options
Extend decompressor framework to handle compression options stored in
the filesystem.  These options can be used by the relevant decompressor
at initialisation time to over-ride defaults.

The presence of compression options in the filesystem is indicated by
the COMP_OPT filesystem flag.  If present the data is read from the
filesystem and passed to the decompressor init function.  The decompressor
init function signature has been extended to take this data.

Also update the init function signature in the glib, lzo and xz
decompressor wrappers.

Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
2011-02-28 18:21:59 +00:00
Phillip Lougher
81bb8debd0 Squashfs: add XZ compression support
Add support for reading file systems compressed with the
XZ compression algorithm.

This patch adds the XZ decompressor wrapper code.

Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
2011-01-13 20:51:20 +00:00
Phillip Lougher
66048c381b Squashfs: fix checkpatch.pl warnings
Checkpatch.pl in 2.6.34 added a check for spaces between tabs.

Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
2010-08-08 22:29:33 +00:00
Phillip Lougher
67f66cc6c7 squashfs: add new extended inode types
Add new extended inode types that store the xattr_id field.
Also add the necessary code changes to make xattrs visibile.

Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
2010-05-17 19:54:06 +01:00
Phillip Lougher
f41d207cbe squashfs: add support for xattr reading
Add support for listxattr and getxattr.  Also add xattr definitions.

Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
2010-05-17 19:54:06 +01:00
Phillip Lougher
4b5397dc24 squashfs: add xattr id support
This patch adds support for mapping xattr ids (stored in inodes)
into the on-disk location of the xattrs themselves.

Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
2010-05-17 19:54:05 +01:00
Phillip Lougher
06862f884d Squashfs: get rid of obsolete definition in header file
Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
2010-03-05 15:35:35 +00:00
Phillip Lougher
dc3256782f Squashfs: add decompressor entries for lzma and lzo
Add knowledge of lzma/lzo compression formats to the decompressor
framework.  For now these are added as unsupported.  Without
these entries lzma/lzo compressed filesystems will be flagged as
having unknown compression which is undesirable.

Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
2010-01-20 21:47:48 +00:00
Qinghuang Feng
1bcbf31337 btrfs & squashfs: Move btrfs and squashfsto's magic number to <linux/magic.h>
Use the standard magic.h for btrfs and squashfs.

Signed-off-by: Qinghuang Feng <qhfeng.kernel@gmail.com>
Cc: Phillip Lougher <phillip@lougher.demon.co.uk>
Cc: Chris Mason <chris.mason@oracle.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2009-01-15 16:39:38 -08:00
Phillip Lougher
ffae2cd73a Squashfs: header files
Signed-off-by: Phillip Lougher <phillip@lougher.demon.co.uk>
2009-01-05 08:46:27 +00:00