tree-wide: remove Emacs lines from all files This should be handled fine now by .dir-locals.el, so need to carry that stuff in every file.

journal: add dst_allocated_size parameter for compress_blob compress_blob took src, src_size, dst and *dst_size, but dst_size wasn't used as an input parameter with the size of dst, but only as an output parameter. dst was implicitly assumed to be at least src_size-1. This code wasn't *wrong*, because the only real caller in journal-file.c got it right. But it was misleading, and the tests in test-compress.c got it wrong, and worked only because the output buffer happened to be the same size as input buffer. So add a seperate dst_allocated_size parameter to make it explicit what the size of the buffer is, and to allow test to proceed with different output buffer sizes.

util-lib: split out allocation calls into alloc-util.[ch]

util-lib: split string parsing related calls from util.[ch] into parse-util.[ch]

util-lib: split our string related calls from util.[ch] into its own file string-util.[ch] There are more than enough calls doing string manipulations to deserve its own files, hence do something about it. This patch also sorts the #include blocks of all files that needed to be updated, according to the sorting suggestions from CODING_STYLE. Since pretty much every file needs our string manipulation functions this effectively means that most files have sorted #include blocks now. Also touches a few unrelated include files.

test-compress-benchmark: properly initialize buffer We were compressing unitialized memory, which should not result in any problems, but is inelegant.

test-compress-benchmark: limit default runtime to 2 seconds per subtest If both lz4 and xz are enabled, this results in a limit of 2×3×2 s ~= 12 s runtime. Previous implementation started with really small buffer sizes. When combined with a short time limit this resulteded in abysmal results for xz. It seems that the initialization overead is really significant for small buffers. Since xz will not be used by default anymore, this does not seem worth fixing. Instead buffer sizes are changed to run a pseudo-random non-repeating pattern. This should allow reasonable testing for all buffer sizes. For testing, both runtime and the buffer size seed can be specified on the command line. Sufficiently large runtime allows all buffer sizes up to 1MB to be tested.

test-compress-benchmark: test three cases (zeros, simple, semi-random) Existing test would use highly-compressible repeatable input. Two types of input are added: - zeros - random blocks interspersed with zeros The idea is to get more information about behaviour in various cases. On Intel Xeon the results are: % ./test-compress-benchmark XZ/zeros: compressed & decompressed 2535301373 bytes in 32.56s (74.26MiB/s), mean compresion 99.96%, skipped 3160 bytes LZ4/zeros: compressed & decompressed 2535304362 bytes in 1.16s (2088.69MiB/s), mean compresion 99.60%, skipped 171 bytes XZ/simple: compressed & decompressed 2535300963 bytes in 30.42s (79.48MiB/s), mean compresion 99.95%, skipped 3570 bytes LZ4/simple: compressed & decompressed 2535303543 bytes in 1.22s (1978.86MiB/s), mean compresion 99.60%, skipped 990 bytes XZ/random: compressed & decompressed 381756649 bytes in 60.02s (6.07MiB/s), mean compresion 39.64%, skipped 27813723 bytes LZ4/random: compressed & decompressed 2507385036 bytes in 0.97s (2477.52MiB/s), mean compresion 54.77%, skipped 27919497 bytes If someone has ideas for more realistic test cases, they can be easily added to this framework.

tests: use assert_se instead of assert Otherwise they can be optimized away with -DNDEBUG

Fix a few more typos

Fix misuse of uint64_t as size_t They have different size on 32 bit, so they are really not interchangable.

test-compress-benchmark: add missing % before PRIu64 format

test-compress-benchmark: silence warnings and btw make it pass for 32bits where size_t != uint64_t

compress: add benchmark-style test This is useful to test the behaviour of the compressor for various buffer sizes. Time is limited to a minute per compression, since otherwise, when LZ4 takes more than a second which is necessary to reduce the noise, XZ takes more than 10 minutes. % build/test-compress-benchmark (without time limit) XZ: compressed & decompressed 2535300963 bytes in 794.57s (3.04MiB/s), mean compresion 99.95%, skipped 3570 bytes LZ4: compressed & decompressed 2535303543 bytes in 1.56s (1550.07MiB/s), mean compresion 99.60%, skipped 990 bytes % build/test-compress-benchmark (with time limit) XZ: compressed & decompressed 174321481 bytes in 60.02s (2.77MiB/s), mean compresion 99.76%, skipped 3570 bytes LZ4: compressed & decompressed 2535303543 bytes in 1.63s (1480.83MiB/s), mean compresion 99.60%, skipped 990 bytes It appears that there's a bug in lzma_end where it leaks 32 bytes.