tree-wide: remove Emacs lines from all files This should be handled fine now by .dir-locals.el, so need to carry that stuff in every file.

journalctl: add match for the current boot when called with devpath

journalctl: make "journalctl /dev/sda" work Currently when journalctl is called with path to block device node we add following match _KERNEL_DEVICE=b$MAJOR:$MINOR. That is not sufficient to actually obtain logs about the disk because dev_printk() kernel helper puts to /dev/kmsg information about the device in following format, +$SUBSYSTEM:$ADDRESS, e.g. "+pci:pci:0000:00:14.0". Now we will walk upward the syspath and add match for every device in format produced by dev_printk() as well as match for its device node if it exists.

journalctl: add new --fields switch to dump all currently used field names Fixes #2176

journalctl: improve error messages when the specified boot is not found

journalctl: show friendly info when using -b on runtime journal only Make it clear that specifing boot when there is actually only one has no effect. This cosmetic patch improves user experience a bit.

basic/terminal-util: introduce SYSTEMD_COLORS environment variable ... to determine if color output should be enabled. If the variable is not set, fall back to using on_tty(). Also, rewrite existing code to use colors_enabled() where appropriate.

tree-wide: expose "p"-suffix unref calls in public APIs to make gcc cleanup easy GLIB has recently started to officially support the gcc cleanup attribute in its public API, hence let's do the same for our APIs. With this patch we'll define an xyz_unrefp() call for each public xyz_unref() call, to make it easy to use inside a __attribute__((cleanup())) expression. Then, all code is ported over to make use of this. The new calls are also documented in the man pages, with examples how to use them (well, I only added docs where the _unref() call itself already had docs, and the examples, only cover sd_bus_unrefp() and sd_event_unrefp()). This also renames sd_lldp_free() to sd_lldp_unref(), since that's how we tend to call our destructors these days. Note that this defines no public macro that wraps gcc's attribute and makes it easier to use. While I think it's our duty in the library to make our stuff easy to use, I figure it's not our duty to make gcc's own features easy to use on its own. Most likely, client code which wants to make use of this should define its own: #define _cleanup_(function) __attribute__((cleanup(function))) Or similar, to make the gcc feature easier to use. Making this logic public has the benefit that we can remove three header files whose only purpose was to define these functions internally. See #2008.

journalctl: don't print -- No entries -- in quiet mode

journald: rework --sync/--rotate logic to use CLOCK_MONOTONIC timestamp files Previously, we'd rely on the mtime timestamps of the touch files to see if our sync/rotation requests were already suppressed. This means we rely on CLOCK_REALTIME timestamps. With this patch we instead store the CLOCK_MONOTONIC timestamp *in* the touch files, and avoid relying on mtime. This should make things more reliable when the clock or underlying mtime granularity is not very good. This also adds warning messages if writing any of the flag files fails.

journalctl: change repeated if checks into switch blocks No functional changes.

journalctl: refuse to --machine= in combination with --flush, --sync or --rotate

journalctl: make --rotate synchronous, too Of course, ideally we'd just use normal synchronous bus calls, but this is out of the question as long as we rely on dbus-daemon (which logs to journald, and thus cannot use to avoid cyclic sync loops). Hence, instead, reuse the wait logic already implemented for --sync, and use a signal in one direction, and a mtime watch file for the reply.

journalctl: add new --sync switch for syncing the journal to disk With this new "--sync" switch we add a synchronous way to sync everything queued to disk, and return only after that's complete. This command gives the guarantee that anything queued before has hit the disk before the command returns. While we are at it, also improve the man pages and help text for journalctl a bit.

treewide: use the negative error codes returned by our functions Our functions return negative error codes. Do not rely on errno being set after calling our own functions.

journalctl: when we fail to open a journal file, print why When we enumerate journal files and encounter an invalid one, remember which this, and show it to the user. Note the possibly slightly surprising logic here: we store only one path per error code. This means we show all error kinds but not every actual error we encounter. This has the benefit of not requiring us to keep a potentially unbounded list of errors with their sources around, but can still provide a pretty complete overview on the errors we encountered. Fixes #1669.

journalctl: continue operation, even if we run into an invalid file

util-lib: split out allocation calls into alloc-util.[ch]

util-lib: split out globbing related calls into glob-util.[ch]

util-lib: split out syslog-related calls into syslog-util.[ch]

util-lib: move more locale-related calls to locale-util.[ch]

util-lib: move a number of fs operations into fs-util.[ch]

util-lib: split out file attribute calls to chattr-util.[ch]

util-lib: split out resource limits related calls into rlimit-util.[ch]

util-lib: split string parsing related calls from util.[ch] into parse-util.[ch]

util-lib: split out user/group/uid/gid calls into user-util.[ch]

util-lib: split out IO related calls to io-util.[ch]

util-lib: split out fd-related operations into fd-util.[ch] There are more than enough to deserve their own .c file, hence move them over.

path-util: unify how we process paths specified on the command line Let's introduce a common function that makes relative paths absolute and warns about any errors while doing so.

journalctl: clarify -q option -q suppresses info messages too

journalctl: introduce short options for --since and --until Fixes #1514.

journalctl: enable args for --dump-catalog and --list-catalog `journalctl --dump-catalog ID1 ID2 ...` works fine.

journalctl: open pager for --dump-catalog and --list-catalog

journal: rework vacuuming logic Implement a maximum limit on number of journal files to keep around. Enforcing a limit is useful on this since our performance when viewing pays a heavy penalty for each journal file to interleve. This setting is turned on now by default, and set to 100. Also, actully implement what 348ced909724a1331b85d57aede80a102a00e428 promised: use whatever we find on disk at startup as lower bound on how much disk space we can use. That commit introduced some provisions to implement this, but actually never did. This also adds "journalctl --vacuum-files=" to vacuum files on disk by their number explicitly.

tree-wide: remove a number of invocations of strerror() and replace by %m Let's clean up our tree a bit, and reduce invocations of the thread-unsafe strerror() by replacing it with printf()'s %m specifier.

journalctl: add --rotate option shortcut for `systemctl kill --kill-who main --signal SIGUSR2 systemd-journald`

bus-util: rename bus_open_transport() to bus_connect_transport() In sd-bus, the sd_bus_open_xyz() family of calls allocates a new bus, while sd_bus_default_xyz() family tries to reuse the thread's default bus. bus_open_transport() sometimes internally uses the former, sometimes the latter family, but suggests it only calls the former via its name. Hence, let's avoid this confusion, and generically rename the call to bus_connect_transport(). Similar for all related calls. And while we are at it, also change cgls + cgtop to do direct systemd connections where possible, since all they do is talk to systemd itself.

util: introduce common version() implementation and use it everywhere This also allows us to drop build.h from a ton of files, hence do so. Since we touched the #includes of those files, let's order them properly according to CODING_STYLE.

cgtop: underline table header Let's underline the header line of the table shown by cgtop, how it is customary for tables. In order to do this, let's introduce new ANSI underline macros, and clean up the existing ones as side effect.

tree-wide: never use the off_t unless glibc makes us use it off_t is a really weird type as it is usually 64bit these days (at least in sane programs), but could theoretically be 32bit. We don't support off_t as 32bit builds though, but still constantly deal with safely converting from off_t to other types and back for no point. Hence, never use the type anymore. Always use uint64_t instead. This has various benefits, including that we can expose these values directly as D-Bus properties, and also that the values parse the same in all cases.

tree-wide: make use of log_error_errno() return value in more cases The previous coccinelle semantic patch that improved usage of log_error_errno()'s return value, only looked for log_error_errno() invocations with a single parameter after the error parameter. Update the patch to handle arbitrary numbers of additional arguments.

tree-wide: make use of log_error_errno() return value Turns this: r = -errno; log_error_errno(errno, "foo"); into this: r = log_error_errno(errno, "foo"); and this: r = log_error_errno(errno, "foo"); return r; into this: return log_error_errno(errno, "foo");

journalctl: make sure 'journalctl -f -t unmatched' blocks Previously the following command: $ journalctl -f -t unmatchedtag12345 ... would block when called with criteria that did not match any journal lines. Once log lines appeared that matched the criteria they were displayed. Commit 02ab86c732576a71179ce12e97d44c289833236d broke this behavior and the journal was not followed, but the command exits with '-- No entries --' displayed. This commit fixes the issue. More information downstream: https://bugzilla.redhat.com/show_bug.cgi?id=1253649

hostname-util: get rid of unused parameter of hostname_cleanup() All users are now setting lowercase=false.

tree-wide: introduce mfree() Pretty trivial helper which wraps free() but returns NULL, so we can simplify this: free(foobar); foobar = NULL; to this: foobar = mfree(foobar);

journalctl: properly detect empty journal files When we encounter a journal file with exactly zero entries, print a nice message and exit, and don't print a weird error message.

sd-bus: introduce new sd_bus_flush_close_unref() call sd_bus_flush_close_unref() is a call that simply combines sd_bus_flush() (which writes all unwritten messages out) + sd_bus_close() (which terminates the connection, releasing all unread messages) + sd_bus_unref() (which frees the connection). The combination of this call is used pretty frequently in systemd tools right before exiting, and should also be relevant for most external clients, and is hence useful to cover in a call of its own. Previously the combination of the three calls was already done in the _cleanup_bus_close_unref_ macro, but this was only available internally. Also see #327

journalctl: unify how we free boot id lists a bit Instead of use LIST_FOREACH_SAFE, just use the same, seperate destructor everywhere.

journalctl: clean up how we log errors All functions should either log the errors they run into, or only return them in which case the caller should log them. Make sure this rule is followed, so that each error is logged precisely once, and neither never, nor more than once.

journalctl: rename boot_id_t to BootId So far we tried to reserve the _t suffix to types we use like a value in contrast to types we use as objects, hence let's do this in journalctl too.

journalctl: lstat() should suffice if we call canonicalize_file_name() first

journalctl: free all command line argument objects let's try to be valgrind clean

journalctl: only have a single exit path from main() That way we can be sure we execute the destructors properly, and can be valgrind-clean.

journalctl: Improve boot ID lookup This method should greatly improve offset based lookup, by simply jumping from one boot to the next boot. It starts at the journal head to get the a boot ID, makes a _BOOT_ID match and then comes from the opposite journal direction (tail) to get to the end that boot. After flushing the matches and advancing the journal from that exact position, we arrive at the start of next boot. Rinse and repeat. This is faster than the old method of aggregating the full boot listing just so we can jump to a specific boot, which can be a real pain on big journals just for a mere "-b -1" case. As an additional benefit --list-boots should improve slightly too, because it does less seeking. Note that there can be a change in boot order with this lookup method because it will use the order of boots in the journal, not the realtime stamp stored in them. That's arguably better, though. Another deficiency is that it will get confused with boots interleaving in the journal, therefore, it will refuse operation in --merge, --file and --directory mode. https://bugs.freedesktop.org/show_bug.cgi?id=72601

util: split all hostname related calls into hostname-util.c

core: rework unit name validation and manipulation logic A variety of changes: - Make sure all our calls distuingish OOM from other errors if OOM is not the only error possible. - Be much stricter when parsing escaped paths, do not accept trailing or leading escaped slashes. - Change unit validation to take a bit mask for allowing plain names, instance names or template names or an combination thereof. - Refuse manipulating invalid unit name

journalctl: rework code that checks whether we have access to /var/log/journal - fix some memory leaks on error conditions - handle all error cases properly, and log about failures - move HAVE_ACL and no-HAVE_ACL code closer to each other

shared: add terminal-util.[ch]

util: merge change_attr_fd() and chattr_fd()

fix gcc warnings about uninitialized variables like: src/shared/install.c: In function ‘unit_file_lookup_state’: src/shared/install.c:1861:16: warning: ‘r’ may be used uninitialized in this function [-Wmaybe-uninitialized] return r < 0 ? r : state; ^ src/shared/install.c:1796:13: note: ‘r’ was declared here int r; ^

tree-wide: there is no ENOTSUP on linux Replace ENOTSUP by EOPNOTSUPP as this is what linux actually uses.

Introduce loop_read_exact helper Usually when using loop_read(), we want to read the full buffer. Add a helper that mirrors loop_write(), and returns 0 when full buffer was read, and an error otherwise. Use -ENODATA for the short read, to distinguish it from a read error.

journalctl: unlink without checking with access first It is more elegant to do this in one step. Coverity complains about the TOCTOU difference, but it is not an actual problem (CID #1237777).

journalctl: update hint now that we set ACL everywhere

remove unused includes This patch removes includes that are not used. The removals were found with include-what-you-use which checks if any of the symbols from a header is in use.

util: rework strappenda(), and rename it strjoina() After all it is now much more like strjoin() than strappend(). At the same time, add support for NULL sentinels, even if they are normally not necessary.

tmpfiles: add 'a' type to set ACLs

journalctl: trim --help to fit in 80 columns Terminals tend to be 80 columns wide by default, and the help text is only supposed to be a terse reminder anyway. https://bugzilla.redhat.com/show_bug.cgi?id=1183771

journal: bump RLIMIT_NOFILE when journal files to 16K (if possible) When there are a lot of split out journal files, we might run out of fds quicker then we want. Hence: bump RLIMIT_NOFILE to 16K if possible. Do these even for journalctl. On Fedora the soft RLIMIT_NOFILE is at 1K, the hard at 4K by default for normal user processes, this code hence bumps this up for users to 4K. https://bugzilla.redhat.com/show_bug.cgi?id=1179980

journald: turn off COW for journal files on btrfs btrfs' COW logic results in heavily fragment journal files, which is detrimental for perfomance. Hence, turn off COW for journal files as we create them. Turning off COW comes at the cost of data integrity guarantees, but this should be acceptable, given that we do our own checksumming, and generally have a pretty conservative write pattern. Also see discussion on linux-btrfs: http://www.spinics.net/lists/linux-btrfs/msg41001.html

tree-wide: remove unnecessary LOG_PRI LOG_DEBUG is already a log level, there is no need to use LOG_PRI which is for filtering out the facility.

journal: install sigbus handler for journal tools too This makes them robust regarding truncation. Ideally, we'd export this as an API, but given how messy SIGBUS handling is, and the uncertain ownership logic of signal handlers we should not do this (unless libc one day invents a scheme how to sanely install SIGBUS handlers for specific memory areas only). However, for now we can still make all our own tools robust. Note that external tools will only have read-access to the journal anyway, where SIGBUS is much more unlikely, given that only writes are subject to disk full problems.

tree-wide: spelling fixes https://github.com/vlajos/misspell_fixer https://github.com/torstehu/systemd/commit/b6fdeb618cf2f3ce1645b3315f15f482710c7ffa Thanks to Torstein Husebo <torstein@huseboe.net>.

treewide: sanitize loop_write loop_write() didn't follow the usual systemd rules and returned status partially in errno and required extensive checks from callers. Some of the callers dealt with this properly, but many did not, treating partial writes as successful. Simplify things by conforming to usual rules.

journalctl: respect --after-cursor semantics with --follow in all cases In the case where no entries have been added to the journal after the specified cursor, set need_seek before the main loop to prevent display of the entry at said cursor.

treewide: another round of simplifications Using the same scripts as in f647962d64e "treewide: yet more log_*_errno + return simplifications".

treewide: use log_*_errno whenever %m is in the format string If the format string contains %m, clearly errno must have a meaningful value, so we might as well use log_*_errno to have ERRNO= logged. Using: find . -name '*.[ch]' | xargs sed -r -i -e \ 's/log_(debug|info|notice|warning|error|emergency)\((".*%m.*")/log_\1_errno(errno, \2/' Plus some whitespace, linewrap, and indent adjustments.

treewide: more log_*_errno + return simplifications

treewide: more log_*_errno() conversions, multiline calls Basically: find . -name '*.[ch]' | while read f; do perl -i.mmm -e \ 'local $/; local $_=<>; s/log_(debug|info|notice|warning|error|emergency)\("([^"]*)%s"([^;]*),\s*strerror\(-?([->a-zA-Z_]+)\)\);/log_\1_errno(\4, "\2%m"\3);/gms;print;' \ $f; done Plus manual indentation fixups.

treewide: more log_*_errno() conversions

treewide: no need to negate errno for log_*_errno() It corrrectly handles both positive and negative errno values.

treewide: auto-convert the simple cases to log_*_errno() As a followup to 086891e5c1 "log: add an "error" parameter to all low-level logging calls and intrdouce log_error_errno() as log calls that take error numbers", use sed to convert the simple cases to use the new macros: find . -name '*.[ch]' | xargs sed -r -i -e \ 's/log_(debug|info|notice|warning|error|emergency)\("(.*)%s"(.*), strerror\(-([a-zA-Z_]+)\)\);/log_\1_errno(-\4, "\2%m"\3);/' Multi-line log_*() invocations are not covered. And we also should add log_unit_*_errno().

journalctl: print all possible lines immediately with --follow + --since When I tryed to run journalctl with --follow and --since arguments it behaved very strangely. First It prints logs from what I specified in --since argument, then printed 10 lines (as is default in --follow) and when app put something new in to log journalctl printed everithing from the last printed line. How to reproduce: 1. run: journalctl -m --since 14:00 --follow Then you'll see 10 lines of logs since 14:00. After that wait until some app add something in the journal or just run `systemd-cat echo test` 2. After that journalctl will print every single line since 14:00 and will follow as expected. As long as --since and --follow will eventually print all relevant lines, I seen no reason why not to print them right away and not after first new message in journal. Relevant bugzillas: https://bugs.freedesktop.org/show_bug.cgi?id=71546 https://bugs.freedesktop.org/show_bug.cgi?id=64291

journalctl: add new --vacuum-size= and --vacuum-time= commands to clean up journal files based on a size/time limit This is equivalent to the effect of SystemMaxUse= and RetentionSec=, however can be invoked directly instead of implicitly.

journalctl: correct help text for --until http://bugs.debian.org/766598

journalctl: Unify boot id lookup into common function get_boots

journal: make sd_journal::files a OrderedHashmap Anything that uses hashmap_next() almost certainly cares about the order and needs to be an OrderedHashmap.

journalctl: add new --flush command and make use of it in systemd-journal-flush.service This new command will ask the journal daemon to flush all log data stored in /run to /var, and wait for it to complete. This is useful, so that in case of Storage=persistent we can order systemd-tmpfiles-setup afterwards, to ensure any possibly newly created directory in /var/log gets proper access mode and owners.

journalctl: don't introduce numeric constants with special names, give them names

journalctl: use pager for --list-boots

sd-journal: change check to assert As of 0f99f74a14 'sd-journal: verify that object start with the field name' this condition should never happen.

time: functions named "internal" really shouldn't be exported Also, let's try to make function names descriptive, instead of using bools for flags.

journalctl: make --utc work everywhere The --utc option was introduced by commit 9fd290443f5f99fca0dcd4216b1de70f7d3b8db1. Howerver, the implementation was incomplete.

journalctl: add --utc option Introduce option to display time in UTC.

journalctl: do not output --reboot-- markers when running non-interactively They are not legal in the export format.

hashmap: introduce hash_ops to make struct Hashmap smaller It is redundant to store 'hash' and 'compare' function pointers in struct Hashmap separately. The functions always comprise a pair. Store a single pointer to struct hash_ops instead. systemd keeps hundreds of hashmaps, so this saves a little bit of memory.

journalctl: Allow to disable line cap with --lines=all

journalctl: add "-t --identifier=STRING" option This turns journalctl to the counterpart of systemd-cat. Messages sent with systemd-cat --identifier foo --prioritiy debug can now be shown with journalctl --identifier foo --prioritiy debug "--identifier" is not merged with "--unit" to make a clear distinction between syslog and systemd units. syslog identifiers can be chosen freely by anyone.

Unify parse_argv style getopt is usually good at printing out a nice error message when commandline options are invalid. It distinguishes between an unknown option and a known option with a missing arg. It is better to let it do its job and not use opterr=0 unless we actually want to suppress messages. So remove opterr=0 in the few places where it wasn't really useful. When an error in options is encountered, we should not print a lengthy help() and overwhelm the user, when we know precisely what is wrong with the commandline. In addition, since help() prints to stdout, it should not be used except when requested with -h or --help. Also, simplify things here and there.

Always prefer our headers to system headers In practice this shouldn't make much difference, but sometimes our headers might be newer, and we want to test them.

Add utility function to append root to path

Always check asprintf return code There is a small number of the places in sources where we don't check asprintf() return code and assume that after error the function returns NULL pointer via the first argument. That's wrong, after error the content of pointer is undefined.

journalctl,man: allow + only between terms https://bugzilla.redhat.com/show_bug.cgi?id=1110712

journal-upload: use journal as the source

journalctl: free arg_file on exit

util: replace close_nointr_nofail() by a more useful safe_close() safe_close() automatically becomes a NOP when a negative fd is passed, and returns -1 unconditionally. This makes it easy to write lines like this: fd = safe_close(fd); Which will close an fd if it is open, and reset the fd variable correctly. By making use of this new scheme we can drop a > 200 lines of code that was required to test for non-negative fds or to reset the closed fd variable afterwards.

journalctl: refuse extra arguments with --verify and similar Positional arguments only make sense with the default action. For other actions, complain instead of ignoring them silently.

make gcc shut up If -flto is used then gcc will generate a lot more warnings than before, among them a number of use-without-initialization warnings. Most of them without are false positives, but let's make them go away, because it doesn't really matter.

doc: update punctuation Resolve spotted issues related to missing or extraneous commas, dashes.

always use the same code for creating temporary files Let's unify our code here, and also always specifiy O_CLOEXEC.

journalctl: allow globbing in --unit and --user-unit This is a continuation of e3e0314b systemctl: allow globbing in commands which take multiple unit names. Multiple patterns can be specified, as separate arguments, or as one argument with patterns seperated by commas. If patterns are given, at least one unit must be matched (by any of the patterns). This is different behaviour than systemctl, but here it is necessary because otherwise anything would be matched, which is unlikely to be the intended behaviour. https://bugs.freedesktop.org/show_bug.cgi?id=59336

Use enums to make it obvious what boolean params mean Suggested-by: Russ Allbery <rra@debian.org>

systemctl: allow globbing in commands which take multiple unit names

journalctl: make sure -b --foobar cannot be misunderstood as --boot=--foobar

Revert "journalctl: remove unexpected behavior of journalctl -b" This reverts commit cf5bccc2bb9569030cb04debbc4208aaca0fe5b4. We should fix thinks properly if they aren't perfect, not just break other things...

journalctl: remove unexpected behavior of journalctl -b This flag shouldn't try and consume the following argument. It should behave like every other flag which takes an optional argument when parsed by getopt_long.

journalctl,zsh-completion: fix several issues in --help message text -- fix grammar and reword some descriptions for clarity -- add a useful description of what --follow does -- fix the description for --after-cursor -- properly introduce the FSS acronym for "Forward Secure Sealing" in both sections -- clarify the --disk-usage command [zj: perform similar changes to zsh completions] squash! journalctl: fix several issues in --help message text

journal: pipe journalctl help output into a pager journalctl help output might run off the screen, so be consistent as other systemd tools do and pipe it into a pager.

journal: add ability to browse journals of running OS containers This adds the new library call sd_journal_open_container() and a new "-M" switch to journalctl. Particular care is taken that journalctl's "-b" switch resolves to the current boot ID of the container, not the host.

Get rid of our reimplementation of basename The only problem is that libgen.h #defines basename to point to it's own broken implementation instead of the GNU one. This can be fixed by #undefining basename.

Remove duplicate includes

clients: unify how we invoke getopt_long() Among other things this makes sure we always expose a --version command and show it in the help texts.

journalctl: add --list-boots to show boot IDs and times Suggested by David Wilkins <dwilkins@maths.tcd.ie> in https://bugzilla.redhat.com/show_bug.cgi?id=967521: > [Specific boot ID is a] bit of a palaver to obtain. I consulted the > verbose dump of the journal to discover the _BOOT_ID for the > timestamp, and then generated the journal dump for that boot using > journalctl _BOOT_ID=foo -o short-monotonic.

Never call qsort on potentially NULL arrays This extends 62678ded 'efi: never call qsort on potentially NULL arrays' to all other places where qsort is used and it is not obvious that the count is non-zero.

journalctl: flip to --full by default We already shew lines in full when using a pager or not on a tty. The commit disables ellipsization in the sole remaining case, namely when --follow is used. This has been a popular request for a long time, and indeed, full output seems much more useful. Old behaviour can still be requested by using --no-full. Old options retain their behaviour for compatiblity, but aren't advertised as much. This change applies only to jornalctl, not to systemctl, when ellipsization is useful to keep the layout. https://bugzilla.redhat.com/show_bug.cgi?id=984758

journald: be a bit more verbose when vacuuming Vacuuming behaviour is a bit confusing, and/or we have some bugs, so those additional messages should help to find out what's going on. Also, rotation of journal files shouldn't be happening too often, so the level of the messages is bumped to info, so that they'll be logged under normal operation.

logs-show: add short-precise mode with us timestamps Also, always show us timestamps in verbose mode. https://bugzilla.redhat.com/show_bug.cgi?id=991678

systemctl: show hint about --full when lines don't fit

systemd-delta: Only print colors when on a tty This make systemd-delta follow the behaviour of systemctl and journalctl. https://bugs.freedesktop.org/show_bug.cgi?id=67656 [zj: unify color query methods between those three programs.]

journalctl: use _COMM= match for scripts In case of scripts, _EXE is set to the interpreter name, and _COMM is set based on the file name. Add a match for _COMM, and _EXE if the interpreter is not a link (e.g. for yum, the interpreter is /usr/bin/python, but it is a link to /usr/bin/python2, which in turn is a link to /usr/bin/python2.7, at least on Fedora, so we end up with _EXE=/usr/bin/python2.7). I don't think that such link chasing makes sense, because the final _EXE name is more likely to change.

journalctl: add ”short-iso” output format with verbose ISO8601 timestamps Example: 2013-07-18T10:10:01+0200 sandworm CROND[20957]: (root) CMD (/usr/lib64/sa/sa1 1 1)

journalctl: fix signedness warning and boot-id syntax check

journalctl: remove ":" from the --boot syntax Instead of :-0, :1, :5, etc., use -0, 1 or +1, 5, etc. For BOOT_ID+OFFSET, use BOOT_ID+offset or BOOT_ID-offset (either + or - is required). Also make error handling a bit more robust and verbose. Modify the man page to describe the most common case (-b) first, and the second most common case (-b -1) second.

journalctl,systemctl: fix tiny memleak

journalctl: augment short mode with a cursor at the end Two options are added: --show-cursor to print the cursor at the end, and --after-cursor to resume logs on the next line after the previous one.

journalctl: Add support for showing messages from a previous boot Hi, I redid the boot ID look up to use enumerate_unique. This is quite fast if the cache is warm but painfully slow if it isn't. It has a slight chance of returning the wrong order if realtime clock jumps around. This one has to do n searches for every boot ID there is plus a sort, so it depends heavily on cache hotness. This is in contrast to the other way of look-up through filtering by a MESSAGE_ID, which only needs about 1 seek + whatever amount of relative IDs you want to walk. I also have a linked-list + (in-place) mergesort version of this patch, which has pretty much the same runtime. But since this one is using libc sorting and armortized allocation, I prefer this one. To summarize: The MESSAGE_ID way is a *lot* faster but can be incomplete due to rotation, while the enumerate+sort will find every boot ID out there but will be painfully slow for large journals and cold caches. You choose :P Jan

journalctl: add --force option to recreate FSS

journalctl: have a useful --setup-keys error message when using non-persistant logging Generating seed... Generating key pair... Generating sealing key... Failed to open /var/log/journal/33f46101703a10c5fc6fa4f451840101/fss.tmp.k2wDDU: No such file or directory

journalctl,loginctl,systemctl,systemd-cgls: add -l as alias for --full https://bugs.freedesktop.org/show_bug.cgi?id=65850

journalctl: allow the user to specify the file(s) to use This is useful for debugging and feels pretty natural. For example answering the question "is this big .journal file worth keeping?" is made easier.

journalctl: no color for --reboot-- when not on tty

journalctl: add --system/--user flags --user basically gives messages from your own systemd --user services. --system basically gives messages from PID 1, kernel, and --system services. Those two options are not exahustive, because a priviledged user might be able to see messages from other users, and they will not be shown with either or both of those flags.

journalctl: fix verbose output when no logs are found $ journalctl -o verbose _EXE=/quiet/binary -f -- Logs begin at Sun 2013-03-17 17:28:22 EDT. -- Failed to get realtime timestamp: Cannot assign requested address JOURNAL_FOREACH_DATA_RETVAL is added, which allows the caller to get the return value from sd_journal_enumerate_data. I think we might want to expose this macro like SD_JOURNAL_FOREACH_DATA, but for now it is in journal-internal.h. There's a change in behaviour for output_*, not only in output_verbose, that errors in sd_j_enumerate_data are not silently ignored anymore. https://bugs.freedesktop.org/show_bug.cgi?id=56459

systemctl: limit logs in status to current boot Also reworded a few debug messages for brevity, and added a log statement which prints out the filter at debug level: Journal filter: (((UNIT=sys-module-configfs.device AND _PID=1) OR (COREDUMP_UNIT=sys-module-configfs.device AND MESSAGE_ID=fc2e22bc6ee647b6b90729ab34a250b1) OR _SYSTEMD_UNIT=sys-module-configfs.device) AND _BOOT_ID=4e3c518ab0474c12ac8de7896fe6b154)

journalctl: add -k/--dmesg

hostname: only suppress setting of pretty hostname if it is non-equal to the static hostname and if the static hostname is set, too https://bugzilla.redhat.com/show_bug.cgi?id=957814

move _cleanup_ attribute in front of the type http://lists.freedesktop.org/archives/systemd-devel/2013-April/010510.html

journalctl: specify "--unit=" and "--user-unit" multiple times Previously only one "--unit=" or "--user-unit" could be specified. With this patch, journalcrtl can show multiple units. $ journalctl -u systemd-udevd.service -u sshd.service -u crond.service -b -- Logs begin at Sa 2013-03-23 11:08:45 CET, end at Fr 2013-04-12 09:10:22 CEST. -- Apr 12 08:41:37 lenovo systemd[1]: Started udev Kernel Device Manager. Apr 12 08:41:37 lenovo systemd[1]: Stopped udev Kernel Device Manager. Apr 12 08:41:38 lenovo systemd[1]: Started udev Kernel Device Manager. Apr 12 08:41:38 lenovo crond[291]: (CRON) INFO (Syslog will be used instead of sendmail.) Apr 12 08:41:38 lenovo crond[291]: (CRON) INFO (running with inotify support) Apr 12 08:41:39 lenovo systemd[1]: Starting OpenSSH server daemon... Apr 12 08:41:39 lenovo systemd[1]: Started OpenSSH server daemon. Apr 12 08:41:39 lenovo sshd[355]: Server listening on 0.0.0.0 port 22. Apr 12 08:41:39 lenovo sshd[355]: Server listening on :: port 22. Apr 12 08:41:39 lenovo mtp-probe[373]: checking bus 1, device 8: "/sys/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.5/1-1.5.6/1-1.5.6.2/1-1.5.6.2.1"

journal: add one more level on top with AND When using "-p" and "-b" in combination with "-u", the output is not what you would expect. The reason is the sd_journal_add_disjunction() call in add_matches_for_unit() and add_matches_for_user_unit(), which adds two ORs without taking the other conditions to every OR. Adding another level on top with AND and sd_journal_add_conjunction() solves the problem. Output before: $ journalctl -o short-monotonic -ab -p 0 -u sshd.service -- Reboot -- [ 3.216305] lenovo systemd[1]: Starting OpenSSH server daemon... -- Reboot -- [ 3.168666] lenovo systemd[1]: Starting OpenSSH server daemon... [ 3.169639] lenovo systemd[1]: Started OpenSSH server daemon. [36285.635389] lenovo systemd[1]: Stopped OpenSSH server daemon. -- Reboot -- [ 10.838657] lenovo systemd[1]: Starting OpenSSH server daemon... [ 10.913698] lenovo systemd[1]: Started OpenSSH server daemon. [ 6881.035183] lenovo systemd[1]: Stopped OpenSSH server daemon. -- Reboot -- [ 6.636228] lenovo systemd[1]: Starting OpenSSH server daemon... [ 6.662573] lenovo systemd[1]: Started OpenSSH server daemon. [ 6.681148] lenovo sshd[397]: Server listening on 0.0.0.0 port 22. [ 6.681379] lenovo sshd[397]: Server listening on :: port 22. As we see, the output is from _every_ boot and priority 0 is not taken into account. Output after patch: $ journalctl -o short-monotonic -ab -p 0 -u sshd.service -- Logs begin at Sun 2013-02-24 20:54:44 CET, end at Tue 2013-03-19 14:58:21 CET. -- Increasing the priority: $ journalctl -o short-monotonic -ab -p 6 -u sshd.service -- Logs begin at Sun 2013-02-24 20:54:44 CET, end at Tue 2013-03-19 14:59:12 CET. -- [ 6.636228] lenovo systemd[1]: Starting OpenSSH server daemon... [ 6.662573] lenovo systemd[1]: Started OpenSSH server daemon. [ 6.681148] lenovo sshd[397]: Server listening on 0.0.0.0 port 22. [ 6.681379] lenovo sshd[397]: Server listening on :: port 22.

journalctl: show fields requested with --field in full I see little point in silently truncating fields when they are explictly requested. With this change e.g. journalctl -b MESSAGE_ID=9f26aa562cf440c2b16c773d0479b518 --field=BOOTCHART works as expected.

journalctl: check return of strjoin

journal: add public API call sd_journal_get_events() This function should be used when filling in "struct pollfd"'s .events field for watching the journal. It will always return POLLIN for now, but we should keep our options open to change this later on. This mimics libsystemd-bus' sd_bus_get_events() call with the same purpose.

util: make time formatting a bit smarter Instead of outputting "5h 55s 50ms 3us" we'll now output "5h 55.050003s". Also, while outputting the accuracy is configurable. Basically we now try use "dot notation" for all time values > 1min. For >= 1s we use 's' as unit, otherwise for >= 1ms we use 'ms' as unit, and finally 'us'. This should give reasonably values in most cases.

util: rename parse_usec() to parse_sec() sinds the default unit is seconds Internally we store all time values in usec_t, however parse_usec() actually was used mostly to parse values in seconds (unless explicit units were specified to define a different unit). Hence, be clear about this and name the function about what we pass into it, not what we get out of it.

journalctl: reword error message Sentence seemed to suggest that all three conditions must be true.

journalctl: fix --update-catalog with not --root arg grawity> `journalctl --update-catalog` from latest git prints: "Recursive mkdir .: Invalid argument" and "Failed to write : Invalid argument"

journalctl: support --root for message catalogs

catalog: open up catalog internals In order to write tests for the catalog functions, they are made non-static and start taking a 'database' parameter, which is the name of a file with the preprocessed catalog entries. This makes it possible to make test-catalog part of the normal test suite, since it now only operates on files in /tmp. Some more tests are added.

journalctl: various fixes to the access check logic - Reword messages a bit - Correct check whether EACCES is in the set of errors - Don't complain if no journal files are found - allocate Set object for errors lazily since in the best case we don't need it at all. - don't consider it an error if /run/log/journal doesn't exist (because that's the usual case actually, if storage is enabled)

journalctl: be smarter about journal error checks There are many ways in which we can get those checks wrong, so it is better to warn and then error out on a real access failure. The error messages are wrapped to <80 lines, because their primary use is to be displayed in the terminal, and it is easier to read them this way. Reading them in the journal can be a bit trickier, but this is a bug in logs-show.c.

build-sys: move acl searching code into libsystemd-acl This loop over acls is a bit too much to keep inside of another loop.

journalctl: give a nice hint about group membership based on ACLs of /var/log/journal If we notice that we unprivileged and not in any of the groups which have access to /var/log/journal, print a nice message about which groups do. This checks and prints all groups that are in the default ACL for /var/log/journal, which is not necessarily correct for all journal files, but pretty close.

Remove some unused variables

journalct: beef up entry listing The ability to dump catalog entries in full and by id is added.

journalctl: use _cleanup_ in one function

journal,shared: add _cleanup_journal_close_

journalctl: show systemd messages about unit for -u journalctl -u unit is not very useful, because it doesn't show systemd messages about starting, stopping, coredumps, etc, like systemctl status unit does. Make it show the same information using the same rules. If somebody really want to see just messages from by the unit, it is easy enough to use _SYSTEMD_UNIT=...

journalctl: imply -n1000 when -e is used Make sure the pager does not have to buffer an unbounded number of log messages, by default.

pager: introduce "jump to end" option $ journalctl -be is what you want :) https://bugzilla.redhat.com/show_bug.cgi?id=867841

libsystemd-journal: return 0 on success in get_data() The man page says so. Right now 0 would be returned if the data was encrypted, 1 otherwise.

journald: introduce new "systemd-journal" group and make it own the journal files Previously all journal files were owned by "adm". In order to allow specific users to read the journal files without granting it access to the full "adm" powers, introduce a new specific group for this. "systemd-journal" has to be created by the packaging scripts manually at installation time. It's a good idea to assign a static UID/GID to this group, since /var/log/journal might be shared across machines via NFS. This commit also grants read access to the journal files by default to members of the "wheel" and "adm" groups via file system ACLs, since these "almost-root" groups should be able to see what's going on on the system. These ACLs are created by "make install". Packagers probably need to duplicate this logic in their postinst scripts. This also adds documentation how to grant access to the journal to additional users or groups via fs ACLs.

journalctl: add --user-unit= switch Add --user-unit= to make it possible to query for user logs by the name of the service.

journalctl: add --reverse option to show the newest lines first

journal: fix --until https://bugs.freedesktop.org/show_bug.cgi?id=58946

journalct: also print Python code in --new-id

journalctl: allow both "-n 55" and "-n55" on the command line, as equivalent syntaxes https://bugs.freedesktop.org/show_bug.cgi?id=60596

journalctl: allow --lines=0 i.e. only new Makes it easier to watch just for new entries. Once scenario is where the user starts 'journalctl -qfn0' to watch for changes during some operation.

journalctl: quit on I/O error This makes journalctl quit on ferror() conditions on stdout. It fixes an annoying bug if you pipe its output through 'less' and press 'q'. Without this fix journalctl will continue reading all journal data until EOF which can take quite some time. For instance on my machine: david-nb ~ # time journalctl | wc -l 327240 real 1m13.039s user 1m0.217s sys 0m10.467s However, expected behavior is journalctl to quit when its pager closed the output pipe. Signed-off-by: David Herrmann <dh.herrmann@googlemail.com>

build-sys: drop all distribution specfic checks Yay, we now have a completely generic systemd. No distribution specific checks anymore!

journalctl: add option to forgo ellipsization Sometimes it is better to see messages in full, and the existing set of options didn't allow this easily. E.g. now journalctl -f --full will behave like tail -f /var/log/messages of yore. Long option only for now, since small letters are becoming scarce, and this doesn't feel like a capital-letter-option. '-u' would be nice, and the above command would be spelled journalctl -fu

journal: extend catalog header to 64bit for all fields, just to be safe and follow the rest of the file formats

journal: implement message catalog The message catalog can be used to attach short help texts to log lines, keyed by their MESSAGE_ID= fields. This is useful to help the administrator understand the context and cause of a message, find possible solutions and find further related documentation. Since this is keyed off MESSAGE_ID= this will only work for native journal messages. The message catalog supports i18n, and is useful to augment english language system messages with explanations in the local language. This commit only includes short explanatory messages for a few example message IDs, we'll add more complete documentation for the relevant systemd messages later on.

journalctl: require argument for --priority This fixes a segfault due to a missing value for --priority. -p is unaffected because it is specified in the getopt_long parameter list.

enable localization for common *ctl commands

util: add is_locale_utf8() journalctl and vconsole-setup both implement utf8 locale detection. Let's have a common function for it. The next patch will add another use.

journalctl: remove left-over log message

util: unify line caching and column caching

util: unify usage of on_tty() in util.c

journalctl: unify ellipsation handling between journalctl and systemctl

journalctl: don't ellipsize unless on a tty

journalctl: honour -n if -F is used

journalctl: move access check before the first access to the journal files

journalctl: implement quering field values with new -F switch Example: journalctl -F _SYSTEMD_UNIT will list all units that ever logged to the journal.

journal: add ability to list values a specified field can take in all entries of the journal The new 'unique' API allows listing all unique field values that a field specified by a field name can take in all entries of the journal. This allows answering queries such as "What units logged to the journal?", "What hosts have logged into the journal?", "Which boot IDs have logged into the journal?". Ultimately this allows implementation of tools similar to lastlog based on journal data. Note that listing these field values will not work for journal files created with older journald, as the field values are not indexed in older files.

journalctl: add --unit=/-u to match by unit name This applies unit_name_mangle() to the specified unit names and hence can handle weird characters nicely and will add unit suffixes as necessary.

journalctl: harmonise in-stream comments From now on, always use ANSI-SQL-style comments in log streams, i.e. prefix with --. We also suffix things with this, just to be nice...

journalctl: skip informational messages in export/json modes

journalctl: make --follow really work

journalctl: make --follow work again Stopped working after cfbc22ab 'journalctl: implement --since= and --until for filtering by time'.

journalctl: implement --since= and --until for filtering by time

journal: support epxorting the journal in a format suitable for text/event-stream

journalctl: properly parse --cursor= long getopt in addition to short -c

journal: implement follow mode for HTTP GET in gatewayd

logs-show: various cleanups Among other cleanups this introduces a threshold for the size of binary blobs we serialize as integer arrays in the JSON output. THis can be disabled via --all.

journalctl: add --cursor switch

journald: always pass first entry timestamp back from journal_file_verify()

journalctl: print correct timespan in verify The old code used a timestamp to print a timespan for unsealed journalfiles, incorrectly showing things like 2230 days of unsealed entries. Print the timespan between the first and last entry instead.

journalctl: reset cached column count on SIGWINCH This requires a little bit of tip-toeing around to explicitly avoid touching the environment from a sig handler. Instead, simply create a function to reset the var to its "unset" state, allowing the next call to columns() to recalculate and cache the new value.

journalctl: make the argument to -n optional

journalctl: print QR code only if we are running on an UTF-8 system

journal: add call to determine current journal file disk usage

journalctl: show "Reboot" markers in output only when showing local-only entries

journalctl: replace --local by --merge, i.e. don't interleave remote journals by default

journalctl: rework JSON output mode This splits the JSON output mode into different modes: json and json-pretty. The former printing one entry per line, the latter showing JSON objects nicely indented and in multiple lines to make it easier to read for humans.

journalctl: complain if unprivileged users attempt to access the journal and persistant logging is off

journalctl: be more friendly when informing about seal verification

journalctl: add a bit of color to the output

journal: automatically evolve FSS key even when nothing is logged

journalctl: reword things a bit

journal: rearrange QR code output a bit to fi to 80x25 terminals

journalctl: explain QR code use

journalctl: output FSS key as QR code on generating

journal: make libgcrypt dependency optional

journal: fix tag ordering check

jounral: write bit fiddling test This test goes through every single bit in a journal file, toggles it, and checks if this change is detected by the verification.

journal: set secure deletion flags for FSS file

journal: after verification output validated time range

journal: rework terminology Let's clean up our terminology a bit. New terminology: FSS = Forward Secure Sealing FSPRG = Forward Secure Pseudo-Random Generator FSS is the combination of FSPRG and a HMAC. Sealing = process of adding authentication tags to the journal. Verification = process of checking authentication tags to the journal. Sealing Key = The key used for adding authentication tags to the journal. Verification Key = The key used for checking authentication tags of the journal. Key pair = The pair of Sealing Key and Verification Key Internally, the Sealing Key is the combination of the FSPRG State plus change interval/start time. Internally, the Verification Key is the combination of the FSPRG Seed plus change interval/start time.

journal: add FSPRG journal authentication

journalctl: immeidately terminate on invalid seed

journalctl: add --verify-seed= switch to specify seed value

journal: split up journal-file.c

journal: implement basic journal file verification logic

journal: add all objects we add to HMAC

journald: initial version of FSPRG hookup This adds forward-secure authentication of journal files. This patch includes key generation as well as tagging of journal files, Verification of journal files will be added in a later patch.

journalctl: support device node matches as shortcut

journalctl: add --priority= switch for filtering by priority

journalctl: hightlight log lines by priority warn/notice = bright white < error = red

log.h: new log_oom() -> int -ENOMEM, use it also a number of minor fixups and bug fixes: spelling, oom errors that didn't print errors, not properly forwarding error codes, few more consistency issues, et cetera