tree-wide: remove Emacs lines from all files This should be handled fine now by .dir-locals.el, so need to carry that stuff in every file.

shared: meanor clean-ups for logs-show.c Some minor simplifications. Shouldn't change codepaths.

journal: properly handle an unexpectedly missing field parse_field() checks if the field has the expected format, and returns 0 if it doesn't. In that case, value and size are not set. Nevertheless, we would try to continue, and hit an assert in safe_atou64. This case shouldn't happen, unless sd_j_get_data is borked, so cleanly assert that we got the expected field. Also, oom is the only way that parse_field can fail, which we log already. Instead of outputting a debug statement and carrying on, treat oom as fatal.

shared: include what we use The next step of a general cleanup of our includes. This one mostly adds missing includes but there are a few removals as well.

tree-wide: expose "p"-suffix unref calls in public APIs to make gcc cleanup easy GLIB has recently started to officially support the gcc cleanup attribute in its public API, hence let's do the same for our APIs. With this patch we'll define an xyz_unrefp() call for each public xyz_unref() call, to make it easy to use inside a __attribute__((cleanup())) expression. Then, all code is ported over to make use of this. The new calls are also documented in the man pages, with examples how to use them (well, I only added docs where the _unref() call itself already had docs, and the examples, only cover sd_bus_unrefp() and sd_event_unrefp()). This also renames sd_lldp_free() to sd_lldp_unref(), since that's how we tend to call our destructors these days. Note that this defines no public macro that wraps gcc's attribute and makes it easier to use. While I think it's our duty in the library to make our stuff easy to use, I figure it's not our duty to make gcc's own features easy to use on its own. Most likely, client code which wants to make use of this should define its own: #define _cleanup_(function) __attribute__((cleanup(function))) Or similar, to make the gcc feature easier to use. Making this logic public has the benefit that we can remove three header files whose only purpose was to define these functions internally. See #2008.

util-lib: split out allocation calls into alloc-util.[ch]

util-lib: move string table stuff into its own string-table.[ch]

util-lib: split string parsing related calls from util.[ch] into parse-util.[ch]

util-lib: split out IO related calls to io-util.[ch]

util-lib: split out fd-related operations into fd-util.[ch] There are more than enough to deserve their own .c file, hence move them over.

util-lib: split our string related calls from util.[ch] into its own file string-util.[ch] There are more than enough calls doing string manipulations to deserve its own files, hence do something about it. This patch also sorts the #include blocks of all files that needed to be updated, according to the sorting suggestions from CODING_STYLE. Since pretty much every file needs our string manipulation functions this effectively means that most files have sorted #include blocks now. Also touches a few unrelated include files.

cgtop: underline table header Let's underline the header line of the table shown by cgtop, how it is customary for tables. In order to do this, let's introduce new ANSI underline macros, and clean up the existing ones as side effect.

tree-wide: drop {} from one-line if blocks Patch via coccinelle.

util: make machine_name_is_valid() a macro and move it to hostname-util.h As it turns out machine_name_is_valid() does the exact same thing as hostname_is_valid() these days, as it just invoked that and checked the name length was < 64. However, hostname_is_valid() checks the length against HOST_NAME_MAX anyway (which is 64 on Linux), hence any additional check is redundant. We hence replace machine_name_is_valid() by a macro that simply maps it to hostname_is_valid() but sets the allow_trailing_dot parameter to false. We also move this this call to hostname-util.h, to the same place as the hostname_is_valid() declaration.

journal: fix json output of unicode characters

namespace helpers: Allow entering a UID namespace To be able to use `systemd-run` or `machinectl login` on a container that is in a private user namespace, the sub-process must have entered the user namespace before connecting to the container's D-Bus, otherwise the UID and GID in the peer credentials are garbage. So we extend namespace_open and namespace_enter to support UID namespaces, and we enter the UID namespace in bus_container_connect_{socket,kernel}. namespace_open will degrade to a no-op if user namespaces are not enabled in the kernel. Special handling is required for the setns call in namespace_enter with a user namespace, since transitioning to your own namespace is forbidden, as it would result in re-entering your user namespace as root. Arguably it may be valid to check this at the call site, rather than inside namespace_enter, but it is less code to do it inside, and if the intention of calling namespace_enter is to *be* in the target namespace, rather than to transition to the target namespace, it is a reasonable approach. The check for whether the user namespace is the same must happen before entering namespaces, as we may not be able to access /proc during the intermediate transition stage. We can't instead attempt to enter the user namespace and then ignore the failure from it being the same namespace, since the error code is not distinct, and we can't compare namespaces while mid-transition.

logs-show: print a debug message when we skip entries without MESSAGE= fields

journalctl: clean up how we log errors All functions should either log the errors they run into, or only return them in which case the caller should log them. Make sure this rule is followed, so that each error is logged precisely once, and neither never, nor more than once.

tree-wide: get rid of more strerror() calls

shared: add terminal-util.[ch]

shared: add process-util.[ch]

shared: add formats-util.h

fix gcc warnings about uninitialized variables like: src/shared/install.c: In function ‘unit_file_lookup_state’: src/shared/install.c:1861:16: warning: ‘r’ may be used uninitialized in this function [-Wmaybe-uninitialized] return r < 0 ? r : state; ^ src/shared/install.c:1796:13: note: ‘r’ was declared here int r; ^

logs-show: fix check of loop_read_exact just a typo fix

Introduce loop_read_exact helper Usually when using loop_read(), we want to read the full buffer. Add a helper that mirrors loop_write(), and returns 0 when full buffer was read, and an error otherwise. Use -ENODATA for the short read, to distinguish it from a read error.

remove unused includes This patch removes includes that are not used. The removals were found with include-what-you-use which checks if any of the symbols from a header is in use.

include <poll.h> instead of <sys/poll.h> include-what-you-use automatically does this and it makes finding unnecessary harder to spot. The only content of poll.h is a include of sys/poll.h so should be harmless.

util: rework strappenda(), and rename it strjoina() After all it is now much more like strjoin() than strappend(). At the same time, add support for NULL sentinels, even if they are normally not necessary.

loginctl: show the 10 most recent log user/session log lines in "loginctl user-status" and "loginctl session-status"

tree-wide: remove unnecessary LOG_PRI LOG_DEBUG is already a log level, there is no need to use LOG_PRI which is for filtering out the facility.

treewide: a few more log_*_errno + return simplifications The one in tmpfiles.c:create_item() even looks like it fixes a bug.

treewide: yet more log_*_errno + return simplifications Using: find . -name '*.[ch]' | while read f; do perl -i.mmm -e \ 'local $/; local $_=<>; s/(if\s*\([^\n]+\))\s*{\n(\s*)(log_[a-z_]*_errno\(\s*([->a-zA-Z_]+)\s*,[^;]+);\s*return\s+\g4;\s+}/\1\n\2return \3;/msg; print;' $f done And a couple of manual whitespace fixups.

treewide: more log_*_errno() conversions, multiline calls Basically: find . -name '*.[ch]' | while read f; do perl -i.mmm -e \ 'local $/; local $_=<>; s/log_(debug|info|notice|warning|error|emergency)\("([^"]*)%s"([^;]*),\s*strerror\(-?([->a-zA-Z_]+)\)\);/log_\1_errno(\4, "\2%m"\3);/gms;print;' \ $f; done Plus manual indentation fixups.

treewide: no need to negate errno for log_*_errno() It corrrectly handles both positive and negative errno values.

treewide: auto-convert the simple cases to log_*_errno() As a followup to 086891e5c1 "log: add an "error" parameter to all low-level logging calls and intrdouce log_error_errno() as log calls that take error numbers", use sed to convert the simple cases to use the new macros: find . -name '*.[ch]' | xargs sed -r -i -e \ 's/log_(debug|info|notice|warning|error|emergency)\("(.*)%s"(.*), strerror\(-([a-zA-Z_]+)\)\);/log_\1_errno(-\4, "\2%m"\3);/' Multi-line log_*() invocations are not covered. And we also should add log_unit_*_errno().

journal: when dumping log data with missing COMM fields, show "unknown" instead A small readability improvement...

machine: validate machine names using machine_name_is_valid() instead of string_is_safe() After all, we know have this as generic validator, so let's be correct and use it wherver applicable.

time: functions named "internal" really shouldn't be exported Also, let's try to make function names descriptive, instead of using bools for flags.

journalctl: make --utc work everywhere The --utc option was introduced by commit 9fd290443f5f99fca0dcd4216b1de70f7d3b8db1. Howerver, the implementation was incomplete.

journalctl: add --utc option Introduce option to display time in UTC.

hashmap: introduce hash_ops to make struct Hashmap smaller It is redundant to store 'hash' and 'compare' function pointers in struct Hashmap separately. The functions always comprise a pair. Store a single pointer to struct hash_ops instead. systemd keeps hundreds of hashmaps, so this saves a little bit of memory.

time-util: add and use USEC/NSEC_INFINIY

time-util: make sure USEC_PER_SEC and friends are actually of type usec_t

machined: add logic to query IP addresses of containers

Remove unnecessary casts in printfs No functional change expected :)

journal: fix export of messages containing newlines In "export" format, newlines are significant, and messages containing newlines must be exported as "binary".

util: replace close_pipe() with new safe_close_pair() safe_close_pair() is more like safe_close(), except that it handles pairs of fds, and doesn't make and misleading allusion, as it works similarly well for socketpairs() as for pipe()s...

util: replace close_nointr_nofail() by a more useful safe_close() safe_close() automatically becomes a NOP when a negative fd is passed, and returns -1 unconditionally. This makes it easy to write lines like this: fd = safe_close(fd); Which will close an fd if it is open, and reset the fd variable correctly. By making use of this new scheme we can drop a > 200 lines of code that was required to test for non-negative fds or to reset the closed fd variable afterwards.

systemctl: suppress duplicate newline if there's not log output in "systemctl status"

logs-show: fix corrupt output with empty messages If a message had zero length, journalctl would print no newline, and two output lines would be concatenated. Fix. The problem was introduced in commit 31f7bf199452 ("logs-show: print multiline messages"). Affected short and verbose output modes. Before fix: Feb 09 21:16:17 glyph dhclient[1323]: Feb 09 21:16:17 glyph NetworkManager[788]: <info> (enp4s2): DHCPv4 state changed nbi -> preinit after: Feb 09 21:16:17 glyph dhclient[1323]: Feb 09 21:16:17 glyph NetworkManager[788]: <info> (enp4s2): DHCPv4 state changed nbi -> preinit

bus: when getting a kdbus connection into a container wait first for child, then read message There's no EOF generated for AF_UNIX/SOCK_DGRAM sockets, hence let's wait for the child first to see if it succeeded, only then read the socket.

bus: when entering an existing namespace to connect to a container's system bus also switch over PID namespace This is necessary to ensure that kdbus can collect creds of the destination namespace when connecting.

util: port last code over to new namespace utility calls

shared: add missing include Needed for socketpair, recv

journal: add ability to browse journals of running OS containers This adds the new library call sd_journal_open_container() and a new "-M" switch to journalctl. Particular care is taken that journalctl's "-b" switch resolves to the current boot ID of the container, not the host.

logs-show.c: show all messages for a slice

logs-show.c: fix enum type in function declaration

Advertise hibernation only if there's enough free swap Condition that is checked is taken from upower: active(anon) < free swap * 0.98 This is really stupid, because the kernel knows the situation better, e.g. there could be two swap files, and then hibernation would be impossible despite passing this check, or the kernel could start supporting compressed swap and/or compressed hibernation images, and then this this check would be too stringent. Nevertheless, until we have something better, this should at least return a true negative if there's no swap. Logging of capabilities in the journal is changed to not strip leading zeros. I consider this more readable anyway. http://cgit.freedesktop.org/upower/tree/src/up-daemon.c#n613 https://bugzilla.redhat.com/show_bug.cgi?id=1007059

remove hasprefix(), use startswith()

logs-show: fix condition for ellipsizing multi-line messages falconindy> the ellipsizing seems a bit wrong here.... I got a bit carried away with putting dots everywhere :)

logs-show: show source timestamp in verbose mode This makes verbose behave like short mode, i.e. try to show the source timestamp, and fall back to journald timestamp only if unavailable or unparsable. I think verbose should be like short, only showing more fields, and showing different timestamps would be confusing.

logs-show: add short-precise mode with us timestamps Also, always show us timestamps in verbose mode. https://bugzilla.redhat.com/show_bug.cgi?id=991678

logs-show: limit to 3 lines and use dots if not showing full message So far, we would show up to 128 bytes from a message, simply cutting of the rest. With multiline messages, it is quite common for a message to be longer than that, and this model doesn't really work anymore. A new limit is added: up to 3 lines will be shown, unless --full is used (c.f. first line below). The limit for bytes is extended to 300 bytes. An ellipsis will always be used, if some form of truncation occurs. If the tail of the message is cut off, either because of length or line limit, dots will be shown at the end of the last line. If this last line is short, the dots will be simply appended. If the last line is too long for that, it will be ellipsized with dots at the very end. Note that the limits are in bytes, not characters, and we suck at outputting unicode strings (c.f. last three lines below). Aug 11 10:46:21 fedora python[67]: test message line line... Aug 11 10:50:47 fedora python[76]: test message word word word word word word word word word word word wor... Aug 11 10:55:11 fedora python[83]: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx... Aug 11 11:03:21 fedora python[90]: ąąąąąąąąąąąąąąąąąąąąąąąąąąąąąą... Aug 11 11:03:53 fedora python[97]: aąąąąąąąąąąąąąąąąąąąąąąąąąąąąąą... Aug 11 11:25:45 fedora python[121]: aąąąąąąąąąąąąąąąąąąąąąąąąąąąąąąąąąą�...

logs-show: fix indentation for 2nd and later lines, show lines in full Now --full will show long fields in full, like it already did with --all.

systemctl: show hint about --full when lines don't fit

journalctl: add ”short-iso” output format with verbose ISO8601 timestamps Example: 2013-07-18T10:10:01+0200 sandworm CROND[20957]: (root) CMD (/usr/lib64/sa/sa1 1 1)

journalctl,systemctl: fix tiny memleak

journalctl: highlight MESSAGE= in verbose output When looking at verbose output, additional "work" is required to pick out the interesting MESSAGE= lines from all the fields. Also, show long fields in full in verbose output mode when OUTPUT_FULL_WIDTH is specified.

journalctl,systemctl: show messages tagged with OBJECT_SYSTEMD_[USER_]_UNIT Replace mallocs with alloca while at it.

Make sure we only show authentic coredump messages Before we only checked the MESSAGE_ID and COREDUMP_UNIT. Those are both user-controlled fields. For COREDUMP_USER_UNIT, relax the rules a bit, and also allow messages from _UID=0.

Add hasprefix macro to check prefixes of fixed length

journalctl: show lines in full with --all In 31f7bf1 "logs-show: print multiline messages", I forgot to take into account the fact that --all implies --full for journalctl.

journal: add ability to filter by current user This is the just the library part. SD_JOURNAL_CURRENT_USER flags is added to sd_j_open(), to open files from current user. SD_JOURNAL_SYSTEM_ONLY is renamed to SD_JOURNAL_SYSTEM, and changed to mean to (also) open system files. This way various flags can be combined, which gives them nicer semantics, especially if other ones are added later. Backwards compatibility is kept, because SD_JOURNAL_SYSTEM_ONLY is equivalent to SD_JOURNAL_SYSTEM if used alone, and before there we no other flags.

journalctl: fix verbose output when no logs are found $ journalctl -o verbose _EXE=/quiet/binary -f -- Logs begin at Sun 2013-03-17 17:28:22 EDT. -- Failed to get realtime timestamp: Cannot assign requested address JOURNAL_FOREACH_DATA_RETVAL is added, which allows the caller to get the return value from sd_journal_enumerate_data. I think we might want to expose this macro like SD_JOURNAL_FOREACH_DATA, but for now it is in journal-internal.h. There's a change in behaviour for output_*, not only in output_verbose, that errors in sd_j_enumerate_data are not silently ignored anymore. https://bugs.freedesktop.org/show_bug.cgi?id=56459

logs-show: print multiline messages [ 0.019862] fedora kernel: CPU0: Thermal monitoring enabled (TM1) [ 0.019900] fedora kernel: Last level iTLB entries: 4KB 512, 2MB 0, 4MB 0 Last level dTLB entries: 4KB 512, 2MB 32, 4MB 32 tlb_flushall_shift: 5 [ 0.020118] fedora kernel: Freeing SMP alternatives: 24k freed

systemctl: limit logs in status to current boot Also reworded a few debug messages for brevity, and added a log statement which prints out the filter at debug level: Journal filter: (((UNIT=sys-module-configfs.device AND _PID=1) OR (COREDUMP_UNIT=sys-module-configfs.device AND MESSAGE_ID=fc2e22bc6ee647b6b90729ab34a250b1) OR _SYSTEMD_UNIT=sys-module-configfs.device) AND _BOOT_ID=4e3c518ab0474c12ac8de7896fe6b154)

move _cleanup_ attribute in front of the type http://lists.freedesktop.org/archives/systemd-devel/2013-April/010510.html

macro: rework how we define cleanup macros There's now a generic _cleanup_ macro with an argument. The macros for specific types are now defined using this macro, and in the header files where they belong. All cleanup handlers are now inline functions.

journal,shared: add _cleanup_journal_close_

logs-show: export logic to add matches for units After that functions which add matches, show_journal_by_unit and show_journal_by_user_unit, become nearly identical, so I merged them into one function.

logs-show: show messages for all unit types in systemctl status I can't find a reason why we shouldn't try to output messages for other unit types than .service, .socket, .mount and .swap as well. It's probably a leftover from before we started logging UNIT= from inside PID 1.

journal: log user units for coredumps and show them in systemctl status

logs-show: add show_journal_by_user_unit Print the journal for a user session unit. For now this filters by _SYSTEMD_USER_UNIT and USER_UNIT and additionally _UID.

logs-show: automatic cleanup

journalctl: strip TABs and ANSI color sequences from log messages when displaying them

journal: by default do not decompress dat objects larger than 64K This introduces a new data threshold setting for sd_journal objects which controls the maximum size of objects to decompress. This is relieves the library from having to decompress full data objects even if a client program is only interested in the initial part of them. This speeds up "systemd-coredumpctl" drastically when invoked without parameters.

journal: implement message catalog The message catalog can be used to attach short help texts to log lines, keyed by their MESSAGE_ID= fields. This is useful to help the administrator understand the context and cause of a message, find possible solutions and find further related documentation. Since this is keyed off MESSAGE_ID= this will only work for native journal messages. The message catalog supports i18n, and is useful to augment english language system messages with explanations in the local language. This commit only includes short explanatory messages for a few example message IDs, we'll add more complete documentation for the relevant systemd messages later on.

journal: properly serialize fields with multiple values into JSON This now matches the JSON serialization spec from: http://www.freedesktop.org/wiki/Software/systemd/json

journalctl: don't ellipsize unless on a tty

journal: add ability to list unique fields to gatewayd

journal: support epxorting the journal in a format suitable for text/event-stream

logs-show: various cleanups Among other cleanups this introduces a threshold for the size of binary blobs we serialize as integer arrays in the JSON output. THis can be disabled via --all.

journalctl: don't choke on entries with no MESSAGE= field https://bugs.freedesktop.org/show_bug.cgi?id=50177

journalctl: rework JSON output mode This splits the JSON output mode into different modes: json and json-pretty. The former printing one entry per line, the latter showing JSON objects nicely indented and in multiple lines to make it easier to read for humans.

shared: logs-show: fflush after each output type journalctl -f redirected to a pipe or file wasn't working for some output formats but was working for json. It turns out only json was doing an fflush. Make all output formats flush.

journalctl: include logs from PID 1 about services in systemctl status

fix a couple of issues found with llvm-analyze

logs-show: fix off-by-one error Ellipsize lines that are one character too long.

journalctl: hightlight log lines by priority warn/notice = bright white < error = red

logs-show: fix OOM path

log.h: new log_oom() -> int -ENOMEM, use it also a number of minor fixups and bug fixes: spelling, oom errors that didn't print errors, not properly forwarding error codes, few more consistency issues, et cetera

use "Out of memory." consistantly (or with "\n") glibc/glib both use "out of memory" consistantly so maybe we should consider that instead of this. Eliminates one string out of a number of binaries. Also fixes extra newline in udev/scsi_id

journalctl: fix assertion failure in ellipsize_mem() When showing the journal through "journalctl --no-pager", if the prefix of the log message (i.e. the date and syslog identifier) is less than 3 characters shorter than the width of the terminal, you get: Assertion 'new_length >= 3' failed at src/shared/util.c:3859, function ellipsize_mem(). Aborting. because there is not enough space for the "...". This patch add the necessary check.

journalctl: do not ellipsize when using pager If a pager is used, ellipsization is redundant — the pager does that better by hiding the part that cannot be shown. Pager's advantage is that the user can press → to view the hidden part of a message, and then ← to return.

logs: Adapt interface in log-show.c (show_journal_by_unit) Convert more flag arguments into one flag variable.

logs: Adapt interface in log-show.c (output_journal) In preparation for adding more output switches, convert a series of flags arguments into one flag variable.

man: document sd_journal_get_cutoff_realtime_usec()

journalctl: show any printable Unicode character This makes sure we are OK in outputting all valid, non-control UTF-8 characters, instead of just printable 7bit ASCII.

journal: introduce sd_journal_wait() to simplify writing synchronous clients

journal: expose and make use of cutoff times of journal This helps explaining when the log output of "systemctl status" is incomplete because the logs got rotated since the service was started.

fix a couple of things found with the llvm static analyzer

move more common files to shared/ and add them to shared.la