tree-wide: remove Emacs lines from all files This should be handled fine now by .dir-locals.el, so need to carry that stuff in every file.

sd-journal: add an API to enumerate known field names of the journal This adds two new calls to get the list of all journal fields names currently in use. This is the low-level support to implement the feature requested in #2176 in a more optimized way.

sd-journal: introduce has_runtime_files and has_persistent_files Also introduce sd_journal_has_runtime_files() and sd_journal_has_persistent_files() to the public API. These functions can be used to easily find out if the open journal files are runtime and/or persistent.

tree-wide: expose "p"-suffix unref calls in public APIs to make gcc cleanup easy GLIB has recently started to officially support the gcc cleanup attribute in its public API, hence let's do the same for our APIs. With this patch we'll define an xyz_unrefp() call for each public xyz_unref() call, to make it easy to use inside a __attribute__((cleanup())) expression. Then, all code is ported over to make use of this. The new calls are also documented in the man pages, with examples how to use them (well, I only added docs where the _unref() call itself already had docs, and the examples, only cover sd_bus_unrefp() and sd_event_unrefp()). This also renames sd_lldp_free() to sd_lldp_unref(), since that's how we tend to call our destructors these days. Note that this defines no public macro that wraps gcc's attribute and makes it easier to use. While I think it's our duty in the library to make our stuff easy to use, I figure it's not our duty to make gcc's own features easy to use on its own. Most likely, client code which wants to make use of this should define its own: #define _cleanup_(function) __attribute__((cleanup(function))) Or similar, to make the gcc feature easier to use. Making this logic public has the benefit that we can remove three header files whose only purpose was to define these functions internally. See #2008.

tree-wide: sort includes in *.h This is a continuation of the previous include sort patch, which only sorted for .c files.

journalctl: when we fail to open a journal file, print why When we enumerate journal files and encounter an invalid one, remember which this, and show it to the user. Note the possibly slightly surprising logic here: we store only one path per error code. This means we show all error kinds but not every actual error we encounter. This has the benefit of not requiring us to keep a potentially unbounded list of errors with their sources around, but can still provide a pretty complete overview on the errors we encountered. Fixes #1669.

util-lib: split our string related calls from util.[ch] into its own file string-util.[ch] There are more than enough calls doing string manipulations to deserve its own files, hence do something about it. This patch also sorts the #include blocks of all files that needed to be updated, according to the sorting suggestions from CODING_STYLE. Since pretty much every file needs our string manipulation functions this effectively means that most files have sorted #include blocks now. Also touches a few unrelated include files.

Add missing includes in header files This fixes various issues found by globally reordering the include sections of all .c files.

journal: move definition of LocationType to journal-file.h In preparation for individual JournalFiles maintaining a location of their own.

journal: make sd_journal::files a OrderedHashmap Anything that uses hashmap_next() almost certainly cares about the order and needs to be an OrderedHashmap.

sd-journal: do not reset sd_j_enumerate_unique position on error systemctl would call sd_j_enumerate_unique() interleaved with sd_j_next(). But the latter can remove a file if it detects an error in it. In those circumstances sd_j_enumerate_unique would restart with the first file in hashmap. With many corrupted files sd_j_enumerate_unique might iterate over the list multiple times. Avoid this by jumping to the next file in unique list if possible, or setting a flag that tells sd_j_enumerate_unique that it is done otherwise.

Always prefer our headers to system headers In practice this shouldn't make much difference, but sometimes our headers might be newer, and we want to test them.

journal: when listing logs of a container make sure we don't accidentally show messages from host too

util: allow trailing semicolons on define_trivial_cleanup_func lines Emacs C indenting really gets confused by these lines if they carry no trailing semicolon, hence let's make this nicer for good old emacs. The other macros which define functions already do this too, so let's copy the scheme here. Also, let's use an uppercase name for the macro. So far our rough rule was that macros that are totally not function-like (like this ones, which define a function) are uppercase. (Well, admittedly it is a rough rule only, for example function and variable decorators are all lower-case SINCE THE CONSTANT YELLING IN THE SOURCES WOULD SUCK, and also they at least got underscore prefixes.) Also, the macros that define functions that we already have are all uppercase, so let's do the same here...

Introduce udev object cleanup functions

journal: return -ECHILD after a fork A few asserts are replaced with 'return -EINVAL'. I think that assert should not be used to check argument in public functions. Fields in struct sd_journal are rearranged to make it less swiss-cheesy.

journal: add sd_journal_open_files This allows the caller to explicitly specify which journal files should be opened. The same functionality could be achieved before by creating a directory and playing around with symlinks. It is useful to debug stuff and explore the journal, and has been requested before. Waiting is supported, the journal will notice modifications on the files supplied when opening the journal, but will not add any new files.

journalctl: fix verbose output when no logs are found $ journalctl -o verbose _EXE=/quiet/binary -f -- Logs begin at Sun 2013-03-17 17:28:22 EDT. -- Failed to get realtime timestamp: Cannot assign requested address JOURNAL_FOREACH_DATA_RETVAL is added, which allows the caller to get the return value from sd_journal_enumerate_data. I think we might want to expose this macro like SD_JOURNAL_FOREACH_DATA, but for now it is in journal-internal.h. There's a change in behaviour for output_*, not only in output_verbose, that errors in sd_j_enumerate_data are not silently ignored anymore. https://bugs.freedesktop.org/show_bug.cgi?id=56459

Rearrange a few fields to reduce holes

journal: add one more level on top with AND When using "-p" and "-b" in combination with "-u", the output is not what you would expect. The reason is the sd_journal_add_disjunction() call in add_matches_for_unit() and add_matches_for_user_unit(), which adds two ORs without taking the other conditions to every OR. Adding another level on top with AND and sd_journal_add_conjunction() solves the problem. Output before: $ journalctl -o short-monotonic -ab -p 0 -u sshd.service -- Reboot -- [ 3.216305] lenovo systemd[1]: Starting OpenSSH server daemon... -- Reboot -- [ 3.168666] lenovo systemd[1]: Starting OpenSSH server daemon... [ 3.169639] lenovo systemd[1]: Started OpenSSH server daemon. [36285.635389] lenovo systemd[1]: Stopped OpenSSH server daemon. -- Reboot -- [ 10.838657] lenovo systemd[1]: Starting OpenSSH server daemon... [ 10.913698] lenovo systemd[1]: Started OpenSSH server daemon. [ 6881.035183] lenovo systemd[1]: Stopped OpenSSH server daemon. -- Reboot -- [ 6.636228] lenovo systemd[1]: Starting OpenSSH server daemon... [ 6.662573] lenovo systemd[1]: Started OpenSSH server daemon. [ 6.681148] lenovo sshd[397]: Server listening on 0.0.0.0 port 22. [ 6.681379] lenovo sshd[397]: Server listening on :: port 22. As we see, the output is from _every_ boot and priority 0 is not taken into account. Output after patch: $ journalctl -o short-monotonic -ab -p 0 -u sshd.service -- Logs begin at Sun 2013-02-24 20:54:44 CET, end at Tue 2013-03-19 14:58:21 CET. -- Increasing the priority: $ journalctl -o short-monotonic -ab -p 6 -u sshd.service -- Logs begin at Sun 2013-02-24 20:54:44 CET, end at Tue 2013-03-19 14:59:12 CET. -- [ 6.636228] lenovo systemd[1]: Starting OpenSSH server daemon... [ 6.662573] lenovo systemd[1]: Started OpenSSH server daemon. [ 6.681148] lenovo sshd[397]: Server listening on 0.0.0.0 port 22. [ 6.681379] lenovo sshd[397]: Server listening on :: port 22.

macro: rework how we define cleanup macros There's now a generic _cleanup_ macro with an argument. The macros for specific types are now defined using this macro, and in the header files where they belong. All cleanup handlers are now inline functions.

journal: add sd_journal_get_timeout() call to public API Let's do the wake-up logic on NFS internally, making things simpler for users.

journalctl: be smarter about journal error checks There are many ways in which we can get those checks wrong, so it is better to warn and then error out on a real access failure. The error messages are wrapped to <80 lines, because their primary use is to be displayed in the terminal, and it is easier to read them this way. Reading them in the journal can be a bit trickier, but this is a bug in logs-show.c.

journal: by default do not decompress dat objects larger than 64K This introduces a new data threshold setting for sd_journal objects which controls the maximum size of objects to decompress. This is relieves the library from having to decompress full data objects even if a client program is only interested in the initial part of them. This speeds up "systemd-coredumpctl" drastically when invoked without parameters.

journal: provide an API that allows client to figure out whether they need to recheck the journal manually for changes in regular intervals Network file systems generally do not offer inotify() that would work across the network. We hence cannot rely on inotify() exclusiely in those case. Provide an API to determine these cases, and suggest doing manual regular rechecks. Note that this is not complete yet, as we need to rescan journal dirs on network file systems explicitly to find new/removed files

journal: add ability to list values a specified field can take in all entries of the journal The new 'unique' API allows listing all unique field values that a field specified by a field name can take in all entries of the journal. This allows answering queries such as "What units logged to the journal?", "What hosts have logged into the journal?", "Which boot IDs have logged into the journal?". Ultimately this allows implementation of tools similar to lastlog based on journal data. Note that listing these field values will not work for journal files created with older journald, as the field values are not indexed in older files.

journal: update comments a bit

journal: make sure sd_journal_seek_cursor() seeks to the specified entry if it exists, not one after it

journal: implement generic sharable mmap caching logic instead of having one simple per-file cache implement an more comprehensive one that works for multiple files and can actually maintain multiple maps per file and per object type.

use #pragma once instead of foo*foo #define guards #pragma once has been "un-deprecated" in gcc since 3.3, and is widely supported in other compilers. I've been using and maintaining (rebasing) this patch for a while now, as it annoyed me to see #ifndef fooblahfoo, etc all over the place, almost arrogant about the annoyance of having to define all these names to perform a commen but neccicary functionality, when a completely superior alternative exists. I havn't sent it till now, cause its kindof a style change, and it is bad voodoo to mess with style that has been established by more established editors. So feel free to lambast me as a crazy bafoon. v2 - preserve externally used headers

journal: when watching directories actually watch the directories asked for

journal: automatically rotate journal files if the data hash table is full > 75% Previously, when the main data hash table grows too full the performance simply started to decrease drastically. Instead, now simply rotate to a new journal file as the hash table gets to full, so that we can start with a new fresh empty hash table.

journal: avoid re-definition of enums for older gcc versions On Sun, Jul 15, 2012 at 2:00 PM, Koen Kooi <koen@dominion.thruhere.net> wrote: > | In file included from src/journal/sd-journal.c:37:0: > | src/journal/journal-internal.h:47:3: error: redefinition of typedef 'MatchType' > | src/journal/journal-internal.h:36:24: note: previous declaration of 'MatchType' was here > | src/journal/journal-internal.h:67:3: error: redefinition of typedef 'LocationType' > | src/journal/journal-internal.h:37:27: note: previous declaration of 'LocationType' was here

journal: beef up journal matches considerably we now can take multiple matches, and they will apply as AND if they apply to different fields and OR if they apply to the same fields. Also, terms of this kind can be combined with an overreaching OR.

journal: rework directory enumeration/watch logic There's now sd_journal_new_directory() for watching specific journal directories. This is exposed in journalctl -D. sd_journal_wait() and sd_journal_process() now return whether changes in the journal are invalidating or just appending. We now create inotify kernel watches only when we actually need them

relicense to LGPLv2.1 (with exceptions) We finally got the OK from all contributors with non-trivial commits to relicense systemd from GPL2+ to LGPL2.1+. Some udev bits continue to be GPL2+ for now, but we are looking into relicensing them too, to allow free copy/paste of all code within systemd. The bits that used to be MIT continue to be MIT. The big benefit of the relicensing is that closed source code may now link against libsystemd-login.so and friends.

add sparse support to detect endianness bug le16/32/64_t type should be used when storing little-endian value header to integrate with sparse from Josh Triplett <josh@joshtriplett.org>

build-sys: move public header files into a dir of their own

journald: add missing header