cgroup: remove support for NetClass= directive Support for net_cls.class_id through the NetClass= configuration directive has been added in v227 in preparation for a per-unit packet filter mechanism. However, it turns out the kernel people have decided to deprecate the net_cls and net_prio controllers in v2. Tejun provides a comprehensive justification for this in his commit, which has landed during the merge window for kernel v4.5: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=bd1060a1d671 As we're aiming for full support for the v2 cgroup hierarchy, we can no longer support this feature. Userspace tool such as nftables are moving over to setting rules that are specific to the full cgroup path of a task, which obsoletes these controllers anyway. This commit removes support for tweaking details in the net_cls controller, but keeps the NetClass= directive around for legacy compatibility reasons.

tree-wide: remove Emacs lines from all files This should be handled fine now by .dir-locals.el, so need to carry that stuff in every file.

core: when determining system state, don't bother with JOB_TRY_RESTART When we determine the current system state we check whether units like emergency.target are running or a job that results in them being run is queued. However, this is not the case for JOB_TRY_RESTART, since that's a NOP if the unit has not been running before. Hence, don't bother with checking for that job type.

core: fix memory leak on reload ==1== HEAP SUMMARY: ==1== in use at exit: 61,728 bytes in 22 blocks ==1== total heap usage: 258,122 allocs, 258,100 frees, 78,219,628 bytes allocated ==1== ==1== 16 bytes in 1 blocks are definitely lost in loss record 1 of 6 ==1== at 0x4C2BBCF: malloc (vg_replace_malloc.c:299) ==1== by 0x1E350E: memdup (alloc-util.c:34) ==1== by 0x135AFB: memdup_multiply (alloc-util.h:74) ==1== by 0x140F97: manager_set_default_rlimits (manager.c:2929) ==1== by 0x1303DA: manager_set_defaults (main.c:737) ==1== by 0x133A02: main (main.c:1718) ==1== ==1== 272 bytes in 17 blocks are definitely lost in loss record 2 of 6 ==1== at 0x4C2BBCF: malloc (vg_replace_malloc.c:299) ==1== by 0x1E350E: memdup (alloc-util.c:34) ==1== by 0x135AFB: memdup_multiply (alloc-util.h:74) ==1== by 0x140F97: manager_set_default_rlimits (manager.c:2929) ==1== by 0x1303DA: manager_set_defaults (main.c:737) ==1== by 0x13480D: main (main.c:1828) ==1== ==1== LEAK SUMMARY: ==1== definitely lost: 288 bytes in 18 blocks ==1== indirectly lost: 0 bytes in 0 blocks ==1== possibly lost: 0 bytes in 0 blocks ==1== still reachable: 61,440 bytes in 4 blocks ==1== suppressed: 0 bytes in 0 blocks ==1== Reachable blocks (those to which a pointer was found) are not shown. ==1== To see them, rerun with: --leak-check=full --show-leak-kinds=all

tree-wide: check if errno is greater than zero (2) Compare errno with zero in a way that tells gcc that (if the condition is true) errno is positive.

core: don't enable special signals in test mode Fixes: $ systemd-analyze verify ... Failed to open /dev/tty0: Permission denied

core: revert "manager: do not set up signals in test mode" This reverts commit 5aa1054521596c3d268db5f4aff9f2b69647ffc9. Fixes test-execute $ sudo make check TESTS=test-execute ... $ cat test-execute.log + test /tmp/test-exec_workingdirectory = /tmp/test-exec_workingdirectory Test timeout when testing exec-workingdirectory.service exec-workingdirectory.service UMask: 0022 WorkingDirectory: /tmp/test-exec_workingdirectory RootDirectory: / NonBlocking: no PrivateTmp: no PrivateNetwork: no PrivateDevices: no ProtectHome: no ProtectSystem: no IgnoreSIGPIPE: yes RuntimeDirectoryMode: 0755 StandardInput: null StandardOutput: inherit StandardError: inherit FAIL test-execute (exit status: 1)

manager: do not set up signals in test mode When we are running in test mode, we don't expect any signals. In fact ^C should end the program. This also avoids permission issues when running systemd-analyze verify.

core: re-sync bus name list after deserializing during daemon-reload When the daemon reloads, it doesn not actually give up its DBus connection, as wrongly stated in an earlier commit. However, even though the bus connection stays open, the daemon flushes out all its internal state. Hence, if there is a NameOwnerChanged signal after the flush and before the deserialization, it cannot be matched against any pending unit. To fix this, rename bus_list_names() to manager_sync_bus_names() and call it explicitly at the end of the daemon reload operation.

manager: log log level changes uniformly Output the same message when a request to change the log level is received over dbus and through a signal. From the user point of view those two operations are very similar and it's easy to think that the dbus operation didn't work when the expected message is not emitted. Also "downgrade" the message level to info, since this is a normal user initiated action.

manager: move status output change debug messages to set function This way we can only print the debug message when the status actually changes. We also means we don't print anything when running in --user mode, where status output is always disabled.

tree-wide: expose "p"-suffix unref calls in public APIs to make gcc cleanup easy GLIB has recently started to officially support the gcc cleanup attribute in its public API, hence let's do the same for our APIs. With this patch we'll define an xyz_unrefp() call for each public xyz_unref() call, to make it easy to use inside a __attribute__((cleanup())) expression. Then, all code is ported over to make use of this. The new calls are also documented in the man pages, with examples how to use them (well, I only added docs where the _unref() call itself already had docs, and the examples, only cover sd_bus_unrefp() and sd_event_unrefp()). This also renames sd_lldp_free() to sd_lldp_unref(), since that's how we tend to call our destructors these days. Note that this defines no public macro that wraps gcc's attribute and makes it easier to use. While I think it's our duty in the library to make our stuff easy to use, I figure it's not our duty to make gcc's own features easy to use on its own. Most likely, client code which wants to make use of this should define its own: #define _cleanup_(function) __attribute__((cleanup(function))) Or similar, to make the gcc feature easier to use. Making this logic public has the benefit that we can remove three header files whose only purpose was to define these functions internally. See #2008.

core: enable TasksMax= for all services by default, and set it to 512 Also, enable TasksAccounting= for all services by default, too. See: http://lists.freedesktop.org/archives/systemd-devel/2015-November/035006.html

core: add new DefaultTasksMax= setting for system.conf This allows initializing the TasksMax= setting of all units by default to some fixed value, instead of leaving it at infinity as before.

core: use SD_EVENT_PRIORITY_NORMAL-n instead on -n

core: unify code that warns about jobs we fail to enqueue This allows us to shorten our code a bit.

core: drop "override" flag when building transactions Now that we don't have RequiresOverridable= and RequisiteOverridable= dependencies anymore, we can get rid of tracking the "override" boolean for jobs in the job engine, as it serves no purpose anymore. While we are at it, fix some error messages we print when invoking functions that take the override parameter.

core: change type of distribute_fds() prototype to return void We can't handle errors of thisc all sanely anyway, and we never actually return any errors from the unit type that implements the call. Hence, let's make this void, in order to simplify things.

core: change return value of the unit's enumerate() call to void We cannot handle enumeration failures in a sensible way, hence let's try hard to continue without making such failures fatal, and log about it with precise error messages.

core: still make progress when throttling the manager loop Don't simply continue after sleeping, it potentially puts us in a state of spinning doing nothing slowly, if the ratelimit_test() keeps detecting the need for limiting. Observed in vms after the host had been suspended for a while, on resume systemd entered a loop of making zero progress spamming the console with: [431942.850090] systemd[1]: Looping too fast. Throttling execution a little. I see no reason to have a continue here, the intention should be to throttle execution, not circumvent it altogether.

sd-daemon: increase sd_notify() socket buffer size Let's make sure we don't start blocking on sd_notify() earlier than necessary, let's bump the socket buffer sizes to 8M. We already do something similar for our logging socket buffers, hence apply a similar bump here.

core: bail our earlier when doing audit Let's make sure we don't even try to create the audit socket

core,nspawn: minor coding style fixes

core: constify a few things

core: fix priority ordering in notify-handling Currently, we dispatch NOTIFY messages in a tight loop. Regardless how much data is incoming, we always dispatch everything that is queued. This, however, completely breaks priority event-handling of sd-event. When dispatching one NOTIFY event, another completely different event might fire, or might be queued by the NOTIFY handling. However, this event will not get dispatched until all other further NOTIFY messages are handled. Those might even arrive _after_ the other event fired, and as such completely break priority ordering of sd-event (which several code paths rely on). Break this by never dispatching multiple messages. Just return after each message that was read and let sd-event handle everything else. (The patch looks scarier that it is. It basically just drops the for(;;) loop and re-indents the loop-content.)

util-lib: split out allocation calls into alloc-util.[ch]

util-lib: split out umask-related code to umask-util.h

util-lib: move string table stuff into its own string-table.[ch]

util-lib: split stat()/statfs()/stavfs() related calls into stat-util.[ch]

util-lib: move a number of fs operations into fs-util.[ch]

util-lib: move more file I/O related calls into fileio.[ch]

util-lib: split string parsing related calls from util.[ch] into parse-util.[ch]

util-lib: split out IO related calls to io-util.[ch]

util-lib: split out fd-related operations into fd-util.[ch] There are more than enough to deserve their own .c file, hence move them over.

util-lib: split our string related calls from util.[ch] into its own file string-util.[ch] There are more than enough calls doing string manipulations to deserve its own files, hence do something about it. This patch also sorts the #include blocks of all files that needed to be updated, according to the sorting suggestions from CODING_STYLE. Since pretty much every file needs our string manipulation functions this effectively means that most files have sorted #include blocks now. Also touches a few unrelated include files.

util: split out escaping code into escape.[ch] This really deserves its own file, given how much code this is now.

tree-wide: add more void casts for various syscall invocations

manager: remove unused function

core: add support for naming file descriptors passed using socket activation This adds support for naming file descriptors passed using socket activation. The names are passed in a new $LISTEN_FDNAMES= environment variable, that matches the existign $LISTEN_FDS= one and contains a colon-separated list of names. This also adds support for naming fds submitted to the per-service fd store using FDNAME= in the sd_notify() message. This also adds a new FileDescriptorName= setting for socket unit files to set the name for fds created by socket units. This also adds a new call sd_listen_fds_with_names(), that is similar to sd_listen_fds(), but also returns the names of the fds. systemd-activate gained the new --fdname= switch to specify a name for testing socket activation. This is based on #1247 by Maciej Wereski. Fixes #1247.

core: sort includes of manager.[ch] according to CODING_STYLE

cgtop: underline table header Let's underline the header line of the table shown by cgtop, how it is customary for tables. In order to do this, let's introduce new ANSI underline macros, and clean up the existing ones as side effect.

util: add safe_closedir() similar to safe_fclose()

cgroup: add support for net_cls controllers Add a new config directive called NetClass= to CGroup enabled units. Allowed values are positive numbers for fix assignments and "auto" for picking a free value automatically, for which we need to keep track of dynamically assigned net class IDs of units. Introduce a hash table for this, and also record the last ID that was given out, so the allocator can start its search for the next 'hole' from there. This could eventually be optimized with something like an irb. The class IDs up to 65536 are considered reserved and won't be assigned automatically by systemd. This barrier can be made a config directive in the future. Values set in unit files are stored in the CGroupContext of the unit and considered read-only. The actually assigned number (which may have been chosen dynamically) is stored in the unit itself and is guaranteed to remain stable as long as the unit is active. In the CGroup controller, set the configured CGroup net class to net_cls.classid. Multiple unit may share the same net class ID, and those which do are linked together.

mount: use libmount to monitor mountinfo & utab The current implementation directly monitor /proc/self/mountinfo and /run/mount/utab files. It's really not optimal because utab file is private libmount stuff without any official guaranteed semantic. The libmount since v2.26 provides API to monitor mount kernel & userspace changes and since v2.27 the monitor is usable for non-root users too. This patch replaces the current implementation with libmount based solution. Signed-off-by: Karel Zak <kzak@redhat.com>

cgroup: unify how we invalidate cgroup controller settings Let's make sure that we follow the same codepaths when adjusting a cgroup property via the dbus SetProperty() call, and when we execute the StartupCPUShares= effect.

core: invalidate idle pipe event source in manager_close_idle_pipe() In all occasions when this function is called we do so anyway, so let's move this inside, to make things easier.

core: allocate sets of startup and failed units on-demand There's a good chance we never needs these sets, hence allocate them only when needed.

tree-wide: drop {} from one-line if blocks Patch via coccinelle.

tree-wide: use coccinelle to patch a lot of code to use mfree() This replaces this: free(p); p = NULL; by this: p = mfree(p); Change generated using coccinelle. Semantic patch is added to the sources.

basic: rework virtualization detection API Introduce a proper enum, and don't pass around string ids anymore. This simplifies things quite a bit, and makes virtualization detection more similar to architecture detection.

core: split up manager_get_unit_by_pid() Let's move the actual cgroup part of it into a new separate function manager_get_unit_by_pid_cgroup(), and then make manager_get_unit_by_pid() just a wrapper that also checks the two pid hashmaps. Then, let's make sure the various calls that want to deliver events to the owners of a PID check both hashmaps and the cgroup and deliver the event to *each* of them. OTOH make sure bus calls like GetUnitByPID() continue to check the PID hashmaps first and the cgroup only as fallback.

macro: introduce new PID_TO_PTR macros and make use of them This adds a new PID_TO_PTR() macro, plus PTR_TO_PID() and makes use of it wherever we maintain processes in a hash table. Previously we sometimes used LONG_TO_PTR() and other times ULONG_TO_PTR() for that, hence let's make this more explicit and clean up things.

core: unified cgroup hierarchy support This patch set adds full support the new unified cgroup hierarchy logic of modern kernels. A new kernel command line option "systemd.unified_cgroup_hierarchy=1" is added. If specified the unified hierarchy is mounted to /sys/fs/cgroup instead of a tmpfs. No further hierarchies are mounted. The kernel command line option defaults to off. We can turn it on by default as soon as the kernel's APIs regarding this are stabilized (but even then downstream distros might want to turn this off, as this will break any tools that access cgroupfs directly). It is possibly to choose for each boot individually whether the unified or the legacy hierarchy is used. nspawn will by default provide the legacy hierarchy to containers if the host is using it, and the unified otherwise. However it is possible to run containers with the unified hierarchy on a legacy host and vice versa, by setting the $UNIFIED_CGROUP_HIERARCHY environment variable for nspawn to 1 or 0, respectively. The unified hierarchy provides reliable cgroup empty notifications for the first time, via inotify. To make use of this we maintain one manager-wide inotify fd, and each cgroup to it. This patch also removes cg_delete() which is unused now. On kernel 4.2 only the "memory" controller is compatible with the unified hierarchy, hence that's the only controller systemd exposes when booted in unified heirarchy mode. This introduces a new enum for enumerating supported controllers, plus a related enum for the mask bits mapping to it. The core is changed to make use of this everywhere. This moves PID 1 into a new "init.scope" implicit scope unit in the root slice. This is necessary since on the unified hierarchy cgroups may either contain subgroups or processes but not both. PID 1 hence has to move out of the root cgroup (strictly speaking the root cgroup is the only one where processes and subgroups are still allowed, but in order to support containers nicey, we move PID 1 into the new scope in all cases.) This new unit is also used on legacy hierarchy setups. It's actually pretty useful on all systems, as it can then be used to filter journal messages coming from PID 1, and so on. The root slice ("-.slice") is now implicitly created and started (and does not require a unit file on disk anymore), since that's where "init.scope" is located and the slice needs to be started before the scope can. To check whether we are in unified or legacy hierarchy mode we use statfs() on /sys/fs/cgroup. If the .f_type field reports tmpfs we are in legacy mode, if it reports cgroupfs we are in unified mode. This patch set carefuly makes sure that cgls and cgtop continue to work as desired. When invoking nspawn as a service it will implicitly create two subcgroups in the cgroup it is using, one to move the nspawn process into, the other to move the actual container processes into. This is done because of the requirement that cgroups may either contain processes or other subgroups.

manager: don't write first-boot flag file all the time Instead, remember that we have already written it.

manager: remove ask-password fd from sd_event before closing it Otherwise we might attempt to remove a non-existing fd from epoll.

core: dbus: track bus names per unit Currently, PID1 installs an unfiltered NameOwnerChanged signal match, and dispatches the signals itself. This does not scale, as right now, PID1 wakes up every time a bus client connects. To fix this, install individual matches once they are requested by unit_watch_bus_name(), and remove the watches again through their slot in unit_unwatch_bus_name(). If the bus is not available during unit_watch_bus_name(), just store name in the 'watch_bus' hashmap, and let bus_setup_api() do the installing later.

tree-wide: port everything over to fflush_and_check() Some places invoked fflush() directly with their own manual error checking, let's unify all that by using fflush_and_check(). This also unifies the general error paths of fflush()+rename() file writers.

turn kdbus support into a runtime option ./configure --enable/disable-kdbus can be used to set the default behavior regarding kdbus. If no kdbus kernel support is available, dbus-dameon will be used. With --enable-kdbus, the kernel command line option "kdbus=0" can be used to disable kdbus. With --disable-kdbus, the kernel command line option "kdbus=1" is required to enable kdbus support.

util: introduce CMSG_FOREACH() macro and make use of it everywhere It's only marginally shorter then the usual for() loop, but certainly more readable.

tree-wide: remove spurious space

kdbus: remove attach_flags_mask module parameter setting

util: split out signal-util.[ch] from util.[ch] No functional changes.

manager: fix finish_timestamp calculation http://lists.freedesktop.org/archives/systemd-devel/2015-May/032100.html

core: don't consider boot-up finished if we are still reloading http://lists.freedesktop.org/archives/systemd-devel/2015-May/032025.html

core: when propagating restart requests due to deps, downgrade restart to try-restart Previously, if a service A depended on a service B via Requires=, and A was not running and B restarted this would trigger a start of A as well, since the restart was propagated as restart independently of the state of A. This patch ensures that a restart of B would be propagated as a try-restart to A, thus not changing its state if it isn't up. http://lists.freedesktop.org/archives/systemd-devel/2015-May/032061.html

core: fix memory leak in manager_run_generators() If systemd is built with GCC address sanitizer or leak sanitizer the following memory leak ocurs: May 12 02:02:46 linux.site systemd[326]: ================================================================= May 12 02:02:46 linux.site systemd[326]: ==326==ERROR: LeakSanitizer: detected memory leaks May 12 02:02:46 linux.site systemd[326]: Direct leak of 101 byte(s) in 3 object(s) allocated from: May 12 02:02:46 linux.site systemd[326]: #0 0x7fd1f504993f in strdup (/usr/lib64/libasan.so.2+0x6293f) May 12 02:02:46 linux.site systemd[326]: #1 0x55d6ffac5336 in strv_new_ap src/shared/strv.c:163 May 12 02:02:46 linux.site systemd[326]: #2 0x55d6ffac56a9 in strv_new src/shared/strv.c:185 May 12 02:02:46 linux.site systemd[326]: #3 0x55d6ffa80272 in generator_paths src/shared/path-lookup.c:223 May 12 02:02:46 linux.site systemd[326]: #4 0x55d6ff9bdb0f in manager_run_generators src/core/manager.c:2828 May 12 02:02:46 linux.site systemd[326]: #5 0x55d6ff9b1a10 in manager_startup src/core/manager.c:1121 May 12 02:02:46 linux.site systemd[326]: #6 0x55d6ff9a78e3 in main src/core/main.c:1667 May 12 02:02:46 linux.site systemd[326]: #7 0x7fd1f394e8c4 in __libc_start_main (/lib64/libc.so.6+0x208c4) May 12 02:02:46 linux.site systemd[326]: Direct leak of 29 byte(s) in 1 object(s) allocated from: May 12 02:02:46 linux.site systemd[326]: #0 0x7fd1f504993f in strdup (/usr/lib64/libasan.so.2+0x6293f) May 12 02:02:46 linux.site systemd[326]: #1 0x55d6ffac5288 in strv_new_ap src/shared/strv.c:152 May 12 02:02:46 linux.site systemd[326]: #2 0x55d6ffac56a9 in strv_new src/shared/strv.c:185 May 12 02:02:46 linux.site systemd[326]: #3 0x55d6ffa80272 in generator_paths src/shared/path-lookup.c:223 May 12 02:02:46 linux.site systemd[326]: #4 0x55d6ff9bdb0f in manager_run_generators src/core/manager.c:2828 May 12 02:02:46 linux.site systemd[326]: #5 0x55d6ff9b1a10 in manager_startup src/core/manager.c:1121 May 12 02:02:46 linux.site systemd[326]: #6 0x55d6ff9a78e3 in main src/core/main.c:1667 May 12 02:02:46 linux.site systemd[326]: #7 0x7fd1f394e8c4 in __libc_start_main (/lib64/libc.so.6+0x208c4) May 12 02:02:46 linux.site systemd[326]: SUMMARY: AddressSanitizer: 130 byte(s) leaked in 4 allocation(s). There is a leak due to the the use of cleanup_free instead _cleanup_strv_free_

core: drop redundant logging about notification messages Before: May 12 17:11:22 tomegun-x2402 systemd[1]: systemd-udevd.service: Got notification message for unit. May 12 17:11:22 tomegun-x2402 systemd[1]: systemd-udevd.service: Got notification message from PID 195 (READY=1) May 12 17:11:22 tomegun-x2402 systemd[1]: systemd-udevd.service: Ggot READY=1 After: May 12 17:11:22 tomegun-x2402 systemd[1]: systemd-udevd.service: Got notification message from PID 195 (READY=1)

core: rename SystemdRunningAs to ManagerRunningAs It's primarily just a property of the Manager object after all, and we try to refer to PID 1 as "manager" instead of "systemd", hence let's to stick to this here too.

core,network: major per-object logging rework This changes log_unit_info() (and friends) to take a real Unit* object insted of just a unit name as parameter. The call will now prefix all logged messages with the unit name, thus allowing the unit name to be dropped from the various passed romat strings, simplifying invocations drastically, and unifying log output across messages. Also, UNIT= vs. USER_UNIT= is now derived from the Manager object attached to the Unit object, instead of getpid(). This has the benefit of correcting the field for --test runs. Also contains a couple of other logging improvements: - Drops a couple of strerror() invocations in favour of using %m. - Not only .mount units now warn if a symlinks exist for the mount point already, .automount units do that too, now. - A few invocations of log_struct() that didn't actually pass any additional structured data have been replaced by simpler invocations of log_unit_info() and friends. - For structured data a new LOG_UNIT_MESSAGE() macro has been added, that works like LOG_MESSAGE() but prefixes the message with the unit name. Similar, there's now LOG_LINK_MESSAGE() and LOG_NETDEV_MESSAGE(). - For structured data new LOG_UNIT_ID(), LOG_LINK_INTERFACE(), LOG_NETDEV_INTERFACE() macros have been added that generate the necessary per object fields. The old log_unit_struct() call has been removed in favour of these new macros used in raw log_struct() invocations. In addition to removing one more function call this allows generated structured log messages that contain two object fields, as necessary for example for network interfaces that are joined into another network interface, and whose messages shall be indexed by both. - The LOG_ERRNO() macro has been removed, in favour of log_struct_errno(). The latter has the benefit of ensuring that %m in format strings is properly resolved to the specified error number. - A number of logging messages have been converted to use log_unit_info() instead of log_info() - The client code in sysv-generator no longer #includes core code from src/core/. - log_unit_full_errno() has been removed, log_unit_full() instead takes an errno now, too. - log_unit_info(), log_link_info(), log_netdev_info() and friends, now avoid double evaluation of their parameters

treewide: Correct typos and spell plural of bus consistent

bus: don't switch to kdbus if not requested Whenever systemd is re-executed, it tries to create a system bus via kdbus. If the system did not have kdbus loaded during bootup, but the module is loaded later on manually, this will cause two system buses running (kdbus and dbus-daemon in parallel). This patch makes sure we never try to create kdbus buses if it wasn't explicitly requested on the command-line.

core: rework unit name validation and manipulation logic A variety of changes: - Make sure all our calls distuingish OOM from other errors if OOM is not the only error possible. - Be much stricter when parsing escaped paths, do not accept trailing or leading escaped slashes. - Change unit validation to take a bit mask for allowing plain names, instance names or template names or an combination thereof. - Refuse manipulating invalid unit name

core: simplify unit type detection logic Introduce a new call unit_type_supported() and make use of it everywhere. Also, drop Manager parameter from per-type supported method prototype.

core: fix event source annotations These looked like a mass-replace gone slightly wrong – two statements with no { }'s, and no error checking.

core: annotate event sources

manager: don't fail fatally if we cannot coldplug a unit It's better to continue as good as we can, than to totally fail. Hence, let's log about the failure and continue.

Revert "core: do not spawn jobs or touch other units during coldplugging" This reverts commit 6e392c9c45643d106673c6643ac8bf4e65da13c1. We really shouldn't invent external state keeping hashmaps, if we can keep this state in the units themselves.

shared: add terminal-util.[ch]

shared: add process-util.[ch]

util: rework cunescape(), improve error handling Change cunescape() to return a normal error code, so that we can distuingish OOM errors from parse errors. This also adds a flags parameter to control whether "relaxed" or normal parsing shall be done. If set no parse failures are generated, and the only reason why cunescape() can fail is OOM.

util: rework rm_rf() logic - Move to its own file rm-rf.c - Change parameters into a single flags parameter - Remove "honour sticky" logic, it's unused these days

core: Remove explicit Plymouth integration Even if plymouth is running, it might have not displayed the splash yet, so we'll see a few lines on fbcon when we should have otherwise had nothing. Plymouth integration was added to systemd in commit 6faa11140bf776cdaeb8d22d01816e6e48296971. That same day, Plymouth got systemd integration [0]. As such, the Plymouth integration has always been obsolete, and was probably only for older Plymouth's. But I can't imagine anybody running a Plymouth from 2011 with a systemd from 2015. Remove the Plymouth/systemd integration, and let Plymouth's code tell systemd to print the details. [0] http://cgit.freedesktop.org/plymouth/commit/?id=537c16422cd49f1beeaab1ad39846a00018faec1 Signed-off-by: Jasper St. Pierre <jstpierre@mecheye.net> Cc: Daniel Drake <dsd@endlessm.com> Cc: Ray Strode <rstrode@redhat.com>

core: remove useless debug message Mar 13 19:48:28 adam.happyassassin.net systemd[1]: Collecting (null) Mar 13 19:48:28 adam.happyassassin.net systemd[1]: Collecting (null) Mar 13 19:48:28 adam.happyassassin.net systemd[1]: Collecting (null) Mar 13 19:48:28 adam.happyassassin.net systemd[1]: Collecting (null) Mar 13 19:48:28 adam.happyassassin.net systemd[1]: Collecting (null) Mar 13 19:48:28 adam.happyassassin.net systemd[1]: Collecting (null) Mar 13 19:48:28 adam.happyassassin.net systemd[1]: Collecting (null)

core: check asprintf return value CID #1261729.

core: ignore any issues with setting time on jobs_in_progress_event_source CID #1237559.

core: issue error on oom we can do nothing about CID #1287142.

core: do not spawn jobs or touch other units during coldplugging Because the order of coldplugging is not defined, we can reference a not-yet-coldplugged unit and read its state while it has not yet been set to a meaningful value. This way, already active units may get started again. We fix this by deferring such actions until all units have been at least somehow coldplugged. Fixes https://bugs.freedesktop.org/show_bug.cgi?id=88401

core: downgrade unit type not supported message Otherwise every daemon reload prints out warnings like: systemd[1]: Unit type .busname is not supported on this system. systemd[1]: Unit type .swap is not supported on this system.

core: emit changes for NFailedUnits property By notifying the clients when this property is changed it's possible to allow "system health monitor" tools to get transitions like running<->degraded. This is an alternative to send changes on the SystemState property since the latter is more difficult to derive.

remove unused includes This patch removes includes that are not used. The removals were found with include-what-you-use which checks if any of the symbols from a header is in use.

include <poll.h> instead of <sys/poll.h> include-what-you-use automatically does this and it makes finding unnecessary harder to spot. The only content of poll.h is a include of sys/poll.h so should be harmless.

util: rework strappenda(), and rename it strjoina() After all it is now much more like strjoin() than strappend(). At the same time, add support for NULL sentinels, even if they are normally not necessary.

manager: fix minor typo

manager: when we immediately reboot due to 7x C-A-D within 2s, mention this on the console too

core: when the user hits Ctrl-Alt-Del more than 7x per 2s, reboot immediately This should be useful for cases where clean rebooting doesn't work, and the user wants to hurry up the reboot.

importd: when listing transfers, show progress percentage With this change the pull protocol implementation processes will pass progress data to importd which then passes this information on via the bus. We use sd_notify() as generic transport for this communication, making importd listen to them, while matching the incoming messages to the right transfer.

core: zero size notify messages are OK

Assorted format fixes Types used for pids and uids in various interfaces are unpredictable. Too bad.

core: write kdbus.attach_flags_mask only on real boot The kernel module system is not namespaced, so no container should ever modify global options. Make sure we set the kdbus attach_flags_mask only on a real boot as PID1.

Implement masking and overriding of generators Sometimes it is necessary to stop a generator from running. Either because of a bug, or for testing, or some other reason. The only way to do that would be to rename or chmod the generator binary, which is inconvenient and does not survive upgrades. Allow masking and overriding generators similarly to units and other configuration files. For the systemd instance, masking would be more common, rather than overriding generators. For the user instances, it may also be useful for users to have generators in $XDG_CONFIG_HOME to augment or override system-wide generators. Directories are searched according to the usual scheme (/usr/lib, /usr/local/lib, /run, /etc), and files with the same name in higher priority directories override files with the same name in lower priority directories. Empty files and links to /dev/null mask a given name. https://bugs.freedesktop.org/show_bug.cgi?id=87230

Simplify execute_directory() Remove the optional sepearate opening of the directory, it would be just too complicated with the change to multiple directories. Move the middle of execute_directory() to a seperate function to make it easier to grok.

core: add new logic for services to store file descriptors in PID 1 With this change it is possible to send file descriptors to PID 1, via sd_pid_notify_with_fds() which PID 1 will store individually for each service, and pass via the usual fd passing logic on next invocation. This is useful for enable daemon reload schemes where daemons serialize their state to /run, push their fds into PID 1 and terminate, restoring their state on next start from the data in /run and passed in from PID 1. The fds are kept by PID 1 as long as no POLLHUP or POLLERR is seen on them, and the service they belong to are either not dead or failed, or have a job queued.

core: rework counting of running jobs Let's unify the code that counts the running jobs a bit, in order to make sure we are less likely to miss one. This is related to this bug: https://bugs.freedesktop.org/show_bug.cgi?id=87349 However, it probably won't fix it fully, and I cannot reproduce the issue. The change also adds an explicit assert change when the counter is off.

util: rename ignore_file() to hidden_file() hidden_file() is a bit more precise, since dot files usually shouldn't be ignored, but certainly be considered hidden.

unit: handle nicely of certain unit types are not supported on specific systems Containers do not really support .device, .automount or .swap units; Systems compiled without support for swap do not support .swap units; Systems without kdbus do not support .busname units. With this change attempts to start a unsupported unit types will result in an immediate "unsupported" job result, which is a lot more descriptive then before. Also, attempts to start device units in containers will now immediately fail instead of causing jobs to be enqueued that never go away.

core: correct spacing near eol in code comments

sd-bus: move common errors src/shared/bus-errors.h → src/libsystemd/sd-bus/bus-common-errors.h Stuff in src/shared/ should not use stuff from src/libsystemd/ really.

manager: log deserialization errors only at LOG_DEBUG level During upgrades and when transitioning between different systemd versions in initrd and on the host we have to expect that some serialization fields are unknown or parse incorrectly. This shouldn't really be considered an error, hence downgrade the log messages about it to debug. This way we can still trace it, but it doesn't confuse users. This kinda reverts 46849c3f.

core: OOM really shouldn't be considered a deserialization parse failure

mount: monitor for utab changes with inotify Parsing the mount table with libmount races against the mount command, which will handle the actual mounting before updating utab. This means the poll event on /proc/self/mountinfo can kick of a reparse in systemd before the utab information is available. This change adds in an additional event source using inotify to watch for changes to utab. It only watches for IN_MOVED_TO events, matching libmount behavior of always overwriting this file using rename(2). This does add a second pass through the mount table parsing when utab is updated.

treewide: another round of simplifications Using the same scripts as in f647962d64e "treewide: yet more log_*_errno + return simplifications".

treewide: use log_*_errno whenever %m is in the format string If the format string contains %m, clearly errno must have a meaningful value, so we might as well use log_*_errno to have ERRNO= logged. Using: find . -name '*.[ch]' | xargs sed -r -i -e \ 's/log_(debug|info|notice|warning|error|emergency)\((".*%m.*")/log_\1_errno(errno, \2/' Plus some whitespace, linewrap, and indent adjustments.

core: fix return value in error path after sd_event_add_io() failure sd_event_add_io() does not set errno, it returns negative errno. Noticed during log_*_errno conversions.

treewide: more log_*_errno + return simplifications

treewide: simplify log_*_errno(r,...) immediately followed by "return r"

treewide: more log_*_errno() conversions, multiline calls Basically: find . -name '*.[ch]' | while read f; do perl -i.mmm -e \ 'local $/; local $_=<>; s/log_(debug|info|notice|warning|error|emergency)\("([^"]*)%s"([^;]*),\s*strerror\(-?([->a-zA-Z_]+)\)\);/log_\1_errno(\4, "\2%m"\3);/gms;print;' \ $f; done Plus manual indentation fixups.

treewide: no need to negate errno for log_*_errno() It corrrectly handles both positive and negative errno values.

treewide: auto-convert the simple cases to log_*_errno() As a followup to 086891e5c1 "log: add an "error" parameter to all low-level logging calls and intrdouce log_error_errno() as log calls that take error numbers", use sed to convert the simple cases to use the new macros: find . -name '*.[ch]' | xargs sed -r -i -e \ 's/log_(debug|info|notice|warning|error|emergency)\("(.*)%s"(.*), strerror\(-([a-zA-Z_]+)\)\);/log_\1_errno(-\4, "\2%m"\3);/' Multi-line log_*() invocations are not covered. And we also should add log_unit_*_errno().

core: only override kdbus attach mask when running as PID 1

log: fix order of log_unit_struct() to match other logging calls Also, while we are at it, introduce some syntactic sugar for creating ERRNO= and MESSAGE= structured logging fields.

kdbus: set kernel attach mask before creating the first bus

log: rearrange log function naming - Rename log_meta() → log_internal(), to follow naming scheme of most other log functions that are usually invoked through macros, but never directly. - Rename log_info_object() to log_object_info(), simply because the object should be before any other parameters, to follow OO-style programming style.

manager: print fatal errors on the console too When booting in quiet mode, fatal messages would not be shown at all to the user. https://bugzilla.redhat.com/show_bug.cgi?id=1155468

manager: let manager_free() handle NULLs This makes the calling code a bit simpler.

dbus: return non-zero return value in the case that prefix won't match strv_extend returns 0 in the case of success which means that else if (bus_track_deserialize_item(&m->deserialized_subscribed, l) == 0) log_warning("Unknown serialization item '%s'", l); will be printed when value is added correctly.

sd-bus: sync with kdbus upstream (ABI break) kdbus has seen a larger update than expected lately, most notably with kdbusfs, a file system to expose the kdbus control files: * Each time a file system of this type is mounted, a new kdbus domain is created. * The layout inside each mount point is the same as before, except that domains are not hierarchically nested anymore. * Domains are therefore also unnamed now. * Unmounting a kdbusfs will automatically also detroy the associated domain. * Hence, the action of creating a kdbus domain is now as privileged as mounting a filesystem. * This way, we can get around creating dev nodes for everything, which is last but not least something that is not limited by 20-bit minor numbers. The kdbus specific bits in nspawn have all been dropped now, as nspawn can rely on the container OS to set up its own kdbus domain, simply by mounting a new instance. A new set of mounts has been added to mount things *after* the kernel modules have been loaded. For now, only kdbus is in this set, which is invoked with mount_setup_late().

manager: allow test run to catch SIGCHLD events Otherwise we cannot know when a service exited

core: remove unused macro GC_QUEUE_USEC_MAX It is unused since cf1265e188e876dda906dca0029248a06dc80c33

core: unify how we create the notify and private dbus socket Use the same robust logic of mkdir + unlink of any existing AF_UNIX socket, ignoring the return value, right before bind().

manager: cast mkdir() result to (void) to make sure coverity is quiet Also simplify the code a bit by moving mkdir to the common path.

manager: Ensure user's systemd runtime directory exists. This mirrors code in dbus.c when creating the private socket and avoids error messages like: systemd[1353]: bind(/run/user/603/systemd/notify) failed: No such file or directory systemd[1353]: Failed to fully start up daemon: No such file or directory

audit: improve the audit messages we generate always pass along comm, as documented by audit. Always set the correct comm value.

manager: do not print timing when running in test mode

Convert the rest to sd_bus_errnomap I tried to preserve most errno values, but in some cases they were inconsistent (different errno values for the same error name) or just mismatched.

manager: print warning on console before reboot It will be printed even if a prompt is blocking other messages.

manager: convert ephemeral to enum In preparation for subsequent changes.

manager: do not print anything while passwords are being queried https://bugs.freedesktop.org/show_bug.cgi?id=73942

core: remove system start timeout logic again The system start timeout as previously implemented would get confused by long-running services that are included in the initial system startup transaction for example by being cron-job-like long-running services triggered immediately at boot. Such long-running jobs would be subject to the default 15min timeout, esily triggering it. Hence, remove this again. In a subsequent commit, introduce per-target job timeouts instead, that allow us to control these timeouts more finegrained.

manager: Linux on hppa has fewer rtsigs, hence avoid using the higher ones there https://bugs.freedesktop.org/show_bug.cgi?id=84931

util: avoid duplication of TIME_T_MAX

systemd: try harder to bind to notify socket Without the socket open we are going to crash and burn. If for whatever reason we fail during deserialization we will fail when trying to open the socket. In this case it is better to unlink the old socket and maybe lose some messages, than to continue without the notification socket. Of course this situation should not happen, but we should handle it as gracefully as possible anyway. https://bugzilla.redhat.com/show_bug.cgi?id=1099299

systemd: log deserialization errors as warnings If we failed to parse something that we wrote ourselves, things are seriously off. This is also likely to lead to problems futher on.

core: fix resource leak in manager_environment_add Second error path must free the (potentially) allocated memory in the first code chunk before returning. Found by coverity. Fixes: CID#1237750

hashmap: introduce hash_ops to make struct Hashmap smaller It is redundant to store 'hash' and 'compare' function pointers in struct Hashmap separately. The functions always comprise a pair. Store a single pointer to struct hash_ops instead. systemd keeps hundreds of hashmaps, so this saves a little bit of memory.

manager: use correct cleanup function Close the dir instead of attempt to free it. Introduced in 874310b7b68c4c0d36ff07397db30a959bb7dae5 Found with coverity. Fixes: CID#996368

core: split up "starting" manager state into "initializing" and "starting" We'll stay in "initializing" until basic.target has reached, at which point we will enter "starting". This is preparation so that we can change the startip timeout to only apply to the first phase of startup, not the full procedure.

core: introduce "poweroff" as new failure action types Also, change the default action on a system start-up timeout to powering off.

core: print 'startup finished' messages even if we log to console

core: add support for a configurable system-wide start-up timeout When this system-wide start-up timeout is hit we execute one of the failure actions already implemented for services that fail. This should not only be useful on embedded devices, but also on laptops which have the power-button reachable when the lid is closed. This devices, when in a backpack might get powered on by accident due to the easily reachable power button. We want to make sure that the system turns itself off if it starts up due this after a while. When the system manages to fully start-up logind will suspend the machine by default if the lid is closed. However, in some cases we don't even get as far as logind, and the boot hangs much earlier, for example because we ask for a LUKS password that nobody ever enters. Yeah, this is a real-life problem on my Yoga 13, which has one of those easily accessible power buttons, even if the device is closed.

notify: send STOPPING=1 from our daemons

manager: don#t dispatch sd_notify() messages and SIGCHLD multiple times to the same units

manager: reuse sockaddr_union instead of redefining our own version of it

log: never ever log to syslog from PID 1, log to the journal again We don't support journal-less systems anyway, so let's avoid the confusion.

tty-ask-password-agent: modernization

time-util: add and use USEC/NSEC_INFINIY

systemd: return the first error from manager_startup()

test-engine: fix access to unit load path Also add a bit of debugging output to help diagnose problems, add missing units, and simplify cppflags. Move test-engine to normal tests from manual tests, it should now work without destroying the system.

Be more verbose when bind or listen fails Also be more verbose in devnode_acl_all().

core: fix oneshot service resource control Oneshot services's cgroup is removed when the service exits. An assert is hit otherwise.

firstboot: get rid of firstboot generator again, introduce ConditionFirstBoot= instead As Zbigniew pointed out a new ConditionFirstBoot= appears like the nicer way to hook in systemd-firstboot.service on first boots (those with /etc unpopulated), so let's do this, and get rid of the generator again.

firstboot: add new component to query basic system settings on first boot, or when creating OS images offline A new tool "systemd-firstboot" can be used either interactively on boot, where it will query basic locale, timezone, hostname, root password information and set it. Or it can be used non-interactively from the command line when prepareing disk images for booting. When used non-inertactively the tool can either copy settings from the host, or take settings on the command line. $ systemd-firstboot --root=/path/to/my/new/root --copy-locale --copy-root-password --hostname=waldi The tool will be automatically invoked (interactively) now on first boot if /etc is found unpopulated. This also creates the infrastructure for generators to be notified via an environment variable whether they are running on the first boot, or not.

cgroups: simplify CPUQuota= logic Only accept cpu quota values in percentages, get rid of period definition. It's not clear whether the CFS period controllable per-cgroup even has a future in the kernel, hence let's simplify all this, hardcode the period to 100ms and only accept percentage based quota values.

cgroup: rework startup logic Introduce a (unsigned long) -1 as "unset" state for cpu shares/block io weights, and keep the startup unit set around all the time.

core: add startup resource control option Similar to CPUShares= and BlockIOWeight= respectively. However only assign the specified weight during startup. Each control group attribute is re-assigned as weight by CPUShares=weight and BlockIOWeight=weight after startup. If not CPUShares= or BlockIOWeight= be specified, then the attribute is re-assigned to each default attribute value. (default cpu.shares=1024, blkio.weight=1000) If only CPUShares=weight or BlockIOWeight=weight be specified, then that implies StartupCPUShares=weight and StartupBlockIOWeight=weight.

core: make sure to serialize jobs for all units Previously we wouldn't serialize jobs for units that themselves have nothing to serialize. http://lists.freedesktop.org/archives/systemd-devel/2014-May/019051.html

Make systemctl --root look for files in the proper places Running systemctl enable/disable/set-default/... with the --root option under strace reveals that it accessed various files and directories in the main fs, and not underneath the specified root. This can lead to correct results only when the layout and configuration in the container are identical, which often is not the case. Fix this by adding the specified root to all file access operations. This patch does not handle some corner cases: symlinks which point outside of the specified root might be interpreted differently than they would be by the kernel if the specified root was the real root. But systemctl does not create such symlinks by itself, and I think this is enough of a corner case not to be worth the additional complexity of reimplementing link chasing in systemd. Also, simplify the code in a few places and remove an hypothetical memory leak on error.

core: expose CFS CPU time quota as high-level unit properties

core: do not read system boot timestamps in systemd --user mode Before: $ systemd-analyze --user Startup finished in 2.810s (firmware) + 48ms (loader) + 122ms (userspace) = 122ms After: $ systemd-analyze --user Startup finished in 122ms (userspace) = 122ms

core: add a setting to globally control the default for timer unit accuracy

util: replace close_pipe() with new safe_close_pair() safe_close_pair() is more like safe_close(), except that it handles pairs of fds, and doesn't make and misleading allusion, as it works similarly well for socketpairs() as for pipe()s...

sd-event: rework API to support CLOCK_REALTIME_ALARM and CLOCK_BOOTTIME_ALARM, too

core: move notify sockets to /run and $XDG_RUNTIME_DIR A service with PrivateNetwork= cannot access abstract namespace sockets of the host anymore, hence let's better not use abstract namespace sockets for this, since we want to make sure that PrivateNetwork= is useful and doesn't break sd_notify().

util: replace close_nointr_nofail() by a more useful safe_close() safe_close() automatically becomes a NOP when a negative fd is passed, and returns -1 unconditionally. This makes it easy to write lines like this: fd = safe_close(fd); Which will close an fd if it is open, and reset the fd variable correctly. By making use of this new scheme we can drop a > 200 lines of code that was required to test for non-negative fds or to reset the closed fd variable afterwards.

manager: use system state enum where appropriate

core: introduce system state enum The system state knows the states starting → running/degraded/maintenance → stopping, where: starting = system startup running = normal operation degraded = at least one unit is currently in failed state maintenance = rescue/emergency mode is active or queued stopping = system shutdown

manager: flush memory stream before using the buffer When the manager receives a SIGUSR2 signal, it opens a memory stream with open_memstream(), uses the returned file handle for logging, and dumps the logged content with log_dump(). However, the char* buffer is only safe to use after the file handle has been flushed with fflush, as the man pages states: When the stream is closed (fclose(3)) or flushed (fflush(3)), the locations pointed to by ptr and sizeloc are updated to contain, respectively, a pointer to the buffer and the current size of the buffer. These values remain valid only as long as the caller performs no further output on the stream. If further output is performed, then the stream must again be flushed before trying to access these variables. Without that call, dump remains NULL and the daemon crashes in log_dump().

util: add timeout to generator execution

missing: if RLIMIT_RTTIME is not defined by the libc, then we need a new define for the max number of rlimits, too

core: introduce new RuntimeDirectory= and RuntimeDirectoryMode= unit settings As discussed on the ML these are useful to manage runtime directories below /run for services.

bus: add sd_bus_track object for tracking peers, and port core over to it This is primarily useful for services that need to track clients which reference certain objects they maintain, or which explicitly want to subscribe to certain events. Something like this is done in a large number of services, and not trivial to do. Hence, let's unify this at one place. This also ports over PID 1 to use this to ensure that subscriptions to job and manager events are correctly tracked. As a side-effect this makes sure we properly serialize and restore the track list across daemon reexec/reload, which didn't work correctly before. This also simplifies how we distribute messages to broadcast to the direct busses: we only track subscriptions for the API bus and implicitly assume that all direct busses are subscribed. This should be a pretty OK simplification since clients connected via direct bus connections are shortlived anyway.

api: in constructor function calls, always put the returned object pointer first (or second) Previously the returned object of constructor functions where sometimes returned as last, sometimes as first and sometimes as second parameter. Let's clean this up a bit. Here are the new rules: 1. The object the new object is derived from is put first, if there is any 2. The object we are creating will be returned in the next arguments 3. This is followed by any additional arguments Rationale: For functions that operate on an object we always put that object first. Constructors should probably not be too different in this regard. Also, if the additional parameters might want to use varargs which suggests to put them last. Note that this new scheme only applies to constructor functions, not to all other functions. We do give a lot of freedom for those. Note that this commit only changes the order of the new functions we added, for old ones we accept the wrong order and leave it like that.

journald: log provenience of signals

manager: fix initialization of plymouth socket I'm not sure why this makes a difference...

core: use automatic cleanup in two functions

core: allow PIDs to be watched by two units at the same time In some cases it is interesting to map a PID to two units at the same time. For example, when a user logs in via a getty, which is reexeced to /sbin/login that binary will be explicitly referenced as main pid of the getty service, as well as implicitly referenced as part of the session scope.

core: watch SIGCHLD more closely to track processes of units with no reliable cgroup empty notifier When a process dies that we can associate with a specific unit, start watching all other processes of that unit, so that we can associate those processes with the unit too. Also, for service units start doing this as soon as we get the first SIGCHLD for either control or main process, so that we can follow the processes of the service from one to the other, as long as process that remain are processes of the ones we watched that died and got reassigned to us as parent. Similar, for scope units start doing this as soon as the scope controller abandons the unit, and thus management entirely reverts to systemd. To abandon a unit introduce a new Abandon() scope unit method call.

transaction: print more information about conflicting jobs Also remove some debug statement that should not have been committed.

bus: update kdbus.h (ABI break)

manager: remove "debugging" "feature"

manager: requeue the cylon eye for 5s later when a job finishes We'd reqeue the next status update very soon after. Change it so that we wait for full 5s without any job status changes until we print anything.

manager: print ephemeral information about running jobs' timeouts (v2) This reverts commit 28c758de94bc8ba97b89d9dab3f517cf466978d0 but makes job_coldplug smarter. In (v1) I changed the job start timestamp to be always set, so the start time can be reported in the cylon eye message. The bug was that when deserializing jobs, they would be ignored if their start timestamp was unset which was synonymous with no timeout. But after the change, jobs would have a start timestamp set despite having no timeout. After deserialization they would be considered immediately expired. Fix this by checking if the timeout is not zero when considering jobs for expiration.

Revert "manager: print ephemeral information about running jobs' timeouts" This reverts commit 2cba2e03524ec0922ddc70f933e8a89b7d23b4ec. It breaks bootup with dracut, the transition to the real rootfs fails.