tree-wide: remove Emacs lines from all files This should be handled fine now by .dir-locals.el, so need to carry that stuff in every file.

basic/terminal-util: introduce SYSTEMD_COLORS environment variable ... to determine if color output should be enabled. If the variable is not set, fall back to using on_tty(). Also, rewrite existing code to use colors_enabled() where appropriate.

bus-util: print "systemctl --user" on user service manager When a unit was started with "systemctl --user" and it failed, error messages is printed as "systemctl status". But it should be "systemctl --user status".

importd: drop dkr support The current code is not compatible with current dkr protocols anyway, and dkr has a different focus ("microservices") than nspawn anyway ("whole machine containers"), hence drop support for it, we cannot reasonably keep this up to date, and it creates the impression we'd actually care for the microservices usecase.

tree-wide: expose "p"-suffix unref calls in public APIs to make gcc cleanup easy GLIB has recently started to officially support the gcc cleanup attribute in its public API, hence let's do the same for our APIs. With this patch we'll define an xyz_unrefp() call for each public xyz_unref() call, to make it easy to use inside a __attribute__((cleanup())) expression. Then, all code is ported over to make use of this. The new calls are also documented in the man pages, with examples how to use them (well, I only added docs where the _unref() call itself already had docs, and the examples, only cover sd_bus_unrefp() and sd_event_unrefp()). This also renames sd_lldp_free() to sd_lldp_unref(), since that's how we tend to call our destructors these days. Note that this defines no public macro that wraps gcc's attribute and makes it easier to use. While I think it's our duty in the library to make our stuff easy to use, I figure it's not our duty to make gcc's own features easy to use on its own. Most likely, client code which wants to make use of this should define its own: #define _cleanup_(function) __attribute__((cleanup(function))) Or similar, to make the gcc feature easier to use. Making this logic public has the benefit that we can remove three header files whose only purpose was to define these functions internally. See #2008.

util-lib: split out allocation calls into alloc-util.[ch]

util-lib: move web-related calls into web-util.[ch]

util-lib: split string parsing related calls from util.[ch] into parse-util.[ch]

util-lib: split out fd-related operations into fd-util.[ch] There are more than enough to deserve their own .c file, hence move them over.

util-lib: get_current_dir_name() can return errors other than ENOMEM get_current_dir_name() can return a variety of errors, not just ENOMEM, hence don't blindly turn its errors to ENOMEM, but return correct errors in path_make_absolute_cwd(). This trickles down into a couple of other functions, some of which receive unrelated minor fixes too with this commit.

machinectl: accept "none" and "infinity" as specifier when dropping quotas using "machinectl set-limit" Previously, we already accepted "-" as special value for dropping limits. Add "infinity", as that's what we support for RLIMITs and hence should support here to. Also add "none" as that's what the btrfs tools use.

machined: when opening a shell via machined, pass tty fds in With this change we'll open the shell's tty right from machined and then pass it to the transient unit we create. This way we make sure the pty is opened exactly as long as the transient service is around, and no longer, and vice versa. This way pty forwarders do not have to deal with EIO problems due to vhangup, as the pty is open all the time from the point we set things up to the point where the service goes away.

machinectl: fix race when opening new shells with "machinectl shell" Previously, we'd allocate the TTY, spawn a service on it, but immediately start processing the TTY and forwarding it to whatever the commnd was started on. This is however problematic, as the TTY might get actually opened only much later by the service. We'll hence first get EIOs on the master as the other side is still closed, and hence considered it hung up and terminated the session. With this change we add a flag to the pty forwarding logic: PTY_FORWARD_IGNORE_INITIAL_VHANGUP. If set, we'll ignore all hangups (i.e. EIOs) on the master PTY until the first byte is successfully read. From that point on we consider a hangup/EIO a regular connection termination. This way, we handle the race: when we get EIO initially we'll ignore it, until the connection is properly set up, at which time we start honouring it.

bus-util: rename bus_open_transport() to bus_connect_transport() In sd-bus, the sd_bus_open_xyz() family of calls allocates a new bus, while sd_bus_default_xyz() family tries to reuse the thread's default bus. bus_open_transport() sometimes internally uses the former, sometimes the latter family, but suggests it only calls the former via its name. Hence, let's avoid this confusion, and generically rename the call to bus_connect_transport(). Similar for all related calls. And while we are at it, also change cgls + cgtop to do direct systemd connections where possible, since all they do is talk to systemd itself.

util: introduce common version() implementation and use it everywhere This also allows us to drop build.h from a ton of files, hence do so. Since we touched the #includes of those files, let's order them properly according to CODING_STYLE.

cgtop: underline table header Let's underline the header line of the table shown by cgtop, how it is customary for tables. In order to do this, let's introduce new ANSI underline macros, and clean up the existing ones as side effect.

tree-wide: never use the off_t unless glibc makes us use it off_t is a really weird type as it is usually 64bit these days (at least in sane programs), but could theoretically be 32bit. We don't support off_t as 32bit builds though, but still constantly deal with safely converting from off_t to other types and back for no point. Hence, never use the type anymore. Always use uint64_t instead. This has various benefits, including that we can expose these values directly as D-Bus properties, and also that the values parse the same in all cases.

tree-wide: drop {} from one-line if blocks Patch via coccinelle.

cgroup: drop "ignore_self" argument from cg_is_empty() In all cases where the function (or cg_is_empty_recursive()) ignoring the calling process is actually wrong, as a process keeps a cgroup busy regardless if its the current one or another. Hence, let's simplify things and drop the "ignore_self" parameter.

core: add unit_dbus_interface_from_type() to unit-name.h Let's add a way to get the type-specific D-Bus interface of a unit from either its type or name to src/basic/unit-name.[ch]. That way we can share it with the client side, where it is useful in tools like cgls or machinectl. Also ports over machinectl to make use of this.

machinectl: pass $TERM into "machinectl shell" sessions

machinectl: remove unused variable

machinectl: extend the "shell" syntax to take user@container names In order to make "machinectl shell" more similar to ssh, allow the following syntax to connect to a container under a specific username: machinectl shell lennart@fedora Also beefs up related man page documentation.

machinectl: make machine name parameters for "shell" and "login" optional If no machine name is specified, imply that we connect to ".host", i.e. the local host.

machinectl: don't show ".host" pseudo-machine in list by default Let's hide all machines whose name begins with "." by default, thus hiding the ".host" pseudo-machine, unless --all is specified. This takes inspiration from the ".host" image handling in "machinectl list-images" which also hides all images whose name starts with ".".

machined: introduce pseudo-machine ".host" refererring to the host system Some of the operations machined/machinectl implement are also very useful when applied to the host system (such as machinectl login, machinectl shell or machinectl status), hence introduce a pseudo-machine by the name of ".host" in machined that refers to the host system, and may be used top execute operations on the host system with. This copies the pseudo-image ".host" machined already implements for image related commands. (This commit also adds a PK privilege for opening a PTY in a container, which was previously not accessible for non-root.)

util: make machine_name_is_valid() a macro and move it to hostname-util.h As it turns out machine_name_is_valid() does the exact same thing as hostname_is_valid() these days, as it just invoked that and checked the name length was < 64. However, hostname_is_valid() checks the length against HOST_NAME_MAX anyway (which is 64 on Linux), hence any additional check is redundant. We hence replace machine_name_is_valid() by a macro that simply maps it to hostname_is_valid() but sets the allow_trailing_dot parameter to false. We also move this this call to hostname-util.h, to the same place as the hostname_is_valid() declaration.

machinectl: add new "machinectl shell" command This makes use of machined's new OpenShell() command and allows opening a new interactive shell in any container.

machinectl: support relative host paths in copy-{from,to}

sd-bus: introduce new sd_bus_flush_close_unref() call sd_bus_flush_close_unref() is a call that simply combines sd_bus_flush() (which writes all unwritten messages out) + sd_bus_close() (which terminates the connection, releasing all unread messages) + sd_bus_unref() (which frees the connection). The combination of this call is used pretty frequently in systemd tools right before exiting, and should also be relevant for most external clients, and is hence useful to cover in a call of its own. Previously the combination of the three calls was already done in the _cleanup_bus_close_unref_ macro, but this was only available internally. Also see #327

everywhere: port everything to sigprocmask_many() and friends This ports a lot of manual code over to sigprocmask_many() and friends. Also, we now consistly check for sigprocmask() failures with assert_se(), since the call cannot realistically fail unless there's a programming error. Also encloses a few sd_event_add_signal() calls with (void) when we ignore the return values for it knowingly.

tree-wide: fix memory leaks in users of bus_map_all_properties() If you use bus_map_all_properties(), you must be aware that it might touch output variables even though it may fail. This is, because we parse many different bus-properties and cannot tell how to clean them up, in case we fail deep down in the parser. Fix all callers of bus_map_all_properties() to correctly cleanup any context structures at all times.

util: split out signal-util.[ch] from util.[ch] No functional changes.

systemctl: introduce --now for enable, disable and mask https://bugs.freedesktop.org/show_bug.cgi?id=42940

machinectl: remove unused variable

core: rework unit name validation and manipulation logic A variety of changes: - Make sure all our calls distuingish OOM from other errors if OOM is not the only error possible. - Be much stricter when parsing escaped paths, do not accept trailing or leading escaped slashes. - Change unit validation to take a bit mask for allowing plain names, instance names or template names or an combination thereof. - Refuse manipulating invalid unit name

sd-bus: drop bus parameter from message callback prototype This should simplify the prototype a bit. The bus parameter is redundant in most cases, and in the few where it matters it can be derived from the message via sd_bus_message_get_bus().

shared: add terminal-util.[ch]

shared: add process-util.[ch]

tree-wide: there is no ENOTSUP on linux Replace ENOTSUP by EOPNOTSUPP as this is what linux actually uses.

man: document "machinectl export-tar" and "export-raw"

machinectl: remove unused variables

importd: add API for exporting container/VM images Also, expose it in machinectl.

machinectl: minor --help text improvements

importd: add new bus calls for importing local tar and raw images This also adds "machinectl import-raw" and "machinectl import-tar" to wrap these new bus calls. THe commands basically do for local files that "machinectl pull-raw" and friends do for remote files.

machinectl: update --help text to clarify that set-limit can also change pool size

machined,machinectl: add calls for changing container/VM quotas

machined/machinectl: when "machinectl image-status" is used without arguments show statistics about pool

remove unused includes This patch removes includes that are not used. The removals were found with include-what-you-use which checks if any of the symbols from a header is in use.

nspawn: when connected to pipes for stdin/stdout, pass them as-is to PID 1 Previously we always invoked the container PID 1 on /dev/console of the container. With this change we do so only if nspawn was invoked interactively (i.e. its stdin/stdout was connected to a TTY). In all other cases we directly pass through the fds unmodified. This has the benefit that nspawn can be added into shell pipelines. https://bugs.freedesktop.org/show_bug.cgi?id=87732

machinectl: issue all bus commands while allowing interactive auth

machined: make "machinectl copy-to" and "machinectl copy-from" server side operations This way, any bus client can make use of these calls.

machined: move logic for bind mounting into containers from machinectl to machined This extends the bus interface, adding BindMountMachine() for bind mounting directories from the host into the container.

tree-wide: whenever we include libgen.h, immediately undefine basename() Also, document in adjacent comments and in CODING_STYLE why we do that.

util: rework strappenda(), and rename it strjoina() After all it is now much more like strjoin() than strappend(). At the same time, add support for NULL sentinels, even if they are normally not necessary.

remove unused variables

treewide: fix multiple typos

machinectl: fix typo

importd: when listing transfers, show progress percentage With this change the pull protocol implementation processes will pass progress data to importd which then passes this information on via the bus. We use sd_notify() as generic transport for this communication, making importd listen to them, while matching the incoming messages to the right transfer.

machinectl: fix handling of --verify= argument for dkr downloads

machinectl: when downloading an image, clarify that C-c will not cancel the download, but continue it in the background

machinectl: minor simplification

machinectl: parse verify setting client-side

machinectl: various minor updates to the --help text

import: rename --verify=sum to --verify=checksum This is how we call it internally, and also a bit more descriptive.

import: introduce new mini-daemon systemd-importd, and make machinectl a client to it The old "systemd-import" binary is now an internal tool. We still use it as asynchronous backend for systemd-importd. Since the import tool might require some IO and CPU resources (due to qcow2 explosion, and decompression), and because we might want to run it with more minimal priviliges we still keep it around as the worker binary to execute as child process of importd. machinectl now has verbs for pulling down images, cancelling them and listing them.

machined: refer to the disk space allocated for an image to "usage" rather than "size" After all, it's closer to the "du"-reported value than to the file sizes...

machinectl: fix minor memory leak

machinectl: use GNU basename, not the XPG version

loginctl: port to generic verbs.h API

man: bring machinectl man page up-to-date

systemctl,loginctl: start polkit agent for all polkit enabled operations

machinectl: given that machinectl invokes a number of polkit enabled methods, start the polkit agent on terminals

machinectl: show most recent log output in "machinectl status", too

machinectl: prettify "machinectl list" output

ptyfwd: simplify how we handle vhangups a bit

machinectl: make sure that "machinectl login" exits immediately when the machine it is connected to dies

machinectl: remove spurious newline

machinectl: Check type instead of path before printing the type Looks like a typo when introduced in fefdc04b38725457a91651218feb7000f6ccc1f4

machinectl: remove dead code 'r' is not touched after the previous error-checking 100 lines above. Drop that code.

machinectl: add "enable" and "disable" verbs for enabling/disabling systemd-nspawn for containers This is basically just a shortcut for "systemctl enable systemd-nspawn@<foobar>.service", but does escaping.

machinectl: add new "start" verb to start a container as a service in nspawn

machined: add support for reporting image size via btrfs quota

machinectl/machined: implement "rename", "clone", "read-only" verbs for machine images

machined: add "machinectl remove" for removing images

machinectl: add status commands

machinectl: mark read-only images when listing in red

machinectl: left-align times

machined: beef up machined image listing with creation/modification times of subvolumes We make use of the btrfs subvol crtime for this, and for gpt images of a manually managed xattr, if we can.

sd-bus: rename sd_bus_open_system_container() to sd_bus_open_system_machine() Pretty much everywhere else we use the generic term "machine" when referring to containers in API, so let's do though in sd-bus too. In particular, since the concept of a "container" exists in sd-bus too, but as part of the marshalling system.

machined: introduce polkit for OpenLogin() call This way "machinectl login" can be opened up to run without privileges.

machinectl: rework 'machinectl login' to use OpenMachineLogin()

systemd-run: support -t mode when combined with -M For that, ask machined for a container PTY and use that.

machined: add OpenMachinePTY() bus call for allocating a PTY device within a container Then, port "machinectl" over to make use of it.

machinectl,nspawn: don't print extra final newline if pty terminal output was newline-terinated anyway

run: add a new "-t" mode for invoking a binary on an allocated TTY

machinectl: port machinectl to new verbs logic

machined/machinectl: add logic to show list of available images This adds a new bus call to machined that enumerates /var/lib/container and returns all trees stored in it, distuingishing three types: - GPT disk images, which are files suffixed with ".gpt" - directory trees - btrfs subvolumes

machinectl: add new commands for copying files from/to containers

machinectl: implement "bind" command to create additional bind mounts from host to container during runtime

treewide: another round of simplifications Using the same scripts as in f647962d64e "treewide: yet more log_*_errno + return simplifications".

treewide: use log_*_errno whenever %m is in the format string If the format string contains %m, clearly errno must have a meaningful value, so we might as well use log_*_errno to have ERRNO= logged. Using: find . -name '*.[ch]' | xargs sed -r -i -e \ 's/log_(debug|info|notice|warning|error|emergency)\((".*%m.*")/log_\1_errno(errno, \2/' Plus some whitespace, linewrap, and indent adjustments.

treewide: yet more log_*_errno + return simplifications Using: find . -name '*.[ch]' | while read f; do perl -i.mmm -e \ 'local $/; local $_=<>; s/(if\s*\([^\n]+\))\s*{\n(\s*)(log_[a-z_]*_errno\(\s*([->a-zA-Z_]+)\s*,[^;]+);\s*return\s+\g4;\s+}/\1\n\2return \3;/msg; print;' $f done And a couple of manual whitespace fixups.

treewide: no need to negate errno for log_*_errno() It corrrectly handles both positive and negative errno values.

treewide: auto-convert the simple cases to log_*_errno() As a followup to 086891e5c1 "log: add an "error" parameter to all low-level logging calls and intrdouce log_error_errno() as log calls that take error numbers", use sed to convert the simple cases to use the new macros: find . -name '*.[ch]' | xargs sed -r -i -e \ 's/log_(debug|info|notice|warning|error|emergency)\("(.*)%s"(.*), strerror\(-([a-zA-Z_]+)\)\);/log_\1_errno(-\4, "\2%m"\3);/' Multi-line log_*() invocations are not covered. And we also should add log_unit_*_errno().

ptyforward: rework PTY forwarder logic used by nspawn to utilize the normal event loop We really should not run manual event loops anymore, but standardize on sd_event, so that we can run sd_bus connections from it eventually.

bus: always explicitly close bus from main programs Since b5eca3a2059f9399d1dc52cbcf9698674c4b1cf0 we don't attempt to GC busses anymore when unsent messages remain that keep their reference, when they otherwise are not referenced anymore. This means that if we explicitly want connections to go away, we need to close them. With this change we will no do so explicitly wherver we connect to the bus from a main program (and thus know when the bus connection should go away), or when we create a private bus connection, that really should go away after our use. This fixes connection leaks in the NSS and PAM modules.

Unify parse_argv style getopt is usually good at printing out a nice error message when commandline options are invalid. It distinguishes between an unknown option and a known option with a missing arg. It is better to let it do its job and not use opterr=0 unless we actually want to suppress messages. So remove opterr=0 in the few places where it wasn't really useful. When an error in options is encountered, we should not print a lengthy help() and overwhelm the user, when we know precisely what is wrong with the commandline. In addition, since help() prints to stdout, it should not be used except when requested with -h or --help. Also, simplify things here and there.

machinectl: make sure we are not reading an unitialized variable

change type for address family to "int" Let's settle on a single type for all address family values, even if UNIX is very inconsitent on the precise type otherwise. Given that socket() is the primary entrypoint for the sockets API, and that uses "int", and "int" is relatively simple and generic, we settle on "int" for this.

machinectl: show network interface name for containers Also, append the if indexes as scope field to the addresses we show. That way they may be used for connecting to the containers directly.

machinectl: show /etc/os-release information of container in status output

machined: add logic to query IP addresses of containers

util: replace close_pipe() with new safe_close_pair() safe_close_pair() is more like safe_close(), except that it handles pairs of fds, and doesn't make and misleading allusion, as it works similarly well for socketpairs() as for pipe()s...

util: replace close_nointr_nofail() by a more useful safe_close() safe_close() automatically becomes a NOP when a negative fd is passed, and returns -1 unconditionally. This makes it easy to write lines like this: fd = safe_close(fd); Which will close an fd if it is open, and reset the fd variable correctly. By making use of this new scheme we can drop a > 200 lines of code that was required to test for non-negative fds or to reset the closed fd variable afterwards.

machinectl: reimplement machinectl's "reboot" verb on top of "kill", and add new verb "poweroff" There's really no point to send the reboot SIGINT from machinectl directly, if machined can do that anyway. This saves code, and makes machinectl network transparent for these verbs. And while we are at it we can easily add a "poweroff" verb in addition to "reboot". Yay!

sd-bus: the bus returned should be the first arg This matches the API of previous headers, such as sd-journal.h.

machinectl: remove unused --no-ask-password

machinectl: add bash completion

machinectl: add new "machinectl reboot" call

machined: optionally, allow registration of pre-existing units (scopes or services) as machine with machined

machinectl: fix success check when getting pty from within container

bus: when getting a kdbus connection into a container wait first for child, then read message There's no EOF generated for AF_UNIX/SOCK_DGRAM sockets, hence let's wait for the child first to see if it succeeded, only then read the socket.

machinectl: show_properties() already logs the error

bus: when entering an existing namespace to connect to a container's system bus also switch over PID namespace This is necessary to ensure that kdbus can collect creds of the destination namespace when connecting.

bus: when connecting to a container's kdbus instance, enter namespace first Previously we'd open the connection in the originating namespace, which meant most peers of the bus would not be able to make sense of the PID/UID/... identity of us since we didn't exist in the namespace they run in. However they require this identity for privilege decisions, hence disallowing access to anything from the host. Instead, when connecting to a container, create a temporary subprocess, make it join the container's namespace and then connect from there to the kdbus instance. This is similar to how we do it for socket conections already. THis also unifies the namespacing code used by machinectl and the bus APIs.

Help output spring cleaning Use [brackets] only for optional elements. Use <optional> in XML sources.

service: add the ability for units to join other unit's PrivateNetwork= and PrivateTmp= namespaces

machinectl: close bus to container early

machinectl: show list headers even if we pipe, since that appears to be the usual way

bus: log message parsing errors everywhere with a generalized bus_log_parse_error()

polkit: don't spawn local client if we access a remote system

bus: make bus_print_all_properties work for non machinectl cases add a destination parameter and skip properties we can't read

machinectl: we need glibc's socket.h, not our own

machinectl: use initialization instead of zeroing

clients: unify how we invoke getopt_long() Among other things this makes sure we always expose a --version command and show it in the help texts.

clients: various simplifications

bus: update bus_map_all_properties()

machinectl: fix warning when compiling with -Og src/machine/machinectl.c: In function ‘openpt_in_namespace’: src/machine/machinectl.c:623:44: warning: ‘master’ may be used uninitialized in this function [-Wmaybe-uninitialized] close_nointr_nofail(master); ^

machinectl: add new command to spawn a getty inside a container

run: add support for executing commands remotely via SSH or in a container Also, unify the transport logic a bit, since we reuse the same scheme in many of our client tools.

machinectl: we don't really just show information, we execute operations

bus: add API call to create bus connection to the system bus of local containers Also, add support for this to machinectl, so that we can enumerate the machines that run inside a container. We must go deeper!

bus: move ssh support into public API of libsystem-bus

machinectl: privileged option is gone

machinectl: port to sd-bus

loginctl: suppress cgroup tree output if cgroup is empty same for machinectl

loginctl: restore cgroup display for status output Same for machinectl.

man: document machinectl and systemd-machined

machined: split out machine registration stuff from logind Embedded folks don't need the machine registration stuff, hence it's nice to make this optional. Also, I'd expect that machinectl will grow additional commands quickly, for example to join existing containers and suchlike, hence it's better keeping that separate from loginctl.