tree-wide: remove Emacs lines from all files This should be handled fine now by .dir-locals.el, so need to carry that stuff in every file.

nspawn: make sure --help fits it 79ch

nspawn: optionally run a stub init process as PID 1 This adds a new switch --as-pid2, which allows running commands as PID 2, while a stub init process is run as PID 1. This is useful in order to run arbitrary commands in a container, as PID1's semantics are different from all other processes regarding reaping of unknown children or signal handling.

nspawn: add new --chdir= switch Fixes: #2192

nspawn: make sure --link-journal=host may be used twice in a row Fixes #2186 This fixes fall-out from 574edc90066c3faeadcf4666928ed9b0ac409c75.

nspawn: make journal linking non-fatal in try and auto modes Fixes #2091

nspawn: fix memory leak

capabilities: keep bounding set in non-inverted format. Change the capability bounding set parser and logic so that the bounding set is kept as a positive set internally. This means that the set reflects those capabilities that we want to keep instead of drop.

tree-wide: expose "p"-suffix unref calls in public APIs to make gcc cleanup easy GLIB has recently started to officially support the gcc cleanup attribute in its public API, hence let's do the same for our APIs. With this patch we'll define an xyz_unrefp() call for each public xyz_unref() call, to make it easy to use inside a __attribute__((cleanup())) expression. Then, all code is ported over to make use of this. The new calls are also documented in the man pages, with examples how to use them (well, I only added docs where the _unref() call itself already had docs, and the examples, only cover sd_bus_unrefp() and sd_event_unrefp()). This also renames sd_lldp_free() to sd_lldp_unref(), since that's how we tend to call our destructors these days. Note that this defines no public macro that wraps gcc's attribute and makes it easier to use. While I think it's our duty in the library to make our stuff easy to use, I figure it's not our duty to make gcc's own features easy to use on its own. Most likely, client code which wants to make use of this should define its own: #define _cleanup_(function) __attribute__((cleanup(function))) Or similar, to make the gcc feature easier to use. Making this logic public has the benefit that we can remove three header files whose only purpose was to define these functions internally. See #2008.

tree-wide: use right cast macros for UIDs, GIDs and PIDs

nspawn: add new --network-veth-extra= switch for defining additional veth links The new switch operates like --network-veth, but may be specified multiple times (to define multiple link pairs) and allows flexible definition of the interface names. This is an independent reimplementation of #1678, but defines different semantics, keeping the behaviour completely independent of --network-veth. It also comes will full hook-up for .nspawn files, and the matching documentation.

treewide: apply errno.cocci with small manual cleanups for style.

nspawn: support custom container service name We were hardcoding "systemd-nspawn" as the value of the $container env variable and "nspawn" as the service string in machined registration. This commit allows the user to configure it by setting the $SYSTEMD_NSPAWN_CONTAINER_SERVICE env variable when calling systemd-nspawn. If $SYSTEMD_NSPAWN_CONTAINER_SERVICE is not set, we use the string "systemd-nspawn" for both, fixing the previous inconsistency.

doc: correct orthography, word forms and missing/extraneous words

doc: correct punctuation and improve typography in documentation

nspawn: save errno before reopening log after exec failure

nspawn: no fake errno The S_ISREG test does not set errno, so don't use it in the error message.

nspawn: simplify error returns Use the "return log_error_errno(...)" idiom to have fewer curly braces. The last hunk also fixes the return value of setup_journal(), but the fix has no practical effect.

treewide: use the negative error codes returned by our functions Our functions return negative error codes. Do not rely on errno being set after calling our own functions.

core,nspawn: minor coding style fixes

nspwan: port to extract_first_word

util-lib: split out allocation calls into alloc-util.[ch]

util-lib: split out printf() helpers to stdio-util.h

src/basic: rename audit.[ch] → audit-util.[ch] and capability.[ch] → capability-util.[ch] The files are named too generically, so that they might conflict with the upstream project headers. Hence, let's add a "-util" suffix, to clarify that this are just our utility headers and not any official upstream headers.

util-lib: split out umask-related code to umask-util.h

util-lib: split stat()/statfs()/stavfs() related calls into stat-util.[ch]

util-lib: move a number of fs operations into fs-util.[ch]

util-lib: move mount related utility calls to mount-util.[ch]

util-lib: split string parsing related calls from util.[ch] into parse-util.[ch]

socket-util: move remaining socket-related calls from util.[ch] to socket-util.[ch]

util-lib: split out user/group/uid/gid calls into user-util.[ch]

util-lib: split out fd-related operations into fd-util.[ch] There are more than enough to deserve their own .c file, hence move them over.

util-lib: split our string related calls from util.[ch] into its own file string-util.[ch] There are more than enough calls doing string manipulations to deserve its own files, hence do something about it. This patch also sorts the #include blocks of all files that needed to be updated, according to the sorting suggestions from CODING_STYLE. Since pretty much every file needs our string manipulation functions this effectively means that most files have sorted #include blocks now. Also touches a few unrelated include files.

path-util: unify how we process paths specified on the command line Let's introduce a common function that makes relative paths absolute and warns about any errors while doing so.

util-lib: get_current_dir_name() can return errors other than ENOMEM get_current_dir_name() can return a variety of errors, not just ENOMEM, hence don't blindly turn its errors to ENOMEM, but return correct errors in path_make_absolute_cwd(). This trickles down into a couple of other functions, some of which receive unrelated minor fixes too with this commit.

nspawn: don't try to resolve passed binary before entering namespace Othewise we might follow the symlinks on the host, instead of the container. Fixes #1400

nspawn: rework how we determine private networking settings Make sure we acquire CAP_NET_ADMIN if we require virtual networking. Make sure we imply virtual ethernet correctly when bridge is request. Fixes: #1511 Fixes: #1554 Fixes: #1590

btrfs: beef-up btrfs support with a limited understanding of quota With this change we understand more than just leaf quota groups for btrfs file systems. Specifically: - When we create a subvolume we can now optionally add the new subvolume to all qgroups its parent subvolume was member of too. Alternatively it is also possible to insert an intermediary quota group between the parent's qgroups and the subvolume's leaf qgroup, which is useful for a concept of "subtree" qgroups, that contain a subvolume and all its children. - The remove logic for subvolumes has been updated to optionally remove any leaf qgroups or "subtree" qgroups, following the logic above. - The snapshot logic for subvolumes has been updated to replicate the original qgroup setup of the source, if it follows the "subtree" design described above. It will not cover qgroup setups that introduce arbitrary qgroups, especially those orthogonal to the subvolume hierarchy. This also tries to be more graceful when setting up /var/lib/machines as btrfs. For example, if mkfs.btrfs is missing we don't even try to set it up as loopback device. Fixes #1559 Fixes #1129

nspawn: skip /sys-as-tmpfs if we don't use private-network Since v3.11/7dc5dbc ("sysfs: Restrict mounting sysfs"), the kernel doesn't allow mounting sysfs if you don't have CAP_SYS_ADMIN rights over the network namespace. So the mounting /sys as a tmpfs code introduced in d8fc6a000fe21b0c1ba27fbfed8b42d00b349a4b doesn't work with user namespaces if we don't use private-net. The reason is that we mount sysfs inside the container and we're in the network namespace of the host but we don't have CAP_SYS_ADMIN over that namespace. To fix that, we mount /sys as a sysfs (instead of tmpfs) if we don't use private network and ignore the /sys-as-a-tmpfs code if we find that /sys is already mounted as sysfs. Fixes #1555

machinectl: fix race when opening new shells with "machinectl shell" Previously, we'd allocate the TTY, spawn a service on it, but immediately start processing the TTY and forwarding it to whatever the commnd was started on. This is however problematic, as the TTY might get actually opened only much later by the service. We'll hence first get EIOs on the master as the other side is still closed, and hence considered it hung up and terminated the session. With this change we add a flag to the pty forwarding logic: PTY_FORWARD_IGNORE_INITIAL_VHANGUP. If set, we'll ignore all hangups (i.e. EIOs) on the master PTY until the first byte is successfully read. From that point on we consider a hangup/EIO a regular connection termination. This way, we handle the race: when we get EIO initially we'll ignore it, until the connection is properly set up, at which time we start honouring it.

nspawn: mount /sys as tmpfs, and then mount only select subdirs of the real sysfs below it This way we can hide things like /sys/firmware or /sys/hypervisor from the container, while keeping the device tree around. While this is a security benefit in itself it also allows us to fix issue #1277. Previously we'd mount /sys before creating the user namespace, in order to be able to mount /sys/fs/cgroup/* beneath it (which resides in it), which we can only mount outside of the user namespace. To ensure that the user namespace owns the network namespace we'd set up the network namespace at the same time as the user namespace. Thus, we'd still see the /sys/class/net/ from the originating network namespace, even though we are in our own network namespace now. With this patch, /sys is mounted before transitioning into the user namespace as tmpfs, so that we can also mount /sys/fs/cgroup/* into it this early. The directories such as /sys/class/ are then later added in from the real sysfs from inside the network and user namespace so that they actually show whatis available in it. Fixes #1277

nspawn: fix user namespace support We didn#t actually pass ownership of /run to the UID in the container since some releases, let's fix that.

nspawn: order includes

util: introduce common version() implementation and use it everywhere This also allows us to drop build.h from a ton of files, hence do so. Since we touched the #includes of those files, let's order them properly according to CODING_STYLE.

util: unify implementation of NOP signal handler This is highly complex code after all, we really should make sure to only keep one implementation of this extremely difficult function around.

tree-wide: take benefit of the fact that fdset_free() returns NULL

tree-wide: port more code to use send_one_fd() and receive_one_fd() Also, make it slightly more powerful, by accepting a flags argument, and make it safe for handling if more than one cmsg attribute happens to be attached.

nspawn, machined: fix comments and error messages A bunch of "Client -> Child" fixes and one barrier-enumerator fix. (David: rebased on master)

nspawn: close unneeded sockets in outer child (David: Note, this is just a cleanup and doesn't fix any bugs)

util: introduce {send,receive}_one_fd() Introduce two new helpers that send/receive a single fd via a unix transport. Also make nspawn use them instead of hard-coding it. Based on a patch by Krzesimir Nowak.

tree-wide: never use the off_t unless glibc makes us use it off_t is a really weird type as it is usually 64bit these days (at least in sane programs), but could theoretically be 32bit. We don't support off_t as 32bit builds though, but still constantly deal with safely converting from off_t to other types and back for no point. Hence, never use the type anymore. Always use uint64_t instead. This has various benefits, including that we can expose these values directly as D-Bus properties, and also that the values parse the same in all cases.

nspawn: also close uid shift socket in the parent We should really close all parent sides of our child/parent socket pairs.

nspawn: short reads do not set errno, hence don't try to print it

inspawn: switch from SOCK_DGRAM to SOCK_SEQPACKET for internal socketpairs SOCK_DGRAM and SOCK_SEQPACKET have very similar semantics when used with socketpair(). However, SOCK_SEQPACKET has the advantage of knowing a hangup concept, since it is inherently connection-oriented. Since we use socket pairs to communicate between the nspawn main process and the nspawn child process, where the child might die abnormally it's interesting to us to learn about this via hangups if the child side of the pair is closed. Hence, let's switch to SOCK_SEQPACKET for these internal communication sockets. Fixes #956.

nspawn: properly propagate errors when we fail to set soemthing up

nspawn: sort and clean up included header list Let's remove unnecessary inclusions, and order the list alphabetically as suggested in CODING_STYLE now.

nspawn: remove nspawn.h, it's empty now

nspawn: split out --uid= logic into nspawn-setuid.[ch]

nspawn: split out machined registration code to nspawn-register.[ch]

nspawn: split out cgroup related calls into nspawn-cgroup.[ch]

nspawn: split out network related code to nspawn-network.[ch]

nspawn: split all port exposure code into nspawn-expose-port.[ch]

nspawn: split out mount related functions into a new nspawn-mount.c file

nspawn: add new .nspawn files for container settings .nspawn fiels are simple settings files that may accompany container images and directories and contain settings otherwise passed on the nspawn command line. This provides an efficient way to attach execution data directly to containers.

nspawn: enable all controllers we can for the "payload" subcgroup we create In the unified hierarchy delegating controller access is safe, hence make sure to enable all controllers for the "payload" subcgroup if we create it, so that the container will have all controllers enabled the nspawn service itself has.

core: unified cgroup hierarchy support This patch set adds full support the new unified cgroup hierarchy logic of modern kernels. A new kernel command line option "systemd.unified_cgroup_hierarchy=1" is added. If specified the unified hierarchy is mounted to /sys/fs/cgroup instead of a tmpfs. No further hierarchies are mounted. The kernel command line option defaults to off. We can turn it on by default as soon as the kernel's APIs regarding this are stabilized (but even then downstream distros might want to turn this off, as this will break any tools that access cgroupfs directly). It is possibly to choose for each boot individually whether the unified or the legacy hierarchy is used. nspawn will by default provide the legacy hierarchy to containers if the host is using it, and the unified otherwise. However it is possible to run containers with the unified hierarchy on a legacy host and vice versa, by setting the $UNIFIED_CGROUP_HIERARCHY environment variable for nspawn to 1 or 0, respectively. The unified hierarchy provides reliable cgroup empty notifications for the first time, via inotify. To make use of this we maintain one manager-wide inotify fd, and each cgroup to it. This patch also removes cg_delete() which is unused now. On kernel 4.2 only the "memory" controller is compatible with the unified hierarchy, hence that's the only controller systemd exposes when booted in unified heirarchy mode. This introduces a new enum for enumerating supported controllers, plus a related enum for the mask bits mapping to it. The core is changed to make use of this everywhere. This moves PID 1 into a new "init.scope" implicit scope unit in the root slice. This is necessary since on the unified hierarchy cgroups may either contain subgroups or processes but not both. PID 1 hence has to move out of the root cgroup (strictly speaking the root cgroup is the only one where processes and subgroups are still allowed, but in order to support containers nicey, we move PID 1 into the new scope in all cases.) This new unit is also used on legacy hierarchy setups. It's actually pretty useful on all systems, as it can then be used to filter journal messages coming from PID 1, and so on. The root slice ("-.slice") is now implicitly created and started (and does not require a unit file on disk anymore), since that's where "init.scope" is located and the slice needs to be started before the scope can. To check whether we are in unified or legacy hierarchy mode we use statfs() on /sys/fs/cgroup. If the .f_type field reports tmpfs we are in legacy mode, if it reports cgroupfs we are in unified mode. This patch set carefuly makes sure that cgls and cgtop continue to work as desired. When invoking nspawn as a service it will implicitly create two subcgroups in the cgroup it is using, one to move the nspawn process into, the other to move the actual container processes into. This is done because of the requirement that cgroups may either contain processes or other subgroups.

nspawn: don't try to extract quotes from option string, glibc doesn't do that either Follow-up regarding #649.

nspawn: add (no)rbind option to --bind and --bind-ro --bind and --bind-ro perform the bind mount non-recursively. It is sometimes (often?) desirable to do a recursive mount. This patch adds an optional set of bind mount options in the form of: --bind=src-path:dst-path:options options are comma separated and currently only "rbind" and "norbind" are allowed. Default value is "rbind".

nspawn: make sure --template= and --machine= my be combined Fixes #1018. Based on a patch from Seth Jennings.

remove unused variables

nspawn: Allow : characters in overlay paths : characters can be entered with the \: escape sequence.

nspawn: escape paths in overlay mount options Overlayfs uses , as an option separator and : as a list separator. These characters are both valid in file paths, so overlayfs allows file paths which contain these characters to backslash escape these values.

nspawn: Allow : characters in nspawn --bind paths : characters in bind paths can be entered as the \: escape sequence.

nspawn: Allow : characters in --tmpfs path This now accepts : characters with the \: escape sequence. Other escape sequences are also interpreted, but having a \ in your file path is less likely than :, so this shouldn't break anyone's existing tools.

hostname-util: get rid of unused parameter of hostname_cleanup() All users are now setting lowercase=false.

tree-wide: introduce mfree() Pretty trivial helper which wraps free() but returns NULL, so we can simplify this: free(foobar); foobar = NULL; to this: foobar = mfree(foobar);

tree-wide: use free_and_strdup() Use free_and_strdup() where appropriate and replace equivalent, open-coded versions.

nspawn: Don't pass uid mount option for devpts Mounting devpts with a uid breaks pty allocation with recent glibc versions, which expect that the kernel will set the correct owner for user-allocated ptys. The kernel seems to be smart enough to use the correct uid for root when we switch to a user namespace. This resolves #337.

Remove repeated 'the's

tree-wide: fix write_string_file() user that should not create files The latest consolidation cleanup of write_string_file() revealed some users of that helper which should have used write_string_file_no_create() in the past but didn't. Basically, all existing users that write to files in /sys and /proc should not expect to write to a file which is not yet existant.

fileio: consolidate write_string_file*() Merge write_string_file(), write_string_file_no_create() and write_string_file_atomic() into write_string_file() and provide a flags mask that allows combinations of atomic writing, newline appending and automatic file creation. Change all users accordingly.

nspawn: Communicate determined UID shift to parent There is logic to determine the UID shift from the file-system, rather than having it be explicitly passed in. However, this needs to happen in the child process that sets up the mounts, as what's important is the UID of the mounted root, rather than the mount-point. Setting up the UID map needs to happen in the parent becuase the inner child needs to have been started, and the outer child is no longer able to access the uid_map file, since it lost access to it when setting up the mounts for the inner child. So we need to communicate the uid shift back out, along with the PID of the inner child process. Failing to communicate this means that the invalid UID shift, which is the value used to specify "this needs to be determined from the file system" is left invalid, so setting up the user namespace's UID shift fails.

nspawn: fix indenting

sd-bus: introduce new sd_bus_flush_close_unref() call sd_bus_flush_close_unref() is a call that simply combines sd_bus_flush() (which writes all unwritten messages out) + sd_bus_close() (which terminates the connection, releasing all unread messages) + sd_bus_unref() (which frees the connection). The combination of this call is used pretty frequently in systemd tools right before exiting, and should also be relevant for most external clients, and is hence useful to cover in a call of its own. Previously the combination of the three calls was already done in the _cleanup_bus_close_unref_ macro, but this was only available internally. Also see #327

Revert "nspawn: determine_uid_shift before forking"

nspawn: determine_uid_shift before forking It is needed in one branch of the fork, but calculated in another branch. Failing to do this means using --private-users without specifying a uid shift always fails because it tries to shift the uid to UID_INVALID.

nspawn: Don't remount with fewer options When we do a MS_BIND mount, it inherits the flags of its parent mount. When we do a remount, it sets the flags to exactly what is specified. If we are in a user namespace then these mount points have their flags locked, so you can't reduce the protection. As a consequence, the default setup of mount_all doesn't work with user namespaces. However if we ensure we add the mount flags of the parent mount when remounting, then we aren't removing mount options, so we aren't trying to unlock an option that we aren't allowed to.

nspawn: suppress warning when /etc/resolv.conf is a valid symlink In such a case let's suppress the warning (downgrade to LOG_DEBUG), under the assumption that the user has no config file to update in its place, but a symlink that points to something like resolved's automatically managed resolve.conf file. While we are at it, also stop complaining if we cannot write /etc/resolv.conf due to a read-only disk, given that there's little we could do about it.

nspawn: when exiting, flush all remaining bytes from the pty to stdout This is a simpler fix for #210, it simply uses copy_bytes() for the copying.

nspawn: check if kernel supports userns as early as possible If the kernel do not support user namespace then one of the children created by nspawn parent will fail at clone(CLONE_NEWUSER) with the generic error EINVAL and without logging the error. At the same time the parent may also try to setup the user namespace and will fail with another error. To improve this, check if the kernel supports user namespace as early as possible.

everywhere: port everything to sigprocmask_many() and friends This ports a lot of manual code over to sigprocmask_many() and friends. Also, we now consistly check for sigprocmask() failures with assert_se(), since the call cannot realistically fail unless there's a programming error. Also encloses a few sd_event_add_signal() calls with (void) when we ignore the return values for it knowingly.

tmpfiles: automatically remove old machine snapshots at boot Remove old temporary snapshots, but only at boot. Ideally we'd have "self-destroying" btrfs snapshots that go away if the last last reference to it does. To mimic a scheme like this at least remove the old snapshots on fresh boots, where we know they cannot be referenced anymore. Note that we actually remove all temporary files in /var/lib/machines/ at boot, which should be safe since the directory has defined semantics. In the root directory (where systemd-nspawn --ephemeral places snapshots) we are more strict, to avoid removing unrelated temporary files. This also splits out nspawn/container related tmpfiles bits into a new tmpfiles snippet to systemd-nspawn.conf

util: when creating temporary file names, allow including extra id string in it This adds a "char *extra" parameter to tempfn_xxxxxx(), tempfn_random(), tempfn_ranomd_child(). If non-NULL this string is included in the middle of the newly created file name. This is useful for being able to distuingish the kind of temporary file when we see one. This also adds tests for the three call. For now, we don't make use of this at all, but port all users over.

firewall: rename fw-util.[ch] → firewall-util.[ch] The names fw-util.[ch] are too ambiguous, better rename the files to firewall-util.[ch]. Also rename the test accordingly.

nspawn: make seccomp loading errors non-fatal seccomp_load returns -EINVAL when seccomp support is not enabled in the kernel [1]. This should be a debug log, not an error that interrupts nspawn. If the seccomp filter can't be set and audit is enabled, the user will get an error message anyway. [1]: http://man7.org/linux/man-pages/man2/prctl.2.html

sd-netlink: rename from sd-rtnl

sd-rtnl: make joining broadcast groups implicit

tree-wide: whenever we fork off a foreign child process reset signal mask/handlers Also, when the child is potentially long-running make sure to set a death signal. Also, ignore the result of the reset operations explicitly by casting them to (void).

util: split out signal-util.[ch] from util.[ch] No functional changes.

path-util: Change path_is_mount_point() symlink arg from bool to flags This makes path_is_mount_point() consistent with fd_is_mount_point() wrt. flags.

nspawn: fix memleak This was a typo, swapping prefix_root() in place of prefix_roota(). Fixes CID 1299640.

nspawn: avoid memleak Simplify the code a bit, at the cost of potentially duplicating some memory unneccessarily. Fixes CID 1299641.

nspawn: drop some debugging code These have no effect. Fixes CID 1299643.

nspawn: make coverity happy Rather than checking the return of asprintf() we are checking if buf gets allocated, make it clear that it is ok to ignore the return value. Fixes CID 1299644.

nspawn: be verbose about interface names Allowed interface name is relatively small. Lets not make users go in to the source code to figure out what happened. --machine=debian-tree conflicts with --machine=debian-tree2 ex: Failed to add new veth \ interfaces (host0, vb-debian-tree): File exists

nspawn: prohibit access to the kernel log buffer by default Unless CAP_SYSLOG is explicitly passed block all access to kmg

util: introduce PERSONALITY_INVALID as macro for 0xffffffffLU

nspawn: finish user namespace support

core,nspawn: unify code that moves the root dir

nspawn: close extra fds before execing init When systemd-nspawn gets exec*()ed, it inherits the followings file descriptors: - 0, 1, 2: stdin, stdout, stderr - SD_LISTEN_FDS_START, ... SD_LISTEN_FDS_START+LISTEN_FDS: file descriptors passed by the system manager (useful for socket activation). They are passed to the child process (process leader). - extra lock fd: rkt passes a locked directory as an extra fd, so the directory remains locked as long as the container is alive. systemd-nspawn used to close all open fds except 0, 1, 2 and the SD_LISTEN_FDS_START..SD_LISTEN_FDS_START+LISTEN_FDS. This patch delays the close just before the exec so the nspawn process (parent) keeps the extra fds open. This patch supersedes the previous attempt ("cloexec extraneous fds"): http://lists.freedesktop.org/archives/systemd-devel/2015-May/031608.html

util: split all hostname related calls into hostname-util.c

nspawn: allow access to device nodes listed in --bind= and --bind-ro= switches https://bugs.freedesktop.org/show_bug.cgi?id=90385

nspawn: skip symlink to a combined cgroup hierarchy if it already exists If a symlink to a combined cgroup hierarchy already exists and points to the right path, skip it. This avoids an error when the cgroups are set manually before calling nspawn.

nspawn: only mount the cgroup root if it's not already mounted This allows the user to set the cgroups manually before calling nspawn.

nspawn: rework custom mount point order, and add support for overlayfs Previously all bind mount mounts were applied in the order specified, followed by all tmpfs mounts in the order specified. This is problematic, if bind mounts shall be placed within tmpfs mounts. This patch hence reworks the custom mount point logic, and alwas applies them in strict prefix-first order. This means the order of mounts specified on the command line becomes irrelevant, the right operation will always be executed. While we are at it this commit also adds native support for overlayfs mounts, as supported by recent kernels.

nspawn: pass on kill signal setting to contaner scope Let's just pass on what the user set for us.

nspawn: when run as a service, don't ask machined for terminatin of ourselves

nspawn: make sure we install the device policy if nspawn is run as unit as on the command line

nspawn: don't inherit read-only flag from disk image if --ephemeral is used When --ephemeral is used there's no need to keep the image read-only, so let's not do that then.

tree-wide: get rid of more strerror() calls

shared: add terminal-util.[ch]

shared: add random-util.[ch]

shared: add process-util.[ch]

shared: add formats-util.h

path-util: fix more path_is_mount e792e890f fallout

btrfs: add support for recursive btrfs snapshotting

btrfs: make btrfs_subvol_snapshot() parameters a flags field

btrfs: support recursively removing btrfs snapshots

util: rework rm_rf() logic - Move to its own file rm-rf.c - Change parameters into a single flags parameter - Remove "honour sticky" logic, it's unused these days

nspawn: fallback on bind mount when mknod fails Some systems abusively restrict mknod, even when the device node already exists in /dev. This is unfortunate because it prevents systemd-nspawn from creating the basic devices in /dev in the container. This patch implements a workaround: when mknod fails, fallback on bind mounts. Additionally, /dev/console was created with a mknod with the same major/minor as /dev/null before bind mounting a pts on it. This patch removes the mknod and creates an empty regular file instead. In order to test this patch, I used the following configuration, which I think should replicate the system with the abusive restriction on mknod: # grep devices /proc/self/cgroup 4:devices:/user.slice/restrict # cat /sys/fs/cgroup/devices/user.slice/restrict/devices.list c 1:9 r c 5:2 rw c 136:* rw # systemd-nspawn --register=false -D . v2: - remove "bind", it is not needed since there is already MS_BIND v3: - fix error management when calling touch() - fix lowercase in error message

nspawn: drop sd_booted() check We have no such check in any of the other tools, hence don't have one in nspawn either. (This should make things nicer for Rocket, among other things) Note: removing this check does not mean that we support running nspawn on non-systemd. We explicitly don't. It just means that we remove the check for running it like that. You are still on your own if you do...

nspawn: change filesystem type from "bind" to NULL in mount() syscalls Try to keep syscalls as minimal as possible.

nspawn: tell coverity that we ignore return value CID #1271353.

tree-wide: there is no ENOTSUP on linux Replace ENOTSUP by EOPNOTSUPP as this is what linux actually uses.

nspawn: fix use-after-free and leak in error paths CID #1257765.

nspawn: Map all seccomp filters to capabilities This change makes it so all seccomp filters are mapped to the appropriate capability and are only added if that capability was not requested when running the container. This unbreaks the remaining use cases broken by the addition of seccomp filters without respecting requested capabilities. Co-Authored-By: Clif Houck <me@clifhouck.com> [zj: - adapt to our coding style, make struct anonymous]

nspawn: make kill signal to use for PID 1 configurable

remove unused includes This patch removes includes that are not used. The removals were found with include-what-you-use which checks if any of the symbols from a header is in use.

nspawn: fix whitespace and typo in partition table blurb

nspawn: chown basic device nodes to userns root

nspawn: fix build on non-selinux systems

nspawn: add basic user namespacing support (This is incomplete, /proc and /sys are still owned by root from outside the container, not inside)

nspawn: when connected to pipes for stdin/stdout, pass them as-is to PID 1 Previously we always invoked the container PID 1 on /dev/console of the container. With this change we do so only if nspawn was invoked interactively (i.e. its stdin/stdout was connected to a TTY). In all other cases we directly pass through the fds unmodified. This has the benefit that nspawn can be added into shell pipelines. https://bugs.freedesktop.org/show_bug.cgi?id=87732

nspawn: add support for --property= to set scope properties This is similar to systemd-run's --property= setting.

nspawn: Allow module loading if CAP_SYS_MODULE is requested nspawn containers currently block module loading in all cases, with no option to disable it. This allows an admin, specifically setting capability=CAP_SYS_MODULE or capability=all to load modules.

util: rework strappenda(), and rename it strjoina() After all it is now much more like strjoin() than strappend(). At the same time, add support for NULL sentinels, even if they are normally not necessary.

remove unused variables

nspawn: when mounting the cgroup hierarchies, use the exact same mount options for the superblock as the host Otherwise we'll generate kernel runtime warnings about non-matching mount options.

nspawn: mount /tmp in the container, don't leave this to the container's init We really want /tmp to be properly mounted, especially in containers that lack CAP_SYS_ADMIN or that are not fully booted up and only get a shell, hence let's do so in nspawn already.

nspawn: allow bind-mounting char and block files

nspawn: work around kernel bug with partition table probing on loopback devices When we set up a loopback device with partition probing, the udev "change" event about the configured device is first passed on to userspace, only the the in-kernel partition prober is started. Since partition probing fails with EBUSY when somebody has the device open, the probing frequently fails since udev starts probing/opening the device as soon as it gets the notification about it, and it might do so earlier than the kernel probing. This patch adds a (hopefully temporary) work-around for this, that compares the number of probed partitions of the kernel with those of blkid and synchronously asks for reprobing until the numebrs are in sync. This really deserves a proper kernel fix.

nspawn: add ipvlan support

nspawn: support dissecting GPT images that contain only a single generic linux partition This should allow running Ubuntu UEFI GPT Images with nspawn, unmodified.

inspawn: wait until udev has probed a loopback device before making us of it

nspawn: fix log typos

import: rename "gpt" disk image type to "raw" After all, nspawn can now dissect MBR partition levels, too, hence ".gpt" appears a misnomer. Moreover, the the .raw suffix for these files is already pretty popular (the Fedora disk images use it for example), hence sounds like an OK scheme to adopt.

spawn: downgrade loopback detach errors to debug Sometimes udev or some other background daemon might keep the loopback devices busy while we already want to detach them. Downgrade the warning about it. Given that we use autodetach downgrading these messages should be with little risk.

nspawn: add support for limited dissecting of MBR disk images with nspawn With this change nspawn's -i switch now can now make sense of MBR disk images too - however only if there's only a single, bootable partition of type 0x83 on the image. For all other cases we cannot really make sense from the partition table alone. The big benefit of this change is that upstream Fedora Cloud Images can now be booted unmodified with systemd-nspawn: # wget http://download.fedoraproject.org/pub/fedora/linux/releases/21/Cloud/Images/x86_64/Fedora-Cloud-Base-20141203-21.x86_64.raw.xz # unxz Fedora-Cloud-Base-20141203-21.x86_64.raw.xz # systemd-nspawn -i Fedora-Cloud-Base-20141203-21.x86_64.raw -b Next stop: teach the import logic to automatically download these images, uncompress and verify them.

nspawn: pass the container's init PID out via sd_notify() This is useful for nspawn managers that want to learn when nspawn is finished with initialiuzation, as well what the PID of the init system in the container is.

nspawn: fix an incorrect assert comparison

nspawn: add file system locks for controlling access to container images This adds three kinds of file system locks for container images: a) a file system lock next to the actual image, in a .lck file in the same directory the image is located. This lock has the benefit of usually being located on the same NFS share as the image itself, and thus allows locking container images across NFS shares. b) a file system lock in /run, named after st_dev and st_ino of the root of the image. This lock has the advantage that it is unique even if the same image is bind mounted to two different places at the same time, as the ino/dev stays constant for them. c) a file system lock that is only taken when a new disk image is about to be created, that ensures that checking whether the name is already used across the search path, and actually placing the image is not interrupted by other code taking the name. a + b are read-write locks. When a container is booted in read-only mode a read lock is taken, otherwise a write lock. Lock b is always taken after a, to avoid ABBA problems. Lock c is mostly relevant when renaming or cloning images.

nspawn: remove the right propagation directory

nspawn: --help typo fix

nspawn: add "-n" shortcut for "--network-veth" Now that networkd's IP masquerading support means that running containers with "--network-veth" will provide network access out of the box for the container, let's add a shortcut "-n" for it, to make it easily accessible.

nspawn: add new option "--port=" for exposing container ports on the local host This exposes an IP port on the container as local port using DNAT.

machined: when cloning a raw disk image, also set the NOCOW flag

nspawn: fix error message when mknod fails

machinectl: make sure that "machinectl login" exits immediately when the machine it is connected to dies

nspawn: mount most of the cgroup tree read-only in nspawn containers except for the container's own subtree in the name=systemd hierarchy More specifically mount all other hierarchies in their entirety and the name=systemd above the container's subtree read-only.

nspawn: report back to systemd only very late whether we are OK That way, systemd can actually figure out if everything is OK with nspawn.

nspawn: use the same image discovery logic in nspawn as in machined

nspawn: remove spurious include of <sys/capability.h> It does not use any functions from libcap directly. The CAP_* constants in use through this file come from "missing.h" which will import <linux/capability.h> and complement it with CAP_* constants not defined by the current kernel headers. Add an explicit import of our "capability.h" since it does use the function capability_bounding_set_drop from that header file. Previously, that header was implicitly imported through through "cap-list.h". Tested that "systemd-nspawn" builds cleanly and works after this change.

nspawn,pty: port over to new ptsname_malloc() helper

machinectl,nspawn: don't print extra final newline if pty terminal output was newline-terinated anyway

run: add a new "-t" mode for invoking a binary on an allocated TTY

machinectl: implement "bind" command to create additional bind mounts from host to container during runtime

nspawn: fix invocation of the raw clone() system call on s390 and cris Since the order of the first and second arguments of the raw clone() system call is reversed on s390 and cris it needs to be invoked differently.

nspawn: when booting in ephemeral mode, append random token to machine name Also, when booting up an ephemeral container of / use the system hostname as default machine name. This way specifiyng -M is unnecessary when booting up an ephemeral container, while allowing any number of ephemeral containers to run from the same tree.

nspawn: allow spawning ephemeral nspawn containers based on the root file system of the OS This works now: # systemd-nspawn -xb -D / -M foobar Which boots up an ephemeral container, based on the host's root file system. Or in other words: you can now run the very same host OS you booted your system with also in a container, on top of it, without having it interfere. Great for testing whether the init system you are hacking on still boots without reboot the system!

nspawn: don't link journals in ephemeral mode

nspawn: properly unset arg_link_journal_try, when --link-journal= is specified

nspawn: beef up nspawn with some btrfs magic This adds --template= to duplicate an OS tree as btrfs snpashot and run it This also adds --ephemeral or -x to create a snapshot of an OS tree and boot that, removing it after exit.

nspawn: properly validate machine names

util: introduce our own gperf based capability list This way, we can ensure we have a more complete, up-to-date list of capabilities around, always.

nspawn: create the macvlan MAC addresses in an arch independent stable way

nspawn: make sure macvlan MAC addresses are stable https://bugs.freedesktop.org/show_bug.cgi?id=85527

nspawn: correct EEXIST check when creating directory to mount /tmp in https://bugs.freedesktop.org/show_bug.cgi?id=86309

nspawn: fix unused variable warning

delta: diff returns 1 when files differ, ignore this https://bugs.debian/org/771397

treewide: another round of simplifications Using the same scripts as in f647962d64e "treewide: yet more log_*_errno + return simplifications".

treewide: use log_*_errno whenever %m is in the format string If the format string contains %m, clearly errno must have a meaningful value, so we might as well use log_*_errno to have ERRNO= logged. Using: find . -name '*.[ch]' | xargs sed -r -i -e \ 's/log_(debug|info|notice|warning|error|emergency)\((".*%m.*")/log_\1_errno(errno, \2/' Plus some whitespace, linewrap, and indent adjustments.

treewide: yet more log_*_errno + return simplifications Using: find . -name '*.[ch]' | while read f; do perl -i.mmm -e \ 'local $/; local $_=<>; s/(if\s*\([^\n]+\))\s*{\n(\s*)(log_[a-z_]*_errno\(\s*([->a-zA-Z_]+)\s*,[^;]+);\s*return\s+\g4;\s+}/\1\n\2return \3;/msg; print;' $f done And a couple of manual whitespace fixups.

treewide: no need to negate errno for log_*_errno() It corrrectly handles both positive and negative errno values.

treewide: auto-convert the simple cases to log_*_errno() As a followup to 086891e5c1 "log: add an "error" parameter to all low-level logging calls and intrdouce log_error_errno() as log calls that take error numbers", use sed to convert the simple cases to use the new macros: find . -name '*.[ch]' | xargs sed -r -i -e \ 's/log_(debug|info|notice|warning|error|emergency)\("(.*)%s"(.*), strerror\(-([a-zA-Z_]+)\)\);/log_\1_errno(-\4, "\2%m"\3);/' Multi-line log_*() invocations are not covered. And we also should add log_unit_*_errno().

nspawn: ignore EEXIST when mounting tmpfs commit 79d80fc1466512d0ca211f4bfcd9de5f2f816a5a introduced a regression that prevents mounting a tmpfs if the mount point already exits in the container's root file system. This commit fixes the problem by ignoring EEXIST.

nspawn: Add try-{host,guest} journal link modes --link-journal={host,guest} fail if the host does not have persistent journalling enabled and /var/log/journal/ does not exist. Even worse, as there is no stdout/err any more, there is no error message to point that out. Introduce two new modes "try-host" and "try-guest" which don't fail in this case, and instead just silently skip the guest journal setup. Change -j to mean "try-guest" instead of "guest", and fix the wrong --help output for it (it said "host" before). Change systemd-nspawn@.service.in to use "try-guest" so that this unit works with both persistent and non-persistent journals on the host without failing. https://bugs.debian.org/770275

sd-bus: sync with kdbus upstream (ABI break) kdbus has seen a larger update than expected lately, most notably with kdbusfs, a file system to expose the kdbus control files: * Each time a file system of this type is mounted, a new kdbus domain is created. * The layout inside each mount point is the same as before, except that domains are not hierarchically nested anymore. * Domains are therefore also unnamed now. * Unmounting a kdbusfs will automatically also detroy the associated domain. * Hence, the action of creating a kdbus domain is now as privileged as mounting a filesystem. * This way, we can get around creating dev nodes for everything, which is last but not least something that is not limited by 20-bit minor numbers. The kdbus specific bits in nspawn have all been dropped now, as nspawn can rely on the container OS to set up its own kdbus domain, simply by mounting a new instance. A new set of mounts has been added to mount things *after* the kernel modules have been loaded. For now, only kdbus is in this set, which is invoked with mount_setup_late().

barrier: explicitly ignore return values of barrier_place() The barrier implementation tracks remote states internally. There is no need to check the return value of any barrier_*() function if the caller is not interested in the result. The barrier helpers only return the state of the remote side, which is usually not interesting as later calls to barrier_sync() will catch this, anyway. Shut up coverity by explicitly ignoring return values of barrier_place() if we're not interested in it.

ptyforward: rework PTY forwarder logic used by nspawn to utilize the normal event loop We really should not run manual event loops anymore, but standardize on sd_event, so that we can run sd_bus connections from it eventually.

units: don't order journal flushing afte remote-fs.target Instead, only depend on the actual file systems we need. This should solve dep loops on setups where remote-fs.target is moved into late boot.

nspawn: don't make up -1 as error code

nspawn: ignore EEXIST when creating mount point A combination of commits f3c80515c and 79d80fc14 cause nspawn to silently fail with a commandline such as: # systemd-nspawn -D /build/extra-x86_64 --bind=/usr strace shows the culprit: [pid 27868] writev(2, [{"Failed to create mount point /build/extra-x86_64/usr: File exists", 82}, {"\n", 1}], 2) = 83

util: introduce sethostname_idempotent Function queries system hostname and applies changes only when necessary. Also, migrate all client of sethostname to sethostname_idempotent while at it.

nspawn: fix DeviceAllow list Commit 864e17068 ("nspawn: actually allow access to /dev/net/tun in the container") added "/dev/net/tun" to the list of allowed devices but forgot to tweak the array length, which caused "/dev/kdbus/*" to be missed.

nspawn: actually allow access to /dev/net/tun in the container It's not sufficient to just copy the device node over, we need to update the policy for it too.

nspawn: copy /dev/net/tun from host This enables tuntap support in the container (assumning the necessary capabilities are in place).

nspawn: log when tearing down of loop device fails

nspawn: check some more return values Most of these failures would anyway get caught later on, but now the error messages are a bit more specific.

nspawn: don't try to create veth link with too long ifname Reported by: James Lott <james@lottspot.com>

nspawn: fix --network-interface Use SETLINK when modifying an existing link.

util: make use of newly added reset_signal_mask() call wherever appropriate

notify: send STOPPING=1 from our daemons

nspawn: make sure that when --network-veth is used both the host and the container side get fixed MAC addresses

bus: always explicitly close bus from main programs Since b5eca3a2059f9399d1dc52cbcf9698674c4b1cf0 we don't attempt to GC busses anymore when unsent messages remain that keep their reference, when they otherwise are not referenced anymore. This means that if we explicitly want connections to go away, we need to close them. With this change we will no do so explicitly wherver we connect to the bus from a main program (and thus know when the bus connection should go away), or when we create a private bus connection, that really should go away after our use. This fixes connection leaks in the NSS and PAM modules.

Unify parse_argv style getopt is usually good at printing out a nice error message when commandline options are invalid. It distinguishes between an unknown option and a known option with a missing arg. It is better to let it do its job and not use opterr=0 unless we actually want to suppress messages. So remove opterr=0 in the few places where it wasn't really useful. When an error in options is encountered, we should not print a lengthy help() and overwhelm the user, when we know precisely what is wrong with the commandline. In addition, since help() prints to stdout, it should not be used except when requested with -h or --help. Also, simplify things here and there.