tree-wide: remove Emacs lines from all files This should be handled fine now by .dir-locals.el, so need to carry that stuff in every file.

resolved: rework what ResolveHostname() with family == AF_UNSPEC means Previously, if a hostanem is resolved with AF_UNSPEC specified, this would be used as indication to resolve both an AF_INET and an AF_INET6 address. With this change this logic is altered: an AF_INET address is only resolved if there's actually a routable IPv4 address on the specific interface, and similar an AF_INET6 address is only resolved if there's a routable IPv6 address. With this in place, it's ensured that the returned data is actually connectable by applications. This logic mimics glibc's resolver behaviour. Note that if the client asks explicitly for AF_INET or AF_INET6 it will get what it asked for. This also simplifies the logic how it is determined whether a specific lookup shall take place on a scope. Specifically, the checks with dns_scope_good_key() are now moved out of the transaction code and into the query code, so that we don't even create a transaction object on a specific scope if we cannot execute the resolution on it anyway.

resolve: fix typos

resolved: fix the rcode to SUCCESS if we find at least one matching RR in a DNS response If we encounter NXDOMAIN, but find at least one matching RR in a response, then patch it to become SUCCESS. This should clean up handling of CNAME/DNAMEs, and makes sure broken servers and those conforming to RFC 6604 are treated the same way. The new behaviour opposes the logic suggested in RFC 6604, but given that some servers don't implement it correctly, and given that in some ways the CNAME/DNAME chains will be incomplete anyway, and given that DNSSEC generally only allows us to prove the first element of a CNAME/DNAME chain, this should simplify things for us.

resolved: don't consider NSEC/NSEC3 RRs as "pimary" for transactions So far, abritrary NSEC and NSEC3 RRs were implicitly consider "primary" for any transaction, meaning we'd abort the transaction immediately if we couldn't validate it. With this patch this logic is removed, and the NSEC/NSEC3 RRs will not be considered primary anymore. This has the effect that they will be dropped from the message if they don't validate, but processing continues. This is safe to do, as they are required anyway to validate positive wildcard and negative responses, and if they are missing then, then message will be considered unsigned, which hence means the outcome is effectively the same. This is benefical in case the server sends us NSEC/NSEC3 RRs that are not directly related to the lookup we did, but simply auxiliary information. Previously, if we couldn't authenticate those RRs we'd fail the entire lookup while with this change we'll simply drop the auxiliary information and proceed without it.

resolved: replace DNS_TRANSACTION_RESOURCES by DNS_TRANSACTION_ERRNO Whenever we encounter an OS error we did not expect, we so far put the transaction into DNS_TRANSACTION_RESOURCES state. Rename this state to DNS_TRANSACTION_ERRNO, and save + propagate the actual system error to the caller. This should make error messages triggered by system errors much more readable by the user.

resolved: log recognizably about DNSSEC downgrades If we downgrade from DNSSEC to non-DNSSEC mode, let's log about this in a recognizable way (i.e. with a message ID), after all, this is of major importance.

resolved: properly handle LLMNR/TCP connection errors The LLMNR spec suggests to do do reverse address lookups by doing direct LLMNR/TCP connections to the indicated address, instead of doing any LLMNR multicast queries. When we do this and the peer doesn't actually implement LLMNR this will result in a TCP connection error, which we need to handle. In contrast to most LLMNR lookups this will give us a quick response on whether we can find a suitable name. Report this as new transaction state, since this should mostly be treated like an NXDOMAIN rcode, except that it's not one.

resolved: log each time we increase the DNSSEC verdict counters Also, don't consider RRs that aren't primary to the lookups we do as relevant to the lookups.

resolved: if we detect a message with incomplete DNSSEC data, consider this an invalid packet event

resolved: also collect statistics about negative DNSSEC proofs We already maintain statistics about positive DNSSEC proofs, and count them up by 1 for each validated RRset. Now, update the same counters each time we validated a negative query, so that the statistics are the combined result of all validation checks, both positive and negative.

resolve: generate a nice clean error when clients try to resolve a name when the network is down

resolved: don't try to print error strings, where errno isn't set

resolved: when restarting a transaction pick a new ID When we restart a transaction because of an incompatible server, pick a new transaction ID. This should increase compatibility with DNS servers that don't like if they get different requests with the same transaction ID.

resolved: enforce maximum limit on DNS transactions given that DNSSEC lookups may result in quite a number of auxiliary transactions, let's better be safe than sorry and also enforce a limit on the number of total transactions, not just on the number of queries.

resolved: fix how we detect whether auxiliary DNSSEC transactions are ready Previously, when getting notified about a completed auxiliary DNSSEC transaction we'd immediately act on it, and possibly abort the main transaction. This is problematic, as DNS transactions that already completed at the time we started using them will never get the notification event, and hence never be acted on in the same way. Hence, introduce a new call dns_transaction_dnssec_ready() that checks the state of auxiliary DNSSEC transactions, and returns 1 when we are ready for the actual DNSSEC validation step. Then, make sure this is invoked when the auxiliary transactions are first acquired (and thus possibly reused) as well when the notifications explained above take place. This fixes problems particularly when doing combined A and AAAA lookups where the auxiliary DNSSEC transactions get reused between them, and where we got confused if we reused an auxiliary DNSSEC transaction from one when it already got completed from the other.

resolved: when we find a DNAME RR, don't insist in a signed CNAME RR If we have a signed DNAME RR response, there's no need to insist on a signature for a CNAME RR response, after all it is unlikely to be signed, given the implicit synthethis of CNAME through DNAME RRs.

resolved: when the server feature level changes between query and response restart transaction In some cases we learn something about a server's feature level through its responses. If we notice that after doing basic checking of a response, and after collecting all auxiliary DNSSEC info the feature level of the server is lower than where we started, restart the whole transaction. This is useful to deal with servers that response rubbish when talked to with too high feature levels.

resolved: check OPT RR before accepting a reply for verification of server feature level Let's make sure we first check if the OPT was lost in the reply, before we accept a reply as successful and use it for verifying the current feature level.

resolved: when restarting a DNS transaction, remove all auxiliary DNSSEC transactions When we restart a DNS transaction, remove all connections to any auxiliary DNSSEC transactions, after all we might acquire completely different data this time, requiring different auxiliary DNSSEC transactions.

resolved: downgrade server feature level more aggressively when we have reason to This adds logic to downgrade the feature level more aggressively when we have reason to. Specifically: - When we get a response packet that lacks an OPT RR for a query that had it. If so, downgrade immediately to UDP mode, i.e. don't generate EDNS0 packets anymore. - When we get a response which we are sure should be signed, but lacks RRSIG RRs, we downgrade to EDNS0 mode, i.e. below DO mode, since DO is apparently not really supported. This should increase compatibility with servers that generate non-sensical responses if they messages with OPT RRs and suchlike, for example the situation described here: https://open.nlnetlabs.nl/pipermail/dnssec-trigger/2014-November/000376.html This also changes the downgrade code to explain in a debug log message why a specific downgrade happened.

resolved: rename dnssec_verify_dnskey() → dnssec_verify_dnskey_by_ds() This should clarify that this is not regular signature-based validation, but validation through DS RR fingerprints.

resolved: when validating an RRset, store information about the synthesizing source and zone in each RR Having this information available is useful when we need to check whether various RRs are suitable for proofs. This information is stored in the RRs as number of labels to skip from the beginning of the owner name to reach the synthesizing source/signer. Simple accessor calls are then added to retrieve the signer/source from the RR using this information. This also moves validation of a a number of RRSIG parameters into a new call dnssec_rrsig_prepare() that as side-effect initializes the two numeric values.

resolved: implement the full NSEC and NSEC3 postive wildcard proofs

resolved: refuse doing queries for known-obsolete RR types Given how fragile DNS servers are with some DNS types, and given that we really should avoid confusing them with known-weird lookups, refuse doing lookups for known-obsolete RR types.

resolved: rename DnsTransaction's current_features field to current_feature_level This is a follow-up for f4461e5641d53f27d6e76e0607bdaa9c0c58c1f6.

resolved: accept rightfully unsigned NSEC responses

resolved: rework how and when we detect whether our chosen DNS server knows DNSSEC Move detection into a set of new functions, that check whether one specific server can do DNSSEC, whether a server and a specific transaction can do DNSSEC, or whether a transaction and all its auxiliary transactions could do so. Also, do these checks both before we acquire additional RRs for the validation (so that we can skip them if the server doesn't do DNSSEC anyway), and after we acquired them all (to see if any of the lookups changed our opinion about the servers). THis also tightens the checks a bit: a server that lacks TCP support is considered incompatible with DNSSEC too.

resolved: rework server feature level logic This changes the DnsServer logic to count failed UDP and TCP failures separately. This is useful so that we don't end up downgrading the feature level from one UDP level to a lower UDP level just because a TCP connection we did because of a TC response failed. This also adds accounting of truncated packets. If we detect incoming truncated packets, and count too many failed TCP connections (which is the normal fall back if we get a trucnated UDP packet) we downgrade the feature level, given that the responses at the current levels don't get through, and we somehow need to make sure they become smaller, which they will do if we don't request DNSSEC or EDNS support. This makes resolved work much better with crappy DNS servers that do not implement TCP and only limited UDP packet sizes, but otherwise support DNSSEC RRs. They end up choking on the generally larger DNSSEC RRs and there's no way to retrieve the full data.

resolved: log why we use TCP when UDP isn't supported by a server

resolved: log about truncated replies before trying again, not after

resolved: don't attempt to send queries for DNSSEC RR types to servers not supporting them If we already degraded the feature level below DO don't bother with sending requests for DS, DNSKEY, RRSIG, NSEC, NSEC3 or NSEC3PARAM RRs. After all, we cannot do DNSSEC validation then anyway, and we better not press a legacy server like this with such modern concepts. This also has the benefit that when we try to validate a response we received using DNSSEC, and we detect a limited server support level while doing so, all further auxiliary DNSSEC queries will fail right-away.

resolved: log about reasons for switching to TCP

resolved: properly handle UDP ICMP errors as lost packets UDP ICMP errors are reported to us via recvmsg() when we read a reply. Handle this properly, and consider this a lost packet, and retry the connection. This also adds some additional logging for invalid incoming packets.

resolved: when we get a TCP connection failure, try again Previously, when we couldn't connect to a DNS server via TCP we'd abort the whole transaction using a "connection-failure" state. This change removes that, and counts failed connections as "lost packet" events, so that we switch back to the UDP protocol again.

resolved: introduce dns_transaction_retry() and use it everywhere The code to retry transactions has been used over and over again, simplify it by replacing it by a new function.

resolved: set a description on all our event sources

resolved: properly look for NSEC/NSEC3 RRs when getting a positive wildcard response This implements RFC 5155, Section 8.8 and RFC 4035, Section 5.3.4: When we receive a response with an RRset generated from a wildcard we need to look for one NSEC/NSEC3 RR that proves that there's no explicit RR around before we accept the wildcard RRset as response. This patch does a couple of things: the validation calls will now identify wildcard signatures for us, and let us know the RRSIG used (so that the RRSIG's signer field let's us know what the wildcard was that generate the entry). Moreover, when iterating trough the RRsets of a response we now employ three phases instead of just two. a) in the first phase we only look for DNSKEYs RRs b) in the second phase we only look for NSEC RRs c) in the third phase we look for all kinds of RRs Phase a) is necessary, since DNSKEYs "unlock" more signatures for us, hence we shouldn't assume a key is missing until all DNSKEY RRs have been processed. Phase b) is necessary since NSECs need to be validated before we can validate wildcard RRs due to the logic explained above. Phase c) validates everything else. This phase also handles RRsets that cannot be fully validated and removes them or lets the transaction fail.

resolved: when validating, first strip revoked trust anchor keys from validated keys list When validating a transaction we initially collect DNSKEY, DS, SOA RRs in the "validated_keys" list, that we need for the proofs. This includes DNSKEY and DS data from our trust anchor database. Quite possibly we learn that some of these DNSKEY/DS RRs have been revoked between the time we request and collect those additional RRs and we begin the validation step. In this case we need to make sure that the respective DS/DNSKEY RRs are removed again from our list. This patch adds that, and strips known revoked trust anchor RRs from the validated list before we begin the actual validation proof, and each time we add more DNSKEY material to it while we are doing the proof.

resolved: rework trust anchor revoke checking Instead of first iterating through all DNSKEYs in the DnsAnswer in dns_transaction_check_revoked_trust_anchors(), and then doing that a second time in dns_trust_anchor_check_revoked(), do so only once in the former, and pass the dnskey we found directly to the latter.

resolved: look for revoked trust anchors before validating a message There's not reason to wait for checking for revoked trust anchors until after validation, after all revoked DNSKEYs only need to be self-signed, but not have a full trust chain. This way, we can be sure that all trust anchor lookups we do during validation already honour that some keys might have been revoked.

resolved: remove one level of indentation in dns_transaction_validate_dnssec() Invert an "if" check, so that we can use "continue" rather than another code block indentation.

resolved: introduce support for per-interface negative trust anchors

resolved: count unsupported dnssec algorithm as indeterminate RRset After all, when we don't support the algorithm we cannot determine validity.

resolved: try to detect fritz.box-style private DNS zones, and downgrade to non-DNSSEC mode for them This adds logic to detect cases like the Fritz!Box routers which serve a private DNS domain "fritz.box" under the TLD "box" that does not exist in the root servers. If this is detected DNSSEC validation is turned off for this private domain, thus improving compatibility with such private DNS zones. This should be fairly secure as we first rely on the proof that .box does not exist before this logic is applied. Nevertheless the logic is only enabled for DNSSEC=allow-downgrade mode. This logic does not work for routers that set up a full DNS zone directly under a non-existing TLD, as in that case we cannot prove that the domain is truly non-existing according to the root servers.

resolved: fix DNSSEC transaction dependency recursion check We followed the wrong connection. This only worked sometimes at all, because we also return the wrong error code.

resolved: rename "downgrade-ok" mode to "allow-downgrade" After discussing this with Tom, we figured out "allow-downgrade" sounds nicer.

resolved: when caching negative responses, honour NSEC/NSEC3 TTLs When storing negative responses, clamp the SOA minimum TTL (as suggested by RFC2308) to the TTL of the NSEC/NSEC3 RRs we used to prove non-existance, if it there is any. This is necessary since otherwise an attacker might put together a faked negative response for one of our question including a high-ttl SOA RR for any parent zone, and we'd use trust the TTL.

resolved: explicitly handle case when the trust anchor is empty Since we honour RFC5011 revoked keys it might happen we end up with an empty trust anchor, or one where there's no entry for the root left. With this patch the logic is changed what to do in this case. Before this patch we'd end up requesting the root DS, which returns with NODATA but a signed NSEC we cannot verify, since the trust anchor is empty after all. Thus we'd return a DNSSEC result of "missing-key", as we lack a verified version of the key. With this patch in place, look-ups for the root DS are explicitly recognized, and not passed on to the DNS servers. Instead, if downgrade-ok mode is on an unsigned NODATA response is synthesized, so that the validator code continues under the assumption the root zone was unsigned. If downgrade-ok mode is off a new transaction failure is generated, that makes this case recognizable.

resolved: explicitly avoid cyclic transaction dependencies We already try hard not to create cyclic transaction dependencies, where a transaction requires another one for DNSSEC validation purposes, which in turn (possibly indirectly) pulls in the original transaction again, thus resulting in a cyclic dependency and ultimately a deadlock since each transaction waits for another one forever. So far we wanted to avoid such cyclic dependencies by only going "up the tree" when requesting auxiliary RRs and only going from one RR type to another, but never back. However this turned out to be insufficient. Consider a domain that publishes one or more DNSKEY but which has no DS for it. A request for the domain's DNSKEY triggers a request for the domain's DS, which will then fail, but return an NSEC, signed by the DNSKEY. To validate that we'd request the DNSKEY again. Thus a DNSKEY request results in a DS request which results in the original DNSKEY request again. If the original lookup had been a DS lookup we'd end up in the same cyclic dependency, hence we cannot statically break one of them, since both requests are of course fully valid. Hence, do full cyclic dependency checking: each time we are about to add a dependency to a transaction, check if the transaction is already a dependency of the dependency (recursively down the tree).

resolved: block transaction GC'ing while dns_transaction_request_dnssec_keys() is running If any of the transactions started by dns_transaction_request_dnssec_keys() finishes promptly without requiring asynchronous operation this is reported back to the issuing transaction from the same stackframe. This might ultimately result in this transaction to be freed while we are still in its _request_dnssec_keys() stack frame. To avoid memory corruption block the transaction GC while in the call, and manually issue a GC after it returned.

resolved: partially implement RFC5011 Trust Anchor support With this patch resolved will properly handle revoked keys, but not augment the locally configured trust anchor database with newly learned keys. Specifically, resolved now refuses validating RRsets with revoked keys, and it will remove revoked keys from the configured trust anchors (only until reboot). This patch does not add logic for adding new keys to the set of trust anchors. This is a deliberate decision as this only can work with persistent disk storage, and would result in a different update logic for stateful and stateless systems. Since we have to support stateless systems anyway, and don't want to encourage two independent upgrade paths we focus on upgrading the trust anchor database via the usual OS upgrade logic. Whenever a trust anchor entry is found revoked and removed from the trust anchor a recognizable log message is written, encouraging the user to update the trust anchor or update his operating system.

resolved: actually make use of message ID when logging about failed DNSSEC validation

resolved: add negative trust anchro support, and add trust anchor configuration files This adds negative trust anchor support and allows reading trust anchor data from disk, from files /etc/systemd/dnssec-trust-anchors.d/*.positive and /etc/systemd/dnssec-trust-anchros.d/*.negative, as well as the matching counterparts in /usr/lib and /run. The positive trust anchor files are more or less compatible to normal DNS zone files containing DNSKEY and DS RRs. The negative trust anchor files contain only new-line separated hostnames for which to require no signing. By default no trust anchor files are installed, in which case the compiled-in root domain DS RR is used, as before. As soon as at least one positive root anchor for the root is defined via trust anchor files this buil-in DS RR is not added though.

resolved: don't conclude NODATA if CNAME exists Instead introduce the new return-code DNSSEC_NSEC_CNAME to indicate this condition. See RFC 6840, Section 4.3.

resolved: explain why we don't check IP addresses/ports of incoming DNS UDP traffic

resolved: clean up dns_transaction_stop() This renames dns_transaction_stop() to dns_transaction_stop_timeout() and makes it only about stopping the transaction timeout. This is safe, as in most occasions we call dns_transaction_stop() at the same time as dns_transaction_close_connection() anyway, which does the rest of what dns_transaction_stop() used to do. And in the one where we don't call it, it's implicitly called by the UDP emission or TCP connection code. This also closes the connections as we enter the validation phase of a transaction, so that no further messages may be received then.

resolved: rename "features" variables to "feature_level" The name "features" suggests an orthogonal bitmap or suchlike, but the variables really encode only a linear set of feature levels. The type used is already called DnsServerFeatureLevel, hence fix up the variables accordingly, too.

resolved: rework OPT RR generation logic This moves management of the OPT RR out of the scope management and into the server and packet management. There are now explicit calls for appending and truncating the OPT RR from a packet (dns_packet_append_opt() and dns_packet_truncate_opt()) as well as a call to do the right thing depending on a DnsServer's feature level (dns_server_adjust_opt()). This also unifies the code to pick a server between the TCP and UDP code paths, and makes sure the feature level used for the transaction is selected at the time the server is picked, and not changed until the next time we pick a server. The server selction code is now unified in dns_transaction_pick_server(). This all fixes problems when changing between UDP and TCP communication for the same server, and makes sure the UDP and TCP codepaths are more alike. It also makes sure we never keep the UDP port open when switchung to TCP, so that we don't have to handle incoming datagrams on the latter we don't expect. As the new code picks the DNS server at the time we make a connection, we don't need to invalidate the DNS server anymore when changing to the next one, thus dns_transaction_next_dns_server() has been removed.

resolved: reuse dns_transaction_stop() when destructing transaction objects

resolved: add dns_transaction_close_connection() This new call unifies how we shut down all connection resources, such as UDP sockets, event sources, and TCP stream objects. This patch just adds the basic hook-up, this function will be used more in later commits.

resolved: make sure we reset the DNSSEC result when we accept a response packet

resolved: improve some log messages a bit Indicate thar we ignore invalid messages

resolved: never proceed processing truncated packets Make sure we don't end up processing packets that are truncated. Instead, actually let the TCP connection do its thing.

resolved: remember explicitly whether we already tried a stream connection On LLMNR we never want to retry stream connections (since local TCP connections should work, and we don't want to unnecessarily delay operation), explicitly remember whether we already tried one, instead of deriving this from a still stored stream object. This way, we can free the stream early, without forgetting that we tried it.

resolved: make sure we GC stream transactions properly Make sure to GC a transaction after dealing with a reply, even if the transaction is not complete yet.

resolved: ignore additional DNS responses we get while validating No need to choke on them.

resolved: introduce dns_transaction_reset_answer() Let's unify how we reset the answer data we collected, after all pretty much every time we do it incompletely so far, let's fix it.

resolved: name TCP and UDP socket calls uniformly Previously the calls for emitting DNS UDP packets were just called dns_{transacion|scope}_emit(), but the one to establish a DNS TCP connection was called dns_transaction_open_tcp(). Clean this up, and rename them dns_{transaction|scope}_emit_udp() and dns_transaction_open_tcp().

resolved: add an automatic downgrade to non-DNSSEC mode This adds a mode that makes resolved automatically downgrade from DNSSEC support to classic non-DNSSEC resolving if the configured DNS server is not capable of DNSSEC. Enabling this mode increases compatibility with crappy network equipment, but of course opens up the system to downgrading attacks. The new mode can be enabled by setting DNSSEC=downgrade-ok in resolved.conf. DNSSEC=yes otoh remains a "strict" mode, where DNS resolving rather fails then allow downgrading. Downgrading is done: - when the server does not support EDNS0+DO - or when the server supports it but does not augment returned RRs with RRSIGs. The latter is detected when requesting DS or SOA RRs for the root domain (which is necessary to do proofs for unsigned data)

resolved: generate an explicit transaction error when we cannot reach server via TCP Previously, if we couldn't reach a server via UDP we'd generate an MAX_ATTEMPTS transaction result, but if we couldn't reach it via TCP we'd generate a RESOURCES transaction result. While it is OK to generate two different errors I think, "RESOURCES" is certainly a misnomer. Introduce a new transaction result "CONNECTION_FAILURE" instead.

resolved: deal with unsigned DS/NSEC/NSEC3 properly Previously, we'd insist on an RRSIG for all DS/NSEC/NSEC3 RRs. With this change we don't do that anymore, but also allow unsigned DS/NSEC/NSEC3 if we can prove that the zone they are located in is unsigned.

resolved: log each dnssec failure, in a recognizable way

resolved: gather statistics about resolved names This collects statistical data about transactions, dnssec verifications and the cache, and exposes it over the bus. The systemd-resolve-host tool learns new options to query these statistics and reset them.

resolved: if we accepted unauthenticated NSEC/NSEC3 RRs, use them for proofs But keep track that the proof is not authenticated.

resolved: don't insist on finding DNSKEYs for RRsets of zones with DNSSEC off

resolved: tighten search for NSEC3 RRs a bit Be stricter when searching suitable NSEC3 RRs for proof: generalize the check we use to find suitable NSEC3 RRs, in nsec3_is_good(), and add additional checks, such as checking whether all NSEC3 RRs use the same parameters, have the same suffix and so on.

resolved: internalize string buffer of dns_resource_record_to_string() Let's simplify usage and memory management of DnsResourceRecord's dns_resource_record_to_string() call: cache the formatted string as part of the object, and return it on subsequent calls, freeing it when the DnsResourceRecord itself is freed.

resolve: fix indentation

resolved: propagate DNSSEC validation status from auxiliary transactions Let's make sure we propagate the DNSSEC validation status from an auxiliary DNSSEC transaction back to the originating transaction, to improve the error messages we generate.

resolved: propagate the DNSSEC result from the transaction to the query and the the bus client It's useful to generate useful errors, so let's do that.

resolved: rename DNS_TRANSACTION_FAILURE → DNS_TRANSACTION_RCODE_FAILURE We have many types of failure for a transaction, and DNS_TRANSACTION_FAILURE was just one specific one of them, if the server responded with a non-zero RCODE. Hence let's rename this, to indicate which kind of failure this actually refers to.

resolved: add support NSEC3 proofs, as well as proofs for domains that are OK to be unsigned This large patch adds a couple of mechanisms to ensure we get NSEC3 and proof-of-unsigned support into place. Specifically: - Each item in an DnsAnswer gets two bit flags now: DNS_ANSWER_AUTHENTICATED and DNS_ANSWER_CACHEABLE. The former is necessary since DNS responses might contain signed as well as unsigned RRsets in one, and we need to remember which ones are signed and which ones aren't. The latter is necessary, since not we need to keep track which RRsets may be cached and which ones may not be, even while manipulating DnsAnswer objects. - The .n_answer_cachable of DnsTransaction is dropped now (it used to store how many of the first DnsAnswer entries are cachable), and replaced by the DNS_ANSWER_CACHABLE flag instead. - NSEC3 proofs are implemented now (lacking support for the wildcard part, to be added in a later commit). - Support for the "AD" bit has been dropped. It's unsafe, and now that we have end-to-end authentication we don't need it anymore. - An auxiliary DnsTransaction of a DnsTransactions is now kept around as least as long as the latter stays around. We no longer remove the auxiliary DnsTransaction as soon as it completed. THis is necessary, as we now are interested not only in the RRsets it acquired but also in its authentication status.

resolved: refuse to add auxiliary transactions loops Let's be safe and explicitly avoid that we add an auxiliary transaction dependency on ourselves.

resolved: stop timeout timer when validating transactions We need no separate timeout anymore as soon as we received a reply, as the auxiliary transactions have their own timeouts.

resolved: make sure we don't get confused when notifying transactions while they are destroyed A failing transaction might cause other transactions to fail too, and thus the set of transactions to notify for a transaction might change while we are notifying them. Protect against that.

resolved: cache stringified transaction key once per transaction We end up needing the stringified transaction key in many log messages, hence let's simplify the logic and cache it inside of the transaction: generate it the first time we need it, and reuse it afterwards. Free it when the transaction goes away. This also updated a couple of log messages to make use of this.

resolved: add basic proof of non-existance support for NSEC+NSEC3 Note that this is not complete yet, as we don't handle wildcard domains correctly, nor handle domains correctly that use empty non-terminals.

resolved: always consider NSEC/NSEC3 RRs as "primary" It's not OK to drop these for our proof of non-existance checks.

treewide: fix typos and indentation

resolved: refactor DNSSEC answer validation This changes answer validation to be more accepting to unordered RRs in responses. The agorithm we now implement goes something like this: 1. populate validated keys list for this transaction from DS RRs 2. as long as the following changes the unvalidated answer list: 2a. try to validate the first RRset we find in unvalidated answer list 2b. if that worked: add to validated answer; if DNSKEY also add to validated keys list; remove from unvalidated answer. 2c. continue at 2a, with the next RRset, or restart from the beginning when we hit the end 3. as long as the following changes the unvalidated answer list: 3a. try to validate the first RRset again. This will necessarily fail, but we learn the precise error 3b. If this was a "primary" response to the question, fail the entire transaction. "Primary" in this context means that it is directly a response to the query, or a CNAME/DNAME for it. 3c. Otherwise, remove the RRset from the unvalidated answer list. Note that we the too loops in 2 + 3 are actually coded as a single one, but the dnskeys_finalized bool indicates which loop we are currently processing. Note that loop 2 does not drop any invalidated RRsets yet, that's something only loop 3 does. This is because loop 2 might still encounter additional DNSKEYS which might validate more stuff, and if we'd already have dropped those RRsets we couldn't validate those anymore. The first loop is hence a "constructive" loop, the second loop a "destructive" one: the first one validates whatever is possible, the second one then deletes whatever still isn't.

resolved: rework how and when the number of answer RRs to cache is determined Instead of figuring out how many RRs to cache right before we do so, determine this at the time we install the answer RRs, so that we can still alter this as we manipulate the answer during validation. The primary purpose of this is to pave the way so that we can drop unsigned RRsets from the answer and invalidate the number of RRs to cache at the same time.

resolved: generalize DNS RR type validity checks Check the validity of RR types as we parse or receive data from IPC clients, and use the same code for all of them.

resolved: cache - do negative caching only on the canonical name Apart from dropping redundant information, this fixes an issue where, due to broken DNS servers, we can only be certain of whether an apparent NODATA response is in fact an NXDOMAIN response after explicitly resolving the canonical name. This issue is outlined in RFC2308. Moreover, by caching NXDOMAIN for an existing name, we would mistakenly return NXDOMAIN for types which should not be redirected. I.e., a query for AAAA on test-nx-1.jklm.no correctly returns NXDOMAIN, but a query for CNAME should return the record and a query for DNAME should return NODATA. Note that this means we will not cache an NXDOMAIN response in the presence of redirection, meaning one redundant roundtrip in case the name is queried again.

resolved: make sure the packet's transaction ID is always 0 for mDNS RFC6762, 18.1: In multicast query messages, the Query Identifier SHOULD be set to zero on transmission.

resolved: rename dns_transaction_prepare_next_attempt() Let's simply call it dns_transaction_prepare(), so that we have the nice cycle for prepare() → go() → emit() → process(). After all it's pretty clear that what we prepare there, and we dont call the others go_next_attempt(), emit_next_attempt() or process_next_attempt().

resolved: don't accept doing queries for invalid RR types

resolved: chase DNSKEY/DS RRs when doing look-ups with DNSSEC enabled This adds initial support for validating RRSIG/DNSKEY/DS chains when doing lookups. Proof-of-non-existance, or proof-of-unsigned-zones is not implemented yet. With this change DnsTransaction objects will generate additional DnsTransaction objects when looking for DNSKEY or DS RRs to validate an RRSIG on a response. DnsTransaction objects are thus created for three reasons now: 1) Because a user asked for something to be resolved, i.e. requested by a DnsQuery/DnsQueryCandidate object. 2) As result of LLMNR RR probing, requested by a DnsZoneItem. 3) Because another DnsTransaction requires the requested RRs for validation of its own response. DnsTransactions are shared between all these users, and are GC automatically as soon as all of these users don't need a specific transaction anymore. To unify the handling of these three reasons for existance for a DnsTransaction, a new common naming is introduced: each DnsTransaction now tracks its "owners" via a Set* object named "notify_xyz", containing all owners to notify on completion. A new DnsTransaction state is introduced called "VALIDATING" that is entered after a response has been receieved which needs to be validated, as long as we are still waiting for the DNSKEY/DS RRs from other DnsTransactions. This patch will request the DNSKEY/DS RRs bottom-up, and then validate them top-down. Caching of RRs is now only done after verification, so that the cache is not poisoned with known invalid data. The "DnsAnswer" object gained a substantial number of new calls, since we need to add/remove RRs to it dynamically now.

resolved: reenable caching for LLMNR This got borked in 547493c5ad5c82032e247609970f96be76c2d661.

resolved: split out check whether reply matches our question It's complicated enough, it deserves its own call. (Also contains some unrelated whitespace, comment and assertion changes)

resolved: add dns_cache_export_to_packet() This new functions exports cached records of type PTR, SRV and TXT into an existing DnsPacket. This is used in order to fill in known records to mDNS queries, for known answer supression.

resolved: implement query coalescing Implement dns_transaction_make_packet_mdns(), a special version of dns_transaction_make_packet() for mDNS which differs in many ways: a) We coalesce queries of currently active transaction on the scope. This is possible because mDNS actually allows many questions in a to be sent in a single packet and it takes some burden from the network. b) Both A and AAAA query keys are broadcast on both IPv4 and IPv6 scopes, because other hosts might only respond on one of their addresses but resolve both types. c) We discard previously sent packages (t->sent) so we can start over and coalesce pending transactions again.

resolved: add 'next_attempt_after' field to DnsTransaction For each transaction, record when the earliest point in time when the query packet may hit the wire. This is the same time stamp for which the timer is scheduled in retries, except for the initial query packets which are delayed by a random jitter. In this case, we denote that the packet may actually be sent at the nominal time, without the jitter. Transactions that share the same timestamp will also have identical values in this field. It is used to coalesce pending queries in a later patch.

resolved: split dns_transaction_go() Split some code out of dns_transaction_go() so we can re-use it later from different context. The new function dns_transaction_prepare_next_attempt() takes care of preparing everything so that a new packet can conditionally be formulated for a transaction. This patch shouldn't cause any functional change.

resolved: handle more mDNS protocol details

resolved: fix debug message

resolved: handle mDNS timeouts per transaction mDNS packet timeouts need to be handled per transaction, not per link. Re-use the n_attempts field for this purpose, as packets timeouts should be determined by starting at 1 second, and doubling the value on each try.

resolved: short-cut jitter callbacks for LLMNR and mDNS When a jitter callback is issued instead of sending a DNS packet directly, on_transaction_timeout() is invoked to 'retry' the transaction. However, this function has side effects. For once, it increases the packet loss counter on the scope, and it also unrefs/refs the server instances. Fix this by tracking the jitter with two bool variables. One saying that the initial jitter has been scheduled in the first place, and one that tells us the delay packet has been sent.

resolved: add mDNS initial jitter The logic is to kick off mDNS packets in a delayed way is mostly identical to what LLMNR needs, except that the constants are different.

resolved: add packet header details for mDNS Validate mDNS queries and responses by looking at some header fields, add mDNS flags.

resolved: add a concept of "authenticated" responses This adds a new SD_RESOLVED_AUTHENTICATED flag for responses we return on the bus. When set, then the data has been authenticated. For now this mostly reflects the DNSSEC AD bit, if DNSSEC=trust is set. As soon as the client-side validation is complete it will be hooked up to this flag too. We also set this bit whenver we generated the data ourselves, for example, because it originates in our local LLMNR zone, or from the built-in trust anchor database. The "systemd-resolve-host" tool has been updated to show the flag state for the data it shows.

resolved: introduce a dnssec_mode setting per scope The setting controls which kind of DNSSEC validation is done: none at all, trusting the AD bit, or client-side validation. For now, no validation is implemented, hence the setting doesn't do much yet, except of toggling the CD bit in the generated messages if full client-side validation is requested.

resolved: add a simple trust anchor database as additional RR source When doing DNSSEC lookups we need to know one or more DS or DNSKEY RRs as trust anchors to validate lookups. With this change we add a compiled-in trust anchor database, serving the root DS key as of today, retrieved from: https://data.iana.org/root-anchors/root-anchors.xml The interface is kept generic, so that additional DS or DNSKEY RRs may be served via the same interface, for example by provisioning them locally in external files to support "islands" of security. The trust anchor database becomes the fourth source of RRs we maintain, besides, the network, the local cache, and the local zone.

resolved: announce support for large UDP packets This is often needed for proper DNSSEC support, and even to handle AAAA records without falling back to TCP. If the path between the client and server is fully compliant, this should always work, however, that is not the case, and overlarge packets will get mysteriously lost in some cases. For that reason, we use a similar fallback mechanism as we do for palin EDNS0, EDNS0+DO, etc.: The large UDP size feature is different from the other supported feature, as we cannot simply verify that it works based on receiving a reply (as the server will usually send us much smaller packets than what we claim to support, so simply receiving a reply does not mean much). For that reason, we keep track of the largest UDP packet we ever received, as this is the smallest known good size (defaulting to the standard 512 bytes). If announcing the default large size of 4096 fails (in the same way as the other features), we fall back to the known good size. The same logic of retrying after a grace-period applies.

resolved: implement minimal EDNS0 support This is a minimal implementation of RFC6891. Only default values are used, so in reality this will be a noop. EDNS0 support is dependent on the current server's feature level, so appending the OPT pseudo RR is done when the packet is emitted, rather than when it is assembled. To handle different feature levels on retransmission, we strip off the OPT RR again after sending the packet. Similarly, to how we fall back to TCP if UDP fails, we fall back to plain UDP if EDNS0 fails (but if EDNS0 ever succeeded we never fall back again, and after a timeout we will retry EDNS0).

resolved: degrade the feature level on explicit failure Previously, we would only degrade on packet loss, but when adding EDNS0 support, we also have to handle the case where the server replies with an explicit error.

resolved: fallback to TCP if UDP fails This is inspired by the logic in BIND [0], follow-up patches will implement the reset of that scheme. If we get a server error back, or if after several attempts we don't get a reply at all, we switch from UDP to TCP for the given server for the current and all subsequent requests. However, if we ever successfully received a reply over UDP, we never fall back to TCP, and once a grace-period has passed, we try to upgrade again to using UDP. The grace-period starts off at five minutes after the current feature level was verified and then grows exponentially to six hours. This is to mitigate problems due to temporary lack of network connectivity, but at the same time avoid flooding the network with retries when the feature attempted feature level genuinely does not work. Note that UDP is likely much more commonly supported than TCP, but depending on the path between the client and the server, we may have more luck with TCP in case something is wrong. We really do prefer UDP though, as that is much more lightweight, that is why TCP is only the last resort. [0]: <https://kb.isc.org/article/AA-01219/0/Refinements-to-EDNS-fallback-behavior-can-cause-different-outcomes-in-Recursive-Servers.html>

resolved: never cache RRs originating from localhost After all, this is likely a local DNS forwarder that caches anyway, hence there's no point in caching twice. Fixes #2038.

resolved: handle properly if there are multiple transactions for the same key per scope When the zone probing code looks for a transaction to reuse it will refuse to look at transactions that have been answered from cache or the zone itself, but insist on the network. This has the effect that there might be multiple transactions around for the same key on the same scope. Previously we'd track all transactions in a hashmap, indexed by the key, which implied that there would be only one transaction per key, per scope. With this change the hashmap will only store the most recent transaction per key, and a linked list will be used to track all transactions per scope, allowing multiple per-key per-scope. Note that the linked list fields for this actually already existed in the DnsTransaction structure, but were previously unused.

resolved: for a transaction, keep track where the answer data came from Let's track where the data came from: from the network, the cache or the local zone. This is not only useful for debugging purposes, but is also useful when the zone probing wants to ensure it's not reusing transactions that were answered from the cache or the zone itself.

resolved: store just the DnsAnswer instead of a DnsPacket as answer in DnsTransaction objects Previously we'd only store the DnsPacket in the DnsTransaction, and the DnsQuery would then take the DnsPacket's DnsAnswer and return it. With this change we already pull the DnsAnswer out inside the transaction. We still store the DnsPacket in the transaction, if we have it, since we still need to determine from which peer a response originates, to implement caching properly. However, the DnsQuery logic doesn't care anymore for the packet, it now only looks at answers and rcodes from the successfuly candidate. This also has the benefit of unifying how we propagate incoming packets, data from the local zone or the local cache.

resolved: fully support DNS search domains This adds support for searching single-label hostnames in a set of configured search domains. A new object DnsQueryCandidate is added that links queries to scopes. It keeps track of the search domain last used for a query on a specific link. Whenever a host name was unsuccessfuly resolved on a scope all its transactions are flushed out and replaced by a new set, with the next search domain appended. This also adds a new flag SD_RESOLVED_NO_SEARCH to disable search domain behaviour. The "systemd-resolve-host" tool is updated to make this configurable via --search=. Fixes #1697

resolved: shortcut lookups names in the local zone Previously, we'd always generate a packet on the wire, even for names that are within our local zone. Shortcut this, and always check the local zone first. This should minimize generated traffic and improve security.

util-lib: split out allocation calls into alloc-util.[ch]

util-lib: move more locale-related calls to locale-util.[ch]

util-lib: move string table stuff into its own string-table.[ch]

util-lib: split out fd-related operations into fd-util.[ch] There are more than enough to deserve their own .c file, hence move them over.

resolved: cache - only allow putting a single question key at a time Only one key is allowed per transaction now, so let's simplify things and only allow putting one question key into the cache at a time.

resolved: rename DNS UDP socket to 'dns_udp_fd' This hopefully makes this a bit more expressive and clarifies that the fd is not used for the DNS TCP socket. This also mimics how the LLMNR UDP fd is named in the manager object.

resolved: move assertion Make a scope with invalid protocol state fail as soon as possible.

resolved: use switch-case statements for protocol details With more protocols to come, switch repetitive if-else blocks with a switch-case statements.

resolved: remove runtime check for previously asserted condition

resolved: change error code when trying to resolve direct LLMNR PTR RRs If we try to resoolve an LLMNR PTR RR we shall connect via TCP directly to the specified IP address. We already refuse to do this if the address to resolve is of a different address family as the transaction's scope. The error returned was EAFNOSUPPORT. Let's change this to ESRCH which is how we indicate "not server available" when connecting for LLMNR or DNS, since that's what this really is: we have no server we could connect to in this address family. This allows us to ensure that no server errors are always handled the same way.

resolved: replace transaction list by hashmap Right now we keep track of ongoing transactions in a linked listed for each scope. Replace this by a hashmap that is indexed by the RR key. Given that all ongoing transactions will be placed in pretty much the same scopes usually this should optimize behaviour. We used to require a list here, since we wanted to do "superset" query checks, but this became obsolete since transactions are now single-key instead of multi-key.

resolved: only maintain one question RR key per transaction Let's simplify things and only maintain a single RR key per transaction object, instead of a full DnsQuestion. Unicast DNS and LLMNR don't support multiple questions per packet anway, and Multicast DNS suggests coalescing questions beyond a single dns query, across the whole system.

resolved: add extra check for family when doing LLMNR TCP connections It shouldn't happen that we try to resolve IPv4 addresses via LLMNR on IPv6 and vice versa, but let's explicitly verify that we don't turn an IPv4 LLMNR lookup into an IPv6 TCP connection.

resolved: minor typo comment fix

resolved: cache - add more detailed cache debug logging

sd-event: make sure sd_event_now() cannot fail Previously, if the event loop never ran before sd_event_now() would fail. With this change it will instead fall back to invoking now(). This way, the function cannot fail anymore, except for programming error when invoking it with wrong parameters. This takes into account the fact that many callers did not handle the error condition correctly, and if the callers did, then they kept simply invoking now() as fall back on their own. Hence let's shorten the code using this call, and make things more robust, and let's just fall back to now() internally. Whether now() is used or the cache timestamp may still be detected via the return value of sd_event_now(). If > 0 is returned, then the fall back to now() was used, if == 0 is returned, then the cached value was returned. This patch also simplifies many of the invocations of sd_event_now(): the manual fall back to now() can be removed. Also, in cases where the call is invoked withing void functions we can now protect the invocation via assert_se(), acknowledging the fact that the call cannot fail anymore except for programming errors with the parameters. This change is inspired by #841.

resolved: transaction - exponentially increase retry timeouts Rather than fixing this to 5s for unicast DNS and 1s for LLMNR, start at a tenth of those values and increase exponentially until the old values are reached. For LLMNR the recommended timeout for IEEE802 networks (which basically means all of the ones we care about) is 100ms, so that should be uncontroversial. For unicast DNS I have found no recommended value. However, it seems vastly more likely that hitting a 500ms timeout is casued by a packet loss, rather than the RTT genuinely being greater than 500ms, so taking this as a startnig value seems reasonable to me. In the common case this greatly reduces the latency due to normal packet loss. Moreover, once we get support for probing for features, this means that we can send more packets before degrading the feature level whilst still allowing us to settle on the correct feature level in a reasonable timeframe. The timeouts are tracked per server (or per scope for the multicast protocols), and once a server (or scope) receives a successfull package the timeout is reset. We also track the largest RTT for the given server/scope, and always start our timouts at twice the largest observed RTT.

resolved: compare dns question arrays properly Let's optimize things a bit and properly compare DNS question arrays, instead of checking if they are mutual supersets. This also makes ANY query handling more accurate.

resolved: transaction - don't explicitly verify packet source This is handled by the kernel now that the socket is connect()ed.

resolved: transaction - don't unref server when creating TCP socket This was a bug.

resolved: transaction - introduce dns_transaction_emit() This function emits the UDP packet via the scope, but first it will determine the current server (and connect to it) and store the server in the transaction. This should not change the behavior, but simplifies the code.

resolved: transaction - move a couple of functions No functional change, but makes follow-up patch clearer.

resolved: transaction - move DNS UDP socket creation to the scope With access to the server when creating the socket, we can connect() to the server and hence simplify message sending and receiving in follow-up patches.

resolved: transaction - close socket when changing server Close the socket when changing the server in a transaction, in order for it to be reopened with the right server when we send the next packet. This fixes a regression where we could get stuck with a failing server.

resolved: transaction - don't request PKTINFO for unicast DNS This was only ever used by LLMNR, so don't request this for unicast DNS packets.

resloved: transaction - unify IPv4 and IPv6 sockets A transaction can only have one socket at a time, so no need to distinguish these.

resolve: transaction - stop processing packet when found to be invalid We were stopping the transaction, but we need to stop processing the packet alltogether.

resolved: use one UDP socket per transaction We used to have one global socket, use one per transaction instead. This has the side-effect of giving us a random UDP port per transaction, and hence increasing the entropy and making cache poisoining significantly harder to achieve. We still reuse the same port number for packets belonging to the same transaction (resent packets).

resolved: implement RFC5452 This improves the resilience against cache poisoning by being stricter about only accepting responses that match precisely the requst they are in reply to. It should be noted that we still only use one port (which is picked at random), rather than one port for each transaction. Port randomization would improve things further, but is not required by the RFC.

resolved: pin the server used in a transaction We want to discover information about the server and use that in when crafting packets to be resent.

resolved: separate LLMNR specific header bits The C and T bits in the DNS packet header definitions are specific to LLMNR. In regular DNS, they are called AA and RD instead. Reflect that by calling the macros accordingly, and alias LLMNR specific macros. While at it, define RA, AD and CD getters as well.

resolved: use a #define for LLMNR port De-duplicate some magic numbers.

shared: add random-util.[ch]

fix gcc warnings about uninitialized variables like: src/shared/install.c: In function ‘unit_file_lookup_state’: src/shared/install.c:1861:16: warning: ‘r’ may be used uninitialized in this function [-Wmaybe-uninitialized] return r < 0 ? r : state; ^ src/shared/install.c:1796:13: note: ‘r’ was declared here int r; ^

hashmap: introduce hash_ops to make struct Hashmap smaller It is redundant to store 'hash' and 'compare' function pointers in struct Hashmap separately. The functions always comprise a pair. Store a single pointer to struct hash_ops instead. systemd keeps hundreds of hashmaps, so this saves a little bit of memory.

resolved: actually, the peer with the lower IP address wins conflicts

resolved: make sure we don't mark the wrong zone RRs conflicting

resolved: be a bit more communicative about conflicts

resolved: implement full LLMNR conflict detection logic

resolved: don't abort if a transaction is aborted because its scope is removed

resolved: add 100ms initial jitter to all LLMNR requests

resolved: when sending fails, don't try connecting to the next DNS server if we actually use LLMNR as protocol

resolved: bypass local cache when we issue a transaction for verification purposes

resolved: flush cache each time we change to a different DNS server

resolved: use CLOCK_BOOTTIME instead of CLOCK_MONOTONIC when aging caches and timeing out transactions That way the cache doens't get confused when the system is suspended.

resolved: implement LLMNR uniqueness verification