tree-wide: remove Emacs lines from all files This should be handled fine now by .dir-locals.el, so need to carry that stuff in every file.

resolved: correctly store interface index of RRs in cache Fixes: #2361

resolved: never store NSEC/NSEC3 RRs from the upper zone of a zone cut in cache When using NSEC/NSEC3 RRs from the cache to derive existance of arbitrary RRs, we should not get confused by the fact that NSEC/NSEC3 RRs exist twice at zone cuts: once in the parent zone, and once in the child zone. For most RR types we should only consult the latter since that's where the beef is. However, for DS lookups we have to check the former. This change makes sure we never cache NSEC/NSEC3 RRs from any parent zone of a zone-cut. It also makes sure that when we look for a DS RR in the cache we never consider any cached NSEC RR, as those are now always from the child zone.

resolved: try to reduce number or DnsResourceKeys we keep around by merging them Quite often we read the same RR key multiple times from the same message. Try to replace them by a single object when we notice this. Do so again when we add things to the cache. This should reduce memory consumption a tiny bit.

resolved: when caching negative responses, honour NSEC/NSEC3 TTLs When storing negative responses, clamp the SOA minimum TTL (as suggested by RFC2308) to the TTL of the NSEC/NSEC3 RRs we used to prove non-existance, if it there is any. This is necessary since otherwise an attacker might put together a faked negative response for one of our question including a high-ttl SOA RR for any parent zone, and we'd use trust the TTL.

resolved: add comments referencing various RFCs to various places

resolved: also use RRSIG expiry for negative caching This makes sure that we also honour the RRSIG expiry for negative caching.

resolved: use RRSIG expiry and original TTL for cache management When we verified a signature, fix up the RR's TTL to the original TTL mentioned in the signature, and store the signature expiry information in the RR, too. Then, use that when adding RRs to the cache.

resolved: gather statistics about resolved names This collects statistical data about transactions, dnssec verifications and the cache, and exposes it over the bus. The systemd-resolve-host tool learns new options to query these statistics and reset them.

resolved: internalize string buffer of dns_resource_record_to_string() Let's simplify usage and memory management of DnsResourceRecord's dns_resource_record_to_string() call: cache the formatted string as part of the object, and return it on subsequent calls, freeing it when the DnsResourceRecord itself is freed.

resolved: when caching NXDOMAIN for an RR, make sure we flush out old ANY entries We use ANY RR keys to store NXDOMAIN information, but we previously didn't flush out old ANY RR items in the cache when adding new entries. Fix that.

resolved: split out a new dns_type_may_redirect() call Let's abstract which RRs shall honour CNAMEs, and which ones should not.

resolved: merge two comments

resolved: rename dns_cache_item_remove_and_free() → _unlink_and_free() In most of the other call, we called similar functions that remove the data structure link-ups to other objects "unlink", hence we should here, too.

resolved: rename dns_cache_remove() → dns_cache_remove_by_key() Given that we already have dns_cache_remove_by_rr() this makes clearer what the operation actually does.

resolved: rework mDNS cache-flush bit handling This adds a new DnsAnswer item flag "DNS_ANSWER_SHARED_OWNER" which is set for mDNS RRs that lack the cache-flush bit. The cache-flush bit is removed from the DnsResourceRecord object in favour of this. This also splits out the code that removes previous entries when adding new positive ones into a new separate call dns_cache_remove_previous().

resolved: properly determine size of DnsAnswer object After all we want to allow NULL DnsAnswer objects as equivalent to empty ones, hence we should use the right checks everywhere.

resolved: pass out precise authenticated bit we got passed in Make sure the cache never altes the authenticated bit of RRs stored in it, and drops it for RRs when passing it out again.

resolve: optimize dns_cache_flush() a bit Let's use dns_cache_remove() rather than dns_cache_item_remove_and_free() to destroy the cache, since the former requires far fewer hash table lookups.

resolved: when receiving a TTL=0 RR, only flush that specific RR When we receieve a TTL=0 RR, then let's only flush that specific RR and not the whole RRset. On mDNS with RRsets that a shared-owner this is how specific RRs are removed from the set, hence support this. And on non-mDNS the whole RRset will already be removed much earlier in dns_cache_put() hence there's no reason remove it again.

resolved: optimize dns_cache_remove() a bit

resolved: use dns_name_parent() where appropriate

resolved: check SOA authentication state when negative caching We should never use the TTL of an unauthenticated SOA to cache an authenticated RR.

resolved: don't call dns_cache_remove() from dns_cache_put_negative() We call it anyway as one of the first calls in dns_cache_put(), hence there's no reason to do this multiple times.

resolved: bump cache size a bit Let's keep entries for longer and more of them. After all, due to the DNSSEC hookup the amount of RRs we need to store is much higher now.

resolved: make use of dns_{class|type}_is_{pseudo|valid_rr}() everywhere

resolved: add support NSEC3 proofs, as well as proofs for domains that are OK to be unsigned This large patch adds a couple of mechanisms to ensure we get NSEC3 and proof-of-unsigned support into place. Specifically: - Each item in an DnsAnswer gets two bit flags now: DNS_ANSWER_AUTHENTICATED and DNS_ANSWER_CACHEABLE. The former is necessary since DNS responses might contain signed as well as unsigned RRsets in one, and we need to remember which ones are signed and which ones aren't. The latter is necessary, since not we need to keep track which RRsets may be cached and which ones may not be, even while manipulating DnsAnswer objects. - The .n_answer_cachable of DnsTransaction is dropped now (it used to store how many of the first DnsAnswer entries are cachable), and replaced by the DNS_ANSWER_CACHABLE flag instead. - NSEC3 proofs are implemented now (lacking support for the wildcard part, to be added in a later commit). - Support for the "AD" bit has been dropped. It's unsafe, and now that we have end-to-end authentication we don't need it anymore. - An auxiliary DnsTransaction of a DnsTransactions is now kept around as least as long as the latter stays around. We no longer remove the auxiliary DnsTransaction as soon as it completed. THis is necessary, as we now are interested not only in the RRsets it acquired but also in its authentication status.

resolve: remove unused variable

resolved: refuse to cache ANY kind of pseudo-RR-type

resolved: cache - only stringify RR keys when in debug mode This is in the fast path, so let's not do all this work unneccessarily.

resolved: cache - don't flush the cache of mDNS records unneccesarily When the DNS_RESOURCE_KEY_CACHE_FLUSH flag is not set for an mDNS packet, we should not flush the cache for RRs with matching keys. However, we were unconditionally flushing the cache also for these packets. Now mark all packets as cache_flush by default, except for these mDNS packets, and respect that flag in the cache handling. This fixes 90325e8c2e559a21ef0bc2f26b844c140faf8020.

resolved: cache - rework which RR types we apply redirection to The logic of dns_cache_get() is now: - look up the precise key; - look up NXDOMAIN item; - if an RR type that may be redirected (i.e., not CNAME, DNAME, RRSIG, NSEC, NSEC3, SIG, KEY, or NXT) look up a correpsonding CNAME or DNAME record; - look up a corresponding NSEC record; Before this change we would give up before potentially finding negative cache entries for DNAME, CNAME and NSEC records, we would return NSEC records for aliases where we had DNAME or CNAME records available and we would incorrectly try to redirect DNSSEC RRs.

resolved: cache - improve logging Some DNS servers will hand out negative answers without SOA records, these can not be cached, so log about that fact.

resolved: cache - don't cache NXDOMAIN by TYPE An NXDOMAIN entry means there are no RRs of any type for a name, so only cache by CLASS + NAME, rather than CLASS + NAME + TYPE.

resolved: cache - do negative caching only on the canonical name Apart from dropping redundant information, this fixes an issue where, due to broken DNS servers, we can only be certain of whether an apparent NODATA response is in fact an NXDOMAIN response after explicitly resolving the canonical name. This issue is outlined in RFC2308. Moreover, by caching NXDOMAIN for an existing name, we would mistakenly return NXDOMAIN for types which should not be redirected. I.e., a query for AAAA on test-nx-1.jklm.no correctly returns NXDOMAIN, but a query for CNAME should return the record and a query for DNAME should return NODATA. Note that this means we will not cache an NXDOMAIN response in the presence of redirection, meaning one redundant roundtrip in case the name is queried again.

resolved: chase DNSKEY/DS RRs when doing look-ups with DNSSEC enabled This adds initial support for validating RRSIG/DNSKEY/DS chains when doing lookups. Proof-of-non-existance, or proof-of-unsigned-zones is not implemented yet. With this change DnsTransaction objects will generate additional DnsTransaction objects when looking for DNSKEY or DS RRs to validate an RRSIG on a response. DnsTransaction objects are thus created for three reasons now: 1) Because a user asked for something to be resolved, i.e. requested by a DnsQuery/DnsQueryCandidate object. 2) As result of LLMNR RR probing, requested by a DnsZoneItem. 3) Because another DnsTransaction requires the requested RRs for validation of its own response. DnsTransactions are shared between all these users, and are GC automatically as soon as all of these users don't need a specific transaction anymore. To unify the handling of these three reasons for existance for a DnsTransaction, a new common naming is introduced: each DnsTransaction now tracks its "owners" via a Set* object named "notify_xyz", containing all owners to notify on completion. A new DnsTransaction state is introduced called "VALIDATING" that is entered after a response has been receieved which needs to be validated, as long as we are still waiting for the DNSKEY/DS RRs from other DnsTransactions. This patch will request the DNSKEY/DS RRs bottom-up, and then validate them top-down. Caching of RRs is now only done after verification, so that the cache is not poisoned with known invalid data. The "DnsAnswer" object gained a substantial number of new calls, since we need to add/remove RRs to it dynamically now.

resolved: partially revert 5eefe54 Quoting @teg: "Contrary to what the comment said, we always verify redirect chains in full, and cache all the CNAME records. There is therefore no need to do extra negative caching along a CNAME chain." This simply steals @teg's commit since we'll touch the SOA matching case in a later patch, and rather want this bit gone, so that we don't have to "fix" it, only to remove it later on.

resolved: add more linked packets for overlong known answers For mDNS, if we're unable to stuff all known answers into the given packet, allocate a new one, push the RR into that one and link it to the current one.

resolved: add dns_cache_export_to_packet() This new functions exports cached records of type PTR, SRV and TXT into an existing DnsPacket. This is used in order to fill in known records to mDNS queries, for known answer supression.

resolved: flush keys when DNS_RESOURCE_KEY_CACHE_FLUSH is set In mDNS, DNS_RESOURCE_KEY_CACHE_FLUSH denotes whether other records with the same key should be flushed from the cache.

resolved: add a concept of "authenticated" responses This adds a new SD_RESOLVED_AUTHENTICATED flag for responses we return on the bus. When set, then the data has been authenticated. For now this mostly reflects the DNSSEC AD bit, if DNSSEC=trust is set. As soon as the client-side validation is complete it will be hooked up to this flag too. We also set this bit whenver we generated the data ourselves, for example, because it originates in our local LLMNR zone, or from the built-in trust anchor database. The "systemd-resolve-host" tool has been updated to show the flag state for the data it shows.

resolved: when synthesizing NODATA from cached NSEC bitmaps, honour CNAME/DNAME When an RR type is not set in an NSEC, then the CNAME/DNAME types might still be, hence check them too. Otherwise we might end up refusing resolving of CNAME'd RRs if we cached an NSEC before.

resolved: optionally, allocate DnsResourceKey objects on the stack Sometimes when looking up entries in hashmaps indexed by a DnsResourceKey it is helpful not having to allocate a full DnsResourceKey dynamically just to use it as search key. Instead, optionally allow allocation of a DnsResourceKey on the stack. Resource keys allocated like that of course are subject to other lifetime cycles than the usual Resource keys, hence initialize the reference counter to to (unsigned) -1. While we are at it, remove the prototype for dns_resource_key_new_dname() which was never implemented.

resolved: synthesize NODATA cache results when we find matching NSEC RRs If we have a precisely matching NSEC RR for a name, we can use its type bit field to synthesize NODATA cache lookup results for all types not mentioned in there. This is useful for mDNS where NSEC RRs are used to indicate missing RRs for a specific type, but is beneficial in other cases too. To test this, consider these two lines: systemd-resolve-host -t NSEC nasa.gov systemd-resolve-host -t SRV nasa.gov The second line will not result in traffic as the first line already cached the NSEC field.

resolved: implement client-side DNAME resolution Most servers apparently always implicitly convert DNAME to CNAME, but some servers don't, hence implement this properly, as this is required by edns0.

util-lib: split out allocation calls into alloc-util.[ch]

resolved: cache - cache what we can of negative redirect chains When a NXDATA or a NODATA response is received for an alias it may include CNAME records from the redirect chain. We should cache the response for each of these names to avoid needless roundtrips in the future. It is not sufficient to do the negative caching only for the canonical name, as the included redirection chain is not guaranteed to be complete. In fact, only the final CNAME record from the chain is guaranteed to be included. We take care not to cache entries that redirects outside the current zone, as the SOA will then not be valid.

resolved: cache - handle CNAME redirection CNAME records are special in the way they are treated by DNS servers, and our cache should mimic that behavior: In case a domain name has an alias, its CNAME record is returned in place of any other. Our cache was not doing this despite caching the CNAME records, this entailed needless lookups to re-resolve the CNAME.

resolved: cache - only allow putting a single question key at a time Only one key is allowed per transaction now, so let's simplify things and only allow putting one question key into the cache at a time.

resolved: cache - clarify logging

tree-wide: drop {} from one-line if blocks Patch via coccinelle.

resolved: dump cache and zone contents to syslog on SIGUSR1

resolved: allow dns_cache_put() without a question Currently, dns_cache_put() does a number of things: 1) It unconditionally removes all keys contained in the passed question before adding keys from the newly arrived answers. 2) It puts positive entries into the cache for all RRs contained in the answer. 3) It creates negative entries in the cache for all keys in the question that are not answered. Allow passing q = NULL in the parameters and skip 1) and 3), so we can use that function for mDNS responses. In this case, the question is irrelevant, we are interested in all answers we got.

resolved: only maintain one question RR key per transaction Let's simplify things and only maintain a single RR key per transaction object, instead of a full DnsQuestion. Unicast DNS and LLMNR don't support multiple questions per packet anway, and Multicast DNS suggests coalescing questions beyond a single dns query, across the whole system.

resolved: add reference to negative caching RFC

resolved: rework synthesizing logic With this change we'll now also generate synthesized RRs for the local LLMNR hostname (first label of system hostname), the local mDNS hostname (first label of system hostname suffixed with .local), the "gateway" hostname and all the reverse PTRs. This hence takes over part of what nss-myhostname already implemented. Local hostnames resolve to the set of local IP addresses. Since the addresses are possibly on different interfaces it is necessary to change the internal DnsAnswer object to track per-RR interface indexes, and to change the bus API to always return the interface per-address rather than per-reply. This change also patches the existing clients for resolved accordingly (nss-resolve + systemd-resolve-host). This also changes the routing logic for queries slightly: we now ensure that the local hostname is never resolved via LLMNR, thus making it trustable on the local system.

resolved: cache - add more detailed cache debug logging

resolved: cache - use clock_boottime_or_monotonic() We cannot rely on CLOCK_BOOTTIME being supported by the kernel, so fallack to CLOCK_MONOTONIC if the former is not supported.

resolved: fix crash when shutting down Reported by Cristian Rodríguez http://lists.freedesktop.org/archives/systemd-devel/2015-May/031626.html

hashmap: introduce hash_ops to make struct Hashmap smaller It is redundant to store 'hash' and 'compare' function pointers in struct Hashmap separately. The functions always comprise a pair. Store a single pointer to struct hash_ops instead. systemd keeps hundreds of hashmaps, so this saves a little bit of memory.

resolved: implement full LLMNR conflict detection logic

resolved: properly check return value of dns_resource_record_equal()

resolved: don't bother caching negative RRs when the SOA TTL is 0 anyway

resolved: use CLOCK_BOOTTIME instead of CLOCK_MONOTONIC when aging caches and timeing out transactions That way the cache doens't get confused when the system is suspended.

resolved: properly handle adding empty replies to cache

resolved: never cache ANY lookups

resolved: only cache answer RRs, never additional or authoritative RRs of responses

resolve: add llmnr responder side for UDP and TCP Name defending is still missing.

resolved: implement negative caching

resolved: rework logic so that we can share transactions between queries of different clients

resolved: various bad memory access fixes to the cache

resolved: add DNS cache