The title of this post refers to a network function commonly known as duplicate address detection (DAD). The complete story will cover a range of seemingly unrelated technologies including ARP, Perl socket programming, systemd, IPv6, and a once-popular LAN technology that if you’ve never seen you probably never will, Token Ring. It all starts with a fake DNS server. Oh yeah, I forgot to mention DNS is here as well, but that is just a happy coincidence.
One of the network listener daemons used by the Dataplane.org sensor network has been a DNS service. One example of such a service is fdns, a minimal DNS network server used to monitor for DNS queries seen in-the-wild on the Internet. fdns listens on both TCP and UDP port 53, observing well-formed, but unsolicited DNS queries. If this daemon sees enough activity it can produce useful DNS signal data for the community.
Awhile ago we redesigned our DNS listener from a customized ISC BIND 9 installation to a smaller, more flexible daemon we coded ourselves. We did this for a number reasons. One was to have a simpler, lightweight daemon that does just what we need it to do. Another was to have control over the code base to avoid surprises if the implementation changes. fdns is an example of a fake DNS server daemon written in Perl. fdns is made possible with the help of Net::DNS, which was “the” library for rapid DNS tool making for many years. There are now many alternatives, but Net::DNS is still maintained by NLnet Labs and continues to work well for those that are familiar with it.
What does any of this have to do with IPv6 DAD?!?!
Good question. We’re getting there.
The fdns code can listen on both IPv4 and IPv6 addresses in one of two
ways. It can listen on any available IP address or it can be given a
list of IP addresses to specifically bind to. One way to do this is to
just bind to the in6addr_any ('::') unspecified IPv6 address. This is
convenient, because the socket will handle both IPv4 and IPv6 transport.
The IPv4 addresses will be represented as IPv4-Mapped IPv6 addresses of
::FFFF:192.0.2.1 (see IETF RFC 4291 IP Version 6 Addressing
Architecture). There is just
one problem with this approach. Perl’s UDP socket handling does not
track the local address when using this type of socket listener. If
your code needs to know the receiving IP address for debugging purposes
or because a customized response requires that information, you will
need to pass a list of specific listener addresses instead. fdns can
operate in either mode, controlled by a command line argument.
Now here’s where the problem with IPv6 DAD comes in. If you setup fdns to listen on a set of specific IPv6 addresses, those address must be in the “preferred” state. IPv6 address state is detailed in IETF RFC 4862 IPv6 Stateless Address Autoconfiguration. In a nutshell, an IPv6 address is available for use when in the preferred state and when DAD is active, this can take a few seconds. Even if your IPv6 address is manually assigned, RFC 4862 states and is reiterated again in IETF RFC 8504 IPv6 Node Requirements:
“Duplicate Address Detection MUST be performed on all unicast addresses prior to assigning them to an interface, regardless of whether they are obtained through stateless autoconfiguration, DHCPv6, or manual configuration […]”
In other words, if you have fdns or similar listener startup at boot and want them bound to a specific IPv6 address, and they don’t wait for DAD to complete, your listener may not be fully operational, and may even fail in interesting ways.
The IETF specifications permit a system administrator to disable DAD.
In Linux, the sysctl setting is accept_dad found under
/proc/sys/net/ipv6/conf/[interface], where [interface] is the
associated IPv6 interface in question. Disabling DAD may be a
reasonable option for many environments. However, if modifying standard
operating system behavior makes you uneasy for fear of potential future
and unforeseen consequences there is an alternative approach. If you
choose to leave DAD enabled you have to make sure DAD completes before
your startup service attempts to bind to any specific IPv6 address. In
systemd, this can be accomplished in at least two ways. One is with
--ipv6 option to systemd-networkd-wait-online.service. A recent
SYSTEMD-NETWORKD-WAIT-ONLINE.SERVICE(8) man page explains this feature:
Waiting for an IPv6 address of each network interface to be configured.
In other words, in your service file for a start up service, you can instruct systemd to wait for DAD to complete, which would presumably be when all assigned IPv6 addresses are in the preferred state. This feature is implemented in the settle-dad.sh script from the ifupdown package. Unfortunately, waiting for IPv6 addresses to be available is a relatively new ifupdown feature and probably not the default setting in distributions that have it as of this writing. For example, Debian 11, the current stable release named bullseye, does not have this capability, and in the testing release, it is not enabled by default.
Another way to wait for DAD to complete, admittedly a hack, is to modify systemd, sysvinit, or the daemon itself to wait a sufficient amount of time for IPv6 DAD to complete before binding to IPv6 addresses. One simple approach to accomplish this with systemd is to set the following parameters in the associated service file:
[Service] ExecStartPre=/bin/sleep 10 TimeoutStartSec=infinity
This will insert an arbitrary 10 second delay before starting the associated service. Ten seconds is a hedge, this should be longer than necessary for practically any Earth-based system. In practice, you will probably see DAD complete within two to three seconds.
What about ARP and Token Ring? How do they factor in?
Investigating IPv6 DAD and how it worked, or didn’t in our test case, eventually led me to wonder about the history of duplicate address tests in networking. I’m old enough, and perhaps lucky (or unlucky depending on your perspective) to have been involved in operating Token Ring networks. The process of “inserting” a node into a Token Ring LAN involved a number of steps before communication to and from the new node could commence. Phase 2, the third of five phases is where the new node performs a relatively simple duplicate address test. The earliest form of duplicate address detection I knew of was in Token Ring'. Was that really the first widely deployed version of a such a function?
ObFact and ObBrag, many years ago I made some major contributions to an early version of Wikipedia’s Token Ring page. There wasn’t much there at the time and I was the one who first added details about those five phases and the token insertion process to the article.
I thought of ARP, because it can be used to perform duplicate IPv4 address detection, and in fact this has been done with mixed success, but this came much later after ARP was first specified. I wondered what, if anything, preceded either this use or Token Ring’s version. I decided to ask this question on the Internet History mailing list. This led to a number of interesting and useful replies, but so far as I know Token Ring appears to be the first formal use of a duplicate address test in computer networking. If you think of anything earlier, please contact me so I can update this blog post and the mailing list accordingly.
The following additional references helped inform this blog post:
- Andrew Ayer’s blog - Beware the IPv6 DAD Race Condition
- systemd issue #2037 - systemd-networkd-wait-online: Wait ALL links to gain a carrier
- University of Cambridge DNS news and blogs - IPv6 DAD-die issues
- The Art of Web - System: IPv6 Networking and DAD
- systemd pull request #16700 - beef up systemd-networkd-wait-online to wait for ipv4 or ipv6 or both addresses
- systemd pull request #19069 - systemd-networkd-wait-online: wait for specific address family