CDDL HEADER START

The contents of this file are subject to the terms of the

Common Development and Distribution License (the "License").

You may not use this file except in compliance with the License.

You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE

or http://www.opensolaris.org/os/licensing.

See the License for the specific language governing permissions

and limitations under the License.

When distributing Covered Code, include this CDDL HEADER in each

file and include the License file at usr/src/OPENSOLARIS.LICENSE.

If applicable, add the following below this CDDL HEADER, with the

fields enclosed by brackets "[]" replaced with your own identifying

information: Portions Copyright [yyyy] [name of copyright owner]

CDDL HEADER END

Copyright 2007 Sun Microsystems, Inc. All rights reserved.

Use is subject to license terms.

Architectural Overview for the DHCP agent

Peter Memishian

ident "%Z%%M% %I% %E% SMI"

INTRODUCTION

============

The Solaris DHCP agent (dhcpagent) is a DHCP client implementation

compliant with RFCs 2131, 3315, and others. The major forces shaping

its design were:

* Must be capable of managing multiple network interfaces.

* Must consume little CPU, since it will always be running.

* Must have a small memory footprint, since it will always be

running.

* Must not rely on any shared libraries outside of /lib, since

it must run before all filesystems have been mounted.

When a DHCP agent implementation is only required to control a single

interface on a machine, the problem is expressed well as a simple

state-machine, as shown in RFC2131. However, when a DHCP agent is

responsible for managing more than one interface at a time, the

problem becomes much more complicated.

This can be resolved using threads or with an event-driven model.

Given that DHCP's behavior can be expressed concisely as a state

machine, the event-driven model is the closest match.

While tried-and-true, that model is subtle and easy to get wrong.

Indeed, much of the agent's code is there to manage the complexity of

programming in an asynchronous event-driven paradigm.

THE BASICS

==========

The DHCP agent consists of roughly 30 source files, most with a

companion header file. While the largest source file is around 1700

lines, most are much shorter. The source files can largely be broken

up into three groups:

* Source files that, along with their companion header files,

define an abstract "object" that is used by other parts of

the system. Examples include "packet.c", which along with

"packet.h" provide a Packet object for use by the rest of

the agent; and "async.c", which along with "async.h" defines

an interface for managing asynchronous transactions within

the agent.

* Source files that implement a given state of the agent; for

instance, there is a "request.c" which comprises all of

the procedural "work" which must be done while in the

REQUESTING state of the agent. By encapsulating states in

files, it becomes easier to debug errors in the

client/server protocol and adapt the agent to new

constraints, since all the relevant code is in one place.

* Source files, which along with their companion header files,

encapsulate a given task or related set of tasks. The

difference between this and the first group is that the

interfaces exported from these files do not operate on

an "object", but rather perform a specific task. Examples

include "defaults.c", which provides a useful interface

to /etc/default/dhcpagent file operations.

OVERVIEW

========

Here we discuss the essential objects and subtle aspects of the

DHCP agent implementation. Note that there is of course much more

that is not discussed here, but after this overview you should be able

to fend for yourself in the source code.

For details on the DHCPv6 aspects of the design, and how this relates

to the implementation present in previous releases of Solaris, see the

README.v6 file.

Event Handlers and Timer Queues

-------------------------------

The most important object in the agent is the event handler, whose

interface is in libinetutil.h and whose implementation is in

libinetutil. The event handler is essentially an object-oriented

wrapper around poll(2): other components of the agent can register to

be called back when specific events on file descriptors happen -- for

instance, to wait for requests to arrive on its IPC socket, the agent

registers a callback function (accept_event()) that will be called

back whenever a new connection arrives on the file descriptor

associated with the IPC socket. When the agent initially begins in

main(), it registers a number of events with the event handler, and

then calls iu_handle_events(), which proceeds to wait for events to

happen -- this function does not return until the agent is shutdown

via signal.

When the registered events occur, the callback functions are called

back, which in turn might lead to additional callbacks being

registered -- this is the classic event-driven model. (As an aside,

note that programming in an event-driven model means that callbacks

cannot block, or else the agent will become unresponsive.)

A special kind of "event" is a timeout. Since there are many timers

which must be maintained for each DHCP-controlled interface (such as a

lease expiration timer, time-to-first-renewal (t1) timer, and so

forth), an object-oriented abstraction to timers called a "timer

queue" is provided, whose interface is in libinetutil.h with a

corresponding implementation in libinetutil. The timer queue allows

callback functions to be "scheduled" for callback after a certain

amount of time has passed.

The event handler and timer queue objects work hand-in-hand: the event

handler is passed a pointer to a timer queue in iu_handle_events() --

from there, it can use the iu_earliest_timer() routine to find the

timer which will next fire, and use this to set its timeout value in

its call to poll(2). If poll(2) returns due to a timeout, the event

handler calls iu_expire_timers() to expire all timers that expired

(note that more than one may have expired if, for example, multiple

timers were set to expire at the same time).

Although it is possible to instantiate more than one timer queue or

event handler object, it doesn't make a lot of sense -- these objects

are really "singletons". Accordingly, the agent has two global

variables, `eh' and `tq', which store pointers to the global event

handler and timer queue.

Network Interfaces

------------------

For each network interface managed by the agent, there is a set of

associated state that describes both its general properties (such as

the maximum MTU) and its connections to DHCP-related state (the

protocol state machines). This state is stored in a pair of

structures called `dhcp_pif_t' (the IP physical interface layer or

PIF) and `dhcp_lif_t' (the IP logical interface layer or LIF). Each

dhcp_pif_t represents a single physical interface, such as "hme0," for

a given IP protocol version (4 or 6), and has a list of dhcp_lif_t

structures representing the logical interfaces (such as "hme0:1") in

use by the agent.

This split is important because of differences between IPv4 and IPv6.

For IPv4, each DHCP state machine manages a single IP address and

associated configuration data. This corresponds to a single logical

interface, which must be specified by the user. For IPv6, however,

each DHCP state machine manages a group of addresses, and is

associated with DUID value rather than with just an interface.

Thus, DHCPv6 behaves more like in.ndpd in its creation of "ADDRCONF"

interfaces. The agent automatically plumbs logical interfaces when

needed and removes them when the addresses expire.

The state for a given session is stored separately in `dhcp_smach_t'.

This state machine then points to the main LIF used for I/O, and to a

list of `dhcp_lease_t' structures representing individual leases, and

each of those points to a list of LIFs corresponding to the individual

addresses being managed.

One point that was brushed over in the preceding discussion of event

handlers and timer queues was context. Recall that the event-driven

nature of the agent requires that functions cannot block, lest they

starve out others and impact the observed responsiveness of the agent.

As an example, consider the process of extending a lease: the agent

must send a REQUEST packet and wait for an ACK or NAK packet in

response. This is done by sending a REQUEST and then returning to the

event handler that waits for an ACK or NAK packet to arrive on the

file descriptor associated with the interface. Note however, that

when the ACK or NAK does arrive, and the callback function called

back, it must know which state machine this packet is for (it must get

back its context). This could be handled through an ad-hoc mapping of

file descriptors to state machines, but a cleaner approach is to have

the event handler's register function (iu_register_event()) take in an

opaque context pointer, which will then be passed back to the

callback. In the agent, the context pointer used depends on the

nature of the event: events on LIFs use the dhcp_lif_t pointer, events

on the state machine use dhcp_smach_t, and so on.

Note that there is nothing that guarantees the pointer passed into

iu_register_event() or iu_schedule_timer() will still be valid when

the callback is called back (for instance, the memory may have been

freed in the meantime). To solve this problem, all of the data

structures used in this way are reference counted. For more details

on how the reference count scheme is implemented, see the closing

comments in interface.h regarding memory management.

Transactions

------------

Many operations performed via DHCP must be performed in groups -- for

instance, acquiring a lease requires several steps: sending a

DISCOVER, collecting OFFERs, selecting an OFFER, sending a REQUEST,

and receiving an ACK, assuming everything goes well. Note however

that due to the event-driven model the agent operates in, these

operations are not inherently "grouped" -- instead, the agent sends a

DISCOVER, goes back into the main event loop, waits for events

(perhaps even requests on the IPC channel to begin acquiring a lease

on another state machine), eventually checks to see if an acceptable

OFFER has come in, and so forth. To some degree, the notion of the

state machine's current state (SELECTING, REQUESTING, etc) helps

control the potential chaos of the event-driven model (for instance,

if while the agent is waiting for an OFFER on a given state machine,

an IPC event comes in requesting that the leases be RELEASED, the

agent knows to send back an error since the state machine must be in

at least the BOUND state before a RELEASE can be performed.)

However, states are not enough -- for instance, suppose that the agent

begins trying to renew a lease. This is done by sending a REQUEST

packet and waiting for an ACK or NAK, which might never come. If,

while waiting for the ACK or NAK, the user sends a request to renew

the lease as well, then if the agent were to send another REQUEST,

things could get quite complicated (and this is only the beginning of

this rathole). To protect against this, two objects exist:

`async_action' and `ipc_action'. These objects are related, but

independent of one another; the more essential object is the

`async_action', which we will discuss first.

In short, an `async_action' represents a pending transaction (aka

asynchronous action), of which each state machine can have at most

one. The `async_action' structure is embedded in the `dhcp_smach_t'

structure, which is fine since there can be at most one pending

transaction per state machine. Typical "asynchronous transactions"

are START, EXTEND, and INFORM, since each consists of a sequence of

packets that must be done without interruption. Note that not all

DHCP operations are "asynchronous" -- for instance, a DHCPv4 RELEASE

operation is synchronous (not asynchronous) since after the RELEASE is

sent no reply is expected from the DHCP server, but DHCPv6 Release is

asynchronous, as all DHCPv6 messages are transactional. Some

operations, such as status query, are synchronous and do not affect

the system state, and thus do not require sequencing.

When the agent realizes it must perform an asynchronous transaction,

it calls async_async() to open the transaction. If one is already

pending, then the new transaction must fail (the details of failure

depend on how the transaction was initiated, which is described in

more detail later when the `ipc_action' object is discussed). If

there is no pending asynchronous transaction, the operation succeeds.

When the transaction is complete, either async_finish() or

async_cancel() must be called to complete or cancel the asynchronous

action on that state machine. If the transaction is unable to

complete within a certain amount of time (more on this later), a timer

should be used to cancel the operation.

The notion of asynchronous transactions is complicated by the fact

that they may originate from both inside and outside of the agent.

For instance, a user initiates an asynchronous START transaction when

he performs an `ifconfig hme0 dhcp start', but the agent will

internally need to perform asynchronous EXTEND transactions to extend

the lease before it expires. Note that user-initiated actions always

have priority over internal actions: the former will cancel the

latter, if necessary.

This leads us into the `ipc_action' object. An `ipc_action'

represents the IPC-related pieces of an asynchronous transaction that

was started as a result of a user request, as well as the `BUSY' state

of the administrative interface. Only IPC-generated asynchronous

transactions have a valid `ipc_action' object. Note that since there

can be at most one asynchronous action per state machine, there can

also be at most one `ipc_action' per state machine (this means it can

also conveniently be embedded inside the `dhcp_smach_t' structure).

One of the main purposes of the `ipc_action' object is to timeout user

events. When the user specifies a timeout value as an argument to

ifconfig, he is specifying an `ipc_action' timeout; in other words,

how long he is willing to wait for the command to complete. When this

time expires, the ipc_action is terminated, as well as the

asynchronous operation.

The API provided for the `ipc_action' object is quite similar to the

one for the `async_action' object: when an IPC request comes in for an

operation requiring asynchronous operation, ipc_action_start() is

called. When the request completes, ipc_action_finish() is called.

If the user times out before the request completes, then

ipc_action_timeout() is called.

Packet Management

-----------------

Another complicated area is packet management: building, manipulating,

sending and receiving packets. These operations are all encapsulated

behind a dozen or so interfaces (see packet.h) that abstract the

unimportant details away from the rest of the agent code. In order to

send a DHCP packet, code first calls init_pkt(), which returns a

dhcp_pkt_t initialized suitably for transmission. Note that currently

init_pkt() returns a dhcp_pkt_t that is actually allocated as part of

the `dhcp_smach_t', but this may change in the future.. After calling

init_pkt(), the add_pkt_opt*() functions are used to add options to

the DHCP packet. Finally, send_pkt() and send_pkt_v6() can be used to

transmit the packet to a given IP address.

The send_pkt() function handles the details of packet timeout and

retransmission. The last argument to send_pkt() is a pointer to a

"stop function." If this argument is passed as NULL, then the packet

will only be sent once (it won't be retransmitted). Otherwise, before

each retransmission, the stop function will be called back prior to

retransmission. The callback may alter dsm_send_timeout if necessary

to place a cap on the next timeout; this is done for DHCPv6 in

stop_init_reboot() in order to implement the CNF_MAX_RD constraint.

The return value from this function indicates whether to continue

retransmission or not, which allows the send_pkt() caller to control

the retransmission policy without making it have to deal with the

retransmission mechanism. See request.c for an example of this in

action.

The recv_pkt() function is simpler but still complicated by the fact

that one may want to receive several different types of packets at

once. The caller registers an event handler on the file descriptor,

and then calls recv_pkt() to read in the packet along with meta

information about the message (the sender and interface identifier).

For IPv6, packet reception is done with a single socket, using

IPV6_PKTINFO to determine the actual destination address and receiving

interface. Packets are then matched against the state machines on the

given interface through the transaction ID.

For IPv4, due to oddities in the DHCP specification (discussed in

PSARC/2007/571), a special IP_DHCPINIT_IF socket option must be used

to allow unicast DHCP traffic to be received on an interface during

lease acquisition. Since the IP_DHCPINIT_IF socket option can only

enable one interface at a time, one socket must be used per interface.

Time

----

The notion of time is an exceptionally subtle area. You will notice

five ways that time is represented in the source: as lease_t's,

uint32_t's, time_t's, hrtime_t's, and monosec_t's. Each of these

types serves a slightly different function.

The `lease_t' type is the simplest to understand; it is the unit of

time in the CD_{LEASE,T1,T2}_TIME options in a DHCP packet, as defined

by RFC2131. This is defined as a positive number of seconds (relative

to some fixed point in time) or the value `-1' (DHCP_PERM) which

represents infinity (i.e., a permanent lease). The lease_t should be

used either when dealing with actual DHCP packets that are sent on the

wire or for variables which follow the exact definition given in the

RFC.

The `uint32_t' type is also used to represent a relative time in

seconds. However, here the value `-1' is not special and of course

this type is not tied to any definition given in RFC2131. Use this

for representing "offsets" from another point in time that are not

DHCP lease times.

The `time_t' type is the natural Unix type for representing time since

the epoch. Unfortunately, it is affected by stime(2) or adjtime(2)

and since the DHCP client is used during system installation (and thus

when time is typically being configured), the time_t cannot be used in

general to represent an absolute time since the epoch. For instance,

if a time_t were used to keep track of when a lease began, and then a

minute later stime(2) was called to adjust the system clock forward a

year, then the lease would appeared to have expired a year ago even

though it has only been a minute. For this reason, time_t's should

only be used either when wall time must be displayed (such as in

DHCP_STATUS ipc transaction) or when a time meaningful across reboots

must be obtained (such as when caching an ACK packet at system

shutdown).

The `hrtime_t' type returned from gethrtime() works around the

limitations of the time_t in that it is not affected by stime(2) or

adjtime(2), with the disadvantage that it represents time from some

arbitrary time in the past and in nanoseconds. The timer queue code

deals with hrtime_t's directly since that particular piece of code is

meant to be fairly independent of the rest of the DHCP client.

However, dealing with nanoseconds is error-prone when all the other

time types are in seconds. As a result, yet another time type, the

`monosec_t' was created to represent a monotonically increasing time

in seconds, and is really no more than (hrtime_t / NANOSEC). Note

that this unit is typically used where time_t's would've traditionally

been used. The function monosec() in util.c returns the current

monosec, and monosec_to_time() can convert a given monosec to wall

time, using the system's current notion of time.

One additional limitation of the `hrtime_t' and `monosec_t' types is

that they are unaware of the passage of time across checkpoint/resume

events (e.g., those generated by sys-suspend(1M)). For example, if

gethrtime() returns time T, and then the machine is suspended for 2

hours, and then gethrtime() is called again, the time returned is not

T + (2 * 60 * 60 * NANOSEC), but rather approximately still T.

To work around this (and other checkpoint/resume related problems),

when a system is resumed, the DHCP client makes the pessimistic

assumption that all finite leases have expired while the machine was

suspended and must be obtained again. This is known as "refreshing"

the leases, and is handled by refresh_smachs().

Note that it appears like a more intelligent approach would be to

record the time(2) when the system is suspended, compare that against

the time(2) when the system is resumed, and use the delta between them

to decide which leases have expired. Sadly, this cannot be done since

through at least Solaris 10, it is not possible for userland programs

to be notified of system suspend events.

Configuration

-------------

For the most part, the DHCP client only *retrieves* configuration data

from the DHCP server, leaving the configuration to scripts (such as

boot scripts), which themselves use dhcpinfo(1) to retrieve the data

from the DHCP client. This is desirable because it keeps the mechanism

of retrieving the configuration data decoupled from the policy of using

the data.

However, unless used in "inform" mode, the DHCP client *does*

configure each IP interface enough to allow it to communicate with

other hosts. Specifically, the DHCP client configures the interface's

IP address, netmask, and broadcast address using the information

provided by the server. Further, for IPv4 logical interface 0

("hme0"), any provided default routes are also configured.

For IPv6, only the IP addresses are set. The netmask (prefix) is then

set automatically by in.ndpd, and routes are discovered in the usual

way by router discovery or routing protocols. DHCPv6 doesn't set

routes.

Since logical interfaces cannot be specified as output interfaces in

the kernel forwarding table, and in most cases, logical interfaces

share a default route with their associated physical interface, the

DHCP client does not automatically add or remove default routes when

IPv4 leases are acquired or expired on logical interfaces.

Event Scripting

---------------

The DHCP client supports user program invocations on DHCP events. The

supported events are BOUND, EXTEND, EXPIRE, DROP, RELEASE, and INFORM

for DHCPv4, and BUILD6, EXTEND6, EXPIRE6, DROP6, LOSS6, RELEASE6, and

INFORM6 for DHCPv6. The user program runs asynchronous to the DHCP

client so that the main event loop stays active to process other

events, including events triggered by the user program (for example,

when it invokes dhcpinfo).

The user program execution is part of the transaction of a DHCP command.

For example, if the user program is not enabled, the transaction of the

DHCP command START is considered over when an ACK is received and the

interface is configured successfully. If the user program is enabled,

it is invoked after the interface is configured successfully, and the

transaction is considered over only when the user program exits. The

event scripting implementation makes use of the asynchronous operations

discussed in the "Transactions" section.

An upper bound of 58 seconds is imposed on how long the user program

can run. If the user program does not exit after 55 seconds, the signal

SIGTERM is sent to it. If it still does not exit after additional 3

seconds, the signal SIGKILL is sent to it. Since the event handler is

a wrapper around poll(), the DHCP client cannot directly observe the

completion of the user program. Instead, the DHCP client creates a

child "helper" process to synchronously monitor the user program (this

process is also used to send the aformentioned signals to the process,

if necessary). The DHCP client and the helper process share a pipe

which is included in the set of poll descriptors monitored by the DHCP

client's event handler. When the user program exits, the helper process

passes the user program exit status to the DHCP client through the pipe,

informing the DHCP client that the user program has finished. When the

DHCP client is asked to shut down, it will wait for any running instances

of the user program to complete.

CDDL HEADER START

The contents of this file are subject to the terms of the

Common Development and Distribution License (the "License").

You may not use this file except in compliance with the License.

You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE

or http://www.opensolaris.org/os/licensing.

See the License for the specific language governing permissions

and limitations under the License.

When distributing Covered Code, include this CDDL HEADER in each

file and include the License file at usr/src/OPENSOLARIS.LICENSE.

If applicable, add the following below this CDDL HEADER, with the

fields enclosed by brackets "[]" replaced with your own identifying

information: Portions Copyright [yyyy] [name of copyright owner]

CDDL HEADER END

Copyright 2007 Sun Microsystems, Inc. All rights reserved.

Use is subject to license terms.

ident "%Z%%M% %I% %E% SMI"

** PLEASE NOTE:

**

** This document discusses aspects of the DHCPv4 client design that have

** since changed (e.g., DLPI is no longer used). However, since those

** aspects affected the DHCPv6 design, the discussion has been left for

** historical record.

DHCPv6 Client Low-Level Design

Introduction

This project adds DHCPv6 client-side (not server) support to

Solaris. Future projects may add server-side support as well as

enhance the basic capabilities added here. These future projects

are not discussed in detail in this document.

This document assumes that the reader is familiar with the following

other documents:

- RFC 3315: the primary description of DHCPv6

- RFCs 2131 and 2132: IPv4 DHCP

- RFCs 2461 and 2462: IPv6 NDP and stateless autoconfiguration

- RFC 3484: IPv6 default address selection

- ifconfig(1M): Solaris IP interface configuration

- in.ndpd(1M): Solaris IPv6 Neighbor and Router Discovery daemon

- dhcpagent(1M): Solaris DHCP client

- dhcpinfo(1): Solaris DHCP parameter utility

- ndpd.conf(4): in.ndpd configuration file

- netstat(1M): Solaris network status utility

- snoop(1M): Solaris network packet capture and inspection

- "DHCPv6 Client High-Level Design"

Several terms from those documents (such as the DHCPv6 IA_NA and

IAADDR options) are used without further explanation in this

document; see the reference documents above for details.

The overall plan is to enhance the existing Solaris dhcpagent so

that it is able to process DHCPv6. It would also have been possible

to create a new, separate daemon process for this, or to integrate

the feature into in.ndpd. These alternatives, and the reason for

the chosen design, are discussed in Appendix A.

This document discusses the internal design issues involved in the

protocol implementation, and with the associated components (such as

in.ndpd, snoop, and the kernel's source address selection

algorithm). It does not discuss the details of the protocol itself,

which are more than adequately described in the RFC, nor the

individual lines of code, which will be in the code review.

As a cross-reference, Appendix B has a summary of the components

involved and the changes to each.

Background

In order to discuss the design changes for DHCPv6, it's necessary

first to talk about the current IPv4-only design, and the

assumptions built into that design.

The main data structure used in dhcpagent is the 'struct ifslist'.

Each instance of this structure represents a Solaris logical IP

interface under DHCP's control. It also represents the shared state

with the DHCP server that granted the address, the address itself,

and copies of the negotiated options.

There is one list in dhcpagent containing all of the IP interfaces

that are under DHCP control. IP interfaces not under DHCP control

(for example, those that are statically addressed) are not included

in this list, even when plumbed on the system. These ifslist

entries are chained like this:

ifsheadp -> ifslist -> ifslist -> ifslist -> NULL

net0 net0:1 net1

Each ifslist entry contains the address, mask, lease information,

interface name, hardware information, packets, protocol state, and

timers. The name of the logical IP interface under DHCP's control

is also the name used in the administrative interfaces (dhcpinfo,

ifconfig) and when logging events.

Each entry holds open a DLPI stream and two sockets. The DLPI

stream is nulled-out with a filter when not in use, but still

consumes system resources. (Most significantly, it causes data

copies in the driver layer that end up sapping performance.)

The entry storage is managed by a insert/hold/release/remove model

and reference counts. In this model, insert_ifs() allocates a new

ifslist entry and inserts it into the global list, with the global

list holding a reference. remove_ifs() removes it from the global

list and drops that reference. hold_ifs() and release_ifs() are

used by data structures that refer to ifslist entries, such as timer

entries, to make sure that the ifslist entry isn't freed until the

timer has been dispatched or deleted.

The design is single-threaded, so code that walks the global list

needn't bother taking holds on the ifslist structure. Only

references that may be used at a different time (i.e., pointers

stored in other data structures) need to be recorded.

Packets are handled using PKT (struct dhcp; <netinet/dhcp.h>),

PKT_LIST (struct dhcp_list; <dhcp_impl.h>), and dhcp_pkt_t (struct

dhcp_pkt; "packet.h"). PKT is just the RFC 2131 DHCP packet

structure, and has no additional information, such as packet length.

PKT_LIST contains a PKT pointer, length, decoded option arrays, and

linkage for putting the packet in a list. Finally, dhcp_pkt_t has a

PKT pointer and length values suitable for modifying the packet.

Essentially, PKT_LIST is a wrapper for received packets, and

dhcp_pkt_t is a wrapper for packets to be sent.

The basic PKT structure is used in dhcpagent, inetboot, in.dhcpd,

libdhcpagent, libwanboot, libdhcputil, and others. PKT_LIST is used

in a similar set of places, including the kernel NFS modules.

dhcp_pkt_t is (as the header file implies) limited to dhcpagent.

In addition to these structures, dhcpagent maintains a set of

internal supporting abstractions. Two key ones involved in this

project are the "async operation" and the "IPC action." An async

operation encapsulates the actions needed for a given operation, so

that if cancellation is needed, there's a single point where the

associated resources can be freed. An IPC action represents the

user state related to the private interface used by ifconfig.

DHCPv6 Inherent Differences

DHCPv6 naturally has some commonality with IPv4 DHCP, but also has

some significant differences.

Unlike IPv4 DHCP, DHCPv6 relies on link-local IP addresses to do its

work. This means that, on Solaris, the client doesn't need DLPI to

perform any of the I/O; regular IP sockets will do the job. It also

means that, unlike IPv4 DHCP, DHCPv6 does not need to obtain a lease

for the address used in its messages to the server. The system

provides the address automatically.

IPv4 DHCP expects some messages from the server to be broadcast.

DHCPv6 has no such mechanism; all messages from the server to the

client are unicast. In the case where the client and server aren't

on the same subnet, a relay agent is used to get the unicast replies

back to the client's link-local address.

With IPv4 DHCP, a single address plus configuration options is

leased with a given client ID and a single state machine instance,

and the implementation binds that to a single IP logical interface

specified by the user. The lease has a "Lease Time," a required

option, as well as two timers, called T1 (renew) and T2 (rebind),

which are controlled by regular options.

DHCPv6 uses a single client/server session to control the

acquisition of configuration options and "identity associations"

(IAs). The identity associations, in turn, contain lists of

addresses for the client to use and the T1/T2 timer values. Each

individual address has its own preferred and valid lifetime, with

the address being marked "deprecated" at the end of the preferred

interval, and removed at the end of the valid interval.

IPv4 DHCP leaves many of the retransmit decisions up to the client,

and some things (such as RELEASE and DECLINE) are sent just once.

Others (such as the REQUEST message used for renew and rebind) are

dealt with by heuristics. DHCPv6 treats each message to the server

as a separate transaction, and resends each message using a common

retransmission mechanism. DHCPv6 also has separate messages for

Renew, Rebind, and Confirm rather than reusing the Request

mechanism.

The set of options (which are used to convey configuration

information) for each protocol are distinct. Notably, two of the

mistakes from IPv4 DHCP have been fixed: DHCPv6 doesn't carry a

client name, and doesn't attempt to impersonate a routing protocol

by setting a "default route."

Another welcome change is the lack of a netmask/prefix length with

DHCPv6. Instead, the client uses the Router Advertisement prefixes

to set the correct interface netmask. This reduces the number of

databases that need to be kept in sync. (The equivalent mechanism

in IPv4 would have been the use of ICMP Address Mask Request /

Reply, but the BOOTP designers chose to embed it in the address

assignment protocol itself.)

Otherwise, DHCPv6 is similar to IPv4 DHCP. The same overall

renew/rebind and lease expiry strategy is used, although the state

machine events must now take into account multiple IAs and the fact

that each can cause RENEWING or REBINDING state independently.

DHCPv6 And Solaris

The protocol distinctions above have several important implications.

For the logical interfaces:

- Because Solaris uses IP logical interfaces to configure

addresses, we must have multiple IP logical interfaces per IA

with IPv6.

- Because we need to support multiple addresses (and thus multiple

IP logical interfaces) per IA and multiple IAs per client/server

session, the IP logical interface name isn't a unique name for

the lease.

As a result, IP logical interfaces will come and go with DHCPv6,

just as happens with the existing stateless address

autoconfiguration support in in.ndpd. The logical interface names

(visible in ifconfig) have no administrative significance.

Fortunately, DHCPv6 does end up with one fixed name that can be used

to identify a session. Because DHCPv6 uses link local addresses for

communication with the server, the name of the IP logical interface

that has this link local address (normally the same as the IP

physical interface) can be used as an identifier for dhcpinfo and

logging purposes.

Dhcpagent Redesign Overview

The redesign starts by refactoring the IP interface representation.

Because we need to have multiple IP logical interfaces (LIFs) for a

single identity association (IA), we should not store all of the

DHCP state information along with the LIF information.

For DHCPv6, we will need to keep LIFs on a single IP physical

interface (PIF) together, so this is probably also a good time to

reconsider the way dhcpagent represents physical interfaces. The

current design simply replicates the state (notably the DLPI stream,

but also the hardware address and other bits) among all of the

ifslist entries on the same physical interface.

The new design creates two lists of dhcp_pif_t entries, one list for

IPv4 and the other for IPv6. Each dhcp_pif_t represents a PIF, with

a list of dhcp_lif_t entries attached, each of which represents a

LIF used by dhcpagent. This structure mirrors the kernel's ill_t

and ipif_t interface representations.

Next, the lease-tracking needs to be refactored. DHCPv6 is the

functional superset in this case, as it has two lifetimes per

address (LIF) and IA groupings with shared T1/T2 timers. To

represent these groupings, we will use a new dhcp_lease_t structure.

IPv4 DHCP will have one such structure per state machine, while

DHCPv6 will have a list. (Note: the initial implementation will

have only one lease per DHCPv6 state machine, because each state

machine uses a single link-local address, a single DUID+IAID pair,

and supports only Non-temporary Addresses [IA_NA option]. Future

enhancements may use multiple leases per DHCPv6 state machine or

support other IA types.)

For all of these new structures, we will use the same insert/hold/

release/remove model as with the original ifslist.

Finally, the remaining items (and the bulk of the original ifslist

members) are kept on a per-state-machine basis. As this is no

longer just an "interface," a new dhcp_smach_t structure will hold

these, and the ifslist structure is gone.

Lease Representation

For DHCPv6, we need to track multiple LIFs per lease (IA), but we

also need multiple LIFs per PIF. Rather than having two sets of

list linkage for each LIF, we can observe that a LIF is on exactly

one PIF and is a member of at most one lease, and then simplify: the

lease structure will use a base pointer for the first LIF in the

lease, and a count for the number of consecutive LIFs in the PIF's

list of LIFs that belong to the lease.

When removing a LIF from the system, we need to decrement the count

of LIFs in the lease, and advance the base pointer if the LIF being

removed is the first one. Inserting a LIF means just moving it into

this list and bumping the counter.

When removing a lease from a state machine, we need to dispose of

the LIFs referenced. If the LIF being disposed is the main LIF for

a state machine, then all that we can do is canonize the LIF

(returning it to a default state); this represents the normal IPv4

DHCP operation on lease expiry. Otherwise, the lease is the owner

of that LIF (it was created because of a DHCPv6 IA), and disposal

means unplumbing the LIF from the actual system and removing the LIF

entry from the PIF.

Main Structure Linkage

For IPv4 DHCP, the new linkage is straightforward. Using the same

system configuration example as in the initial design discussion:

+- lease +- lease +- lease

| ^ | ^ | ^

| | | | | |

\ smach \ smach \ smach

\ ^| \ ^| \ ^|

v|v v|v v|v

lif ----> lif -> NULL lif -> NULL

net0 net0:1 net1

^ ^

| |

v4root -> pif --------------------> pif -> NULL

net0 net1

This diagram shows three separate state machines running (with

backpointers omitted for clarity). Each state machine has a single

"main" LIF with which it's associated (and named). Each also has a

single lease structure that points back to the same LIF (count of

1), because IPv4 DHCP controls a single address allocation per state

machine.

DHCPv6 is a bit more complex. This shows DHCPv6 running on two

interfaces (more or fewer interfaces are of course possible) and

with multiple leases on the first interface, and each lease with

multiple addresses (one with two addresses, the second with one).

lease ----------------> lease -> NULL lease -> NULL

^ \(2) |(1) ^ \ (1)

| \ | | \

smach \ | smach \

^ | \ | ^ | \

| v v v | v v

lif --> lif --> lif --> lif --> NULL lif --> lif -> NULL

net0 net0:1 net0:4 net0:2 net1 net1:5

^ ^

| |

v6root -> pif ----------------------------------> pif -> NULL

net0 net1

Note that there's intentionally no ordering based on name in the

list of LIFs. Instead, the contiguous LIF structures in that list

represent the addresses in each lease. The logical interfaces

themselves are allocated and numbered by the system kernel, so they

may not be sequential, and there may be gaps in the list if other

entities (such as in.ndpd) are also configuring interfaces.

Note also that with IPv4 DHCP, the lease points to the LIF that's

also the main LIF for the state machine, because that's the IP

interface that dhcpagent controls. With DHCPv6, the lease (one per

IA structure) points to a separate set of LIFs that are created just

for the leased addresses (one per IA address in an IAADDR option).

The state machine alone points to the main LIF.

Packet Structure Extensions

Obviously, we need some DHCPv6 packet data structures and

definitions. A new <netinet/dhcp6.h> file will be introduced with

the necessary #defines and structures. The key structure there will

be:

struct dhcpv6_message {

uint8_t d6m_msg_type;

uint8_t d6m_transid_ho;

uint16_t d6m_transid_lo;

};

typedef struct dhcpv6_message dhcpv6_message_t;

This defines the usual (non-relay) DHCPv6 packet header, and is

roughly equivalent to PKT for IPv4.

Extending dhcp_pkt_t for DHCPv6 is straightforward, as it's used

only within dhcpagent. This structure will be amended to use a

union for v4/v6 and include a boolean to flag which version is in

use.

For the PKT_LIST structure, things are more complex. This defines

both a queuing mechanism for received packets (typically OFFERs) and

a set of packet decoding structures. The decoding structures are

highly specific to IPv4 DHCP -- they have no means to handle nested

or repeated options (as used heavily in DHCPv6) and make use of the

DHCP_OPT structure which is specific to IPv4 DHCP -- and are

somewhat expensive in storage, due to the use of arrays indexed by

option code number.

Worse, this structure is used throughout the system, so changes to

it need to be made carefully. (For example, the existing 'pkt'

member can't just be turned into a union.)

For an initial prototype, since discarded, I created a new

dhcp_plist_t structure to represent packet lists as used inside

dhcpagent and made dhcp_pkt_t valid for use on input and output.

The result is unsatisfying, though, as it results in code that

manipulates far too many data structures in common cases; it's a sea

of pointers to pointers.

The better answer is to use PKT_LIST for both IPv4 and IPv6, adding

the few new bits of metadata required to the end (receiving ifIndex,

packet source/destination addresses), and staying within the overall

existing design.

For option parsing, dhcpv6_find_option() and dhcpv6_pkt_option()

functions will be added to libdhcputil. The former function will

walk a DHCPv6 option list, and provide safe (bounds-checked) access

to the options inside. The function can be called recursively, so

that option nesting can be handled fairly simply by nested loops,

and can be called repeatedly to return each instance of a given

option code number. The latter function is just a convenience

wrapper on dhcpv6_find_option() that starts with a PKT_LIST pointer

and iterates over the top-level options with a given code number.

There are two special considerations for the use of these library

interfaces: there's no "pad" option for DHCPv6 or alignment

requirements on option headers or contents, and nested options

always follow a structure that has type-dependent length. This

means that code that handles options must all be written to deal

with unaligned data, and suboption code must index the pointer past

the type-dependent part.

Packet Construction

Unlike DHCPv4, DHCPv6 places the transaction timer value in an

option. The existing code sets the current time value in

send_pkt_internal(), which allows it to be updated in a

straightforward way when doing retransmits.

To make this work in a simple manner for DHCPv6, I added a

remove_pkt_opt() function. The update logic just does a remove and

re-adds the option. We could also just assume the presence of the

option, find it, and modify in place, but the remove feature seems

more general.

DHCPv6 uses nesting options. To make this work, two new utility

functions are needed. First, an add_pkt_subopt() function will take

a pointer to an existing option and add an embedded option within

it. The packet length and existing option length are updated. If

that existing option isn't a top-level option, though, this means

that the caller must update the lengths of all of the enclosing

options up to the top level. To do this, update_v6opt_len() will be

added. This is used in the special case of adding a Status Code

option to an IAADDR option within an IA_NA top-level option.

Sockets and I/O Handling

DHCPv6 doesn't need or use either a DLPI or a broadcast IP socket.

Instead, a single unicast-bound IP socket on a link-local address

would be the most that is needed. This is roughly equivalent to

if_sock_ip_fd in the existing design, but that existing socket is

bound only after DHCP reaches BOUND state -- that is, when it

switches away from DLPI. We need something different.

This, along with the excess of open file descriptors in an otherwise

idle daemon and the potentially serious performance problems in

leaving DLPI open at all times, argues for a larger redesign of the

I/O logic in dhcpagent.

The first thing that we can do is eliminate the need for the

per-ifslist if_sock_fd. This is used primarily for issuing ioctls

to configure interfaces -- a task that would work as well with any

open socket -- and is also registered to receive any ACK/NAK packets

that may arrive via broadcast. Both of these can be eliminated by

creating a pair of global sockets (IPv4 and IPv6), bound and

configured for ACK/NAK reception. The only functional difference is

that the list of running state machines must be scanned on reception

to find the correct transaction ID, but the existing design

effectively already goes to this effort because the kernel

replicates received datagrams among all matching sockets, and each

ifslist entry has a socket open.

(The existing code for if_sock_fd makes oblique reference to unknown

problems in the system that may prevent binding from working in some

cases. The reference dates back some seven years to the original

DHCP implementation. I've observed no such problems in extensive

testing and if any do show up, they will be dealt with by fixing the

underlying bugs.)

This leads to an important simplification: it's no longer necessary

to register, unregister, and re-register for packet reception while

changing state -- register_acknak() and unregister_acknak() are

gone. Instead, we always receive, and we dispatch the packets as

they arrive. As a result, when receiving a DHCPv4 ACK or DHCPv6

Reply when in BOUND state, we know it's a duplicate, and we can

discard.

The next part is in minimizing DLPI usage. A DLPI stream is needed

at most for each IPv4 PIF, and it's not needed when all of the

DHCP instances on that PIF are bound. In fact, the current

implementation deals with this in configure_bound() by setting a

"blackhole" packet filter. The stream is left open.

To simplify this, we will open at most one DLPI stream on a PIF, and

use reference counts from the state machines to determine when the

stream must be open and when it can be closed. This mechanism will

be centralized in a set_smach_state() function that changes the

state and opens/closes the DLPI stream when needed.

This leads to another simplification. The I/O logic in the existing

dhcpagent makes use of the protocol state to select between DLPI and

sockets. Now that we keep track of this in a simpler manner, we no

longer need to switch out on state in when sending a packet; just

test the dsm_using_dlpi flag instead.

Still another simplification is in the handling of DHCPv4 INFORM.

The current code has separate logic in it for getting the interface

state and address information. This is no longer necessary, as the

LIF mechanism keeps track of the interface state. And since we have

separate lease structures, and INFORM doesn't acquire a lease, we no

longer have to be careful about canonizing the interface on

shutdown.

Although the default is to send all client messages to a well-known

multicast address for servers and relays, DHCPv6 also has a

mechanism that allows the client to send unicast messages to the

server. The operation of this mechanism is slightly complex.

First, the server sends the client a unicast address via an option.

We may use this address as the destination (rather than the

well-known multicast address for local DHCPv6 servers and relays)

only if we have a viable local source address. This means using

SIOCGDSTINFO each time we try to send unicast. Next, the server may

send back a special status code: UseMulticast. If this is received,

and if we were actually using unicast in our messages to the server,

then we need to forget the unicast address, switch back to

multicast, and resend our last message.

Note that it's important to avoid the temptation to resend the last

message every time UseMulticast is seen, and do it only once on

switching back to multicast: otherwise, a potential feedback loop is

created.

Because IP_PKTINFO (PSARC 2006/466) has integrated, we could go a

step further by removing the need for any per-LIF sockets and just

use the global sockets for all but DLPI. However, in order to

facilitate a Solaris 10 backport, this will be done separately as CR

6509317.

In the case of DHCPv6, we already have IPV6_PKTINFO, so we will pave

the way for IPv4 by beginning to using this now, and thus have just

a single socket (bound to "::") for all of DHCPv6. Doing this

requires switching from the old BSD4.2 -lsocket -lnsl to the

standards-compliant -lxnet in order to use ancillary data.

It may also be possible to remove the need for DLPI for IPv4, and

incidentally simplify the code a fair amount, by adding a kernel

option to allow transmission and reception of UDP packets over

interfaces that are plumbed but not marked IFF_UP. This is left for

future work.

The State Machine

Several parts of the existing state machine need additions to handle

DHCPv6, which is a superset of DHCPv4.

First, there are the RENEWING and REBINDING states. For IPv4 DHCP,

these states map one-to-one with a single address and single lease

that's undergoing renewal. It's a simple progression (on timeout)

from BOUND, to RENEWING, to REBINDING and finally back to SELECTING

to start over. Each retransmit is done by simply rescheduling the

T1 or T2 timer.

For DHCPv6, things are somewhat more complex. At any one time,

there may be multiple IAs (leases) that are effectively in renewing

or rebinding state, based on the T1/T2 timers for each IA, and many

addresses that have expired.

However, because all of the leases are related to a single server,

and that server either responds to our requests or doesn't, we can

simplify the states to be nearly identical to IPv4 DHCP.

The revised definition for use with DHCPv6 is:

- Transition from BOUND to RENEWING state when the first T1 timer

(of any lease on the state machine) expires. At this point, as

an optimization, we should begin attempting to renew any IAs

that are within REN_TIMEOUT (10 seconds) of reaching T1 as well.

We may as well avoid sending an excess of packets.

- When a T1 lease timer expires and we're in RENEWING or REBINDING

state, just ignore it, because the transaction is already in

progress.

- At each retransmit timeout, we should check to see if there are

more IAs that need to join in because they've passed point T1 as

well, and, if so, add them. This check isn't necessary at this

time, because only a single IA_NA is possible with the initial

design.

- When we reach T2 on any IA and we're in BOUND or RENEWING state,

enter REBINDING state. At this point, we have a choice. For

those other IAs that are past T1 but not yet at T2, we could

ignore them (sending only those that have passed point T2),

continue to send separate Renew messages for them, or just

include them in the Rebind message. This isn't an issue that

must be dealt with for this project, but the plan is to include

them in the Rebind message.

- When a T2 lease timer expires and we're in REBINDING state, just

ignore it, as with the corresponding T1 timer.

- As addresses reach the end of their preferred lifetimes, set the

IFF_DEPRECATED flag. As they reach the end of the valid

lifetime, remove them from the system. When an IA (lease)

becomes empty, just remove it. When there are no more leases

left, return to SELECTING state to start over.

Note that the RFC treats the IAs as separate entities when

discussing the renew/rebind T1/T2 timers, but treats them as a unit

when doing the initial negotiation. This is, to say the least,

confusing, especially so given that there's no reason to expect that

after having failed to elicit any responses at all from the server

on one IA, the server will suddenly start responding when we attempt

to renew some other IA. We rationalize this behavior by using a

single renew/rebind state for the entire state machine (and thus

client/server pair).

There's a subtle timing difference here between DHCPv4 and DHCPv6.

For DHCPv4, the client just sends packets more and more frequently

(shorter timeouts) as the next state gets nearer. DHCPv6 treats

each as a transaction, using the same retransmit logic as for other

messages. The DHCPv6 method is a cleaner design, so we will change

the DHCPv4 implementation to do the same, and compute the new timer

values as part of stop_extending().

Note that it would be possible to start the SELECTING state earlier

than waiting for the last lease to expire, and thus avoid a loss of

connectivity. However, it this point, there are other servers on

the network that have seen us attempting to Rebind for quite some

time, and they have not responded. The likelihood that there's a

server that will ignore Rebind but then suddenly spring into action

on a Solicit message seems low enough that the optimization won't be

done now. (Starting SELECTING state earlier may be done in the

future, if it's found to be useful.)

Persistent State

IPv4 DHCP has only minimal need for persistent state, beyond the

configuration parameters. The state is stored when "ifconfig dhcp

drop" is run or the daemon receives SIGTERM, which is typically done

only well after the system is booted and running.

The daemon stores this state in /etc/dhcp, because it needs to be

available when only the root file system has been mounted.

Moreover, dhcpagent starts very early in the boot process. It runs

as part of svc:/network/physical:default, which runs well before

root is mounted read/write:

svc:/system/filesystem/root:default ->

svc:/system/metainit:default ->

svc:/system/identity:node ->

svc:/network/physical:default

svc:/network/iscsi_initiator:default ->

svc:/network/physical:default

and, of course, well before either /var or /usr is mounted. This

means that any persistent state must be kept in the root file

system, and that if we write before shutdown, we have to cope

gracefully with the root file system returning EROFS on write

attempts.

For DHCPv6, we need to try to keep our stable DUID and IAID values

stable across reboots to fulfill the demands of RFC 3315.

The DUID is either configured or automatically generated. When

configured, it comes from the /etc/default/dhcpagent file, and thus

does not need to be saved by the daemon. If automatically

generated, there's exactly one of these created, and it will

eventually be needed before /usr is mounted, if /usr is mounted over

IPv6. This means a new file in the root file system,

/etc/dhcp/duid, will be used to hold the automatically generated

DUID.

The determination of whether to use a configured DUID or one saved

in a file is made in get_smach_cid(). This function will

encapsulate all of the DUID parsing and generation machinery for the

rest of dhcpagent.

If root is not writable at the point when dhcpagent starts, and our

attempt fails with EROFS, we will set a timer for 60 second

intervals to retry the operation periodically. In the unlikely case

that it just never succeeds or that we're rebooted before root

becomes writable, then the impact will be that the daemon will wake

up once a minute and, ultimately, we'll choose a different DUID on

next start-up, and we'll thus lose our leases across a reboot.

The IAID similarly must be kept stable if at all possible, but

cannot be configured by the user. To do make these values stable,

we will use two strategies. First the IAID value for a given

interface (if not known) will just default to the IP ifIndex value,

provided that there's no known saved IAID using that value. Second,

we will save off the IAID we choose in a single /etc/dhcp/iaid file,

containing an array of entries indexed by logical interface name.

Keeping it in a single file allows us to scan for used and unused

IAID values when necessary.

This mechanism depends on the interface name, and thus will need to

be revisited when Clearview vanity naming and NWAM are available.

Currently, the boot system (GRUB, OBP, the miniroot) does not

support installing over IPv6. This could change in the future, so

one of the goals of the above stability plan is to support that

event.

When running in the miniroot on an x86 system, /etc/dhcp (and the

rest of the root) is mounted on a read-only ramdisk. In this case,

writing to /etc/dhcp will just never work. A possible solution

would be to add a new privileged command in ifconfig that forces

dhcpagent to write to an alternate location. The initial install

process could then do "ifconfig <x> dhcp write /a" to get the needed

state written out to the newly-constructed system root.

This part (the new write option) won't be implemented as part of

this project, because it's not needed yet.

Router Advertisements

IPv6 Router Advertisements perform two functions related to DHCPv6:

- they specify whether and how to run DHCPv6 on a given interface.

- they provide a list of the valid prefixes on an interface.

For the first function, in.ndpd needs to use the same DHCP control

interfaces that ifconfig uses, so that it can launch dhcpagent and

trigger DHCPv6 when necessary. Note that it never needs to shut

down DHCPv6, as router advertisements can't do that.

However, launching dhcpagent presents new problems. As a part of

the "Quagga SMF Modifications" project (PSARC 2006/552), in.ndpd in

Nevada is now privilege-aware and runs with limited privileges,

courtesy of SMF. Dhcpagent, on the other hand, must run with all

privileges.

A simple work-around for this issue is to rip out the "privileges="

clause from the method_credential for in.ndpd. I've taken this

direction initially, but the right longer-term answer seems to be

converting dhcpagent into an SMF service. This is quite a bit more

complex, as it means turning the /sbin/dhcpagent command line

interface into a utility that manipulates the service and passes the

command line options via IPC extensions.

Such a design also begs the question of whether dhcpagent itself

ought to run with reduced privileges. It could, but it still needs

the ability to grant "all" (traditional UNIX root) privileges to the

eventhook script, if present. There seem to be few ways to do this,

though it's a good area for research.

The second function, prefix handling, is also subtle. Unlike IPv4

DHCP, DHCPv6 does not give the netmask or prefix length along with

the leased address. The client is on its own to determine the right

netmask to use. This is where the advertised prefixes come in:

these must be used to finish the interface configuration.

We will have the DHCPv6 client configure each interface with an

all-ones (/128) netmask by default. In.ndpd will be modified so

that when it detects a new IFF_DHCPRUNNING IP logical interface, it

checks for a known matching prefix, and sets the netmask as

necessary. If no matching prefix is known, it will send a new

Router Solicitation message to try to find one.

When in.ndpd learns of a new prefix from a Router Advertisement, it

will scan all of the IFF_DHCPRUNNING IP logical interfaces on the

same physical interface and set the netmasks when necessary.

Dhcpagent, for its part, will ignore the netmask on IPv6 interfaces

when checking for changes that would require it to "abandon" the

interface.

Given the way that DHCPv6 and in.ndpd control both the horizontal

and the vertical in plumbing and removing logical interfaces, and

users do not, it might be worthwhile to consider roping off any

direct user changes to IPv6 logical interfaces under control of

in.ndpd or dhcpagent, and instead force users through a higher-level

interface. This won't be done as part of this project, however.

ARP Hardware Types

There are multiple places within the DHCPv6 client where the mapping

of DLPI MAC type to ARP Hardware Type is required:

- When we are constructing an automatic, stable DUID for our own

identity, we prefer to use a DUID-LLT if possible. This is done

by finding a link-layer interface, opening it, reading the MAC

address and type, and translating in the make_stable_duid()

function in libdhcpagent.

- When we translate a user-configured DUID from

/etc/default/dhcpagent into a binary representation, we may have

to deal with a physical interface name. In this case, we must

open that interface and read the MAC address and type.

- As part of the PIF data structure initialization, we need to read

out the MAC type so that it can be used in the BOOTP/DHCPv4

'htype' field.

Ideally, these would all be provided by a single libdlpi

implementation. However, that project is on-going at this time and

has not yet integrated. For the time being, a dlpi_to_arp()

translation function (taking dl_mac_type and returning an ARP

Hardware Type number) will be placed in libdhcputil.

This temporary function should be removed and this section of the

code updated when the new libdlpi from Clearview integrates.

Field Mappings

Old (all in ifslist) New

next dhcp_smach_t.dsm_next

prev dhcp_smach_t.dsm_prev

if_hold_count dhcp_smach_t.dsm_hold_count

if_ia dhcp_smach_t.dsm_ia

if_async dhcp_smach_t.dsm_async

if_state dhcp_smach_t.dsm_state

if_dflags dhcp_smach_t.dsm_dflags

if_name dhcp_smach_t.dsm_name (see text)

if_index dhcp_pif_t.pif_index

if_max dhcp_lif_t.lif_max and dhcp_pif_t.pif_max

if_min (was unused; removed)

if_opt (was unused; removed)

if_hwaddr dhcp_pif_t.pif_hwaddr

if_hwlen dhcp_pif_t.pif_hwlen

if_hwtype dhcp_pif_t.pif_hwtype

if_cid dhcp_smach_t.dsm_cid

if_cidlen dhcp_smach_t.dsm_cidlen

if_prl dhcp_smach_t.dsm_prl

if_prllen dhcp_smach_t.dsm_prllen

if_daddr dhcp_pif_t.pif_daddr

if_dlen dhcp_pif_t.pif_dlen

if_saplen dhcp_pif_t.pif_saplen

if_sap_before dhcp_pif_t.pif_sap_before

if_dlpi_fd dhcp_pif_t.pif_dlpi_fd

if_sock_fd v4_sock_fd and v6_sock_fd (globals)

if_sock_ip_fd dhcp_lif_t.lif_sock_ip_fd

if_timer (see text)

if_t1 dhcp_lease_t.dl_t1

if_t2 dhcp_lease_t.dl_t2

if_lease dhcp_lif_t.lif_expire

if_nrouters dhcp_smach_t.dsm_nrouters

if_routers dhcp_smach_t.dsm_routers

if_server dhcp_smach_t.dsm_server

if_addr dhcp_lif_t.lif_v6addr

if_netmask dhcp_lif_t.lif_v6mask

if_broadcast dhcp_lif_t.lif_v6peer

if_ack dhcp_smach_t.dsm_ack

if_orig_ack dhcp_smach_t.dsm_orig_ack

if_offer_wait dhcp_smach_t.dsm_offer_wait

if_offer_timer dhcp_smach_t.dsm_offer_timer

if_offer_id dhcp_pif_t.pif_dlpi_id

if_acknak_id dhcp_lif_t.lif_acknak_id

if_acknak_bcast_id v4_acknak_bcast_id (global)

if_neg_monosec dhcp_smach_t.dsm_neg_monosec

if_newstart_monosec dhcp_smach_t.dsm_newstart_monosec

if_curstart_monosec dhcp_smach_t.dsm_curstart_monosec

if_disc_secs dhcp_smach_t.dsm_disc_secs

if_reqhost dhcp_smach_t.dsm_reqhost

if_recv_pkt_list dhcp_smach_t.dsm_recv_pkt_list

if_sent dhcp_smach_t.dsm_sent

if_received dhcp_smach_t.dsm_received

if_bad_offers dhcp_smach_t.dsm_bad_offers

if_send_pkt dhcp_smach_t.dsm_send_pkt

if_send_timeout dhcp_smach_t.dsm_send_timeout

if_send_dest dhcp_smach_t.dsm_send_dest

if_send_stop_func dhcp_smach_t.dsm_send_stop_func

if_packet_sent dhcp_smach_t.dsm_packet_sent

if_retrans_timer dhcp_smach_t.dsm_retrans_timer

if_script_fd dhcp_smach_t.dsm_script_fd

if_script_pid dhcp_smach_t.dsm_script_pid

if_script_helper_pid dhcp_smach_t.dsm_script_helper_pid

if_script_event dhcp_smach_t.dsm_script_event

if_script_event_id dhcp_smach_t.dsm_script_event_id

if_callback_msg dhcp_smach_t.dsm_callback_msg

if_script_callback dhcp_smach_t.dsm_script_callback

Notes:

- The dsm_name field currently just points to the lif_name on the

controlling LIF. This may need to be named differently in the

future; perhaps when Zones are supported.

- The timer mechanism will be refactored. Rather than using the

separate if_timer[] array to hold the timer IDs and

if_{t1,t2,lease} to hold the relative timer values, we will

gather this information into a dhcp_timer_t structure:

dt_id timer ID value

dt_start relative start time

New fields not accounted for above:

dhcp_pif_t.pif_next linkage in global list of PIFs

dhcp_pif_t.pif_prev linkage in global list of PIFs

dhcp_pif_t.pif_lifs pointer to list of LIFs on this PIF

dhcp_pif_t.pif_isv6 IPv6 flag

dhcp_pif_t.pif_dlpi_count number of state machines using DLPI

dhcp_pif_t.pif_hold_count reference count

dhcp_pif_t.pif_name name of physical interface

dhcp_lif_t.lif_next linkage in per-PIF list of LIFs

dhcp_lif_t.lif_prev linkage in per-PIF list of LIFs

dhcp_lif_t.lif_pif backpointer to parent PIF

dhcp_lif_t.lif_smachs pointer to list of state machines

dhcp_lif_t.lif_lease backpointer to lease holding LIF

dhcp_lif_t.lif_flags interface flags (IFF_*)

dhcp_lif_t.lif_hold_count reference count

dhcp_lif_t.lif_dad_wait waiting for DAD resolution flag

dhcp_lif_t.lif_removed removed from list flag

dhcp_lif_t.lif_plumbed plumbed by dhcpagent flag

dhcp_lif_t.lif_expired lease has expired flag

dhcp_lif_t.lif_declined reason to refuse this address (string)

dhcp_lif_t.lif_iaid unique and stable 32-bit identifier

dhcp_lif_t.lif_iaid_id timer for delayed /etc writes

dhcp_lif_t.lif_preferred preferred timer for v6; deprecate after

dhcp_lif_t.lif_name name of logical interface

dhcp_smach_t.dsm_lif controlling (main) LIF

dhcp_smach_t.dsm_leases pointer to list of leases

dhcp_smach_t.dsm_lif_wait number of LIFs waiting on DAD

dhcp_smach_t.dsm_lif_down number of LIFs that have failed

dhcp_smach_t.dsm_using_dlpi currently using DLPI flag

dhcp_smach_t.dsm_send_tcenter v4 central timer value; v6 MRT

dhcp_lease_t.dl_next linkage in per-state-machine list of leases

dhcp_lease_t.dl_prev linkage in per-state-machine list of leases

dhcp_lease_t.dl_smach back pointer to state machine

dhcp_lease_t.dl_lifs pointer to first LIF configured by lease

dhcp_lease_t.dl_nlifs number of configured consecutive LIFs

dhcp_lease_t.dl_hold_count reference counter

dhcp_lease_t.dl_removed removed from list flag

dhcp_lease_t.dl_stale lease was not updated by Renew/Rebind

Snoop

The snoop changes are fairly straightforward. As snoop just decodes

the messages, and the message format is quite different between

DHCPv4 and DHCPv6, a new module will be created to handle DHCPv6

decoding, and will export a interpret_dhcpv6() function.

The one bit of commonality between the two protocols is the use of

ARP Hardware Type numbers, which are found in the underlying BOOTP

message format for DHCPv4 and in the DUID-LL and DUID-LLT

construction for DHCPv6. To simplify this, the existing static

show_htype() function in snoop_dhcp.c will be renamed to arp_htype()

(to better reflect its functionality), updated with more modern

hardware types, moved to snoop_arp.c (where it belongs), and made a

public symbol within snoop.

While I'm there, I'll update snoop_arp.c so that when it prints an

ARP message in verbose mode, it uses arp_htype() to translate the

ar_hrd value.

The snoop updates also involve the addition of a new "dhcp6" keyword

for filtering. As a part of this, CR 6487534 will be fixed.

IPv6 Source Address Selection

One of the customer requests for DHCPv6 is to be able to predict the

address selection behavior in the presence of both stateful and

stateless addresses on the same network.

Solaris implements RFC 3484 address selection behavior. In this

scheme, the first seven rules implement some basic preferences for

addresses, with Rule 8 being a deterministic tie breaker.

Rule 8 relies on a special function, CommonPrefixLen, defined in the

RFC, that compares leading bits of the address without regard to

configured prefix length. As Rule 1 eliminates equal addresses,

this always picks a single address.

This rule, though, allows for additional checks:

Rule 8 may be superseded if the implementation has other means of

choosing among source addresses. For example, if the implementation

somehow knows which source address will result in the "best"

communications performance.

We will thus split Rule 8 into three separate rules:

- First, compare on configured prefix. The interface with the

longest configured prefix length that also matches the candidate

address will be preferred.

- Next, check the type of address. Prefer statically configured

addresses above all others. Next, those from DHCPv6. Next,

stateless autoconfigured addresses. Finally, temporary addresses.

(Note that Rule 7 will take care of temporary address preferences,

so that this rule doesn't actually need to look at them.)

- Finally, run the check-all-bits (CommonPrefixLen) tie breaker.

The result of this is that if there's a local address in the same

configured prefix, then we'll prefer that over other addresses. If

there are multiple to choose from, then will pick static first, then

DHCPv6, then dynamic. Finally, if there are still multiples, we'll

use the "closest" address, bitwise.

Also, this basic implementation scheme also addresses CR 6485164, so

a fix for that will be included with this project.

Minor Improvements

Various small problems with the system encountered during

development will be fixed along with this project. Some of these

are:

- List of ARPHRD_* types is a bit short; add some new ones.

- List of IPPORT_* values is similarly sparse; add others in use by

snoop.

- dhcpmsg.h lacks PRINTFLIKE for dhcpmsg(); add it.

- CR 6482163 causes excessive lint errors with libxnet; will fix.

- libdhcpagent uses gettimeofday() for I/O timing, and this can

drift on systems with NTP. It should use a stable time source

(gethrtime()) instead, and should return better error values.

- Controlling debug mode in the daemon shouldn't require changing

the command line arguments or jumping through special hoops. I've

added undocumented ".DEBUG_LEVEL=[0-3]" and ".VERBOSE=[01]"

features to /etc/default/dhcpagent.

- The various attributes of the IPC commands (requires privileges,

creates a new session, valid with BOOTP, immediate reply) should

be gathered together into one look-up table rather than scattered

as hard-coded tests.

- Remove the event unregistration from the command dispatch loop and

get rid of the ipc_action_pending() botch. We'll get a

zero-length read any time the client goes away, and that will be

enough to trigger termination. This fix removes async_pending()

and async_timeout() as well, and fixes CR 6487958 as a

side-effect.

- Throughout the dhcpagent code, there are private implementations

of doubly-linked and singly-linked lists for each data type.

These will all be removed and replaced with insque(3C) and

remque(3C).

Testing

The implementation was tested using the TAHI test suite for DHCPv6

(www.tahi.org). There are some peculiar aspects to this test suite,

and these issues directed some of the design. In particular:

- If Renew/Rebind doesn't mention one of our leases, then we need to

allow the message to be retransmitted. Real servers are unlikely

to do this.

- We must look for a status code within IAADDR and within IA_NA, and

handle the paradoxical case of "NoAddrAvail." That doesn't make

sense, as a server with no addresses wouldn't use those options.

That option makes more sense at the top level of the message.

- If we get "UseMulticast" when we were already using multicast,

then ignore the error code. Sending another request would cause a

loop.

- TAHI uses "NoBinding" at the top level of the message. This

status code only makes sense within an IA, as it refers to the

GUID:IAID binding, which doesn't exist outside an IA. We must

ignore such errors -- treat them as success.

Interactions With Other Projects

Clearview UV (vanity naming) will cause link names, and thus IP

interface names, to become changeable over time. This will break

the IAID stability mechanism if UV is used for arbitrary renaming,

rather than as just a DR enhancement.

When this portion of Clearview integrates, this part of the DHCPv6

design may need to be revisited. (The solution will likely be

handled at some higher layer, such as within Network Automagic.)

Clearview is also contributing a new libdlpi that will work for

dhcpagent, and is thus removing the private dlpi_io.[ch] functions

from this daemon. When that Clearview project integrates, the

DHCPv6 project will need to adjust to the new interfaces, and remove

or relocate the dlpi_to_arp() function.

Futures

Zones currently cannot address any IP interfaces by way of DHCP.

This project will not fix that problem, but the DUID/IAID could be

used to help fix it in the future.

In particular, the DUID allows the client to obtain separate sets of

addresses and configuration parameters on a single interface, just

like an IPv4 Client ID, but it includes a clean mechanism for vendor

extensions. If we associate the DUID with the zone identifier or

name through an extension, then we have a really simple way of

allocating per-zone addresses.

Moreover, RFC 4361 describes a handy way of using DHCPv6 DUID/IAID

values with IPv4 DHCP, which would quickly solve the problem of

using DHCP for IPv4 address assignment in non-global zones as well.

(One potential risk with this plan is that there may be server

implementations that either do not implement the RFC correctly or

otherwise mishandle the DUID. This has apparently bitten some early

adopters.)

Implementing the FQDN option for DHCPv6 would, given the current

libdhcputil design, require a new 'type' of entry for the inittab6

file. This is because the design does not allow for any simple

means to ``compose'' a sequence of basic types together. Thus,

every type of option must either be a basic type, or an array of

multiple instances of the same basic type.

If we implement FQDN in the future, it may be useful to explore some

means of allowing a given option instance to be a sequence of basic

types.

This project does not make the DNS resolver or any other subsystem

use the data gathered by DHCPv6. It just makes the data available

through dhcpinfo(1). Future projects should modify those services

to use configuration data learned via DHCPv6. (One of the reasons

this is not being done now is that Network Automagic [NWAM] will

likely be changing this area substantially in the very near future,

and thus the effort would be largely wasted.)

Appendix A - Choice of Venue

There are three logical places to implement DHCPv6:

- in dhcpagent

- in in.ndpd

- in a new daemon (say, 'dhcp6agent')

We need to access parameters via dhcpinfo, and should provide the

same set of status and control features via ifconfig as are present

for IPv4. (For the latter, if we fail to do that, it will likely

confuse users. The expense for doing it is comparatively small, and

it will be useful for testing, even though it should not be needed

in normal operation.)

If we implement somewhere other than dhcpagent, then we need to give

that new daemon (in.ndpd or dhcp6agent) the same basic IPC features

as dhcpagent already has. This means either extracting those bits

(async.c and ipc_action.c) into a shared library or just copying

them. Obviously, the former would be preferred, but as those bits

depend on the rest of the dhcpagent infrastructure for timers and

state handling, this means that the new process would have to look a

lot like dhcpagent.

Implementing DHCPv6 as part of in.ndpd is attractive, as it

eliminates the confusion that the router discovery process for

determining interface netmasks can cause, along with the need to do

any signaling at all to bring DHCPv6 up. However, the need to make

in.ndpd more like dhcpagent is unattractive.

Having a new dhcp6agent daemon seems to have little to recommend it,

other than leaving the existing dhcpagent code untouched. If we do

that, then we end up with two implementations that do many similar

things, and must be maintained in parallel.

Thus, although it leads to some complexity in reworking the data

structures to fit both protocols, on balance the simplest solution

is to extend dhcpagent.

Appendix B - Cross-Reference

in.ndpd

- Start dhcpagent and issue "dhcp start" command via libdhcpagent

- Parse StatefulAddrConf interface option from ndpd.conf

- Watch for M and O bits to trigger DHCPv6

- Handle "no routers found" case and start DHCPv6

- Track prefixes and set prefix length on IFF_DHCPRUNNING aliases

- Send new Router Solicitation when prefix unknown

- Change privileges so that dhcpagent can be launched successfully

libdhcputil

- Parse new /etc/dhcp/inittab6 file

- Handle new UNUMBER24, SNUMBER64, IPV6, DUID and DOMAIN types

- Add DHCPv6 option iterators (dhcpv6_find_option and

dhcpv6_pkt_option)

- Add dlpi_to_arp function (temporary)

libdhcpagent

- Add stable DUID and IAID creation and storage support

functions and add new dhcp_stable.h include file

- Support new DECLINING and RELEASING states introduced by DHCPv6.

- Update implementation so that it doesn't rely on gettimeofday()

for I/O timeouts

- Extend the hostconf functions to support DHCPv6, using a new

".dh6" file

snoop

- Add support for DHCPv6 packet decoding (all types)

- Add "dhcp6" filter keyword

- Fix known bugs in DHCP filtering

ifconfig

- Remove inet-only restriction on "dhcp" keyword

netstat

- Remove strange "-I list" feature.

- Add support for DHCPv6 and iterating over IPv6 interfaces.

ip

- Add extensions to IPv6 source address selection to prefer DHCPv6

addresses when all else is equal

- Fix known bugs in source address selection (remaining from TX

integration)

other

- Add ifindex and source/destination address into PKT_LIST.

- Add more ARPHDR_* and IPPORT_* values.