5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * CDDL HEADER START
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * The contents of this file are subject to the terms of the
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Common Development and Distribution License (the "License").
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * You may not use this file except in compliance with the License.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * See the License for the specific language governing permissions
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * and limitations under the License.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * When distributing Covered Code, include this CDDL HEADER in each
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * file and include the License file at usr/src/OPENSOLARIS.LICENSE.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * If applicable, add the following below this CDDL HEADER, with the
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * fields enclosed by brackets "[]" replaced with your own identifying
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * information: Portions Copyright [yyyy] [name of copyright owner]
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * CDDL HEADER END
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Copyright 2009 Sun Microsystems, Inc. All rights reserved.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Use is subject to license terms.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Softmac data-path switching:
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * - Fast-path model
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * When the softmac fast-path is used, a dedicated lower-stream
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * will be opened over the legacy device for each IP/ARP (upper-)stream
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * over the softMAC, and all DLPI messages (including control messages
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * and data messages) will be exchanged between the upper-stream and
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * the corresponding lower-stream directly. Therefore, the data
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * demultiplexing, filtering and classification processing will be done
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * by the lower-stream, and the GLDv3 DLS/MAC layer processing will be
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * no longer needed.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * - Slow-path model
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Some GLDv3 features requires the GLDv3 DLS/MAC layer processing to
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * not be bypassed to assure its function correctness. For example,
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * softmac fast-path must be disabled to support GLDv3 VNIC functionality.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * In this case, a shared lower-stream will be opened over the legacy
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * device, which is responsible for implementing the GLDv3 callbacks
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * and passing RAW data messages between the legacy devices and the GLDv3
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * framework.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * By default, the softmac fast-path mode will be used to assure the
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * performance; MAC clients will be able to request to disable the softmac
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * fast-path mode to support certain features, and if that succeeds,
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * the system will fallback to the slow-path softmac data-path model.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * The details of the softmac data fast-path model is stated as below
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * 1. When a stream is opened on a softMAC, the softmac module will takes
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * over the DLPI processing on this stream;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * 2. For IP/ARP streams over a softMAC, softmac data fast-path will be
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * used by default, unless fast-path is disabled by any MAC client
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * explicitly. The softmac module first identifies an IP/ARP stream
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * by seeing whether there is a SIOCSLIFNAME ioctl sent from upstream,
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * if there is one, this stream is either an IP or an ARP stream
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * and will use fast-path potentially;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * 3. When the softmac fast-path is used, an dedicated lower-stream will
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * be setup for each IP/ARP stream (1-1 mapping). From that point on,
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * all control and data messages will be exchanged between the IP/ARP
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * upper-stream and the legacy device through this dedicated
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * lower-stream. As a result, the DLS/MAC layer processing in GLDv3
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * will be skipped, and this greatly improves the performance;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * 4. When the softmac data fast-path is disabled by a MAC client (e.g.,
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * by a VNIC), all the IP/ARP upper streams will try to switch from
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * the fast-path to the slow-path. The dedicated lower-stream will be
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * destroyed, and all the control and data-messages will go through the
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * existing GLDv3 code path and (in the end) the shared lower-stream;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * 5. On the other hand, when the last MAC client cancels its fast-path
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * disable request, all the IP/ARP streams will try to switch back to
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * the fast-path mode;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Step 5 and 6 both rely on the data-path mode switching process
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * described below:
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * 1) To switch the softmac data-path mode (between fast-path and slow-path),
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * softmac will first send a DL_NOTE_REPLUMB DL_NOTIFY_IND message
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * upstream over each IP/ARP streams that needs data-path mode switching;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * 2) When IP receives this DL_NOTE_REPLUMB message, it will bring down
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * all the IP interfaces on the corresponding ill (IP Lower level
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * structure), and bring up those interfaces over again; this will in
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * turn cause the ARP to "replumb" the interface.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * During the replumb process, both IP and ARP will send downstream the
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * necessary DL_DISABMULTI_REQ and DL_UNBIND_REQ messages and cleanup
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * the old state of the underlying softMAC, following with the necessary
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * DL_BIND_REQ and DL_ENABMULTI_REQ messages to setup the new state.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Between the cleanup and re-setup process, IP/ARP will also send down
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * a DL_NOTE_REPLUMB_DONE DL_NOTIFY_CONF messages to the softMAC to
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * indicate the *switching point*;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * 3) When softmac receives the DL_NOTE_REPLUMB_DONE message, it either
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * creates or destroys the dedicated lower-stream (depending on which
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * data-path mode the softMAC switches to), and change the softmac
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * data-path mode. From then on, softmac will process all the succeeding
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * control messages (including the DL_BIND_REQ and DL_ENABMULTI_REQ
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * messages) and data messages based on new data-path mode.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhoustatic list_t softmac_taskq_list; /* List of softmac_upper_t */
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhoustatic int softmac_fastpath_setup(softmac_upper_t *);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhoustatic mac_tx_cookie_t softmac_fastpath_wput_data(softmac_upper_t *, mblk_t *,
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhoustatic void softmac_datapath_switch_done(softmac_upper_t *);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou mutex_init(&softmac_taskq_lock, NULL, MUTEX_DRIVER, NULL);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou cv_init(&softmac_taskq_cv, NULL, CV_DRIVER, NULL);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou list_create(&softmac_taskq_list, sizeof (softmac_upper_t),
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou (void) thread_create(NULL, 0, softmac_taskq_dispatch, NULL, 0,
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Request the softmac_taskq thread to quit and wait for it to be done.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou if (strcmp(next_q->q_qinfo->qi_minfo->mi_idname, "ip") != 0)
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou/* ARGSUSED */
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhousoftmac_capab_perim(softmac_upper_t *sup, void *data, uint_t flags)
79eeb64527c881659e23b60088c32ae3736a7a2cCathy Zhousoftmac_client_tx_notify(softmac_upper_t *sup, mac_tx_notify_t func, void *arg)
79eeb64527c881659e23b60088c32ae3736a7a2cCathy Zhou * Wait for all tx_notify_func call to be done.
79eeb64527c881659e23b60088c32ae3736a7a2cCathy Zhousoftmac_tx_is_flow_blocked(softmac_upper_t *sup, mac_tx_cookie_t cookie)
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhousoftmac_capab_direct(softmac_upper_t *sup, void *data, uint_t flags)
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou sup->su_direct_rxinfo.slr_rx = (softmac_rx_t)direct->di_rx_cf;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou sup->su_direct_rxinfo.slr_arg = direct->di_rx_ch;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou direct->di_tx_df = (uintptr_t)softmac_fastpath_wput_data;
79eeb64527c881659e23b60088c32ae3736a7a2cCathy Zhou direct->di_tx_fctl_df = (uintptr_t)softmac_tx_is_flow_blocked;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou direct->di_tx_cb_df = (uintptr_t)softmac_client_tx_notify;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhousoftmac_dld_capab(softmac_upper_t *sup, uint_t type, void *data, uint_t flags)
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Don't enable direct callback capabilities unless the caller is
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * the IP client. When a module is inserted in a stream (_I_INSERT)
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * the stack initiates capability disable, but due to races, the
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * module insertion may complete before the capability disable
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * completes. So we limit the check to DLD_ENABLE case.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou if ((flags == DLD_ENABLE && type != DLD_CAPAB_PERIM) &&
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhousoftmac_capability_advertise(softmac_upper_t *sup, mblk_t *mp)
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Initially assume no capabilities.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Direct capability negotiation interface between IP and softmac
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Check if checksum offload is supported on this MAC.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou if (softmac->smac_capab_flags & MAC_CAPAB_HCKSUM) {
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Check if zerocopy is supported on this interface.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou if (!(softmac->smac_capab_flags & MAC_CAPAB_NO_ZCOPY)) {
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * If there are no capabilities to advertise or if we
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * can't allocate a response, send a DL_ERROR_ACK.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou sizeof (dl_capability_ack_t) + subsize, 0)) == NULL) {
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou dlerrorack(q, mp, DL_CAPABILITY_REQ, DL_NOTSUPPORTED, 0);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou mp->b_wptr = mp->b_rptr + sizeof (dl_capability_ack_t) + subsize;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou dlap->dl_sub_offset = sizeof (dl_capability_ack_t);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * IP polling interface.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * TCP/IP checksum offload.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou hcksum.hcksum_txflags = softmac->smac_hcksum_txflags;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou dlcapabsetqid(&(zcopy.zerocopy_mid), sup->su_rq);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou bcopy(&zcopy, ptr, sizeof (dl_capab_zerocopy_t));
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou mdt.mdt_hdr_head = softmac->smac_mdt_capab.mdt_hdr_head;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou mdt.mdt_hdr_tail = softmac->smac_mdt_capab.mdt_hdr_tail;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou mdt.mdt_max_pld = softmac->smac_mdt_capab.mdt_max_pld;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou mdt.mdt_span_limit = softmac->smac_mdt_capab.mdt_span_limit;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou ASSERT(ptr == mp->b_rptr + sizeof (dl_capability_ack_t) + subsize);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhousoftmac_capability_req(softmac_upper_t *sup, mblk_t *mp)
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou dl_capability_req_t *dlp = (dl_capability_req_t *)mp->b_rptr;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * This request is overloaded. If there are no requested capabilities
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * then we just want to acknowledge with all the capabilities we
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * support. Otherwise we enable the set of capabilities requested.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou if (!MBLKIN(mp, dlp->dl_sub_offset, dlp->dl_sub_length)) {
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Walk the list of capabilities to be enabled.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou size = sizeof (dl_capability_sub_t) + sp->dl_length;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * TCP/IP checksum offload to hardware.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Copy for alignment.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou bcopy(hcksump, &hcksum, sizeof (dl_capab_hcksum_t));
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou bcopy(&hcksum, hcksump, sizeof (dl_capab_hcksum_t));
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhousoftmac_bind_req(softmac_upper_t *sup, mblk_t *mp)
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Allocate ackmp incase the underlying driver does not ack timely.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou if ((mp1 = allocb(sizeof (dl_error_ack_t), BPRI_HI)) == NULL) {
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou dlerrorack(sup->su_wq, mp, DL_BIND_REQ, DL_SYSERR, ENOMEM);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou err = softmac_output(slp, mp, DL_BIND_REQ, DL_BIND_ACK, &ackmp);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * The driver does not ack timely.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Enable capabilities the underlying driver claims to support.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Check whether this softmac is already marked as exclusively used,
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * e.g., an aggregation is created over it. Fail the BIND_REQ if so.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou dlerrorack(sup->su_wq, ackmp, DL_BIND_REQ, DL_SYSERR, err);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhousoftmac_unbind_req(softmac_upper_t *sup, mblk_t *mp)
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou dlerrorack(sup->su_wq, mp, DL_UNBIND_REQ, DL_OUTSTATE, 0);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Allocate ackmp incase the underlying driver does not ack timely.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou if ((mp1 = allocb(sizeof (dl_error_ack_t), BPRI_HI)) == NULL) {
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou dlerrorack(sup->su_wq, mp, DL_UNBIND_REQ, DL_SYSERR, ENOMEM);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou err = softmac_output(slp, mp, DL_UNBIND_REQ, DL_OK_ACK, &ackmp);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * The driver does not ack timely.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou dlerrorack(sup->su_wq, ackmp, DL_UNBIND_REQ, DL_SYSERR, err);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Process the non-data mblk.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhousoftmac_wput_single_nondata(softmac_upper_t *sup, mblk_t *mp)
bd670b35a010421b6e1a5536c34453a827007c81Erik Nordmark /* FALLTHROUGH */
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou if (((struct iocblk *)(mp->b_rptr))->ioc_cmd != SIOCSLIFNAME)
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Nak the M_IOCTL based on the STREAMS specification.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * This stream is either IP or ARP. See whether
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * we need to setup a dedicated-lower-stream for it.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Setup the fast-path dedicated lower stream if fast-path
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * is expected. Note that no lock is held here, and if
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * smac_expected_mode is changed from SOFTMAC_FASTPATH to
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * SOFTMAC_SLOWPATH, the DL_NOTE_REPLUMB message used for
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * data-path switching would already be queued and will
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * be processed by softmac_wput_single_nondata() later.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou prim = ((union DL_primitives *)mp->b_rptr)->dl_primitive;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou ((dl_notify_ind_t *)mp->b_rptr)->dl_notification !=
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * This DL_NOTE_REPLUMB message is initiated
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * and queued by the softmac itself, when the
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * sup is trying to switching its datapath mode
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * between SOFTMAC_SLOWPATH and SOFTMAC_FASTPATH.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Send this message upstream.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou ((dl_notify_conf_t *)mp->b_rptr)->dl_notification !=
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * This is an indication from IP/ARP that the
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * fastpath->slowpath switch is done.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * No need to hold lock to check su_mode, since su_mode updating only
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * operation is is serialized by softmac_wput_nondata_task().
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Fastpath non-data message processing. Most of non-data messages
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * can be directly passed down to the dedicated-lower-stream, aside
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * from the following M_PROTO/M_PCPROTO messages.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * The worker thread which processes non-data messages. Note we only process
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * one message at one time in order to be able to "flush" the queued message
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * and serialize the processing.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * If the stream is closing, flush all queued messages and inform
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * the stream to be closed.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou sup->su_pending_head = sup->su_pending_tail = NULL;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Kernel thread to handle taskq dispatch failures in softmac_wput_nondata().
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * This thread is started when the softmac module is first loaded.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou CALLB_CPR_INIT(&cprinfo, &softmac_taskq_lock, callb_generic_cpr,
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou "softmac_taskq_dispatch");
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou softmac_wput_nondata_task, sup, TQ_SLEEP) != NULL);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou CALLB_CPR_SAFE_END(&cprinfo, &softmac_taskq_lock);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhousoftmac_wput_nondata(softmac_upper_t *sup, mblk_t *mp)
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * The processing of the message might block. Enqueue the
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * message for later processing.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou if (taskq_dispatch(system_taskq, softmac_wput_nondata_task,
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Setup the dedicated-lower-stream (fast-path) for the IP/ARP upperstream.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Wait for all data messages to be processed so that we can change
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * the su_mode.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Tear down the dedicated-lower-stream (fast-path) for the IP/ARP upperstream.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Wait for all data messages in the dedicated-lower-stream
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * to be processed.
79eeb64527c881659e23b60088c32ae3736a7a2cCathy Zhou * Note that this function is called either when the stream is closed,
79eeb64527c881659e23b60088c32ae3736a7a2cCathy Zhou * or the stream is unbound (fastpath-slowpath-switch). Therefore,
79eeb64527c881659e23b60088c32ae3736a7a2cCathy Zhou * No need to call the tx_notify callback.
79eeb64527c881659e23b60088c32ae3736a7a2cCathy Zhou VERIFY((sup->su_tx_flow_mp = getq(sup->su_wq)) != NULL);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Destroy the dedicated-lower-stream. Note that slp is destroyed
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * when lh is closed.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou (void) ldi_close(sup->su_slp->sl_lh, FREAD|FWRITE, kcred);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhousoftmac_wput_data(softmac_upper_t *sup, mblk_t *mp)
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * No lock is required to access the su_mode field since the data
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * traffic is quiesce by IP when the data-path mode is in the
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * process of switching.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou (void) softmac_fastpath_wput_data(sup, mp, NULL, 0);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhousoftmac_fastpath_wput_data(softmac_upper_t *sup, mblk_t *mp, uintptr_t f_hint,
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * This function is called from IP, only the MAC_DROP_ON_NO_DESC
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * flag can be specified.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Check wether the dedicated-lower-stream is able to handle more
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * messages, and enable the flow-control if it is not.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Note that in order not to introduce any packet reordering, we
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * always send the message down to the dedicated-lower-stream:
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * If the flow-control is already enabled, but we still get
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * the messages from the upper-stream, it means that the upper
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * stream does not respect STREAMS flow-control (e.g., TCP). Simply
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * pass the message down to the lower-stream in that case.
79eeb64527c881659e23b60088c32ae3736a7a2cCathy Zhou * If DLD_CAPAB_DIRECT is enabled, the notify callback will be
79eeb64527c881659e23b60088c32ae3736a7a2cCathy Zhou * called when the flow control can be disabled. Otherwise,
79eeb64527c881659e23b60088c32ae3736a7a2cCathy Zhou * put the tx_flow_mp into the wq to make use of the old
79eeb64527c881659e23b60088c32ae3736a7a2cCathy Zhou * streams flow control.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou ASSERT(softmac->smac_active && (softmac->smac_nactive == 0));
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Disable/reenable fastpath on given softmac. This request could come from a
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * MAC client or directly from administrators.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhousoftmac_datapath_switch(softmac_t *softmac, boolean_t disable, boolean_t admin)
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou if (softmac->smac_fastpath_admin_disabled == disable) {
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou } else if (disable) {
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * The expected mode is different from whatever datapath mode
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * this softmac is expected from last request, enqueue the data-path
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * switch request.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou list_create(&reqlist, sizeof (softmac_switch_req_t),
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou offsetof(softmac_switch_req_t, ssq_req_list_node));
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Allocate all DL_NOTIFY_IND messages and request structures that
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * are required to switch each IP/ARP stream to the expected mode.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou for (sup = list_head(&softmac->smac_sup_list); sup != NULL;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou req = kmem_alloc(sizeof (softmac_switch_req_t), KM_NOSLEEP);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Allocate the DL_NOTE_REPLUMB message.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou if ((mp = allocb(sizeof (dl_notify_ind_t), BPRI_LO)) == NULL) {
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou mp->b_wptr = mp->b_rptr + sizeof (dl_notify_ind_t);
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Note that it is fine if the expected data-path mode is fast-path
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * and some of streams fails to switch. Only return failure if we
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * are expected to switch to the slow-path.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou if (sup != NULL && expected_mode == SOFTMAC_SLOWPATH) {
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Start switching for each IP/ARP stream. The switching operation
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * will eventually succeed and there is no need to wait for it
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * to finish.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou for (sup = list_head(&softmac->smac_sup_list); sup != NULL;
bd670b35a010421b6e1a5536c34453a827007c81Erik Nordmark * Add the switch request to the requests list of the stream.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou softmac->smac_fastpath_admin_disabled = !disable;
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou } else if (disable) {
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou return (softmac_datapath_switch((softmac_t *)arg, B_TRUE, B_FALSE));
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou VERIFY(softmac_datapath_switch((softmac_t *)arg, B_FALSE,
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Cleanup all the switch requests queueed on this stream.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou while ((req = list_head(&sup->su_req_list)) != NULL) {
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * Handle the DL_NOTE_REPLUMB_DONE indication from IP/ARP. Change the upper
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * stream from the fastpath mode to the slowpath mode.
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhousoftmac_datapath_switch_done(softmac_upper_t *sup)
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * It is fine if the expected mode is fast-path and we fail
5d460eafffba936e81c4dd5ebe0f59b238f09121Cathy Zhou * to enable fastpath on this stream.