9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * CDDL HEADER START
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The contents of this file are subject to the terms of the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Common Development and Distribution License (the "License").
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * You may not use this file except in compliance with the License.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * See the License for the specific language governing permissions
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * and limitations under the License.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * When distributing Covered Code, include this CDDL HEADER in each
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * file and include the License file at usr/src/OPENSOLARIS.LICENSE.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * If applicable, add the following below this CDDL HEADER, with the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * fields enclosed by brackets "[]" replaced with your own identifying
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * information: Portions Copyright [yyyy] [name of copyright owner]
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * CDDL HEADER END
e9dc6bff6e018821c8c8ac7fe3e3b42e621e93aeRamaswamy Tummala * Copyright 2009 Sun Microsystems, Inc. All rights reserved.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Use is subject to license terms.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Contains all the structure definitions and #defines for all Tavor
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * hardware resources and registers (as defined by the Tavor register
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * specification). Wherever possible, the names in the Tavor spec
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * have been preserved in the structure and field names below.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Offsets into the CMD BAR (BAR 0) for many of the more interesting hardware
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * registers. These registers include the HCR (more below), the Event Cause
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Register (ECR) and its related clear register, the Interrupt Clear register
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * (CLR_INT), and the software reset register (SW_RESET).
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Ownership flags used to define hardware or software ownership for
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * various Tavor resources
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Determines whether or not virtual-to-physical address translation is
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * required. Several of the Tavor hardware structures can be optionally
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * accessed by Tavor without going through the TPT address translation
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * HCA Command Register (HCR)
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The HCR command interface provides privileged access to the HCA in
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * order to query, configure and modify HCA execution. It is the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * primary mechanism through which mailboxes may be posted to Tavor
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * firmware. To use this interface software fills the HCR with pointers
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * to input and output mailboxes. Some commands support immediate
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * parameters, however, and for these commands the HCR will contain the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * input or output parameters. Command execution completion can be
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * detected either by the software polling the HCR or by waiting for a
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * command completion event.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor "QUERY_DEV_LIM" command
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The QUERY_DEV_LIM command returns the device limits and capabilities
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * supported by the Tavor device. This command should be run before
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * running the INIT_HCA command (below) in order to determine the maximum
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * capabilities of the device and which optional features are supported.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor "QUERY_FW" command
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The QUERY_FW command retrieves the firmware revision and the Command
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Interface revision. The command also returns the HCA attached local
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * memory area (DDR) which is used by the firmware. Below we also
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * include some defines which are used to enforce a minimum firmware
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * version check (see tavor_fw_version_check() for more details).
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor "QUERY_DDR" command
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The QUERY_DDR command retrieves information regarding the HCA attached
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * local memory area (DDR). This information includes: the DIMM PCI BAR,
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * the total address space provided by the HCA attached local memory, and
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * some DIMM-specific information. Note: Some of the HCA attached local
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * memory is reserved for use by firmware. This extent of this reserved
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * area can be obtained through the QUERY_FW command (above).
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Below we first define the tavor_hw_queryddr_dimm_t or "Logical DIMM
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Information" structure. Four of these are present in the QUERY_DDR
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor "QUERY_ADAPTER" command
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The QUERY_ADAPTER command retrieves adapter specific parameters. The
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * command also retrieves the PCI(X) interrupt pin routing for each of
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * the INTx# pins supported by the device. This information is used by
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * the driver during interrupt processing in order to clear the appropriate
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * interrupt bit.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor "INIT_HCA" and "QUERY_HCA" commands
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The INIT_HCA command configures all HCA resources in HCA attached local
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * memory and some system relevant information. The same mailbox output
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * format is used by the QUERY_HCA command. All parameters, which are
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * specifically the output of the QUERY_HCA command are marked as
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * "QUERY_HCA only". These parameters are not configurable through the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * INIT_HCA command, but can be retrieved as read-only through the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * QUERY_HCA command.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Below we first define several structures which help make up the whole
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * of the INIT_HCA/QUERY_HCA command. These are:
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * tavor_hw_qp_ee_cq_eq_rdb_t for "QPC/EEC/CQC/EQC/RDB Parameters",
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * tavor_udav_mem_param_t for "Memory Access Parameters for UDAV Table",
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * tavor_multicast_param_t for "Multicast Support Parameters",
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * tavor_tpt_param_t for "Translation and Protection Table Parameters",
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * and tavor_uar_param_t for Tavor "UAR Parameters".
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor uint32_t uar_baseaddr_l :12; /* QUERY_HCA only */
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor uint32_t uar_baseaddr_l :12; /* QUERY_HCA only */
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor uint32_t hca_core_clock :8; /* QUERY_HCA only */
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor uint32_t hca_core_clock :8; /* QUERY_HCA only */
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor "INIT_IB" command
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The INIT_IB command enables the physical layer of a given IB port.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * It provides control over the IB port attributes. The capabilities
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * requested here should not exceed the device limits, as retrieved by
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * the QUERY_DEV_LIM command (above). To query information about the IB
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * port or node, the driver may submit GetPortInfo or GetNodeInfo MADs
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * through the Tavor MAD_IFC command.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor Memory Protection Table (MPT) entries
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The Memory Protection Table (MPT) contains the information associated
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * with all the regions and windows. The MPT table resides in a physically-
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * contiguous area in HCA attached local memory, and the memory key (R_Key
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * or L_Key) is used to calculate the physical address for accessing the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * entries in the table.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The following structure is used in the SW2HW_MPT, QUERY_MPT, and
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * HW2SW_MPT commands.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The SW2HW_MPT command transfers ownership of an MPT entry from software
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * to hardware. The command takes the MPT entry from the input mailbox and
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * stores it in the MPT in the hardware. The command will fail if the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * requested MPT entry is already owned by the hardware or if the MPT index
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * given in the command is inconsistent with the MPT entry memory key.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The QUERY_MPT command retrieves a snapshot of an MPT entry. The command
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * takes the current state of an MPT entry from the hardware and stores it
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * in the output mailbox. The command will fail if the requested MPT entry
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * is already owned by software.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Finally, the HW2SW_MPT command transfers ownership of an MPT entry from
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * the hardware to the software. The command takes the MPT entry from the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * hardware, invalidates it, and stores it in the output mailbox. The
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * command will fail if the requested entry is already owned by software.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The command will also fail if the MPT entry in question is a Memory
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Region which has Memory Windows currently bound to it.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor Memory Translation Table (MTT) entries
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * After accessing the MPT table (above) and validating the access rights
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * to the region/window, Tavor address translation moves to the next step
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * where it translates the virtual address to a physical address. This
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * translation is performed using the Memory Translation Table entries
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * (MTT). Note: The MTT in hardware is organized into segments and each
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * segment contains multiple address translation pages (MTT entries).
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Each memory region (MPT above) points to the first segment in the MTT
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * that corresponds to that region.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor Event Queue Context Table (EQC) entries
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor supports 64 Event Queues, and the status of Event Queues is stored
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * in the Event Queue Context (EQC) table. The EQC table is a physically-
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * contiguous memory structure in the HCA attached local memory. Each EQC
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * table entry contains Event Queue status and information required by
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * the hardware in order to access the event queue.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The following structure is used in the SW2HW_EQ, QUERY_EQ, and HW2SW_EQ
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The SW2HW_EQ command transfers ownership of an EQ context from software
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * to hardware. The command takes the EQC entry from the input mailbox and
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * stores it in the EQC in the hardware. The command will fail if the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * requested EQC entry is already owned by the hardware.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The QUERY_EQ command retrieves a snapshot of an EQC entry. The command
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * stores the snapshot in the output mailbox. The EQC state and its values
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * are not affected by the QUERY_EQ command.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Finally, the HW2SW_EQ command transfers ownership of an EQC entry from
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * the hardware to the software. The command takes the EQC entry from the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * hardware and stores it in the output mailbox. The EQC entry will be
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * invalidated as a result of the command. It is the responsibility of the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * software to unmap all the events, which might have been previously
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * mapped to the EQ, prior to issuing the HW2SW_EQ command.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor Event Queue Entries (EQE)
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Each EQE contains enough information for the software to identify the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * source of the event. The following structures are used to define each
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * of the various kinds of events that the Tavor hardware will generate.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Note: The tavor_hw_eqe_t below is the generic "Event Queue Entry". All
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * other EQEs differ only in the contents of their "event_data" field.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Below we first define several structures which define the contents of
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * the "event_data" fields:
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * tavor_hw_eqe_cq_t for "Completion Queue Events"
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * tavor_hw_eqe_cqerr_t for "Completion Queue Error Events"
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * tavor_hw_eqe_portstate_t for "Port State Change Events"
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * tavor_hw_eqe_cmdcmpl_t for "Command Interface Completion Events"
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * tavor_hw_eqe_qp_evt_t for "Queue Pair Events" such as Path Migration
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Succeeded, Path Migration Failed, Communication Established, Send
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Queue Drained, Local WQ Catastrophic Error, Invalid Request Local
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * WQ Error, and Local Access Violation WQ Error.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * tavor_hw_eqe_operr_t for "Operational and Catastrophic Error Events"
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * such as EQ Overflow, Misbehaved UAR page, Internal Parity Error,
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Uplink bus error, and DDR data error.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * tavor_hw_eqe_pgflt_t for "Not-present Page Fault on WQE or Data
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Buffer Access". (Note: Currently, this event is unsupported).
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Note also: The following structures are not #define'd with both
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * little-endian and big-endian definitions. This is because their
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * individual fields are not directly accessed except through the macros
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * defined below.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The following macros are used for extracting (and in some cases filling in)
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * information from EQEs
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor &((uint32_t *)(eqe))[0]) & TAVOR_EQE_EVTTYPE_MASK) >> \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor &((uint32_t *)(eqe))[0]) & TAVOR_EQE_EVTSUBTYPE_MASK) >> \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor &((uint32_t *)(eqe))[1]) & TAVOR_EQE_CQNUM_MASK) >> \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor &((uint32_t *)(eqe))[1]) & TAVOR_EQE_QPNUM_MASK) >> \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor &((uint32_t *)(eqe))[3]) & TAVOR_EQE_PORTNUM_MASK) >> \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor &((uint32_t *)(eqe))[1]) & TAVOR_EQE_CMDTOKEN_MASK) >> \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor &((uint32_t *)(eqe))[3]) & TAVOR_EQE_CMDSTATUS_MASK) >> \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (ddi_get32((eq)->eq_eqinfo.qa_acchdl, &((uint32_t *)(eqe))[4]))
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (ddi_get32((eq)->eq_eqinfo.qa_acchdl, &((uint32_t *)(eqe))[5]))
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor &((uint32_t *)(eqe))[3]) & TAVOR_EQE_OPERRTYPE_MASK) >> \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (ddi_get32((eq)->eq_eqinfo.qa_acchdl, &((uint32_t *)(eqe))[4]))
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor &((uint32_t *)(eqe))[7]) & TAVOR_EQE_OWNER_MASK) >> \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (ddi_put32((eq)->eq_eqinfo.qa_acchdl, &((uint32_t *)(eqe))[7], \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor Completion Queue Context Table (CQC) entries
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The CQC table is a physically-contiguous memory area residing in HCA
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * attached local memory. Each CQC table entry contains information
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * required by the hardware to access the completion queue to post
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * completions (CQE).
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The following structure is used in the SW2HW_CQ, QUERY_CQ, RESIZE_CQ,
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * and HW2SW_CQ commands.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The SW2HW_CQ command transfers ownership of an CQ context from software
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * to hardware. The command takes the CQC entry from the input mailbox and
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * stores it in the CQC in the hardware. The command will fail if the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * requested CQC entry is already owned by the hardware.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The QUERY_CQ command retrieves a snapshot of a CQC entry. The command
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * stores the snapshot in the output mailbox. The CQC state and its values
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * are not affected by the QUERY_CQ command.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Finally, the HW2SW_CQ command transfers ownership of a CQC entry from
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * the hardware to the software. The command takes the CQC entry from the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * hardware and stores it in the output mailbox. The CQC entry will be
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * invalidated as a result of the command.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor Completion Queue Entries (CQE)
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Each CQE contains enough information for the software to associate the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * completion with the Work Queue Element (WQE) to which it corresponds.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Note: The following structure is not #define'd with both little-endian
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * and big-endian definitions. This is because each CQE's individual
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * fields are not directly accessed except through the macros defined below.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The following macros are used for extracting (and in some cases filling in)
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * information from CQEs
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor &((uint32_t *)(cqe))[0]) & TAVOR_CQE_QPNUM_MASK) >> \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor &((uint32_t *)(cqe))[2]) & TAVOR_CQE_DQPN_MASK) >> \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor &((uint32_t *)(cqe))[3]) & TAVOR_CQE_SL_MASK) >> \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor &((uint32_t *)(cqe))[3]) & TAVOR_CQE_GRH_MASK) >> \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor &((uint32_t *)(cqe))[3]) & TAVOR_CQE_PATHBITS_MASK) >> \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor &((uint32_t *)(cqe))[3]) & TAVOR_CQE_DLID_MASK) >> \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_CQE_IMM_ETH_PKEY_CRED_GET(cq, cqe) \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (ddi_get32((cq)->cq_cqinfo.qa_acchdl, &((uint32_t *)(cqe))[4]))
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_CQE_IMM_ETH_PKEY_CRED_SET(cq, cqe, arg) \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (ddi_put32((cq)->cq_cqinfo.qa_acchdl, &((uint32_t *)(cqe))[4], \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (ddi_get32((cq)->cq_cqinfo.qa_acchdl, &((uint32_t *)(cqe))[5]))
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (ddi_get32((cq)->cq_cqinfo.qa_acchdl, &((uint32_t *)(cqe))[6]))
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (ddi_put32((cq)->cq_cqinfo.qa_acchdl, &((uint32_t *)(cqe))[6], \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor &((uint32_t *)(cqe))[7]) & TAVOR_CQE_OPCODE_MASK) >> \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor &((uint32_t *)(cqe))[7]) & TAVOR_CQE_SENDRECV_MASK) >> \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor &((uint32_t *)(cqe))[7]) & TAVOR_CQE_OWNER_MASK) >> \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor Shared Receive Queue (SRQ) Context Entry Format
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor MOD_STAT_CFG input mailbox structure
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor UD Address Vector (UDAV)
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor UDAV are used in conjunction with Unreliable Datagram (UD) send
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * WQEs. Each UD send message specifies an address vector that denotes its
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * link and (optional) network layer destination address. The IBA verbs
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * interface enables the separation of the address administration from the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * send WQE posting. The verbs consumer must use special verbs to create
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * and modify address handles (which represent hardware address vectors).
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * When posting send WQEs to UD QP, the verbs consumer must supply a
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * valid address handle/UDAV.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_UDAV_MODIFY_MASK0 0xFCFFFFFFFF000000ULL
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_UDAV_MODIFY_MASK1 0xFFC0F80000000000ULL
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor Queue Pair Context Table (QPC) entries
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The QPC table is a physically-contiguous memory area residing in HCA
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * attached local memory. Each QPC entry is accessed for reads and writes
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * by the HCA while executing work requests on the associated QP.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The following structure is used in the RST2INIT_QP, INIT2INIT_QP,
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * INIT2RTR_QP, RTR2RTS_QP, RTS2RTS_QP, SQERR2RTS_QP, TOERR_QP, RTS2SQD_QP,
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * SQD2RTS_QP, TORST_QP, and QUERY_QP commands.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * With the exception of the QUERY_QP command, each of these commands reads
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * from some portion of the QPC in the input mailbox and modified the QPC
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * stored in the hardware. The QUERY_QP command retrieves a snapshot of a
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * QPC entry. The command stores the snapshot in the output mailbox. The
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * QPC state and its values are not affected by the QUERY_QP command.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Below we first define the tavor_hw_addr_path_t or "Tavor Address Path"
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * structure. This structure is used to provide address path information
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * (both primary and secondary) for each QP context. Note: Since this
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * structure is _very_ similar to the tavor_hw_udav_t structure above,
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * we are able to leverage the similarity with filling in and reading from
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * the two types of structures. See tavor_get_addr_path() and
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * tavor_set_addr_path() in tavor_misc.c for more details.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor Multicast Group Member (MCG)
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor MCG are organized in a physically-contiguous memory table (the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Multicast Group Table) in the HCA attached local memory. This table is
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * actually comprised of two consecutive tables: the Multicast Group Hash
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Table (MGHT) and the Additional Multicast Group Members Table (AMGM).
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Each such entry contains an MGID and a list of QPs that are attached to
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * the multicast group. Each such entry may also include an index to an
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Additional Multicast Group Member Table (AMGM) entry. The AMGMs are
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * used to form a linked list of MCG entries that all map to the same hash
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * value. The MCG entry size is configured through the INIT_HCA command.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Note: An MCG actually consists of a single tavor_hw_mcg_t and some
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * number of tavor_hw_mcg_qp_list_t (such that the combined structure is a
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * power-of-2).
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The following structures are used in the READ_MGM and WRITE_MGM commands.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The READ_MGM command reads an MCG entry from the multicast table and
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * returns it in the output mailbox. Note: This operation does not affect
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * the MCG entry state or values.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The WRITE_MGM command retrieves an MCG entry from the input mailbox and
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * stores it in the multicast group table at the index specified in the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * command. Once the command has finished execution, the multicast group
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * table is updated. The old entry contents are lost.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor/* Multicast Group Member - QP List entries */
e9dc6bff6e018821c8c8ac7fe3e3b42e621e93aeRamaswamy Tummala * Structure for getting the peformance counters from the HCA
e9dc6bff6e018821c8c8ac7fe3e3b42e621e93aeRamaswamy Tummala#else /* BIG ENDIAN */
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor User Access Region (UAR)
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor doorbells are each rung by writing to the doorbell registers that
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * form a User Access Region (UAR). A doorbell is a write-only hardware
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * register which enables passing information from software to hardware
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * with minimum software latency. A write operation from the host software
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * to these doorbell registers passes information about the HCA resources
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * and initiates processing of the doorbell data. There are 6 types of
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * doorbells in Tavor.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * "Send Doorbell" for synchronizing the attachment of a WQE (or a chain
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * of WQEs) to the send queue.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * "RD Send Doorbell" (Same as above, except for RD QPs) is not supported.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * "Receive Doorbell" for synchronizing the attachment of a WQE (or a chain
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * of WQEs) to the receive queue.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * "CQ Doorbell" for updating the CQ consumer index and requesting
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * completion notifications.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * "EQ Doorbell" for updating the EQ consumer index, arming interrupt
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * triggering, and disarming CQ notification requests.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * "InfiniBlast" (which would have enabled access to the "InfiniBlast
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * buffer") is not supported.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Note: The tavor_hw_uar_t below is the container for all of the various
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * doorbell types. Below we first define several structures which make up
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * the contents of those doorbell types.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Note also: The following structures are not #define'd with both little-
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * endian and big-endian definitions. This is because each doorbell type
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * is not directly accessed except through a single ddi_put64() operation
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * (see tavor_qp_send_doorbell, tavor_qp_recv_doorbell, tavor_cq_doorbell,
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * or tavor_eq_doorbell)
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor/* Max descriptors per Tavor doorbell */
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor/* Default value for use in NOTIFY_CQ doorbell */
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor uint32_t iblast[256]; /* "InfiniBlast" unsupported */
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor Send Work Queue Element (WQE)
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * A Tavor Send WQE is built of the following segments, each of which is a
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * multiple of 16 bytes. Note: Each individual WQE may contain only a
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * subset of these segments described below (according to the operation type
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * and transport type of the QP).
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The first 16 bytes of ever WQE are formed from the "Next/Ctrl" segment.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * This segment contains the address of the next WQE to be executed and the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * information required in order to allocate the resources to execute the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * next WQE. The "Ctrl" part of this segment contains the control
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * information required to execute the WQE, including the opcode and other
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * control information.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The "Datagram" segment contains address information required in order to
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * form a UD message.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The "Bind" segment contains the parameters required for a Bind Memory
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Window operation.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The "Remote Address" segment is present only in RDMA or Atomic WQEs and
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * specifies remote virtual addresses and RKey, respectively. Length of
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * the remote access is calculated from the scatter/gather list (for
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * RDMA-write/RDMA-read) or set to eight (for Atomic).
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The "Atomic" segment is present only in Atomic WQEs and specifies
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Swap/Add and Compare data.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Note: The following structures are not #define'd with both little-endian
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * and big-endian definitions. This is because their individual fields are
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * not directly accessed except through macros defined below.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_SEND_SIGNALED_MASK 0x0000000800000000ull
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_SEND_SOLICIT_MASK 0x0000000200000000ull
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_SEND_IMMEDIATE_MASK 0x0000000100000000ull
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_SENDHDR_UD_AV_MASK 0xFFFFFFFFFFFFFFE0ull
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_SENDHDR_BIND_ATOM 0x8000000000000000ull
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_SENDHDR_BIND_WR 0x4000000000000000ull
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_SENDHDR_BIND_RD 0x2000000000000000ull
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor "MLX transport" Work Queue Element (WQE)
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The format of the MLX WQE is similar to that of the Send WQE (above)
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * with the following exceptions. MLX WQEs are used for sending MADs on
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * special QPs 0 and 1. Everything following the "Next/Ctrl" header
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * (defined below) consists of scatter-gather list entries. The contents
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * of these SGLs (also defined below) will be put on the wire exactly as
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * they appear in the buffers. In addition, the VCRC and the ICRC of each
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * sent packet can be modified by changing values in the following header
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * or in the payload of the packet itself.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_MLXHDR_VL15_MASK 0x0002000000000000ull
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_MLXHDR_SLR_MASK 0x0001000000000000ull
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_MLXHDR_SIGNALED_MASK 0x0000000800000000ull
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Tavor Receive Work Queue Element (WQE)
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Like the Send WQE, the Receive WQE is built of 16-byte segments. The
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * segment is the "Next/Ctrl" segment (defined below). It is followed by
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * some number of scatter list entries for the incoming message.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The format of the scatter-gather list entries is also shown below. For
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Receive WQEs the "inline_data" field must be cleared (i.e. data segments
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * cannot contain inline data).
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * This bit must be set in the next/ctrl field of all Receive WQEs
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * as a workaround to a Tavor hardware erratum related to having
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * the first 32-bits in the WQE set to zero.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_RCV_WQE_NDA0_WA_MASK 0x0000000100000000ull
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The following defines are used when building descriptors for special QP
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * work requests (i.e. MLX transport WQEs). Note: Because Tavor MLX transport
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * requires the driver to build actual IB packet headers, we use these defines
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * for the most common fields in those headers.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The following macros are used for building each of the individual
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * segments that can make up a Tavor WQE. Note: We try not to use the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * structures (with their associated bitfields) here, instead opting to
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * build and put 64-bit or 32-bit chunks to the WQEs as appropriate,
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * primarily because using the bitfields appears to force more read-modify-
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * write operations.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * TAVOR_WQE_BUILD_UD - Builds Unreliable Datagram Segment
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * TAVOR_WQE_BUILD_REMADDR - Builds Remote Address Segment using
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * RDMA info from the work request
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * TAVOR_WQE_BUILD_RC_ATOMIC_REMADDR - Builds Remote Address Segment
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * for RC Atomic work requests
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * TAVOR_WQE_BUILD_ATOMIC - Builds Atomic Segment using atomic
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * info from the work request
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * TAVOR_WQE_BUILD_BIND - Builds the Bind Memory Window
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Segment using bind info from the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * work request
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * TAVOR_WQE_BUILD_DATA_SEG - Builds the individual Data Segments
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * for Send, Receive, and MLX WQEs
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * TAVOR_WQE_BUILD_INLINE - Builds an "inline" Data Segment
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * (primarily for MLX transport)
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * TAVOR_WQE_BUILD_INLINE_ICRC - Also builds an "inline" Data Segment
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * (but used primarily in the ICRC
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * portion of MLX transport WQEs)
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * TAVOR_WQE_LINKNEXT - Links the current WQE to the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * previous one
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * TAVOR_WQE_LINKFIRST - Links the first WQE on the current
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * chain to the previous WQE
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * TAVOR_WQE_BUILD_MLX_LRH - Builds the inline LRH header for
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * MLX transport MADs
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * TAVOR_WQE_BUILD_MLX_GRH - Builds the inline GRH header for
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * MLX transport MADs
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * TAVOR_WQE_BUILD_MLX_BTH - Builds the inline BTH header for
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * MLX transport MADs
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * TAVOR_WQE_BUILD_MLX_DETH - Builds the inline DETH header for
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * MLX transport MADs
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put64((qp)->qp_wqinfo.qa_acchdl, &tmp[2], 0x0); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put64((qp)->qp_wqinfo.qa_acchdl, &tmp[3], 0x0); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (((uint64_t)((wr)->wr.ud.udwr_dest->ud_dst_qpn & \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_BUILD_REMADDR(qp, ra, wr_rdma) \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_BUILD_RC_ATOMIC_REMADDR(qp, rc, wr) \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (uint64_t)(wr)->wr.rc.rcwr.atomic->atom_rkey << 32); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_BUILD_ATOMIC(qp, at, wr_atom) \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put64((qp)->qp_wqinfo.qa_acchdl, &tmp[0], bn0_tmp); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor TAVOR_WQE_SGL_BYTE_CNT_MASK) << 32) | (sgl)->ds_key)); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put64((qp)->qp_wqinfo.qa_acchdl, &tmp[1], (sgl)->ds_va); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_BUILD_DATA_SEG_SRQ(srq, ds, sgl) \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put64((srq)->srq_wqinfo.qa_acchdl, &tmp[0], \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor TAVOR_WQE_SGL_BYTE_CNT_MASK) << 32) | (sgl)->ds_key)); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put64((srq)->srq_wqinfo.qa_acchdl, &tmp[1], (sgl)->ds_va); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put32((qp)->qp_wqinfo.qa_acchdl, &tmp[0], inline_tmp); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_BUILD_INLINE_ICRC(qp, ds, sz, icrc) \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put32((qp)->qp_wqinfo.qa_acchdl, &tmp[0], inline_tmp); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put32((qp)->qp_wqinfo.qa_acchdl, &tmp[1], icrc); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_LINKNEXT(qp, prev, ctrl, next) \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put64((qp)->qp_wqinfo.qa_acchdl, &((uint64_t *)(prev))[1], \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put64((qp)->qp_wqinfo.qa_acchdl, &((uint64_t *)(prev))[0], \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_LINKNEXT_SRQ(srq, prev, ctrl, next) \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put64((srq)->srq_wqinfo.qa_acchdl, &((uint64_t *)(prev))[1],\
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put64((srq)->srq_wqinfo.qa_acchdl, &((uint64_t *)(prev))[0],\
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put64((qp)->qp_wqinfo.qa_acchdl, &((uint64_t *)(prev))[0], \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_BUILD_MLX_LRH(lrh, qp, udav, pktlen) \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor lrh_tmp = TAVOR_MLX_VL0_LVER | ((udav).sl << 20); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put32((qp)->qp_wqinfo.qa_acchdl, &tmp[0], lrh_tmp); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put32((qp)->qp_wqinfo.qa_acchdl, &tmp[1], lrh_tmp); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Note: The GRH payload length, calculated below, is the overall packet
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * length (in bytes) minus LRH header and GRH headers.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Also note: Filling in the GIDs in the way we do below is helpful because
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * it avoids potential alignment restrictions and/or conflicts.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_BUILD_MLX_GRH(state, grh, qp, udav, pktlen) \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor grh_tmp |= (udav).tclass << TAVOR_MLX_TC_SHIFT; \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put32((qp)->qp_wqinfo.qa_acchdl, &tmp[0], grh_tmp); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor grh_tmp = (((pktlen) << 2) - (sizeof (ib_lrh_hdr_t) + \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put32((qp)->qp_wqinfo.qa_acchdl, &tmp[1], grh_tmp); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor TAVOR_SGID_FROM_INDX_GET((state), (qp)->qp_portnum, \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor bcopy(&(udav).rgid_h, &tmp[6], sizeof (ib_gid_t)); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_WQE_BUILD_MLX_BTH(state, bth, qp, wr) \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor bth_tmp |= (IB_BTH_SOLICITED_EVENT_MASK << 16); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put32((qp)->qp_wqinfo.qa_acchdl, &tmp[0], bth_tmp); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put32((qp)->qp_wqinfo.qa_acchdl, &tmp[2], 0x0); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put32((qp)->qp_wqinfo.qa_acchdl, &tmp[0], 0x0); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put32((qp)->qp_wqinfo.qa_acchdl, &tmp[1], 0x0); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ddi_put32((qp)->qp_wqinfo.qa_acchdl, &tmp[1], 0x1); \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Undocumented:
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The following registers (and the macros to access them) are not defined
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * in the Tavor PRM. But we have high confidence that these offsets are
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * unlikely to change in the lifetime of the Tavor hardware.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_HW_PORTINFO_MASTERSMLID_OFFSET 0x10010
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_HW_PORTINFO_LINKWIDTH_OFFSET 0x1001C
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_HW_PMEG_PORTXMITDATA_OFFSET 0x10120
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_HW_PMEG_PORTRECVDATA_OFFSET 0x10124
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_HW_PMEG_PORTXMITPKTS_OFFSET 0x10128
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_HW_PMEG_PORTRECVPKTS_OFFSET 0x1012C
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_HW_PMEG_PORTXMITDISCARD_OFFSET 0x10134
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_HW_PMEG_PORTXMITWAIT_OFFSET 0x1013C
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_HW_PMEG_PORTRECVREMPHYSERR_OFFSET 0x10144
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_HW_PMEG_PORTXMITCONSTERR_OFFSET 0x10148
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_HW_PMEG_PORTRECVCONSTERR_OFFSET 0x1014C
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_HW_PMEG_SYMBOLERRCNT_OFFSET 0x10150
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_HW_PMEG_LINKERRRECOVERCNT_OFFSET 0x10154
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_HW_PMEG_LINKDOWNEDCNT_OFFSET 0x10154
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_HW_PMEG_EXCESSBUFOVERRUN_OFFSET 0x10164
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_HW_PMEG_LOCALLINKINTERR_OFFSET 0x10164
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (uint32_t *)((uintptr_t)(state)->ts_reg_cmd_baseaddr + \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (uint32_t *)((uintptr_t)(state)->ts_reg_cmd_baseaddr + \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_PORT_MASTERSMLID_GET(state, port) \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (uint32_t *)((uintptr_t)(state)->ts_reg_cmd_baseaddr + \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_PORT_LINKWIDTH_ACTIVE_GET(state, port) \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (uint32_t *)((uintptr_t)(state)->ts_reg_cmd_baseaddr + \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_SGID_FROM_INDX_GET(state, port, sgid_ix, sgid) \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (sgid)->gid_prefix = ddi_get64((state)->ts_reg_cmdhdl, \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (uint64_t *)((uintptr_t)(state)->ts_reg_cmd_baseaddr + \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (sgid)->gid_guid = ddi_get64((state)->ts_reg_cmdhdl, \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (uint64_t *)((uintptr_t)(state)->ts_reg_cmd_baseaddr + \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#define TAVOR_PKEY_FROM_INDX_GET(state, port, pkey_ix) \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor (uint32_t *)((uintptr_t)(state)->ts_reg_cmd_baseaddr + \
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor ((pkey_ix) * TAVOR_HW_PKEYTABLE_PKEY_SIZE))) & 0xFFFF)
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Flash interface:
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Below we have PCI config space space offsets for flash interface
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * access, offsets within Tavor CR space for accessing flash-specific
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * information or settings, masks used for flash settings, and
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * timeout values for flash operations.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor/* Intel Command Set */
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor#endif /* _SYS_IB_ADAPTERS_TAVOR_HW_H */