dapl_tavor_hw.h revision 2
283N/A * The contents of this file are subject to the terms of the 283N/A * Common Development and Distribution License (the "License"). 283N/A * You may not use this file except in compliance with the License. 283N/A * See the License for the specific language governing permissions 283N/A * and limitations under the License. 283N/A * When distributing Covered Code, include this CDDL HEADER in each 283N/A * If applicable, add the following below this CDDL HEADER, with the 283N/A * fields enclosed by brackets "[]" replaced with your own identifying 283N/A * information: Portions Copyright [yyyy] [name of copyright owner] 283N/A * Copyright 2008 Sun Microsystems, Inc. All rights reserved. 283N/A * Use is subject to license terms. 459N/A * Contains all the structure definitions and #defines for all Tavor 283N/A * hardware resources and registers. 283N/A * header file used by the tavor device driver. 283N/A * Ownership flags used to define hardware or software ownership for 283N/A * various Tavor resources 283N/A * Tavor Completion Queue Entries (CQE) 283N/A * Each CQE contains enough information for the software to associate the 357N/A * completion with the Work Queue Element (WQE) to which it corresponds. 347N/A * Note: The following structure is not #define'd with both little-endian 347N/A * and big-endian definitions. This is because each CQE's individual 347N/A * fields are not directly accessed except through the macros defined below. 421N/A * The following defines are used for Tavor CQ error handling. Note: For 456N/A * CQEs which correspond to error events, the Tavor device requires some 421N/A * special handling by software. These defines are used to identify and 421N/A * extract the necessary information from each error CQE, including status 347N/A * code (above), doorbell count, and whether a error completion is for a 347N/A * send or receive work request. 436N/A * These are the defines for the Tavor CQ entry types. They are also 347N/A * specified by the Tavor register specification. They indicate what type 426N/A * of work request is completing (for successful completions). Note: The 347N/A * "SND" or "RCV" in each define is used to indicate whether the completion 347N/A * work request was from the Send work queue or the Receive work queue on 347N/A * These are the defines for the Tavor CQ completion statuses. They are 343N/A * specified by the Tavor register specification. * The following macros are used for extracting (and in some cases filling in) * Tavor User Access Region (UAR) * Tavor doorbells are each rung by writing to the doorbell registers that * form a User Access Region (UAR). A doorbell is a write-only hardware * register which enables passing information from software to hardware * with minimum software latency. A write operation from the host software * to these doorbell registers passes information about the HCA resources * and initiates processing of the doorbell data. There are 6 types of * "Send Doorbell" for synchronizing the attachment of a WQE (or a chain * of WQEs) to the send queue. * "RD Send Doorbell" (Same as above, except for RD QPs) is not supported. * "Receive Doorbell" for synchronizing the attachment of a WQE (or a chain * of WQEs) to the receive queue. * "CQ Doorbell" for updating the CQ consumer index and requesting * completion notifications. * "EQ Doorbell" for updating the EQ consumer index, arming interrupt * triggering, and disarming CQ notification requests. * "InfiniBlast" (which would have enabled access to the "InfiniBlast * buffer") is not supported. * Note: The tavor_hw_uar_t below is the container for all of the various * doorbell types. Below we first define several structures which make up * the contents of those doorbell types. * Note also: The following structures are not #define'd with both little- * endian and big-endian definitions. This is because each doorbell type * is not directly accessed except through a single ddi_put64() operation * (see tavor_qp_send_doorbell, tavor_qp_recv_doorbell, tavor_cq_doorbell, /* Max descriptors per Tavor doorbell */ /* Default value for use in NOTIFY_CQ doorbell */ * Tavor Send Work Queue Element (WQE) * A Tavor Send WQE is built of the following segments, each of which is a * multiple of 16 bytes. Note: Each individual WQE may contain only a * subset of these segments described below (according to the operation type * and transport type of the QP). * The first 16 bytes of ever WQE are formed from the "Next/Ctrl" segment. * This segment contains the address of the next WQE to be executed and the * information required in order to allocate the resources to execute the * next WQE. The "Ctrl" part of this segment contains the control * information required to execute the WQE, including the opcode and other * The "Datagram" segment contains address information required in order to * The "Bind" segment contains the parameters required for a Bind Memory * The "Remote Address" segment is present only in RDMA or Atomic WQEs and * specifies remote virtual addresses and RKey, respectively. Length of * the remote access is calculated from the scatter/gather list (for * The "Atomic" segment is present only in Atomic WQEs and specifies * Note: The following structures are not #define'd with both little-endian * and big-endian definitions. This is because their individual fields are * not directly accessed except through macros defined below. * Tavor Receive Work Queue Element (WQE) * Like the Send WQE, the Receive WQE is built of 16-byte segments. The * segment is the "Next/Ctrl" segment (defined below). It is followed by * some number of scatter list entries for the incoming message. * The format of the scatter-gather list entries is also shown below. For * Receive WQEs the "inline_data" field must be cleared (i.e. data segments * cannot contain inline data). * This bit must be set in the next/ctrl field of all Receive WQEs * as a workaround to a Tavor hardware erratum related to having * the first 32-bits in the WQE set to zero. * The tavor_sw_wqe_dbinfo_t structure is used internally by the Tavor * driver to return information (from the tavor_wqe_mlx_build_nextctl() and * tavor_wqe_send_build_nextctl() routines) regarding the type of Tavor * The following macros are used for building each of the individual * segments that can make up a Tavor WQE. Note: We try not to use the * structures (with their associated bitfields) here, instead opting to * build and put 64-bit or 32-bit chunks to the WQEs as appropriate, * primarily because using the bitfields appears to force more read-modify- * TAVOR_WQE_BUILD_REMADDR - Builds Remote Address Segment using * RDMA info from the work request * TAVOR_WQE_BUILD_BIND - Builds the Bind Memory Window * Segment using bind info from the * TAVOR_WQE_LINKNEXT - Links the current WQE to the * TAVOR_WQE_LINKFIRST - Links the first WQE on the current * chain to the previous WQE * The following macro is used to convert WQE address and size into the * "wqeaddrsz" value needed in the tavor_wrid_entry_t (see below). * The following macros are used to calculate pointers to the Send or Receive * WQEs on a given QP, respectively * Maximum header before the data bytes when inlining data. * "Header" includes the link (nextctrl) struct, a remote address struct * (only for RDMA Write, not for Send) and the 32-bit byte count field. #
endif /* _DAPL_TAVOR_HW_H */