9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * CDDL HEADER START
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The contents of this file are subject to the terms of the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Common Development and Distribution License (the "License").
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * You may not use this file except in compliance with the License.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * See the License for the specific language governing permissions
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * and limitations under the License.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * When distributing Covered Code, include this CDDL HEADER in each
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * file and include the License file at usr/src/OPENSOLARIS.LICENSE.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * If applicable, add the following below this CDDL HEADER, with the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * fields enclosed by brackets "[]" replaced with your own identifying
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * information: Portions Copyright [yyyy] [name of copyright owner]
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * CDDL HEADER END
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Copyright (c) 2002-2003, Network Appliance, Inc. All rights reserved.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Copyright 2005 Sun Microsystems, Inc. All rights reserved.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Use is subject to license terms.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * PURPOSE: Manage CQE cookie structures
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The DAPL spec requires that all a cookies passed to a posting operation
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * be returned in the operation's corresponding completion.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Implementing this feature is complicated by the user's ability to
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * suppress event generation for specific operations. When these operations
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * complete successfully, the provider does not have an easy way to
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * deallocate resources devoted to storing context data for these operations.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * To support this feature, a pool of memory is allocated up front large
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * enough to hold cookie data for the maximum number of operations possible
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * on an endpoint.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Two pieces of information are maintained to manage cookie allocation:
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * head index : index of next unallocated cookie
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * tail index : index of last unallocated cookie
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Each cookie store its index in this memory pool.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * When an event is received, the index stored in the event's cookie will be
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * used to update the tail. This will implicitly deallocate all of the cookies
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * "between" the old tail and the new tail.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * The implementation relies on the following assumptions:
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * - there can be only 1 thread in dat_ep_post_send(), dat_ep_post_rdma_write(),
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * dat_ep_post_rdma_read(), or dat_rmr_bind() at a time, therefore
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * dapls_cb_get() does not need to be thread safe when manipulating
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * request data structures.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * - there can be only 1 thread in dat_ep_post_recv(), therefore
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * dapls_cb_get() does not need to be thread safe when manipulating
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * receive data structures.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * - there can be only 1 thread generating completions for a given EP's request
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * opeartions, therefore dapls_cb_put() does not need to be thread safe when
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * manipulating request data structures.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * - there can be only 1 thread generating completions for a given EP's receive
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * opeartions therefore dapls_cb_put() does not need to be thread safe when
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * manipulating receive data structures.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * - completions are delivered in order
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * $Id: dapl_cookie.c,v 1.13 2003/06/16 17:53:32 sjs2 Exp $
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Function Prototypes
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Function Definitions
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * dapls_cb_create
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Given a DAPL_COOKIE_BUFFER, allocate and initialize memory for
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * the data structure.
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * buffer pointer to DAPL_COOKIE_BUFFER
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * ep endpoint to associate with cookies
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * size number of elements to allocate & manage
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * DAT_SUCCESS
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * DAT_INSUFFICIENT_RESOURCES
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * allocate one additional entry so that the tail
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * can always point at an empty location
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor /* round up to multiple of 2 */
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor while (size > i) {
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor buffer->pool = dapl_os_alloc(size * sizeof (DAPL_COOKIE));
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor for (i = 0; i < size; i++) {
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * dapls_cb_resize
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Given a DAPL_COOKIE_BUFFER, reallocate a larger buffer and initialize
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * memory for the data structure from an old one
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * curr_buffer pointer to existing DAPL_COOKIE_BUFFER
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * new_size new number of elements to allocate & manage,
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * has to be > current buffer's size
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * new_buffer pointer to the newly allocated cookie buffer
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * DAT_SUCCESS
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * DAT_INVALID_PARAMETER
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * DAT_INSUFFICIENT_RESOURCES
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * create a new cookie buffer, the queue type and queue ptr remain the
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * same as the curr_buffer so use the values from there
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor curr_buffer->pool[0].queue.ptr, curr_buffer->pool[0].queue_type,
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor /* copy all the free cookies to the new buffer */
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor new_buffer->pool[index] = curr_buffer->pool[head];
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * dapls_cb_free
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Free the data structure
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * buffer pointer to DAPL_COOKIE_BUFFER
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * dapls_cb_get
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Remove an entry from the buffer
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * buffer pointer to DAPL_COOKIE_BUFFER
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * cookie_ptr pointer to pointer to cookie
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * DAT_SUCCESS
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * DAT_INVALID_PARAMETER
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * DAT_INSUFFICIENT_RESOURCES
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor new_head = (buffer->head + 1) % buffer->pool_size;
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * dapls_cb_put
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Add entry(s) to the buffer
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * buffer pointer to DAPL_COOKIE_BUFFER
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * cookie pointer to cookie
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * entry entry removed from the ring buffer
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * DAT_SUCCESS
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * DAT_INSUFFICIENT_EMPTY
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * dapls_rmr_cookie_alloc
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Allocate an RMR Bind cookie
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * buffer pointer to DAPL_COOKIE_BUFFER
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * rmr rmr to associate with the cookie
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * user_cookie user's cookie data
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * cookie_ptr pointer to pointer to allocated cookie
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * DAT_SUCCESS
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * DAT_INSUFFICIENT_EMPTY
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor if (DAT_SUCCESS != dapls_cb_get(buffer, &cookie)) {
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor dat_status = DAT_ERROR(DAT_INSUFFICIENT_RESOURCES,
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * dapls_dto_cookie_alloc
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * Allocate a DTO cookie
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * buffer pointer to DAPL_COOKIE_BUFFER
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * type DTO type
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * user_cookie user's cookie data
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * cookie_ptr pointer to pointer to allocated cookie
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * DAT_SUCCESS
9e39c5ba00a55fa05777cc94b148296af305e135Bill Taylor * DAT_INSUFFICIENT_EMPTY