2N/A * The contents of this file are subject to the terms of the 2N/A * Common Development and Distribution License (the "License"). 2N/A * You may not use this file except in compliance with the License. 2N/A * See the License for the specific language governing permissions 2N/A * and limitations under the License. 2N/A * When distributing Covered Code, include this CDDL HEADER in each 2N/A * If applicable, add the following below this CDDL HEADER, with the 2N/A * fields enclosed by brackets "[]" replaced with your own identifying 2N/A * information: Portions Copyright [yyyy] [name of copyright owner] 2N/A * Copyright (c) 1996, 2010, Oracle and/or its affiliates. All rights reserved. 2N/A/* defines for timedwait in __aio_waitn() and __aio_suspend() */ 2N/A/* value in aio_returned to indicate that aio_return() has *not* been called */ 2N/A * __lio_listio() cancellation handler. 2N/A * If LIO_WAIT, or notification required, allocate a list head. 2N/A }
else {
/* SIGEV_SIGNAL */ 2N/A * submit an AIO request with flags AIO_NO_KAIO 2N/A * to avoid the kaio() syscall in _aio_rw() 2N/A * call kaio(AIOLIOWAIT) to get all outstanding 2N/A * kernel AIO requests 2N/A * __aio_suspend() cancellation handler. 2N/A (*
counter)--;
/* _aio_kernel_suspend or _aio_suscv_cnt */ 2N/A int kerr;
/* error code from _kaio(AIOSUSPEND) */ 2N/A aio_panic(
"__aio_suspend: largefile set when _LP64 defined");
2N/A /* Initialize start time if time monitoring desired */ 2N/A /* content of timeout = 0 : polling */ 2N/A /* timeout pointer = NULL : wait indefinitely */ 2N/A * The next "if -case" is required to accelerate the 2N/A * access to completed RAW-IO requests. 2N/A /* Only kernel requests pending */ 2N/A * _aio_kernel_suspend is used to detect completed non RAW-IO 2N/A * As long as this thread resides in the kernel (_kaio) further 2N/A * asynchronous non RAW-IO requests could be submitted. 2N/A * Always do the kaio() call without using the KAIO_SUPPORTED() 2N/A * checks because it is not mandatory to have a valid fd 2N/A * set in the list entries, only the resultp must be set. 2N/A * _kaio(AIOSUSPEND ...) return values : 2N/A * 0: everythink ok, completed request found 2N/A * 1: no error : _aiodone awaked the _kaio(AIOSUSPEND,,) 2N/A * system call using _kaio(AIONOTIFY). It means, that some 2N/A * non RAW-IOs completed inbetween. 2N/A kerr =
1;
/* simulation: _kaio detected AIONOTIFY */ 2N/A * Return kernel error code if no other IOs are outstanding. 2N/A /* no IOs outstanding in the thread pool */ 2N/A /* return "no IOs completed" */ 2N/A * IOs using the thread pool are outstanding. 2N/A /* time monitoring */ 2N/A /* first scan file system requests */ 2N/A * If there aren't outstanding I/Os in the thread pool then 2N/A * we have to return here, provided that all kernel RAW-IOs 2N/A * If the kernel was notified to return, then we have to check 2N/A * possible pending RAW-IOs. 2N/A * There are outstanding IOs in the thread pool or the kernel 2N/A * was notified to return. 2N/A * Check pending RAW-IOs first. 2N/A * _aiodone just notified the kernel about 2N/A * completed non RAW-IOs (AIONOTIFY was detected). 2N/A /* Update remaining timeout for the kernel */ 2N/A /* Update remaining timeout */ 2N/A /* wait indefinitely */ 2N/A /* this decrements _aio_suscv_cnt and drops __aio_mutex */ 2N/A * Always do the kaio() call without using the 2N/A * KAIO_SUPPORTED() checks because it is not 2N/A * mandatory to have a valid fd set in the 2N/A * aiocb, only the resultp must be set. 2N/A /* aio_return() was called or aiocb is uninitialized */ 2N/A * The _aiodone() function stores resultp->aio_return before 2N/A * storing resultp->aio_errno (with an membar_producer() in 2N/A * between). We use membar_consumer() below to ensure proper 2N/A * memory ordering between _aiodone() and ourself. 2N/A * Before we return, mark the result as being returned so that later 2N/A * calls to aio_return() or aio_error() will return the fact that 2N/A * the result has already been returned. 2N/A /* aio_return() already called or aiocb is uninitialized */ 2N/A * This function returns the number of asynchronous I/O requests submitted. 2N/A * Kernel aio_fsync() is not supported. 2N/A * We force user-level aio_fsync() just 2N/A * for the notification side-effect. 2N/A * The first asynchronous I/O request in the current process will 2N/A * create a bunch of workers (via __uaio_init()). If the number 2N/A * of workers is zero then the number of pending asynchronous I/O 2N/A * requests is zero. In such a case only execute the standard 2N/A * fsync(3C) or fdatasync(3RT) as appropriate. 2N/A * re-use aio_offset as the op field. 2N/A * O_DSYNC - fdatasync() 2N/A * Create a list of fsync requests. The worker that 2N/A * gets the last request will do the fsync request. 2N/A * Insert an fsync request on every worker's queue. 2N/A * Fewer fsync requests than workers means that it was 2N/A * not possible to submit fsync requests to all workers. 2N/A * a) number of fsync requests submitted is 0: 2N/A * => free allocated memory (aio_lio_t). 2N/A * b) number of fsync requests submitted is > 0: 2N/A * => the last worker executing the fsync request 2N/A * will free the aio_lio_t struct. 2N/A * __aio_waitn() cancellation handler. 2N/A /* check for pending aio_waitn() calls */ 2N/A * aio_waitn can be used to reap the results of several I/O operations that 2N/A * were submitted asynchronously. The submission of I/Os can be done using 2N/A * existing POSIX interfaces: lio_listio, aio_write or aio_read. 2N/A * aio_waitn waits until "nwait" I/Os (supplied as a parameter) have 2N/A * completed and it returns the descriptors for these I/Os in "list". The 2N/A * maximum size of this list is given by "nent" and the actual number of I/Os 2N/A * completed is returned in "nwait". Otherwise aio_waitn might also 2N/A * return if the timeout expires. Additionally, aio_waitn returns 0 if 2N/A * successful or -1 if an error occurred. 2N/A int kerrno = 0;
/* save errno from _kaio() call */ 2N/A * Only one running aio_waitn call per process allowed. 2N/A * Further calls will be blocked here until the running 2N/A * If both counters are still set to zero, then only 2N/A * kernel requests are currently outstanding (raw-I/Os). 2N/A /* File system I/Os outstanding ... */ 2N/A * Calculate sum of active non RAW-IO requests (sum_reqs). 2N/A * If the expected amount of completed requests (*nwait) is 2N/A * greater than the calculated sum (sum_reqs) then 2N/A * use _kaio to check pending RAW-IO requests. 2N/A /* possibly some kernel I/Os outstanding */ 2N/A /* don't wait for kernel I/Os */ 2N/A /* just scan for completed LIB I/Os */ 2N/A break;
/* fatal kernel error */ 2N/A /* check completed FS requests in the "done" queue */ 2N/A /* get done requests */ 2N/A /* min. requested amount of completed I/Os satisfied */ 2N/A * If some I/Os are outstanding and we have to wait for them, 2N/A * then sleep here. _aiodone() will call _aio_waitn_wakeup() 2N/A * to wakeup this thread as soon as the required amount of 2N/A * _aio_waitn_wakeup() will wake up this thread when: 2N/A * - _aio_waitncnt requests are completed or 2N/A * - _aio_outstand_cnt becomes zero. 2N/A * sig_cond_reltimedwait() could also return with 2N/A * a timeout error (ETIME). 2N/A /* polling or timer expired */ 2N/A * __aio_waitn() sets AIO_IO_WAITING to notify _aiodone() that 2N/A * it is waiting for completed I/Os. The number of required 2N/A * completed I/Os is stored into "_aio_waitncnt". 2N/A * aio_waitn() is woken up when 2N/A * - there are no further outstanding I/Os 2N/A * (_aio_outstand_cnt == 0) or 2N/A * - the expected number of I/Os has completed. 2N/A * Only one __aio_waitn() function waits for completed I/Os at 2N/A * __aio_suspend() increments "_aio_suscv_cnt" to notify 2N/A * _aiodone() that at least one __aio_suspend() call is 2N/A * There could be more than one __aio_suspend() function 2N/A * waiting for completed I/Os. Because every function should 2N/A * be waiting for different I/Os, _aiodone() has to wake up all 2N/A * __aio_suspend() functions each time. 2N/A * Every __aio_suspend() function will compare the recently 2N/A * completed I/O with its own list. 2N/A /* Wake up waiting aio_suspend calls */ 2N/A * timedwait values : 2N/A * AIO_TIMEOUT_POLL : polling 2N/A * AIO_TIMEOUT_WAIT : timeout 2N/A * AIO_TIMEOUT_INDEF : wait indefinitely 2N/A * If LIO_WAIT, or notification required, allocate a list head. 2N/A }
else {
/* SIGEV_SIGNAL */ 2N/A * submit an AIO request with flags AIO_NO_KAIO 2N/A * to avoid the kaio() syscall in _aio_rw() 2N/A * call kaio(AIOLIOWAIT) to get all outstanding 2N/A * kernel AIO requests 2N/A * Always do the kaio() call without using the 2N/A * KAIO_SUPPORTED() checks because it is not 2N/A * mandatory to have a valid fd set in the 2N/A * aiocb, only the resultp must be set. 2N/A /* aio_return() was called or aiocb is uninitialized */ 2N/A * The _aiodone() function stores resultp->aio_return before 2N/A * storing resultp->aio_errno (with an membar_producer() in 2N/A * between). We use membar_consumer() below to ensure proper 2N/A * memory ordering between _aiodone() and ourself. 2N/A * Before we return, mark the result as being returned so that later 2N/A * calls to aio_return() or aio_error() will return the fact that 2N/A * the result has already been returned. 2N/A /* aio_return() already called or aiocb is uninitialized */ 2N/A * Kernel aio_fsync() is not supported. 2N/A * We force user-level aio_fsync() just 2N/A * for the notification side-effect. 2N/A * The first asynchronous I/O request in the current process will 2N/A * create a bunch of workers (via __uaio_init()). If the number 2N/A * of workers is zero then the number of pending asynchronous I/O 2N/A * requests is zero. In such a case only execute the standard 2N/A * fsync(3C) or fdatasync(3RT) as appropriate. 2N/A * re-use aio_offset as the op field. 2N/A * O_DSYNC - fdatasync() 2N/A * Create a list of fsync requests. The worker that 2N/A * gets the last request will do the fsync request. 2N/A * Insert an fsync request on every worker's queue. 2N/A * Fewer fsync requests than workers means that it was 2N/A * not possible to submit fsync requests to all workers. 2N/A * a) number of fsync requests submitted is 0: 2N/A * => free allocated memory (aio_lio_t). 2N/A * b) number of fsync requests submitted is > 0: 2N/A * => the last worker executing the fsync request 2N/A * will free the aio_lio_t struct. 2N/A#
endif /* !defined(_LP64) */