lib: istream: Allow switching to a specific ioloop.

lib: i_stream_read_more() - assert it can't return -2

lib: Add i_stream_get_last_read_time()

global: Fix spelling mistakes in comments Original work by @andreasschulze and @jsoref

lib: i_stream_read() - Use snapshots This guarantees that data returned by i_stream_get_data() won't be freed until the next i_stream_read() returns >0. This is mainly important, because often the error handling for <=0 doesn't update the buffer pointers correctly and it leads to complicated bugs where already freed memory is attempted to be read (but never written).

global: Rename i_stream_is_eof() to i_stream_read_eof() This describes its behavior a bit better.

lib: Improve i_stream_is_eof() comment

lib: Update struct istream comments

lib: Add i_stream_set_blocking()

lib: Implemented i_stream_create_copy_from_data(). Unlike i_stream_create_from_data(), this function makes a copy of the provided data block. This way, the application does not need to worry about keeping it allocated for the lifetime of the stream. The copied data is allocated durably on the system pool and freed once the stream is destroyed.

lib: return non-NULL pointer in i_stream_get_data when size is 0

lib: Updated istream's stream_errno comments.

global: unsigned int:1 -> bool:1 perl -i -pe 's/unsigned int ([^,:;]+):1;/bool $1:1;/' **/*.[ch]

lib: remove autoclose parameter from [io]_stream_create_fd Use [io]_stream_create_fd_autoclose() for autoclose.

lib: i_stream_get_max_buffer_size() checks also parents' max sizes This fixes i_stream_get_max_buffer_size() to work correctly with istream-chain.

lib: istream - more migration to i_stream_read_bytes() This time a coccinelle semantic patch, again hand checked: git grep 'i_stream_read_data' | sed s/:.*// | \ while read f; do spatch --sp-file istream.cocci --in-place "$f" ; done -- 8< --- istream.cocci --- @@ expression e1, e2, e3, e4; @@ - i_stream_read_data(e1, e2, e3, e4 - 1) + i_stream_read_bytes(e1, e2, e3, e4) -- 8< --- end Signed-off-by: Phil Carmody <phil@dovecot.fi>

lib: istream - migrate to i_stream_read_bytes() Scripted to pick off the low hanging fruit, one liners with an explicit '-1' in the final parameter, but hand-checked. git grep 'i_stream_read_data.*, [^,]*- \?1)' | sed s/:.*// | \ xargs sed -i -e 's/i_stream_read_data(\(.*\), \(.*\), \(.*\), \(.*[^ ]\) \?- \?1)/i_stream_read_bytes(\1, \2, \3, \4)/' Signed-off-by: Phil Carmody <phil@dovecot.fi>

lib: istream - provide alternatives to i_stream_read_data() Providing as a parameter the number of bytes that you would be disappointed to receive was confusing compared to providing the number that you would be happy to receive. This lets us get rid of all the '-1's in the callers. The callers which used 0 as a parameter were so common that it's worth providing a separate API just to simplify their demands. Deep down, these are still the same underlying function. Currently we route the new API through the old one, but eventually, the old API can be routed through the new one, so that the friendlier interface has the simplest implementation. Signed-off-by: Phil Carmody <phil@dovecot.fi>

lib: Added i_stream_get_disconnect_reason()

lib: Added i_stream_set_persistent_buffers()

lib: i_stream_get_error() now returns "EOF" if stream_errno==0 and eof==TRUE. This can be used to replace a lot of code like: input->stream_errno == 0 ? "EOF" : i_stream_get_error(input) with simply: i_stream_get_error(input)

lib: istream create helpers for common cases Several clients want to create streams from buffer_t and string_t, we may as well make it easy for them Signed-off-by: Phil Carmody <phil@dovecot.fi>

lib: i_stream_get_data() should also reset eof=FALSE if it truncates the output.

lib: Added [io]_stream_create_fd_*autoclose() These make it clearer that the fd parameter will be closed automatically when the stream is closed. Eventually (in v2.3) we'll want to get rid of the autoclose boolean parameter in [io]_stream_create_fd().

Added io_add_istream() and related functionality for combining the ioloop/istream. The idea here is that it's possible to implement asynchronous istreams when there isn't a file descriptor that can be used to reliably receive updates.

lib: Added [io]_stream_create_error_str()

iostream: Added ability to set/get error strings for streams.

istream API change: Added support for multiple destroy callbacks.

iostreams: Added close_parent flag to close() handler and clarified close/destroy APIs. This makes it unambiguous how things work: Unless you explicitly call [io]_stream_close(), the parent streams won't be closed. This is what most (hopefully all!) of the existing code expects. I was wondering a bit if [io]_stream_destroy() should simply have been removed and replaced with [io]_stream_unref() calls, since they would have worked basically everywhere, but there might be some places where it's better to have explicitly closed the stream (and where closing the parent stream doesn't matter).

i_stream_read_next_line() now sets stream_errno=ENOBUFS if input buffer gets full. Previously the caller couldn't easily separate this condition from "more data needed", potentially causing infinite loops.

Added [io]_stream_create_error() for creating streams that always fail reads/writes.

i_stream_stat() API changed. This fixes also one missing error check.

Added i_stream_create_range()

Removed CONTEXT_TYPE_SAFETY macro and reimplemented its functionality better. gcc/clang now gives a compiler error in many places if callback isn't exactly what was expected. It's also now much easier to add more of these checks.

Added i_stream_last_line_crlf()

Added i_stream_get_data_size(). Used it where possible.

Marked functions parameters that are allowed to be NULL. Some APIs were also changed. The non-obvious APIs where NULL parameter was changed to "" are master_service_init() and auth_master_user_list_init(). These checks can currently be enabled only on a patched clang: http://llvm.org/bugs/show_bug.cgi?id=6786

Added i_stream_unset_destroy_callback()

Added i_stream_get_absolute_offset().

Added i_stream_get_max_buffer_size().

Added i_stream_create_file() for creating istream from lazily opened file.

i/ostreams can now have a name (e.g. file path). --HG-- branch : HEAD

Added i_stream_set_init_buffer_size(). --HG-- branch : HEAD

Added struct istream.readable_fd, which is used to determine if sendfile() can be used. --HG-- branch : HEAD

Added i_stream_is_eof(). --HG-- branch : HEAD

Added i_stream_get_size(). Use it instead of i_stream_stat() where possible. --HG-- branch : HEAD

Added i_stream_set_return_partial_line(). --HG-- branch : HEAD

i_stream_read_data() comment update. --HG-- branch : HEAD

Added i_stream_add_data(). --HG-- branch : HEAD

Added more consts, ATTR_CONSTs and ATTR_PUREs. --HG-- branch : HEAD

Removed v_start_offset parameter from i_stream_create_limit(). We'll always use input->v_offset now. --HG-- branch : HEAD

Added i_stream_set_destroy_callback(). --HG-- branch : HEAD

Renamed struct _[io]stream to struct [io]stream_private. Also removed _ prefix from _[io]stream_*() functions. --HG-- branch : HEAD

Changed .h ifdef/defines to use <NAME>_H format. --HG-- branch : HEAD

Renamed i_stream_create_file() to i_stream_create_fd(). --HG-- branch : HEAD

Removed memory pool parameter from iostreams. Default pool was almost always used, and the stream usually required destroying anyway so it didn't even make freeing memory easier. --HG-- branch : HEAD

Added istream->blocking setting. It's now used to assert-crash early if a blocking stream unexpectedly returns "need more data". --HG-- branch : HEAD

Array API redesigned to work using unions. It now provides type safety without having to enable DEBUG, as long as the compiler supports typeof(). Its API changed a bit. It now allows directly accessing the array contents, although that's not necessarily recommended. Changed existing array usage to be type safe in a bit more places. Removed array_t completely. Also did s/modifyable/modifiable/. --HG-- branch : HEAD

Include <sys/stat.h> before using struct stat, because some systems may use a macro to define struct stat to something else. --HG-- branch : HEAD

Added i_stream_destroy() and o_stream_destroy() and used them instead of *_stream_unref() where possible. Fixes at least one problem with io_remove() being called after socket was closed, which caused problems with epoll. --HG-- branch : HEAD

deinit, unref, destroy, close, free, etc. functions now take a pointer to their data pointer, and set it to NULL. This makes double-frees less likely to cause security holes. --HG-- branch : HEAD

Added "bool" type and changed all ints that were used as booleans to bool. --HG-- branch : HEAD

API description update --HG-- branch : HEAD

Added exact parameter to i_stream_stat() --HG-- branch : HEAD

Added i_stream_seek_mark() and used it --HG-- branch : HEAD

s/occured/occurred/ --HG-- branch : HEAD

Replaced i_stream_get_size() with i_stream_stat(). Added i_stream_sync(). --HG-- branch : HEAD

i_stream_next_line() works now even if the stream buffer can't be directly modified. --HG-- branch : HEAD

Added seekable variable to struct istream. --HG-- branch : HEAD

Added i_stream_have_bytes_left(). --HG-- branch : HEAD

i_stream_read_data(): return 0 if i_stream_read() returned 0. --HG-- branch : HEAD

Removed istream.disconnected, it's too much of a special case and the only place where it's needed will be removed soon. Added istream.eof back, it's more useful. --HG-- branch : HEAD

We never do blocking reads/writes to network anymore. Changed imap and pop3 processes to use a single I/O loop. Not much tested yet, and currently LIST/LSUB may eat too much memory and APPEND eats all CPU. --HG-- branch : HEAD

Renamed istream->eof to istream->disconnected which is what it currently does. --HG-- branch : HEAD

istream rewrite. instead of directly setting any limits to stream, you now have to use i_stream_create_limit() to existing stream. this should make the istreams much easier to create and understand how they work. --HG-- branch : HEAD

Added istream->eof. istream->v_size is now set to 0 with files. --HG-- branch : HEAD

Added i_stream_read_next_line() --HG-- branch : HEAD

o_stream_send_istream() is now safe to use for moving data within file. --HG-- branch : HEAD

More type safety to i/o streams --HG-- branch : HEAD

Naming change for function typedefs. --HG-- branch : HEAD

Naming style changes, finally got tired of most of the typedefs. Also the previous enum -> macro change reverted so that we don't use the highest bit anymore, that's incompatible with old indexes so they will be rebuilt. --HG-- branch : HEAD

Use unsigned char* when accessing non-NUL terminating strings. Compiler warnings would then notify about accidentally passing them to functions which require them NUL-terminated. Changed a few functions to use void* to avoid unneeded casting. --HG-- branch : HEAD

Renamed IBuffer and OBuffer to IStream and OStream which describes their functionality better. I tried to keep the variable names and comments also sensible. --HG-- branch : HEAD