lib-http: client: Fix request statistics text to properly report send attempts. If the request was first sent in the same ioloop cycle in which the text is generated, the text would claim it was not sent at all yet. With this commit the text now explicitly makes the distinction between request attempts and actual send attempts. The number of attempts is increased at each retry, while the send attempts are increased each time the request is actually being sent to a server.

lib-http: Create http-common.h, which contains global definitions shared by client and server. Currently contains only the default HTTP port definitions that used to be private to the client.

lib-http: Add http_client_init_private() This allows creating a HTTP client without a shared context, in case it's needed for some reason.

lib-http: Change http_client_init() to use the global shared client context This is most likely what is usually wanted.

lib-http: Add http_client_get_global_context()

lib-http: client: Made http_client record the current ioloop it is switched to. This prevents http_client_wait() from switching the client to an ioloop it was never explicitly switched to.

lib-http: client: Add support for event API

lib-http: client: Created basic shared context between clients. In this basic form it only shares default settings.

lib-http: client: Send empty payload (Content-Length: 0) for requests that normally expect a payload. This includes the standard POST and PUT methods. Others need to use the new http_client_request_set_payload_empty() function to force sending an empty payload.

lib-http: Add http_client_request_stats.first_sent_msecs Also rename sent_msecs to last_sent_msecs.

lib-http: client: Implemented means to obtain request statistics.

lib-http: client: Implemented http_client_request_url_str() function that accepts an URL string rather than a pre-parsed URL object. If the provided HTTP URL is invalid, the callback with the error is called some time later from the ioloop. This change also amends the test-http-client-errors test suite with a new test for this new feature.

lib-http: Explicitly define the start of the range of internal response status codes.

lib-http: Improve http_client_request_set_destroy_callback It now allows using non-void* context

lib-http: client: Implemented host name lookup TTL. Host name lookups will now be performed again when the results have expired. Without access to TTL information from DNS lookups, all lookups will use the same default TTL for now.

lib-http: Support per-request timeout and max_attempts

lib-http: client: Implemented no_auto_retry setting that disables all automatic request retries. This currently only applies to requests sent over a connection that is subsequently lost before a response is received. Before, such requests were always implicitly resumbitted for a new connection, without the application knowing about it. By enabling the no_auto_retry client setting, the application is always notified of connection loss through the request's response callback. As a consequence, requests need to be retried explicitly using the http_client_request_try_retry().

lib-http: client: Added settings to configure the connection's socket kernel buffer sizes. This is mainly useful for use in the lib-http test suite.

lib-http: use ssl_iostream_settings in http_client_settings

lib-http: client: Added ability to configure a specific proxy for individual requests. This way, a request can be routed to a specific proxy (or origin server). The destination can also be a unix socket.

lib-http: Include information about number of request attempts and its timing in response reason. Because the reason is usually logged as part of the error string, this causes all of the error messages to include the attempts count and how long the requests took in total. This should make it easier to understand problems in error logs. http_client_request_set_preserve_exact_reason() can be used to disable modifying the reason string. This may also apply to other reason modifications that may be done in the future.

lib-http: client: Added pure attribute to some of the request property functions.

lib-http: client: Cleaned up headers and added some more documentation there.

lib-http: client: Implemented means to set request payload buffer rather than an input stream. This is not purely a convenience function: there have been bugs caused by allocating a data input stream from a datastack buffer. With this function, the buffer is copied to the request pool, so that it is durably allocated while the request exists. This prevents futher mishaps. The server already has an equivalent function for its response object.

lib-http: Clarify http_client_request_*_payload() API and minor change to it The earlier behavior was pretty confusing, and potentially could have caused double-freeing memory in some situations. Now it's clear that req is set to NULL always when the request is finished, regardless of whether it has any references left. Changed http_client_request_finish_payload() to return 0 on success instead of 1. This could have been left alone, but it's unlikely that there is any code outside Dovecot core that calls it and this way is cleaner.

http-client: Added support for using an HTTP proxy running on a unix socket.

lib-http client: Implemented proxy authentication using Basic scheme.

lib-http client: Implemented simple authentication using Basic scheme.

lib-http: client: Implemented maximum for connection backoff time. Without a maximum, the backoff time grows exponentially to enormous values, because it isn't reset until the connection succeeds. This causes recovery from connection failures to become very slow. Current maximum is one minute.

Compiler warning fix

lib-http: client: Added support for absolute request timeout. Requests cannot survive beyond this deadline.

lib-http: client: Added support for delaying requests in milliseconds.

lib-http: client: Implemented support for connection failure backoff.

lib-http: client: Added support for attempting a single IP several times. Also limits the number of attempts when there are many IPs.

lib-http: Added http_client_request_get_method()

lib-http: Added http_client_request_get_target() This can be useful in error message logging without having to duplicate the string.

lib-http: Added http_client_request_remove_header()

http: Implemented delayed requests scheduling. Requests can now be (re)submitted with a delay. The request is not sent until the delay time expires. This facilitates handling the Retry-After header in responses. This can either be performed automatically if the indicated delay is not too long or explicitly by the code using lib-http.

lib-http: Support DNS lookups via the new dns-client API.

lib-http: http-client: Added support for tunneling SSL conntections through proxy.

lib-http: Added support for creating CONNECT tunnels through HTTP.

lib-http: http-client: Implemented proxy support.

lib-http: http-client: Added request error code for broken payload input stream. This error is triggered when reading from the provided payload input stream fails while sending the request. Previously this would yield the same error code as for a failure to write to the connection output.

lib-http: Added support for disabling automatic redirects.

lib-http: Added setting for User-Agent header.

lib-http: Implemented limits on overall HTTP header size, size of individual header fields and the number of fields in the header.

lib-http: http-client: Requests now automatically generate a Date header. The used date value is normally the submission time of the request, but it can be set explicitly.

lib-http: http-client: Fixed request scheduling and connection management.

lib-http: Moved struct http_response* to http-response.h Also http-client.h now #includes only http-response.h

lib-http: Added http_client_get_pending_request_count()

lib-http: Added soft_connect_timeout_msecs setting to connect to multiple IPs in parallel. Based on patch by Stephan Bosch.

lib-http: Added connect and request timeout settings.

lib-http: Added ssl_cert|key|key_password settings to be passed to ssl-iostream. These are used for sending client's SSL certificate.

Added ssl_client_ca_file to specify the CA certs as a file instead of as a dir. This is required for Redhat-based systems where there isn't a CA directory like in Debian/Ubuntu.

lib-ssl-iostream: If handshake callback fails, close the iostreams immediately. This way the callback itself doesn't have to do it. Also fixes errors caused by it, since they didn't close the ostream.

lib-http: Added ssl_ca setting to specify the CA certificate directly.

lib-http: Callback can now request a retry with http_client_request_try_retry() This can be useful for handling "try again" errors from HTTP servers.

lib-http: Allow caller to find out what state a request is in.

lib-http: Minor API change: Added http_client_request_finish_payload()

lib-http: Added http_client_request_set_destroy_callback() This is useful for io_remove()ing the payload stream's fd at the right time.

lib-http: Various bugfixes

lib-http: Added initial HTTP client implementation.