$Id: libidnkit.3.in,v 1.1 2003/06/04 00:27:15 marka Exp $

Copyright (c) 2001,2002 Japan Network Information Center.
All rights reserved.

By using this file, you agree to the terms and conditions set forth bellow.

LICENSE TERMS AND CONDITIONS

The following License Terms and Conditions apply, unless a different
license is obtained from Japan Network Information Center ("JPNIC"),
a Japanese association, Kokusai-Kougyou-Kanda Bldg 6F, 2-3-4 Uchi-Kanda,
Chiyoda-ku, Tokyo 101-0047, Japan.

1. Use, Modification and Redistribution (including distribution of any
modified or derived work) in source and/or binary forms is permitted
under this License Terms and Conditions.

2. Redistribution of source code must retain the copyright notices as they
appear in each source code file, this License Terms and Conditions.

3. Redistribution in binary form must reproduce the Copyright Notice,
this License Terms and Conditions, in the documentation and/or other
materials provided with the distribution. For the purposes of binary
distribution the "Copyright Notice" refers to the following language:
"Copyright (c) 2000-2002 Japan Network Information Center. All rights reserved."

4. The name of JPNIC may not be used to endorse or promote products
derived from this Software without specific prior written approval of
JPNIC.

5. Disclaimer/Limitation of Liability: THIS SOFTWARE IS PROVIDED BY JPNIC
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL JPNIC BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

libidnkit 3 "Mar 11, 2002"

NAME
libidnkit, libidnkitlite - Internationalized Domain Name Handling Libraries
SYNOPSIS
#include <idn/api.h>

idn_result_t
idn_nameinit(int load_file)

idn_result_t
idn_encodename(int actions, const char *from, char *to, size_t tolen)

idn_result_t
idn_decodename(int actions, const char *from, char *to, size_t tolen)

idn_result_t
idn_decodename2(int actions, const char *from, char *to, size_t tolen,
 const char *auxencoding)

idn_result_t
idn_enable(int on_off)

#include <idn/result.h>

char *
idn_result_tostring(idn_result_t result)


OVERVIEW
The libidnkit and libidnkitlite libraries support various manipulations of internationalized domain names, including:

- 2
encoding convesion
- 2
name preparation

They are designed according to IDNA framework where each application must do necessary preparations for the internationalized domain names before passing them to the resolver.

To help applications do the preparation, the libraries provide easy-to-use, high-level interface for the work.

Both libraries provide almost the same API. The difference between them is that libidnkit internally uses iconv function to provide encoding conversion from UTF-8 to the local encoding (such as iso-8859-1, usually determined by the current locale), and vise versa. libidnkitlite is lightweight version of libidnkit. It assumes local encoding is UTF-8 so that it never uses iconv.

This manual describes only a small subset of the API that the libraries provide, most important functions for application programmers. For other API, please refer to the idnkit's specification document (which is not yet available) or the header files typically found under `/usr/local/include/idn/' on your system.

DESCRIPTION

The idn_nameinit function initializes the library. It also sets default configuration if load_file is 0, otherwise it tries to read a configuration file. If idn_nameinit is called more than once, the library initialization will take place only at the first call while the actual configuration procedure will occur at every call.

If there are no errors, idn_nameinit returns idn_success. Otherwise, the returned value indicates the cause of the error. See the section ``RETURN VALUES'' below for the error codes.

Usually you don't have to call this function explicitly because it is implicitly called when idn_encodename or idn_decodename is first called without prior calling of idn_nameinit. In such case, initialization without the configuration file takes place.

idn_encodename function performs name preparation and encoding conversion on the internationalized domain name specified by from, and stores the result to to, whose length is specified by tolen. actions is a bitwise-OR of the following macros, specifying which subprocesses in the encoding process are to be employed.

IDN_LOCALCONV Local encoding to UTF-8 conversion
IDN_DELIMMAP Delimiter mapping
IDN_LOCALMAP Local mapping
IDN_NAMEPREP NAMEPREP mapping, normalization,
 prohibited character check and bidirectional
 string check
IDN_UNASCHECK NAMEPREP unassigned codepoint check
IDN_ASCCHECK ASCII range character check
IDN_IDNCONV UTF-8 to IDN encoding conversion
IDN_LENCHECK Label length check

Details of this encoding process can be found in the section ``NAME ENCODING''.

For convenience, also IDN_ENCODE_QUERY, IDN_ENCODE_APP and IDN_ENCODE_STORED macros are provided. IDN_ENCODE_QUERY is used to encode a ``query string'' (see the IDNA specification). It is equal to

(IDN_LOCALCONV | IDN_DELIMMAP | IDN_LOCALMAP | IDN_NAMEPREP
 | IDN_IDNCONV | IDN_LENCHECK)

if you are using libidnkit, and equal to

(IDN_DELIMMAP | IDN_LOCALMAP | IDN_NAMEPREP | IDN_IDNCONV
 | IDN_LENCHECK)

if you are using libidnkitlite.

IDN_ENCODE_APP is used for ordinary application to encode a domain name. It performs IDN_ASCCHECK in addition with IDN_ENCODE_QUERY. IDN_ENCODE_STORED is used to encode a ``stored string'' (see the IDNA specification). It performs IDN_ENCODE_APP plus IDN_UNASCHECK.

idn_decodename function performs the reverse of idn_encodename. It converts the internationalized domain name given by from, which is represented in a special encoding called ACE, to the application's local codeset and stores into to, whose length is specified by tolen. As in idn_encodename, actions is a bitwise-OR of the following macros.

IDN_DELIMMAP Delimiter mapping
IDN_NAMEPREP NAMEPREP mapping, normalization,
 prohibited character check and bidirectional
 string check
IDN_UNASCHECK NAMEPREP unassigned codepoint check
IDN_IDNCONV UTF-8 to IDN encoding conversion
IDN_RTCHECK Round trip check
IDN_ASCCHECK ASCII range character check
IDN_LOCALCONV Local encoding to UTF-8 conversion

Details of this decoding process can be found in the section ``NAME DECODING''.

For convenience, also IDN_DECODE_QUERY, IDN_DECODE_APP and IDN_DECODE_STORED macros are provided. IDN_DECODE_QUERY is used to decode a ``qeury string'' (see the IDNA specification). It is equal to

(IDN_DELIMMAP | IDN_NAMEPREP | IDN_IDNCONV | IDN_RTCHECK
 | IDN_LOCALCONV)

if you are using libidnkit, and equal to

(IDN_DELIMMAP | IDN_NAMEPREP | IDN_IDNCONV | IDN_RTCHECK)

if you are using libidnkitlite.

IDN_DECODE_APP is used for ordinary application to decode a domain name. It performs IDN_ASCCHECK in addition with IDN_DECODE_QUERY. IDN_DECODE_STORED is used to decode a ``stored string'' (see the IDNA specification). It performs IDN_DECODE_APP plus IDN_UNASCHECK.

idn_decodename2 function provides the same functionality as idn_decodename except that character encoding of from is supposed to be auxencoding. If IDN encoding is Punycode and auxencoding is ISO 8859-2 for example, it is assumed that the Punycode string stored in from is written in ISO 8859-2.

In the IDN decode procedure, IDN_NAMEPREP is done before IDN_IDNCONV, and some non-ASCII characters are converted to ASCII characters as the result of IDN_NAMEPREP. Therefore, ACE string given by from may contains those non-ASCII characters. That is the reason docode_name2 exists.

All of the functions above return error code of type idn_result_t. All codes other than idn_success indicates some kind of failure. idn_result_tostring function takes an error code result and returns a pointer to the corresponding message string.

"NAME ENCODING"
Name encoding is a process that transforms the specified internationalized domain name to a certain string suitable for name resolution. For each label in a given domain name, the encoding processor performs:
"(1) Convert to UTF-8 (IDN_LOCALCONV)"
Convert the encoding of the given domain name from application's local encoding (e.g. ISO-8859-1) to UTF-8. Note that libidnkitlite doesn't support this step.
"(2) Delimiter mapping (IDN_DELIMMAP)"
Map domain name delimiters to `.' (U+002E). The recoginzed delimiters are: U+3002 (ideographic full stop), U+FF0E (fullwidth full stop), U+FF61 (halfwidth ideographic full stop).
"(3) Local mapping (IDN_LOCALMAP)"
Apply character mapping whose rule is determined by the TLD of the name.
"(4) NAMEPREP (IDN_NAMEPREP, IDN_UNASCHECK)"
Perform name preparation (NAMEPREP), which is a standard process for name canonicalizaion of internationalized domain names.

NAMEPREP consists of 5 steps: mapping, normalization, prohibited character check, bidirectional text check and unassigned codepoint check. The first four steps are done by IDN_NAMEPREP, and the last step is done by IDN_UNASCHECK.

"(5) ASCII range character check (IDN_ASCCHECK)"
Checks if the domain name contains non-LDH ASCII character (not alpha-numeric or hyphen), or it begins or end with hyphen.
"(6) Convert to ACE (IDN_IDNCONV)"
Convert the NAMEPREPed name to a special encoding designed for representing internationalized domain names.

The encoding is also known as ACE (ASCII Compatible Encoding) since a string in the encoding is just like a traditional ASCII domain name consisting of only letters, numbers and hyphens.

"(7) Label length check (IDN_LENCHECK)"
For each label, check the number of characters in it. It must be in the range 1 to 63.

There are many configuration parameters for this process, such as the ACE or the local mapping rules. These parameters are read from the default idnkit's configuration file, idn.conf. See idn.conf(5) for details.

"NAME DECODING"
Name decoding is a reverse process of the name encoding. It transforms the specified internationalized domain name in a special encoding suitable for name resolution to the normal name string in the application's current codeset. However, name encoding and name decoding are not symmetric.

For each label in a given domain name, the decoding processor performs:

"(1) Delimiter mapping (IDN_DELIMMAP)"
Map domain name delimiters to `.' (U+002E). The recoginzed delimiters are: U+3002 (ideographic full stop), U+FF0E (fullwidth full stop), U+FF61 (halfwidth ideographic full stop).
"(2) NAMEPREP (IDN_NAMEPREP, IDN_UNASCHECK)"
Perform name preparation (NAMEPREP), which is a standard process for name canonicalizaion of internationalized domain names.
"(3) Convert to UTF-8 (IDN_IDNCONV)"
Convert the encoding of the given domain name from ACE to UTF-8.
"(4) Round trip check (IDN_RTCHECK)"
Encode the result of (3) using the ``NAME ENCODING'' scheme, and then compare it with the result of the step (2). If they are different, the check is failed. If IDN_UNASCHECK, IDN_ASCCHECK or both are specified, also they are done in the encoding processes.
"(5) Convert to local encoding"
Convert the result of (3) from UTF-8 to the application's local encoding (e.g. ISO-8859-1). Note that libidnkitlite doesn't support this step.

If prohibited character check, unassigned codepoint check or bidirectional text check at step (2) is failed, or round trip check at step (4) is failed, the original input label is returned.

The configuration parameters for this process, are also read from the configuration file idn.conf.

"IDN_DISABLE"
If the IDN_DISABLE environ variable is defined at run-time, the libraries disable internationalized domain name support, by default. In this case, idn_encodename and idn_decodename don't encode/decode an input name, but instead they simply ouput a copy of the input name as the result of encoding/decoding.

If your application should always enable mulitilingual domain name support regardless of definition of IDN_DISABLE, call

idn_enable(1)

before performing encoding/decoding.

"RETURN VALUES"
Most of the API functions return values of type idn_result_t in order to indicate the status of the call. The following is a complete list of the status codes. Note that some of them are never returned by the functions described in this manual.

15 idn_success Not an error. The call succeeded.

idn_notfound Specified information does not exist.

idn_invalid_encoding The encoding of the specified string is invalid.

idn_invalid_syntax There is a syntax error in the configuration file.

idn_invalid_name The specified name is not valid.

idn_invalid_message The specified DNS message is not valid.

idn_invalid_action The specified action contains invalid flags.

idn_invalid_codepoint The specified Unicode code point value is not valid.

idn_invalid_length The number of characters in an ACE label is not in the range 1 to 63.

idn_buffer_overflow The specified buffer is too small to hold the result.

idn_noentry The specified key does not exist in the hash table.

idn_nomemory Memory allocation using malloc failed.

idn_nofile The specified file could not be opened.

idn_nomapping Some characters do not have the mapping to the target character set.

idn_context_required Context information is required.

idn_prohibited The specified string contains some prohibited characters.

idn_failure Generic error which is not covered by the above codes.

EXAMPLES
To get the address of a internationalized domain name in the application's local codeset, use idn_encodename to convert the name to the format suitable for passing to resolver functions.

idn_result_t r;
char ace_name[256];
struct hostent *hp;

...
r = idn_encodename(IDN_ENCODE_APP, name, ace_name,
 sizeof(ace_name));
if (r != idn_success) {
 fprintf(stderr, "idn_encodename failed: %s\en",
 idn_result_tostring(r));
 exit(1);
}

hp = gethostbyname(ace_name);
...

To decode the internationalized domain name returned from a resolver function, use idn_decodename.

idn_result_t r;
char local_name[256];
struct hostent *hp;

...
hp = gethostbyname(name);
r = idn_decodename(IDN_DECODE_APP, hp->h_name, local_name,
 sizeof(local_name));
if (r != idn_success) {
 fprintf(stderr, "idn_decodename failed: %s\en",
 idn_result_tostring(r));
 exit(1);
}
printf("name: %s\en", local_name);
...

"SEE ALSO"
idn.conf(5)