.fp 5 CW

.de Af

.ds ;G \\*(;G\\f\\$1\\$3\\f\\$2

.if !\\$4 .Af \\$2 \\$1 "\\$4" "\\$5" "\\$6" "\\$7" "\\$8" "\\$9"

..

.de aF

.ie \\$3 .ft \\$1

.el \{\

.ds ;G \&

.nr ;G \\n(.f

.Af "\\$1" "\\$2" "\\$3" "\\$4" "\\$5" "\\$6" "\\$7" "\\$8" "\\$9"

\\*(;G

.ft \\n(;G \}

..

.de L

.aF 5 \\n(.f "\\$1" "\\$2" "\\$3" "\\$4" "\\$5" "\\$6" "\\$7"

..

.de LR

.aF 5 1 "\\$1" "\\$2" "\\$3" "\\$4" "\\$5" "\\$6" "\\$7"

..

.de RL

.aF 1 5 "\\$1" "\\$2" "\\$3" "\\$4" "\\$5" "\\$6" "\\$7"

..

.de EX \" start example

.ta 1i 2i 3i 4i 5i 6i

.PP

.RS

.PD 0

.ft 5

.nf

..

.de EE \" end example

.fi

.ft

.PD

.RE

.PP

..

.TH TOK 3

.SH NAME

tok \- space separated token stream routines

.SH SYNOPSIS

.L "#include <ast.h>"

.sp

.L "void* tokopen(char* string)"

.L "char* tokread(void* tok)"

.L "void tokclose(void* tok)"

.sp

.L "int tokscan(char* string, char** next, const char* format, ...);"

.sp

.L "Sfio_t* tokline(const char* input, int flags, int* line);"

.SH DESCRIPTION

.L tokopen

returns a pointer to a space separated token stream on the 0 terminated

string

.LR string .

.L tokread

returns a pointer to the next

space separated token in the token stream

.L tok

as returned by

.LR tokopen .

0 is returned when no tokens remain.

.L tokread

temporarily modifies

.L string

by inserting 0's to terminate each token.

.L tokclose

closes the token stream and restores

.L string

to its original state.

.PP

.L tokscan

scans the string

.L string

for tokens specified in

.LR format .

It is a more forgiving

.IR sscanf (3).

If

.L "next != 0"

then it will point to the next unread character in

.L string

on return.

The number of scanned tokens is returned.

.L \-1

is returned if

.L string

was not empty and

.L format

failed to match and tokens.

.PP

.I space

in

.L format

matches 0 or more

.I space

or

.I tab

characters.

.I newline

in format eats the remainder of the current line in

.LR string .

"...", '...' and \e\fIcharacter\fP quotes are interpreted.

A quoted

.I carriage-return

is converted to

.IR newline .

.I newline

in

.L string

is equivalent to end of string except when quoted.

.I \enewline

is a line splice.

.PP

.L %

in

.L format

prefixes format conversion characters; each conversion character

corresponds to a

.L tokscan

argument following the

.L format

argument.

The format conversions are:

.TP

.L %c

A single

.LR char .

.TP

.L "%hd %d %ld"

[short, int, long] base 10 integer.

.TP

.L "%hn %n %ln"

[short, int, long] C-style base integer.

.TP

.L "%ho %o %lo"

[short, int, long] base 8 integer.

.TP

.L %s

String.

.TP

.L "%hu %u %lu"

[short, int, long] C-style base unsigned integer.

.TP

.L %v

The next two arguments are a pointer to a

.L char**

argument vector and the maximum number of elements in the vector.

.TP

.L "%hx %x %lx"

[short, int, long] base 16 integer.

.PP

.L %s

and

.L %v

data may also be counted length strings of the form

\f5(\fIcount\fP:\fIdata\fP)\fR

where

.I count

is the number of characters in

.I data

and the terminating

.L )

may also be a

.IR tab ,

or the data may be

.L (null)

which represents the

.L NULL

string.

.PP

.L tokline

returns an

.IR sfio (3)

stream to a file or string that splices

.I \enewline

into single lines,

allows "..." and '...' to quotes to span

.I newlines

(done by translating quoted

.I newline

to

.IR carriage-return ;

.L tokscan

above converts quoted

.I carriage-return

back to

.IR newline ),

and deletes

.I "# ... newline"

comments.

This is done by pushing an

.I sfio

discipline onto a string or file stream.

Seeks are disabled on the resulting stream.

If

.L "flags == SF_READ"

then

.L input

is a file name;

If

.L "flags == SF_STRING"

then

.L input

is a 0 terminated string;

otherwise

.L input

is an open

.L Sfio_t*

stream.

If

.L "line != 0"

then it points to a line count that is initialized to 0

and is incremented for each input line.

.SH "SEE ALSO"

sfio(3)