22.14

java.io.StreamTokenizer

THE PACKAGE JAVA.IO

22.14 The Class

java.io.StreamTokenizer

A

StreamTokenizer

 takes an input stream and parses it into  tokens,  allowing

the tokens to be read one at a time. The parsing process is controlled by a table

and a number of flags that can be set to various states, allowing recognition of

identifiers, numbers, quoted strings, and comments in a standard style.

public class StreamTokenizer {

public static final int TT_EOF =  1;

public static final int TT_EOL =  \n ;

public static final int TT_NUMBER =  2;

public static final int TT_WORD =  3;

public int ttype;

public String sval;

public double nval;

public StreamTokenizer(InputStream in);

public void resetSyntax();

public void wordChars(int low, int hi);

public void whitespaceChars(int low, int hi);

public void ordinaryChars(int low, int hi);

public void ordinaryChar(int ch);

public void commentChar(int ch);

public void quoteChar(int ch);

public void parseNumbers();

public void eolIsSignificant(boolean flag);

public void slashStarComments(boolean flag);

public void slashSlashComments(boolean flag);

public void lowerCaseMode(boolean flag);

public int nextToken() throws IOException;

public void pushBack();

public int lineno();

public String toString();

}

Each byte read from the input stream is regarded as a character in the range

 \u0000 

 through

 \u00FF 

. The character value is used to look up five possible

attributes of the character: whitespace, alphabetic, numeric, string quote, and

comment character (a character may have more than one of these attributes, or

none at all). In addition, there are three flags controlling whether line terminators

are to be recognized as tokens, whether Java style end of line comments that start

with

//

 should be recognized and skipped, and whether Java style  traditional 

comments delimited by

/*

 and

*/

 should be recognized and skipped. One more

flag controls whether all the characters of identifiers are converted to lowercase.

712