[fpc-devel]Re: question about tokens.pas

Pierre Muller pierre at freepascal.org
Thu May 31 10:35:31 CEST 2001

>Can I declare a TOKEN of length 1 such as "y" or "o"?

  Currently, the scanner knows that there are no single chars that
are tokens :

See this text extracted from
tscannerfile.readtoken (line 1227 of current CVS state)

          { keyword or any other known token,
            pattern is always uppercased }
            if (pattern[1]<>'_') and (length(pattern) in [2..tokenidlen]) then

If you have one char tokens, you need to change the [2..tokenidlen]
into a [1..tokenidlen].

The following lines also explain why you need to have
alphabetically ordered tokens.

More information about the fpc-devel mailing list