[fpc-devel]Hi for first time, and a question about tokens.pas

Peter Vreman peter at freepascal.org
Thu May 31 17:04:08 CEST 2001

> > > Can I declare a TOKEN of length 1 such as "y" or "o"?
> > 
> > It should be possible if you change the length check in scanner in the 
> function readtoken.
> << Sorry, for the previous message I didn't look for scanner file in CVS enough.
> Now, I'm seeing this file exists.
> And I've studied it I have to change 2 to 1 in:
> tscannerfile.readtoken procedure 
> and in
> is_keyword function
> And I've found InitScanner procedure which I supposed is the one that proccess 
> such things
> as "defined parametres", i.e. {$ifdef TP}. 
> Am I right? Is this the last thing about InitScanner correct?

That is from the 1.1 branch. The 1.0.x branch doesn't have this function. But the function
adds the directives like {$I} and {$ifdef}, not the symbols, those are mostly added from
options.pas (search for def_symbol).

> how have you managed to know if it detected "repite" that's to 
> > > say.
> > 
> > By debugging the compiler and checking the value that the token variable gets 
> after
> > reading the token.
> << You're very lucky. I tried to debug it with IDE from 1.0.4 and it didn't work
> (see http://www.freepascal.org/bugs/showrec.php3?ID=1515 ), and then I took the 
> IDE from 1.0.5 and I overwrote it in my current 1.0.4 installation and I didn't 
> work neither.

I'm at work and used plain gdbpas.exe to debug. But using the latest go32v2 1.0.5 IDEs
should also be possible under Win9X. Under WinNT/2K there can be problems as those have
strange Dos support.

> > > The first thing I do is change tokens.pas and recompile the compiler, and 
> then
> > > I try to compile a very easy file (writing hello in screen) but it doesn't 
> > > compile it automatically, it needs system unit, and many others in inc and 
> > > i386. And so... these files have to have the new tokens, that's to say, 
> > > 
> > > I have to put unidad instead of unit. Programa instead of program and this 
> with 
> > > a lot of files.
> > > 
> > > You seem to have made it easier. How have done it? Because myself I had to 
> > > made a program (www.geocities.com/beatlesnap/pashis.zip) in order to 
> translate 
> > > the basic rtl into my new tokens rtl.
> > 
> > Rename the innr.inc to compinnr.inc and fix the compiler accordingly to 
> include that file
> > instead of innr.inc. This was already done in the 1.0.5 development compiler.
> > 
> << I think you haven't understood me, I don't find anything in this file
> (http://www.freepascal.org/cgi-bin/cvsweb/fpc/rtl/inc/innr.inc?rev=1.3&content-
> type=text/x-cvsweb-markup), to make the compiler understand an rtl with old 
> tokens as they were new ones. 

Sorry, it was innrcomp.inc

> And I've been searching compinnr.inc in (http://www.freepascal.org/cgi-
> bin/cvsweb/projects/lazarus/) and I haven't found it, although I must confess I 
> haven't searched in all the folders. (Perhaps should I find it at 
> http://lazarus.freepascal.org snapshot?)

Maybe downloading fpc10.zip from the pub/fpc/source/ dir is better suited for you. Then
you've all the latest files.

Btw. Translating the assembler tokens is dangerous as they are also used for writing
external assembler files. If you translate them you can only use the internal assembler.

More information about the fpc-devel mailing list