[fpc-devel]Hi for first time, and a question about tokens.pas

[adrian15 at europe.com]

Well, as most of you perhaps know I am trying to translate freepascal to 
spanish. Well, I have many problems with tokens.pas file.

And the boards, aren't as much as I wanted, so I came to this emaillist.

The one that is in pp\source\compiler

What I try is to recompile the compiler, with a changed tokens.pas file

I've noticed that in tokens.pas there's a procedure that orders the normal 
words, which are treated by this procedure because they are not special 
(boolean special is false). But it needs the token str to be ordered as they 
are... you know first by length and when the same length, you order it 
alphabetacally.

I've seen many files that treat tokens defined in tokens.pas, that means that 
PERHAPS they have to be ordered in tokens.pas BUT I'M NOT SURE. 

And I'm also not sure of these files being the unique to access tokens.pas and 
need something ordered.

htypechk.pas THIS ONE I'M QUITE SURE IT DOESN'T GO TO TOKENS.PAS AND BOTHER 
ABOUT ORDER BUT... PERHAPS IT IS


tcadd.pas THIS ONE I'M QUITE SURE IT DOESN'T GO TO TOKENS.PAS AND BOTHER ABOUT 
ORDER BUT... PERHAPS IT IS


In pexpr.pas I found this:

  const
     { Warning these stay be ordered !! }
     operator_levels:array[Toperator_precedence] of set of Ttoken=
        ([_LT,_LTE,_GT,_GTE,_EQUAL,_UNEQUAL,_OP_IN,_OP_IS],
         [_PLUS,_MINUS,_OP_OR,_OP_XOR],
         [_CARET,_SYMDIF,_STARSTAR,_STAR,_SLASH,
          _OP_AS,_OP_AND,_OP_DIV,_OP_MOD,_OP_SHL,_OP_SHR],
         [_STARSTAR] );

in line 2027 if you want to check.

WHAT DOES MEAN "Warning these stay be ordered "?
They have to be ordered as they are in tokens.pas
like the original pexpr.pas or what?


psub.pas
in line 425 I have this:


if not(token in [_PLUS..last_overloaded]) then
  Message(parser_e_overload_operator_failed);
optoken:=token;
consume(Token);

What means that goes from _PLUS (I think it is because
it is supposed to be always the first token )

and to last_overloaded, which is defined somewhere else:

in symtable.pas that has in line 404:


 first_overloaded = _PLUS;
 last_overloaded  = _ASSIGNMENT;

this is because these are supposed to be first and the last
operators.

Well,... this is not very good programming, i think
first_overloaded and last_overloaded should be defined
in tokens.pas with an integer const (myconst) which says how many
operators that can be overloaded do we have.

Something like first token is first_overloaded and
myconst-th token is last_overloaded.

In this way we will able to change the tokens order easier,
but now that I think, normal programmers don't do it
usually.

Well...

I know that tokens.pas NORMAL WORDS have to be ordered but...

In fact the question is what about operators? How must them be ordered?
Like they are in the original version? With the same order pattern
than Normal words? Or what?

And another thing about the procedure that orders (well ok, in fact
it creates an index) I don't know if it works ok.

With this tokens.pas (www.geocities.com/beatlesnap/tokens.pas) "Repite" token 
(I change repit token and i put repite) is not detected as a known identifier.

Is it that I don't order it ok, or it's a bug?

Thank you for all, "repite" not known as a token is driving me
mad.

Adrian15.

P.S.: If you are curious and want to know why I'm not using Y,O, instead of YY 
and OO, the answer is that my intuition tells me that you cannot have tokens of 
one character in
freepascal.

---------------------------------------------
This message was sent using Endymion MailMan.
http://www.endymion.com/products/mailman/