[fpc-devel] [patch] pscanner: differentiate between EOL & Tab characters from general Whitespace

[Graeme Geldenhuys]

Op 2010-08-23 01:17, Michael Van Canneyt het geskryf:
>> Well, does it hurt having them tokenized?
> 
> Of course. 
> It slows the code. Marginally, I agree, but nevertheless.

Sorry, but I disagree here. Before it got tokenized as a tkWhitespace; now
it gets tokenized as tkLineEnding or tkTab. It's still being assigned a
token, just one that describes it better. As for the parsers NextToken
method now having two extra tokens it needs to ignore. I profiled that, and
the difference was too small to notice.


> I never said it is bad; I just don't see why you need it.
> 
> You could perfectly do it differently, e.g. insert a newline after certain tokens:
> then
> begin
> ;

Vincent hit the nail on the head. Now the compiled code with profiling has
very different line numbers (now much more than before) to the real code.
So line number info reported by LineInfo unit is now even more useless. And
considering that under 64-bit systems, method names are not reported by
LineInfo for some reason, the line number info is rather important.


> needed patch, even for your purposes. I'm surprised you didn't think of 
> the above ways to do what you need.

See two paragraphs back. ;-)



Regards,
  - Graeme -

-- 
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://opensoft.homeip.net/fpgui/