[fpc-devel] [patch] pscanner: differentiate between EOL & Tab characters from general Whitespace

[Vincent Snijders]

2010/8/23 Michael Van Canneyt <michael at freepascal.org>:
>> I don't see why introducing such new tokens is bad? It can only make
>> the tokenizer and parser more useful in the long term.
>
> I never said it is bad; I just don't see why you need it.

I guess one wants the line numbers between the orginal source to be
the same, for example to have the same line info in case of a
backtrace when you compare before and after the changes.

>
> You could perfectly do it differently, e.g. insert a newline after certain
> tokens:
> then
> begin
> ;
> end
> else
> var
> type
> const
>
So, programs compiled with profile info generate different backtraces
with line info as programs without them. That is not nice.

> That would perfectly be enough for your purposes, which is, to debug your
> profiler.
> And once that is done, it is no longer needed at all.
>
Why is it not longer needed?

> Don't worry, I'll apply the patch, but you have not convinced me it is a
> needed patch, even for your purposes. I'm surprised you didn't think of the
> above ways to do what you need.

I think with the new tokens a profiler can do a better job.

Vincent