[fpc-devel] Macro Processing

[Hans-Peter Diettrich]

Joerg Schuelke schrieb:

> So, why not? In the further reading the main thesis.
> 1.	As far as possible simple syntax, which fits into the up to
> 	date implemented language.
> 2.	Maximal implementation of this syntax. No needless
> 	restrictions.
> 3.	Effectiveness in view of the implementation.
> 4.	Effectiveness in view of the compiling process.
> 5.	Features:
> 	A view at the C-preprocessor tells us "stringification,
> 	concatenation, variadic macros" (I hate C too, but it is a
> 	starting point to make it better)
> 	See:http://gcc.gnu.org/onlinedocs/cpp/ 
> 	Macro definitions inside a macro definition are possible this
> 	time. For me it would be the export from unit interfaces and
> 	macro parameters.
[...]
> at issue 4.
> 	A construct of the kind {$I mac_id(mac_paramlist)} instead
> 	of the simpler mac_id(mac_paramlist) for the use of an macro
> 	would solve some of the problems. Even in respect of the
> 	separation of true pascal code and preprocessor code it would
> 	be nice.

A general decision is required: do we *want* explicit or implicit macro 
expansion?

IMO an important use for macros is the ability to modify parts of the 
code, without touching the code to be modified. E.g. replacing DebugLn 
by a NOP macro allows to remove all debug output from a unit - much 
harder to do with $IFDEF.


> The use of macros takes place in contexts which are different from those
> in ordinary pascal constructs. For example the handing over of a
> unknown number of parameters would be helpful. Obviously this leads to
> the use of .. as ellipsis.
> 
> Issue variadic macros:
> 
> 	{$define dp(x,y..):=
> 		dbgstr(%x%,[%y..%]);
> 	}

The same effect can be achieved by an invocation like
   dp(x,[y,z]);
just as in Format().


> Remark: Up to now it is not clear what a token is for the preprocessor.
> The simplest is to only recognize the tokens for the preprocessor
> himself and to pass on the others unchanged as chars. The advantage of
> this: maybe it fits better in the present scanner, maybe efficiency.

Macros should be stored as tokens, compatible with the scanner's Record 
and Replay methods. The preprocesor must not distinguish between 
identifiers and reserved words, see the implementation of 
context-sensitive directive handling. The detection of macro identifiers 
can be done in the keyword recognition, eventually resulting a tkMacro 
token.

> What about syntactical checks while reading the macro replacement text?
> No need for that, only the parameters are recognized and some kind of
> mark inserted instead of them.

Macro arguments can be stored as identifiers, which then are found 
(during macro expansion) in the top-level symbol table. Eventually a new 
symbol type "MacroArgument" has to be introduced, that enforces argument 
substition, stringification etc.


> Detail work:
> 
> 1.	What about , and ) delimiting token in mac_paramlist.

IMO a macro parameter list should follow the language conventions, so 
that a procedure identifier can be replaced by a macro identifier, 
forcing macro expansion instead of generating an procedure call. I.e. 
the preprocessor is a *postprocessor* for scanner tokens.


> One problem which I can see is the structure of the fpc scanner. His
> interface is "really fat". Maybe further thinkings brings it out:
> It is impossible to do that without rewriting the whole scanner. But I
> will do that thinkings.

That's one more reason to position the preprocessor *after* the scanner, 
or at the end of the tokenizer to prevent another call level.


> Sorry the bad English folks, but it was the harder work for me to
> translate it, then it is for you, to read it.

Nice work :-)

As mentioned at the begin, I don't like special syntax for macro 
identifiers, invocation and parameters. The worth of your attempt, to 
speed up macro processing, must be determined after it has been 
implemented. IMO similar effects can be obtained by removing (not 
introducing) some of the C preprocessor misfeatures, which have not been 
considered yet. Candidates are recursive macro expansion and (also 
recursive?) expansion of macro parameters, as well as concatenation.

DoDi