[fpc-devel] External assemblers (also modular discussion about free pascal compiler)

[Skybuck Flying]

Hello,

Perhaps some last silly questions about "external assemblers".

First I'd like to say that I saw some tutorial where the compiler itself 
produced some form of assembly output in text... probably real assembly...

So that's kinda interesting to output text this makes it flexible for other 
tools to then compile that...

It reduces complexity of the compiler because it doesn't actually need to 
fully assembly the assembly... this can be left to other tools.


So far I understand free pascal has "internal assemblers" (probably mostly 
for i386/x86 target, does it have others as well ?)


But there is also this "external assembler" thingy.... perhaps it's being 
used to implement internal assemblers as well...


But my question is the following:

What exactly is being "fed" towards the "external assembler" via the 
classes/api ?


(I probably should take a closer look... )

But from what I can remember I saw some kind of node structure... (maybe fp 
list/node and stuff like that ?)


To me it seemed like some kind of "free pascal classes/data structures" 
which might not be commonly supported by other languages like 
C/C++/Delphi/Etc...


So just in case I am wrong I am asking the following question:


Is there perhaps some kind of "standardized intermediate form (data 
structure like)" which all compilers could choose to output ? (Which I am 
not aware off ?)


(Or is the external assembler thingy indeed something conceived by the 
pascal developers themselfes ;))


Also one last question: Is there actually any other external tool which is 
actually invoked/call via this mechanism ?

(If the mechanism was ment to function via text then I can imagine that 
ofcourse... but then the question would be: what kind of text would be fed ? 
(some early form of assembly which needs to be fixed up ? )


Sending assembly towards an assembler seems kind weird... because first the 
compiler seems to use these nodes/data structures which are then apperently 
send towards some kind of cpu which seems to use assembly like data 
structures as well (but I could be wrong) which would then be turned back 
into text ????

(Why then not compile directly to binary ??? Perhaps because the binaries 
involve operating system specific structures and the CPU can actually be 
used for different operating systems ? (So I think that's probably the 
answer here...) The compiler compiles towards a certain CPU and makes sure 
the assembly is suited for the CPU... but it doesn't necessarily want to 
know about the operating system and thus sends it off to an operating system 
specific assembler ?)

(But then again this conflicts a bit with the RTL which does seem a bit 
operating system aware... but maybe that is not related to this and the RTL 
is ment for input towards the compiler and perhaps output as welll but in a 
more file like manner...)

A guess what I described above is the roll of the linker.... The linker 
links the assemblies and adds some operating system specific binary 
headers/structures for towards/for the final binary.


Does free pascal actually have an internal linker as well ?


Perhaps all these different aspects of a compiler are confusing things a 
little bit for me/noobies.... but perhaps also for the developers itself...

Therefore I wonder if free pascal could benefit from a more modular approach 
where everything is nicely split up into modules... which somehow 
communicate with each other...

This could be executables (though would get a bit difficult via files) or 
(dll's but that's windows specific) or (sockets... network communications 
seem odd for this...) and finally simply compiled units which would be 
placed in seperated module folders or so. (with the necessary headers... 
could also simply include full source as well).


Currently I wonder how "modular" free pascal is... since I am not acquinted 
with the code (yet)... from the looks of it it seems to use classes here and 
there... so at least it's "class-modular".... but classes can also starts to 
"mingle" with each other a bit... and might not form a "clean cut". Probably 
tempting for developers to mingle stuff up a bit... None-the-less that's 
probably easily avoidable or fixeable...


The problem remains that all source code is in pretty much one huge folder 
called: "compiler" with some "sub folders" for platforms.

At least the platforms are seperated from it... but this seems also a bit 
out of necessity for the search paths to not produce naming-space-conflicts 
and such.

The rest of the compiler code seems to be in one huge folder...

Some of the unit names are pretty short and not very descriptive... 
fortunately most of the unit files have a little comment in the header 
explaining what each file is.


Perhaps it would be interesting to try and split up the compiler folder into 
subfolders where each subfolder would represent a "module".

For example tokenizer/lexer, parser, assemblers and what else you can think 
of which would/should be modular.

The platforms/cpu's can then be moved to their own modular sub folder which 
could be called:

"platforms" or perhaps "cpu's" or perhaps "os" or perhaps "targets" which 
you think is best.


With such a more modular approach via subfolders it could then be easier to 
spot where unwanted depedency might exist within the module itself...

So suppose that a certain module is to be "self standing" and "self 
compileable" and this would immediatly show up if this was not to work 
because of
search paths pointing towards unwanted folders.

At first it might seem the compiler is to much linked towards all modules 
because ofcourse it goes from phase to phase... but that's where hooking 
into modules could come into play.


It could work like this: each module gets it's own input interface (input 
from the higher phase/modules) and it's own output interface (towards the 
lower modules)

Perhaps this could also be in both ways... However each module simply copies 
these interfaces so that they can be self standing...

Only the final compiler main program/executable needs to link the modules up 
towards each other...  perhaps test programs for the modules too. (These 
test programs should then be in seperate folders as well could be higher/sub 
folders to the module itself)

One final module which seems to be in play in general theory is the "symbol 
table" this could also have it's own interface which would be used by all 
modules as well.

So all modules would need to be hook-up to this symbol table as well.


(Think of "hooking-up" as delphi's event properties/method pointers/function 
pointers stuff like that...I am pretty sure you know that already ! ;) =D)



Bye,
  Skybuck.