[fpc-devel] Building compiler, rtl, host, target... (cross compiling)

[Skybuck Flying]

Ok,

I thought about it a bit and I think I now logically understand what's going 
on and what needs to happen when one wants to build a new compiler, possibly 
for cross compiling and such in relation to the RTL.

First of all "free pascal compiler" is not a "multi-target cross compiler" 
but it is a "dual-target cross compiler" (dual meaning host and target can 
be different ;))

So starting with the basics: Suppose somebody would get the free pascal 
source codes only and now executables then that person would first need to 
compile:

1. The compiler
and
2. The run time library

For let's say windows.


The compiler is a little bit like an application, applications need a system 
unit, the runtime library provides that system unit.

However the "external/user compiler" might already have a system unit... so 
for that purposes the "external/user compiler run time library/system unit 
might be used". The free pascal compiler dependancies on it's own RTL could 
be fixed/modified/temporarely changed to use the external compiler's system 
unit to get the first compiler build working.


So step 1 would then produce:

A free pascal compiler executable which runs on windows.

The target to keep things easy was set to windows to first produce a "native 
compiler" (which runs on windows and compiles to windows)


Now the mission/job/idea is to produce a cross-compiler which can 
cross-compile to a new architecture/processor/platform let's called it 
NewCPU and NewRTL in short: NewTarget.


Step 2 is a repeat of step 1. However it would be helpfull if the RTL itself 
was also compileable to the new target... but let's see how far we can go 
without it.


Step 2 specifies somehow to the build enviroment that the compiler sources 
are to be re-compiled but this time the compiler which is build from it 
should compile to the NewTarget... for that the new compiler needs "new 
architecture support"... So that's where the new instructions and new 
assembler come in... and perhaps new linker... but let's start with new 
assembler and such...

Since the executable to be produced must still run on windows it can and 
probably must still use the windows RTL to be able to produce the 
executable.

If the same RTL was used to specify the target build... then it would be a 
bit strange because then NewCPU would try to use a Windows RTL...

So this would then produce a "half-cross compiler" which would be kinda 
funny... it would probably be "defunct"/"not working" perhaps such 
executables produce by such a half compiler could be patched etc... but 
that's probably a bit stupid to do.


What is needed is a "full-cross compiler" which also uses the NewRTL for the 
NewPlatform.

Apperently specifieing the NewRTL can be done by compiler command line 
options or configuration files.

Thus there are now probably two RTL's involved:

One RTL for the host, One RTL for the target.


How the compiler makes sense of this I am not yet sure... Perhaps it's a 
question of linking in the NewRTL when a NewPlatform application is being 
build by the new compiler.

So perhaps it's the "internal linker" which needs the NewRTL to be present 
to do it's work ?!? How else could it work ??


However perhaps the new compiler also needs to do type checking against the 
NewRTL for NewPlatform applications...

Thus somehow the new compiler must be using the new RTL.

The new RTL didn't exist yet... so it needs to be compiled.

Apperently this is where a "half-cross compiler comes in ?"

The new RTL cannot depend on "other platform RTL's" ?

So if I am correct in my assumption then the any RTL for that matter must be 
"self-standing" and not rely on anything else except it's own platform that 
it was written for.

It must also not depend on RTL functionality itself ?

How would otherwise a compiler be able to compile it ? (without creating 
some kind of platform depedency)



So I guess this is where a "half-cross-compiler" comes in... The half-cross 
compiler is capable of compiling a NewRTL and producing some kind of 
assembly text which can then be used to compile the NewRTL to some kind of 
binary (a pascal unit binary?)

This is perhaps where the linker also comes into play or maybe not...


Anyway once the NewRTL is compiled into pascal unit binaries it can be 
used/include in a new compiler build.


So the compiler sources are again recompiled... but this time the compiler 
sources are "told" or "modified via includes/search paths" to use the NewRTL 
units so that it itself is compiled towards the new platform and uses the 
new rtl for that platform.

This will then produce a "full cross compiler" which is capable of compiling 
from running on host towards a target.


Once that's done... it's possible to recompile the compiler towards a native 
compiler... produce native executable and native target (compilation 
mode)... so that last step is the easy step.



So if I am correct "porting free pascal compiler and rtl towards a new 
platform" requires the following steps:


1. Step 1 (preparing for first native free pascal compiler)

Install a "external pascal compiler" so that free pascal compiler sources 
can be compiled.
(ignore free pascal's own rtl if necessary), make changes to the free pascal 
compiler source code if necessary to use the external pascal compiler's RTL.

2. Step 2 (building free pascal compiler)

Build the free pascal compiler with the "external pascal compiler". This 
produces a free pascal compiler executable for whatever platform step1 was 
running on.

(These two steps can be skipped if "free pascal compiler executable" already 
available (in this case external compiler=free pascal compiler executable))

3. Step 3 (building half-cross compiler)

Add new target's cpu instructions/assembler to free pascal compiler sources 
so that the free pascal compiler can compile source code into free pascal 
unit binaries.

Re-build the compiler. (This creates what I would call a "half-cross 
compiler")

A half-cross compiler is a compiler which cannot yet produce 
"target-executables" but it can produce "target-free pascal binary units".

(Seems to make logical sense: target-executables not yet possible because 
target-rtl missing)


4. Step 4 (building full-cross compiler)

Add new target's rtl to free pascal compiler sources so that the free pascal 
compiler can compile source into into target-(executable or 
package/dll)-binaries.

Re-build the compiler and tell it to use the new-target's-rtl as it's uses 
clausule. (This can be via "target-free pascal binary units in combination 
with the interface" or simply new-target's RTL full source code, since the 
compiler should now be able to handle/compile it.

This will produce a "full-cross compiler". The full cross compiler knows how 
to compile towards target-cpu and it also knows how to use the target-rtl.


Optional repeating steps (fixing up):

These steps can then be repeated to undo the changes that where made at step 
1, a native RTL could first be made which works with the native/external 
compiler and perhaps with free pascal compiler itself as well.

Perhaps these steps are even manditory to be able to easily switch 
targets... all code should in principe follow the same RTL layout/interfaces 
;)
(But this might be conflicting with external compilers which already have 
their own RTL, so that's why step1 might/would be necessary)

It might also be necessary to change all other target code's rtl's to use a 
new-non-conflicting api/layout for external compilers.


5. Step 5 (building any native compiler)

Use the cross compiler to rebuild the free pascal compiler source codes.

Specify the target's cpu and target's rtl. (Which in this case should be the 
cpu and rtl for which a native compiler should be build).


The reason for the native compiler is so that user's of this new target can 
now be used to develop on itself... (assuming the target has such 
capabilities)

(Some targets might not have such capabilities like embedded system, perhaps 
console, gameboys, phones, refrigderators/whatever so form them step 4 is 
already enough ;))




Missing from this list is ofcourse:

debuggers/cross debuggers which would still be needed for serious platform 
development but that is beyond the scope of this posting ;)

And unfortunately me very little to almost no experience with writing 
debuggers myself... perhaps a nice thing for me to explore in the future ;)


Bye,
  Skybuck ;)