[fpc-devel] Building compiler, rtl, host, target... (cross compiling)

[Hans-Peter Diettrich]

Skybuck Flying schrieb:

> First of all "free pascal compiler" is not a "multi-target cross 
> compiler" but it is a "dual-target cross compiler" (dual meaning host 
> and target can be different ;))

FPC is a single target cross compiler. "cross" indicates that the target 
can be different from the host system.

> So starting with the basics: Suppose somebody would get the free pascal 
> source codes only and now executables then that person would first need 
> to compile:
> 
> 1. The compiler
> and
> 2. The run time library

No. First the RTL is compiled, then the compiler is built with that RTL.

> The compiler is a little bit like an application, applications need a 
> system unit, the runtime library provides that system unit.

That's why the RTL is compiled before. The compiler is built *exactly* 
like any other application, in this step.

In the next step the new compiler is used to compile itself. This way 
new features can be included, which the old (bootstrap) compiler could 
not compile (e.g. class helpers...).

This step is repeated until the last two executables are binary 
identical. This procedure allows for further extensions, that even could 
not be compiled by an earlier stage compiler, without the need to 
specify an exact number of required recompilations.


> So step 1 would then produce:
> 
> A free pascal compiler executable which runs on windows.
> 
> The target to keep things easy was set to windows to first produce a 
> "native compiler" (which runs on windows and compiles to windows)
... and which can compile the new language version, including all added 
features.


> Now the mission/job/idea is to produce a cross-compiler which can 
> cross-compile to a new architecture/processor/platform let's called it 
> NewCPU and NewRTL in short: NewTarget.
> 
> 
> Step 2 is a repeat of step 1. However it would be helpfull if the RTL 
> itself was also compileable to the new target... but let's see how far 
> we can go without it.
> 
> 
> Step 2 specifies somehow to the build enviroment that the compiler 
> sources are to be re-compiled but this time the compiler which is build 
> from it should compile to the NewTarget... for that the new compiler 
> needs "new architecture support"... So that's where the new instructions 
> and new assembler come in... and perhaps new linker... but let's start 
> with new assembler and such...
> 
> Since the executable to be produced must still run on windows it can and 
> probably must still use the windows RTL to be able to produce the 
> executable.

Right. Every application must use the RTL of its host platform.

> What is needed is a "full-cross compiler" which also uses the NewRTL for 
> the NewPlatform.

No, why should the compiler need to use a different RTL, for its *own* 
operation? Every compiler can compile the RTL for its target, as well as 
other applications.

> Apperently specifieing the NewRTL can be done by compiler command line 
> options or configuration files.

As is, the specification of the target (hard coded in the compiler) is 
sufficient to locate target-specific parts of the source code, in 
target-specific directories. I really like that idea :-)

> Thus there are now probably two RTL's involved:
> 
> One RTL for the host, One RTL for the target.
> 
> 
> How the compiler makes sense of this I am not yet sure... Perhaps it's a 
> question of linking in the NewRTL when a NewPlatform application is 
> being build by the new compiler.

Every target e.g. has its own data types, like 32 or 64 bit pointers. 
That's why the compiler has to use the target-specific version of the 
RTL, in cross-compiling any code.

> So perhaps it's the "internal linker" which needs the NewRTL to be 
> present to do it's work ?!? How else could it work ??

Every linker needs binaries that have been compiled for the target, and 
includes them into the final executable (for the same target).


> However perhaps the new compiler also needs to do type checking against 
> the NewRTL for NewPlatform applications...
> 
> Thus somehow the new compiler must be using the new RTL.

That's fully automatic. The compiler compiles or loads precompiled units 
recursively, by evaluating the Uses clauses, so that it traverses the 
target specific RTL and ends up in the System unit.

All parts of the target, that are required for a compilation, are 
already built into the compiler. That includes code generation for the 
target CPU, and linking executables for the target OS (what may actually 
occur in an external assembler and linker).


> So if I am correct "porting free pascal compiler and rtl towards a new 
> platform" requires the following steps:
> 
> 
> 1. Step 1 (preparing for first native free pascal compiler)
> 
> Install a "external pascal compiler" so that free pascal compiler 
> sources can be compiled.
> (ignore free pascal's own rtl if necessary), make changes to the free 
> pascal compiler source code if necessary to use the external pascal 
> compiler's RTL.
> 
> 2. Step 2 (building free pascal compiler)
> 
> Build the free pascal compiler with the "external pascal compiler". This 
> produces a free pascal compiler executable for whatever platform step1 
> was running on.
> 
> (These two steps can be skipped if "free pascal compiler executable" 
> already available (in this case external compiler=free pascal compiler 
> executable))
> 
> 3. Step 3 (building half-cross compiler)

This builds any compiler, for the same or a different platform. Nothing 
"half" here, the second half only adds the RTL for the target OS.

> Add new target's cpu instructions/assembler to free pascal compiler 
> sources so that the free pascal compiler can compile source code into 
> free pascal unit binaries.
> 
> Re-build the compiler. (This creates what I would call a "half-cross 
> compiler")


> 4. Step 4 (building full-cross compiler)
> 
> Add new target's rtl to free pascal compiler sources so that the free 
> pascal compiler can compile source into into target-(executable or 
> package/dll)-binaries.

That's all. The new RTL can be compiled now, but it would be compiled as 
well with the first project.


> 5. Step 5 (building any native compiler)
> 
> Use the cross compiler to rebuild the free pascal compiler source codes.
> 
> Specify the target's cpu and target's rtl. (Which in this case should be 
> the cpu and rtl for which a native compiler should be build).

This is not different from step 1, only with a different target machine 
and/or OS.

> The reason for the native compiler is so that user's of this new target 
> can now be used to develop on itself... (assuming the target has such 
> capabilities)

The same reason for cross-building any application for the target system.



> Missing from this list is ofcourse:
> 
> debuggers/cross debuggers which would still be needed for serious 
> platform development but that is beyond the scope of this posting ;)

The debugger is part of the binutils, as are assemblers and linkers. 
While it may make sense to add assembler and linker functionality to the 
compiler itself, it doesn't make sense to add an debugger to the 
compiler. But a debugger-interface can be added to a development system 
(Lazarus), for integrated debugging.

DoDi