[fpc-devel] Building compiler, rtl, host, target... (cross compiling)

[Skybuck Flying]

----- Original Message ----- 
From: "Skybuck Flying" <skybuck2000 at hotmail.com>
To: "FPC developers' list" <fpc-devel at lists.freepascal.org>
Sent: Thursday, 7 April, 2011 03:06 AM
Subject: Re: [fpc-devel] Building compiler, rtl, host,target... (cross 
compiling)


> Ok,
>
> I thought about it a bit and I think I now logically understand what's 
> going on and what needs to happen when one wants to build a new compiler, 
> possibly for cross compiling and such in relation to the RTL.
>
> First of all "free pascal compiler" is not a "multi-target cross compiler" 
> but it is a "dual-target cross compiler" (dual meaning host and target can 
> be different ;))
>
> So starting with the basics: Suppose somebody would get the free pascal 
> source codes only and now executables then that person would first need to

Little typo here... gonna correct it:

source codes only and no executables then that person would first need to

> compile:
>
> 1. The compiler
> and
> 2. The run time library
>
> For let's say windows.
>
>
> The compiler is a little bit like an application, applications need a 
> system unit, the runtime library provides that system unit.
>
> However the "external/user compiler" might already have a system unit... 
> so for that purposes the "external/user compiler run time library/system 
> unit might be used". The free pascal compiler dependancies on it's own RTL 
> could be fixed/modified/temporarely changed to use the external compiler's 
> system unit to get the first compiler build working.
>
>
> So step 1 would then produce:
>
> A free pascal compiler executable which runs on windows.
>
> The target to keep things easy was set to windows to first produce a 
> "native compiler" (which runs on windows and compiles to windows)
>
>
> Now the mission/job/idea is to produce a cross-compiler which can 
> cross-compile to a new architecture/processor/platform let's called it 
> NewCPU and NewRTL in short: NewTarget.
>
>
> Step 2 is a repeat of step 1. However it would be helpfull if the RTL 
> itself was also compileable to the new target... but let's see how far we 
> can go without it.
>
>
> Step 2 specifies somehow to the build enviroment that the compiler sources 
> are to be re-compiled but this time the compiler which is build from it 
> should compile to the NewTarget... for that the new compiler needs "new 
> architecture support"... So that's where the new instructions and new 
> assembler come in... and perhaps new linker... but let's start with new 
> assembler and such...
>
> Since the executable to be produced must still run on windows it can and 
> probably must still use the windows RTL to be able to produce the 
> executable.
>
> If the same RTL was used to specify the target build... then it would be a 
> bit strange because then NewCPU would try to use a Windows RTL...
>
> So this would then produce a "half-cross compiler" which would be kinda 
> funny... it would probably be "defunct"/"not working" perhaps such 
> executables produce by such a half compiler could be patched etc... but 
> that's probably a bit stupid to do.
>
>
> What is needed is a "full-cross compiler" which also uses the NewRTL for 
> the NewPlatform.
>
> Apperently specifieing the NewRTL can be done by compiler command line 
> options or configuration files.
>
> Thus there are now probably two RTL's involved:
>
> One RTL for the host, One RTL for the target.
>
>
> How the compiler makes sense of this I am not yet sure... Perhaps it's a 
> question of linking in the NewRTL when a NewPlatform application is being 
> build by the new compiler.
>
> So perhaps it's the "internal linker" which needs the NewRTL to be present 
> to do it's work ?!? How else could it work ??
>
>
> However perhaps the new compiler also needs to do type checking against 
> the NewRTL for NewPlatform applications...
>
> Thus somehow the new compiler must be using the new RTL.
>
> The new RTL didn't exist yet... so it needs to be compiled.
>
> Apperently this is where a "half-cross compiler comes in ?"
>
> The new RTL cannot depend on "other platform RTL's" ?
>
> So if I am correct in my assumption then the any RTL for that matter must 
> be "self-standing" and not rely on anything else except it's own platform 
> that it was written for.
>
> It must also not depend on RTL functionality itself ?
>
> How would otherwise a compiler be able to compile it ? (without creating 
> some kind of platform depedency)
>
>
>
> So I guess this is where a "half-cross-compiler" comes in... The 
> half-cross compiler is capable of compiling a NewRTL and producing some 
> kind of assembly text which can then be used to compile the NewRTL to some 
> kind of binary (a pascal unit binary?)
>
> This is perhaps where the linker also comes into play or maybe not...
>
>
> Anyway once the NewRTL is compiled into pascal unit binaries it can be 
> used/include in a new compiler build.
>
>
> So the compiler sources are again recompiled... but this time the compiler 
> sources are "told" or "modified via includes/search paths" to use the 
> NewRTL units so that it itself is compiled towards the new platform and 
> uses the new rtl for that platform.
>
> This will then produce a "full cross compiler" which is capable of 
> compiling from running on host towards a target.
>
>
> Once that's done... it's possible to recompile the compiler towards a 
> native compiler... produce native executable and native target 
> (compilation mode)... so that last step is the easy step.
>
>
>
> So if I am correct "porting free pascal compiler and rtl towards a new 
> platform" requires the following steps:
>
>
> 1. Step 1 (preparing for first native free pascal compiler)
>
> Install a "external pascal compiler" so that free pascal compiler sources 
> can be compiled.
> (ignore free pascal's own rtl if necessary), make changes to the free 
> pascal compiler source code if necessary to use the external pascal 
> compiler's RTL.
>
> 2. Step 2 (building free pascal compiler)
>
> Build the free pascal compiler with the "external pascal compiler". This 
> produces a free pascal compiler executable for whatever platform step1 was 
> running on.
>
> (These two steps can be skipped if "free pascal compiler executable" 
> already available (in this case external compiler=free pascal compiler 
> executable))
>
> 3. Step 3 (building half-cross compiler)
>
> Add new target's cpu instructions/assembler to free pascal compiler 
> sources so that the free pascal compiler can compile source code into free 
> pascal unit binaries.
>
> Re-build the compiler. (This creates what I would call a "half-cross 
> compiler")
>
> A half-cross compiler is a compiler which cannot yet produce 
> "target-executables" but it can produce "target-free pascal binary units".
>
> (Seems to make logical sense: target-executables not yet possible because 
> target-rtl missing)
>
>
> 4. Step 4 (building full-cross compiler)
>
> Add new target's rtl to free pascal compiler sources so that the free 
> pascal compiler can compile source into into target-(executable or 
> package/dll)-binaries.
>
> Re-build the compiler and tell it to use the new-target's-rtl as it's uses 
> clausule. (This can be via "target-free pascal binary units in combination 
> with the interface" or simply new-target's RTL full source code, since the 
> compiler should now be able to handle/compile it.
>
> This will produce a "full-cross compiler". The full cross compiler knows 
> how to compile towards target-cpu and it also knows how to use the 
> target-rtl.
>
>
> Optional repeating steps (fixing up):
>
> These steps can then be repeated to undo the changes that where made at 
> step 1, a native RTL could first be made which works with the 
> native/external compiler and perhaps with free pascal compiler itself as 
> well.
>
> Perhaps these steps are even manditory to be able to easily switch 
> targets... all code should in principe follow the same RTL 
> layout/interfaces ;)
> (But this might be conflicting with external compilers which already have 
> their own RTL, so that's why step1 might/would be necessary)
>
> It might also be necessary to change all other target code's rtl's to use 
> a new-non-conflicting api/layout for external compilers.
>
>
> 5. Step 5 (building any native compiler)
>
> Use the cross compiler to rebuild the free pascal compiler source codes.
>
> Specify the target's cpu and target's rtl. (Which in this case should be 
> the cpu and rtl for which a native compiler should be build).
>
>
> The reason for the native compiler is so that user's of this new target 
> can now be used to develop on itself... (assuming the target has such 
> capabilities)
>
> (Some targets might not have such capabilities like embedded system, 
> perhaps console, gameboys, phones, refrigderators/whatever so form them 
> step 4 is already enough ;))
>
>
>
>
> Missing from this list is ofcourse:
>
> debuggers/cross debuggers which would still be needed for serious platform 
> development but that is beyond the scope of this posting ;)
>
> And unfortunately me very little to almost no experience with writing 
> debuggers myself... perhaps a nice thing for me to explore in the future 
> ;)
>
>
> Bye,
>  Skybuck ;)
> _______________________________________________
> fpc-devel maillist  -  fpc-devel at lists.freepascal.org
> http://lists.freepascal.org/mailman/listinfo/fpc-devel
>