[fpc-devel] z370 Cross Compilation, Pass 2 of ....
Mr Paul Robinson
rfc1394 at yahoo.com
Mon Aug 19 13:00:18 CEST 2013
Based on information and belief, on Mon, 19 Aug 2013 10:37:43 +0200, Sven Barth <pascaldragon at googlemail.com> wrote to the Free Pascal list:
> I don't get what you're trying to say here.
I mean I spent the better part of a year writing a cross-reference tool for Free Pascal since the compiler doesn't include one as they used to do (Mainframe and minicomputer Cobol, Fortran and Assembler all have built-in cross-reference as part of the compiler/assembler), then I had problems implementing a piece of it, so I realized that I'll come back to it at some time later, and right now I want to go back and try again with Free Pascal since I now know what I need to tackle. When I wasn't sure how to ask for what I knew I needed, I set this request aside to work on the cross-reference. I had to figure out what was going on and what I had to do.
> Analyzing what a big-endian compiler generates won't help you there.
Possibly, but I have to start somewhere.
> You need to look into the compiler's code and learn how it
> does generate code for a specific platform. That has nothing
> to do with big endian or little endian.
It does if the compiler generates object files directly (the way Pascal 8000 did on the 370, UCSD did, and Delphi does). If the compiler generates assembly language source files (the way OMSI Pascal did on the PDP-11 and the National Bureau of Standards Pascal Compiler did), then no, endianness probably won't matter. But I don't know from direct examination which FPC does, it's a huge application and I have no idea what all 100+ units and 264,000 lines of code do.
I have to start somewhere and I have to take a guess. There are three possibilities. (I don't care what FPC does when it scans the program; that's all magic I probably don't have to worry about or deal with right now.) (1) The compiler, when it generates code, generates an intermediate output file processed by something else (Stanford Pascal, P4, P5, Java Compilers). (2) The compiler, when it generates code, generates an assembly language file (or possibly some other language) for the target that is passed to the Assembler (OMSI Pascal for the PDP-11). (3) The compiler, when it generates code, generates an actual object file and/or an executable (Pascal 8000, Turbo Pascal, Microsoft Pascal, Delphi).
I know not which of these three Free Pascal does.
> Platform independant stuff is normally located in
> the ncg*.pas units and the platform specific stuff
> is in the nXY*.pas unit of the corresponding CPU
> directory where XY is a short form of the CPU, e.g.
> n68kadd.pas for the m68k add/compare node. Best
> is you look how other CPUs are implemented
> (prefereably the simpler ones like > MIPS or m68k)
> and copy that more or less for the new platform.
My spell checker says you spelled "independent" and "preferably" worng. :)
Yeah, and I still have to figure out what compiler switches to set to enable the existing compiler to create a cross-compiler for that target that runs on Windows, and I'm asking for help on that point, because I figure someone might be able to tell me in 10 words which switches I need. The documentation is rather sparse on cross-compiling and the code is, well, to say the least, not very forthcoming.
> Adding a new platform to FPC is not cheesecake
I'm fully aware of this, I'm not stupid and this is not the first compiler I've worked on, just the largest. I've written patches to six different Pascal compilers and I've written two compilers from scratch (not for Pascal), one was a Fortran IV to Visual Basic translator written in 7,000 lines of VB5, the other was a compiler I needed for a different purpose. I'm not someone who just fell off the turnip truck. I know there's (a lot of) work involved.
I have to take a guess on where to start. Given an existing implementation, that I actually get it to work, I can then figure out what is being done, and from that, then I can figure out what files are being used and/or called, and what procedures/methods are invokes. And that's what I'm asking for help. What do I do to get one of the existing compilers that generates code for some other machine to run on Windows and when fed itself, have it generate an output file of the generated code of itself for that machine?
Given that, I can then figure out what to look for.
> and you should know how the compiler's backend work. Just looking
> at the output of a target won't help you!
It will a little. Seeing what is generated means I can look in the source code for where that is generated from, sort of the way the NSA, if it thinks you're calling a terrorist, it can search the billions of phone records it secretly seized to see who else you called!
Given the generated output, I can (presumably) trace it back to whatever program file generated it and thus figure out what files were used. Going further, if necessary, I can insert procedure or unit traces on those files and see what procedures were called in what order and possibly by whom.
Given that information, I can figure out what's going on. I'm just having trouble figuring that out and I wanted any help someone could point me in where to look.
> First step is to get a cross compiler to s370/zSystem working so
> that simple programs can be compiled which you can use to
> implement the RTL. Only when enough of the RTL is implemented
> and the code generator works good enough then you can try to
> cross compile the compiler, but not earlier.
Right. And I need to figure out what procedures in what files are being invoked in order to change those so I take an existing compiler which generates some other target than Windows, and while that compiler is running under Windows, turn the switches and flags and defines on, recompile that compiler so it generates code for the machine it's targeting (MIPS, PPC, whatever) so that, even though it's running on Windows, it creates code for the other target machine. Then, taking that compiler that I know does work and generates PPC or MIPS code or whatever target its for, change its flags so that while I ask it to generate S370, it still
generates PPC or MIPS code. Now, then, knowing that, I can then change its code generator so that now, it will generate the output code for S370 instead of MIPS or PPC.
Now, I have a bootstrap compiler that will, on a Windows box, take a program which is syntactically correct, and when it runs, it will generate S370 code. Now, at that point I can recompile the run-time libraries needed to implement the compiler. Take those generated files over to a real or simulated 370, and (presumably) assemble them to produce the run-time library. At that point, I can feed the compiler into itself, produce its own assembler source, assemble it, put it on the 370, link it against the previously compiled library, and if that's right, I should then get a compiler that runs on the target machine. Then, and at that point, I can then throw the compiler at itself and it should generate the exact same output as it did on the Windows box, and at that point the compiler now is no longer a cross-compiler, it's a native compiler.
As I said, I didn't just fall off the turnip truck, I do know what's involved. But to do that, I first have to get a cross-compiler for something running on Windows, and I'm trying to figure out what switches and defines I need!
>> So how would I get the constants and values added to
>> include the s370 as a target for FPC?
> Search the compiler's source for e.g. m68k to see how other
> platforms are implemented.
Been there, done that. If you read my comments in the WIKI on how I was implementing this, I got to the point where the flags were all inserted, and they did nothing except neither did the compiler because I didn't have the flags necessary to invoke what was needed to create an actual working compiler that could do something, it basically excluded the target generator!
So, once I put all the flags in so that I can have something like -DS370 in the command line recognized as a proposed target, how can I get that symbol and the associated defines for the new processor target added to the source base, even if it's not yet implemented? I basically want to have a fixed target set so that if someone else works on some other target there isn't a conflict.
I mean, if I didn't want to play nice and make this available to everyone else, I could just say, "Okay, I need this done, I can't get the FPC team to accept it for a branch for the 370, I'll fork the code and make an incompatible version and now instead of a unified source tree, there's now an independent branch somewhere else and people are going to be confused. Not my problem." But I'd like to be nice and avoid that.
So, once I have picked the next set of flags to support a new processor, that do not change anything for an existing compile that doesn't invoke or use them, they're just there as hooks for when they are placed into service, how do I get them added to the regular source base so I don't end up creating a fork because what I've changed has not been made available as part of the rest of the code base?
>> Second, how do I create a cross-compiler that runs on Windows,
>> for any of the big-endian processors, so that I can see what it
>> has to generate code for the target machine? Given that, I can
>> change the code to either change the generated object file - if
>> it does direct object files - or the assembly language so it
>> generates S370 assembly.
> No. You should learn how the compiler generates code
That's what I'm trying to figure out! One module calls another, which calls another, and so on, and so on, "lather, rinse, repeat." In the life of some program, the compiler front end calls one module, which calls another, and on and on, and at some point it sees the "PROGRAM" or "UNIT" symbol, or it sees a piece of code indicating that this is a program that has no program statement, so it will then call the backend telling it "You need to start compilation of a program, or a unit" followed by "here's its name," or "it doesn't have a name."
There's a place where the scanner hands off to the generator when it gets a UNIT, VAR, or TYPE or PROGRAM or something else to indicate a program or unit has started. Or an executable code for a BEGIN statement. And I want to be there when it hands off, but I don't know where.
The compiler as it stands is huge, and any hints on where to look is what I'm asking for. And what I am asking for is, given the existing code base, what flags and defines do I set, that, taking the compiler, what flags do I set to change the default so that on this compiler, running on Windows, with no flags set in the source, instead of generating a Windows executable, it generates an executable for MIPS or PPC?
Now, then, once that is completed, taking the unmodified compiler and running it through the new one will generate that new target even though it's running on Windows. Then, I can go through the source tree and see what's involved. But I have to start somewhere and I figure one small question now could save me weeks of work trying to find it, since, again, the documentation rather sparse on the question and is none too open on how to do this.
I figure someone's 30-second answer, well, you set this define, and this switch, and point to this directory, and the compiler will now generate MIPS or PPC code even though it's running on Windows. And so I compile that version, once, now, every time I run that version of the compiler it generates code for that machine instead of Windows. Isolate the specific source files it uses, which could be anywhere from 20 to 50 or more, and then I can go through that to change the target machine. But first, I've got to figure out how to get FPC to create a consistent different-machine target, and I figure someone might be able to help.
> and then implement a new code generator
> backend for s370, maybe based on a copy
> of another platform. Playing around with
> the output won't help you.
I don't want to play around with the output, I want to play around with the files that generate that output! And I have to find out what files are called, and by derivation, what objects, modules, procedures and functions are being invoked.
> First step consists of cross compiled
> hello world programs to implement
> and test the RTL. The compiler is much
> too complicated to get it running at first.
Yeah, and to do that I need to get the compiler to generate that cross-compiled program for a machine it already implements OTHER than Windows/Intel, and I'm asking for any help to find out how!
The Lessons of history teach us - if they teach us anything - that no one learns the lessons that history teaches us.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the fpc-devel