[fpc-devel] RTTI for ProcVar types

[Sven Barth]

On 24.03.2013 21:09, Steve Hildebrandt wrote:
> Am 22.03.2013 13:19, schrieb Sven Barth:
>> Am 22.03.2013 12:43, schrieb Steve Hildebrandt:
>>> Am 21.03.2013 22:00, schrieb Sven Barth:
>>>> On 21.03.2013 15:53, Steve Hildebrandt wrote:
>>>>> Am 21.03.2013 11:51, schrieb Sven Barth:
>>>>>> Out of curiosity: Why did you add this?
>>>>> To implement a less "hacky" way of generic method invokation.
>>>>> Supporting several calling conventions I can call a method based on
>>>>> the
>>>>> address and an array of const as parameters.
>>>>> Without RTTI there would've been a need for hard coded meta
>>>>> information,
>>>>> wich is error prone and rather time consuming.
>>>>> Since tkMethod supports RTTI any method contained in a record, class,
>>>>> object would work without additional meta information.
>>>>
>>>> Out of interest: do you support multiple platforms? Maybe if you
>>>> want to provide your code under modified LGPL we could use it once
>>>> we have the RTTI.TValue type and the extended RTTI to support
>>>> RTTI.Invoke.
>>> Sure i can provide the code. Some modifications would be needed, so
>>> it is easier to adopt it to RTTI.Invoke.
>>> Regarding the platforms:
>>>     Calling conventions 64bit : win64(the microsoft standart thing
>>> dunno how its called right now)
>>>                                        32bit : stdcall, register.
>>>     Since the inline asm code for the actual call differs depending
>>> on witch registers are used,
>>>     this part can't be handled in a generic way and needs to be coded
>>> down for each convention.
>>>     While passing parameters on the Stack works rather generic(well
>>> obtaining the stack pointer is the exception here).
>>>      Calculation of the stack offset/used register too is dependant
>>> on the convention.
>>> Hope this sums up what needs to be done to port the approach I took
>>> to all platforms supported.
>> My idea for the extended RTTI was to let the compiler write out all
>> necessary information for parameter setting (and also retrieval for
>> TVirtualMethodInterceptor support) and then use some kind of mini-meta
>> assembly language to generate the final code.
> The questing is what is harder to maintain a meta assembly or some asm
> snippets doing the calls and managing the stack.

I've not yet decided anyway what I would use...

>> Regarding the stack pointer:
>> http://www.freepascal.org/docs-html/rtl/system/sptr.html
> I did look into the sptr function but either I failed to describe what I
> need or sptr works in weird ways on win64.
> I have the following pice of code the get to the offset of the first
> parameter that will be push :
> // stackSize includes $20 for the shadow space and $08 for the method
> address since the stack should be 16bit aligned
>      asm (INTEL ASM)
>        SUB RSP, stackSize
>        MOV stackPtr, RSP
>      end;
>      // Skip shadow space to obtain the offset
>      stackPtr += $20;
>
> Now if i replace this code with sptr like this :
>      asm (INTEL ASM)
>        SUB RSP, stackSize // needs to be done so the called method
> obtains a correct value from RSP
>        //MOV stackPtr, RSP
>      end;
>      stackPtr := sptr;
>      // Skip shadow space to obtain the offset
>      stackPtr += $20;
>
> I end up with different values while the first one works for my test
> cases the second one fails miserably.

Hmm... the problem could be that "inline" does not yet work with 
assembly functions and thus "sptr" will result in the generation of a 
call which will add another shadow space to the stack pointer...

> Also I think using sptr won't do me much good, since I can't replace all
> platform dependant code with operations based one sptr.
> (e.g. implementing the skipping of the shadow space, or reserving space
> on the stack).

It was worth a try :)

>>>>> I'm currently using this to call pascal procedures and use pascal
>>>>> classes in LUA.
>>>>>> Without you showing what you changed we can not help much...
>>>>> typinfo.pp :
>>>>> tkMethod, tkProcVar: // simply added tkProcVar here
>>>>>                (MethodKind : TMethodKind;
>>>>>                 ParamCount : Byte;
>>>>>                 ParamList : array[0..1023] of Char
>>>>> ncgrtti.pas:(Line 690)
>>>>> procedure procvardef_rtti(def:tprocvardef);
>>>>> ...
>>>>> begin
>>>>>    { write method id and name }
>>>>>    if po_methodpointer in def.procoptions
>>>>>      then write_header(def,tkMethod)
>>>>>      else write_header(def,tkProcVar);
>>>>>    maybe_write_align;
>>>>>    ...
>>>>>
>>>>> So that RTTI generation for tkProcVar and tkMethod would only
>>>>> differentiate in the TTypeKind field.
>>>>
>>>> I can reproduce your problem, but I don't have a solution. At least
>>>> it is enough to only use that unit, so you could copy the unit
>>>> locally and comment everything until the problem disappears. This
>>>> way you could pinpoint what construct the problem is and then try to
>>>> fix the compiler. And if it then works without further problems we
>>>> could add this to trunk.
>>> Narrowed it down to those 2 records :
>>> type
>>>   jpeg_entropy_encoder = record
>>>     //test.lpr(23,1) Error: Undefined symbol: RTTI_$JPEGLIB_$$_DEF2
>>>     //ORIGINAL JPEGLib Line 539
>>>     //parameter MCU_data leads to the error
>>>     encode_mcu : function({cinfo : Pointer; }const MCU_data : array
>>> of Pointer) : boolean;
>>>   end;
>>>
>>>   jpeg_entropy_decoder = record
>>>     //test.lpr(23,1) Error: Undefined symbol: RTTI_$JPEGLIB_$$_DEF5
>>>     //ORIGINAL JPEGLib Line 655
>>>     //parameter MCU_data leads to the error
>>>     decode_mcu : function({cinfo : Pointer; }var MCU_data : array of
>>> Pointer) : boolean;
>>>   end;
>>> But here I'm lost again.
>> Seems to be either related to the const/var, the open array (maybe the
>> hidden high parameter?) or a combination of both.
>>
>> You could try to debug the compiler. Simply copy the above records to
>> a program unit, load the pp.lpi (or ppcx64.lpi for 64-bit), set up the
>> run parameters and the working directory correctly (if you don't have
>> a correct configuration for 2.7.1 you should use "-n -Fu..." to find
>> the correct RTL units) and then place breakpoints in ncgrtti to check
>> whether the correct data is generated.
> I fixed the issues I had with the libjpeg. Attached a patch with the
> changes I made, or should I file an issue in the bugtracker?

Regarding this fix:

+          if po_methodpointer in def.procoptions
+            then write_header(def,tkMethod)
+            else
+              begin
+                write_header(def,tkProcVar);
+                if not (def.owner.symtabletype in [ObjectSymtable, 
staticsymtable,globalsymtable]) then exit;
+              end;

It might not be the best idea to check def.owner.symtabletype, because 
we support nested types as well since 2.6.0, e.g:

=== example begin ===

type
   TTest = class
   public type
     TMyProc = procedure;
   end;

=== example end ===

I don't know immediately how you can differentiate between anonymous 
types and named ones, but that would be the key difference.

Also if you should do a bug report please reformat your patch first, so 
that it adheres to the compiler's format style (and don't change the 
formatting of other code like the one in the case except for overall 
indentation). E.g. for the above code from your patch:

=== example begin ===

if po_methodpointer in def.procoptions then
   write_header(def,tkMethod)
else
   begin
     write_header(def,tkProcVar);
     if not (...) then
       exit;
   end;

=== example end ===

Regards,
Sven