[fpc-devel] Managed Types, Undefined Bhaviour

[Thorsten Engler]

> -----Original Message-----
> From: fpc-devel <fpc-devel-bounces at lists.freepascal.org> On Behalf Of
> Mattias Gaertner
> 
> > [...] COM interface methods can't logically not be virtual,
> 
> I think you are confusing things here. They can be virtual or not
> virtual in COM and CORBA.

I think a lot of people simply don't understand how interfaces are implemented.

A decade ago, I wrote some articles about interfaces:

https://www.nexusdb.com/support/index.php?q=intf-fundamentals
https://www.nexusdb.com/support/index.php?q=intf-advanced
https://www.nexusdb.com/support/index.php?q=intf-aggregation

Looking through these now, I see that there are details missing about how Delphi (and I assume FPC) actually implements interfaces. (The following assumes that the read has some understanding about interfaces that can be gained from the articles above.)

An interface being "a pointer to a pointer to an array of method pointers" raises the question, "where is the memory that the 2nd pointer in that chain is stored in?"

When you define a class like:

type
  ISomeInterfaceA = interface(IInterface)
    ['{7C6DC303-0D93-4212-971F-6EACA1B97015}']
    procedure SomeMethod;
  end;

  TSomeObjectA = class(TInterfacedObject, ISomeInterfaceA)
  protected
    {--- ISomeInterfaceA ---}
    procedure SomeMethod;
  end;

The in-memory layout of that class is something like:

VMT                              : Pointer { to array of method pointer};
InheritedFields                  : Array[x] of Byte; // fields inherited from TInterfacedObject
TSomeObjectA_ISomeInterfaceA_VMT : Pointer { to array of method pointer};

So if you have a TSomeObjectA instance and get an ISomeInterfaceA from that, what you get is a pointer to that TSomeObjectA_ISomeInterfaceA_VMT field of that particular instance of TSomeObjectA.

Which raise the next question, what's the value of that TSomeObjectA_ISomeInterfaceA_VMT and where does it come from?

The compiler, at the time it compiles TSomeObjectA will create a static array of method pointers (for all the methods in the interface, including ancestors) and store a pointer to that in the RTTI of the class.

When the an instance of TSomeObjectA is created, as part of the work done before the constructor is run, the RTL will go through the RTTI of the class, find out about all the "hidden interface VMT pointer fields" and initialize them to point at the array of method pointers the compiler generated. So all instances of TSomeObjectA will always have exactly the same value in the hidden TSomeObjectA_ISomeInterfaceA_VMT field.

Which raise the next question, what code exactly are the method pointers in that compiler generated TSomeObjectA_ISomeInterfaceA_VMT array pointing to?

A valid ISomeInterfaceA VMT is expected to contain a pointer in the SomeMethod slot to something like:

procedure(Self: ISomeInterfaceA);

But the code for TSomeObjectA.SomeMethod is:

procedure(Self: TSomeObjectA);

So the compiler clearly can't just put a pointer to TSomeObjectA.SomeMethod into the SomeMethod slot of the TSomeObjectA_ISomeInterfaceA_VMT. Instead, the compiler needs to create code for a hidden trampoline that looks like this:

procedure TSomeObjectA_ISomeInterfaceA_SomeMethod(Self: ISomeInterfaceA);
begin
  TSomeObjectA(PByte(Self)-Offset_of_TSomeObjectA_ISomeInterfaceA_VMT_in_instance_data).SomeMethod;
end;

So the code can reconstruct the TSomeObjectA pointer because the position of the TSomeObjectA_ISomeInterfaceA_VMT field (which is what Self: ISomeInterfaceA points to) is always at a fixed offset from the class VMT (which is what a TSomeObjectA variable points to).


And now we can get to the "COM interface methods can't logically not be virtual" statement.

A call against ISomeInterfaceA.SomeMethod is always "virtual". Yes. It always involved looking up the appropriate method pointer in the interface VMT and then calling that. So you can have different ISomeInterfaceA that will have different code pointers in the SomeMethod slot of their VMT.

But that virtual call only gets you to the compiler generated trampoline. Which then reconstructs the object pointer, and makes the actual call against the object method. And THAT call can be static or virtual, depending if the method in the class is virtual or not.

Let's continue with the code example from above:

type
  ISomeInterfaceB = interface(ISomeInterfaceA)
    ['{1BB48CC2-A2AF-4C7E-A798-288B0F30F04F}']
    procedure SomeOtherMethod;
  end;

  TSomeObjectB = class(TSomeObjectA, ISomeInterfaceB)
  protected
    {--- ISomeInterfaceA ---}
    procedure SomeMethod; //not virtual!

    {--- ISomeInterfaceB ---}
    procedure SomeOtherMethod;  
  end;

The resulting memory layout of TSomeObjectB will be:

VMT                              : Pointer { to array of method pointer};
InheritedFields                  : Array[x] of Byte; // fields inherited from TInterfacedObject
TSomeObjectA_ISomeInterfaceA_VMT : Pointer { to array of method pointer};
TSomeObjectB_ISomeInterfaceB_VMT : Pointer { to array of method pointer};

TSomeObjectA_ISomeInterfaceA_VMT will be initialized to exactly the same value when creating a TSomeObjectB instance then when creating a TSomeObjectA instance.

The SomeMethod slot in that VMT will point to exactly the same TSomeObjectA_ISomeInterfaceA_SomeMethod code I showed above. And because TSomeObjectA(...).SomeMethod; is a static call, it will really call *TSomeObjectA*.SomeMethod, not TSomeObjectB.SomeMethod.

TSomeObjectB_ISomeInterfaceB_VMT will contain a pointer to a totally different VMT than TSomeObjectA_ISomeInterfaceA_VMT. Because the offset between TSomeObjectB_ISomeInterfaceB_VMT and the class VMT is different than between TSomeObjectA_ISomeInterfaceA_VMT and the class VMT the compiler has to create new trampolines for all methods in ISomeInterfaceB (and ancestors).

So the SomeMethod slot in TSomeObjectB_ISomeInterfaceB_VMT will point to this code:

procedure TSomeObjectB_ISomeInterfaceB_SomeMethod(Self: ISomeInterfaceB);
begin
  TSomeObjectB(PByte(Self)-Offset_of_TSomeObjectB_ISomeInterfaceB_VMT_in_instance_data).SomeMethod;
end;

Again, this is a static call, so TSomeObjectB(...).SomeMethod; will call TSomeObjectB.SomeMethod and nothing else.

IF SomeMethod was defined as virtual in TSomeObjectA and override in TSomeObjectB, then the TSomeObjectA(...).SomeMethod call in TSomeObjectA_ISomeInterfaceA_SomeMethod would be a virtual call, and if the ... is actually a TSomeObjectB, then it would end up calling TSomeObjectB.SomeMethod.


And to have a look at the other possibility, redeclaring the interface:

  TSomeObjectBToo = class(TSomeObjectA, ISomeInterfaceA, ISomeInterfaceB)
  protected
    {--- ISomeInterfaceA ---}
    procedure SomeMethod; //not virtual!

    {--- ISomeInterfaceB ---}
    procedure SomeOtherMethod;  
  end;

The resulting memory layout of TSomeObjectBToo will be:

VMT                                 : Pointer { to array of method pointer};
InheritedFields                     : Array[x] of Byte; // fields inherited from TInterfacedObject
TSomeObjectA_ISomeInterfaceA_VMT    : Pointer { to array of method pointer};
TSomeObjectBToo_ISomeInterfaceA_VMT : Pointer { to array of method pointer}; 
TSomeObjectBToo_ISomeInterfaceB_VMT : Pointer { to array of method pointer};

Notice that there is an additional TSomeObjectB_ISomeInterfaceA_VMT, this does not affect TSomeObjectA_ISomeInterfaceA_VMT, which will continue to be initialized to exactly the same value when creating a TSomeObjectBToo instance then when creating a TSomeObjectA instance.

The compiler will prepare VMTs for both TSomeObjectB_ISomeInterfaceA_VMT and TSomeObjectB_ISomeInterfaceB_VMT, each with their own trampolines.

The SomeMethod slot in TSomeObjectA_ISomeInterfaceA_VMT will continue to point to the TSomeObjectA_ISomeInterfaceA_SomeMethod I showed above.

var
  SomeObjectBToo : TSomeObjectBToo;
  SomeObjectA    : TSomeObjectA;

  IntfB                  : ISomeInterfaceB;
  IntfA1, IntfA2, IntfA3 : ISomeInterfaceA;
begin
  SomeObjectBToo := TSomeObjectBToo.Create;
  SomeObjectA    := SomeObjectBToo;

  IntfB  := SomeObjectBToo; // TSomeObjectBToo_ISomeInterfaceB_VMT

  IntfA1 := SomeObjectBToo; // TSomeObjectBToo_ISomeInterfaceA_VMT
  IntfA2 := SomeObjectA;    // TSomeObjectA_ISomeInterfaceA_VMT
  IntfA3 := IntfB;          // TSomeObjectBToo_ISomeInterfaceB_VMT

You have now managed to get 3 different ISomeInterfaceA, all for the same TSomeObjectBToo instance!


I hope this clears up some of the subtle issues with interfaces...

Cheers,
Thorsten