[fpc-pascal] FPC 3.3.x breaks the Firebird Project's Firebird.pas

[Michael Van Canneyt]

On Thu, 2 May 2024, Tony Whyman via fpc-pascal wrote:

> This is a problem reported to me by an IBX user. I have not yet confirmed it 
> (need to find the time to set up the latest development branch of FPC), but 
> the issue looks solid, and is due to the following addition to TObject 
> (copied from the GitLab master branch)
>
> |TObject = class|{$IFDEF SYSTEM_HAS_FEATURE_MONITOR}
> strictprivate
> _MonitorData : Pointer;
> private
> functionSetMonitorData(aData,aCheckOld : Pointer):Pointer; inline;
> functionGetMonitorData:Pointer; inline;
> {$ENDIF}
>       ...
>
> Since Firebird 3.0, the Firebird project has provided a file called 
> "Firebird.pas" in order to provide an interface to Pascal (both Delphi and 
> FPC) from a C++ environment. There is an underlying assumption in the 
> interface. That is TObject contains no fields.

This assumption is not valid in Delphi either. It also has this field, but
it is 'hidden' at a not-fixed location.

>
> The idea is simple enough. There is a single entry point to the Firebird 
> client library with the Pascal signature
>
> function fb_get_master_interface : IMaster; cdecl;
>
> This returns a pointer to a structure that may contain fields but is 
> primarily a list of pointers to cdecl functions. These functions can perform 
> various tasks. In the case of IMaster, they can also return similar 
> structures representing other "interfaces".
>
> On the Pascal side, IMaster is defined as a class with no virtual methods and 
> a single field called "vTable" and this is set to the pointer to the 
> structure returned by fb_get_master_interface. The rest of the class 
> comprises methods used to call functions from the vTable. This makes IMaster 
> look like a normal class and hides the use of function pointers. The code is 
> generated by a program know as "cloop". All Firebird interfaces declared in 
> Firebird.pas follow the same approach.
>
> Given the assumption that TObject defines no fields, it is possible to coerce 
> the pointer returned by fb_get_master_interface to appear as an object 
> "instance" of the IMaster class, and similarly for every other Firebird 
> interface class. This coercion ceases to be valid as soon as a field, such as 
> _MonitorData is added to TObject.
>
> It is probably a valid argument to say that this is Firebird's problem as the 
> TObject definition is an internal data structure. However, this ignores a 
> serious real world problem.

The real problem is the invalid assumption.

This assumption is invalid for Delphi as well. TObject is not empty there either. 
The memory layout is simply different.

> A fix or workaround is needed and that has to be 
> in Firebird.pas. It would be a logistic nightmare to try and change the C++ 
> Firebird client DLL/SO.

It would be a logistic nightmare in FPC as well.

>
> Also, does anyone know whether a similar problem will arise with Delphi, or 
> is thus just an FPC issue?
>
> One of the following seems to be needed:
>
> a) Rewriting Firebird.pas to set the vTable field explicitly rather than 
> implicitly through coercion - ideally without breaking any user code... 
> (note: no current need to free IMaster etc)
>
> b) Taking a different approach to introducing _MonitorData e.g.

This field was introduced for Delphi compatibility:
The implementation of TMonitor requires this data.

Delphi has this field as well but "hides" the field behind all other actually declared fields.

If you examine the actual memory size allocated for TObject, you'll see that an extra field is actually present.

In FPC this turned out to be not so easy to do without modifying the compiler, 
so it was decided that the cleanest approach is to actually introduce the field as part of TObject.

The firebase people use a trick to save you some work, based on an invalid assumption.
I don't think this is sufficient reason to expect the FPC team to adapt their compiler.

The correct solution is to change the cloop program to produce a 
pascal record definition with procedural type fields at the correct offsets. 
It should be perfectly usable without too many changes.

So something like (adapt to actual procedure types)

Type
   RMaster = Record
     init : procedure(xyz:sometype);
     // etc
   end;
   IMaster = ^RMaster;

   // optionally; although it seems this already exists?
   TMaster = class
     intf : IMaster;
     procedure Init(xyz : sometype);
     constructor create;
   end;


function fb_get_master_interface : IMaster; cdecl;

constructor TMaster.Create;

begin
   intf:=fb_get_master_interface;
end;

Procedure TMaster.Init(xyz : sometype);
begin
   intf^.init(xyz);
end;

if you cannot change cloop, you can use fcl-passrc or some other tool 
to actually parse the definition generated by cloop and change it to a record, 
possibly with a wrapper class for easier access as in the above.

That should not be difficult to do, and it will work in Delphi as well.
The advantage is that it will be more robust against future changes as it
makes no assumptions about the layout of objects.

Michael.