[fpc-devel] TFieldDef.Size vs TField.Size

[Joost van der Sluis]

Please, be patient. I'm working on it, as you can see in the bug reports
and commits.

As I said earlier, widestringfields aren't available in fpc yet. Someone
wrote some code for it, but it is never properly tested and probably
only worked by accident.

mse is another story, they have their own implementation.

But I don't commit everything in one go, it's work-in-progress. And I
need to write tests for everything that I commit. You mixed two things
in one patch, so I had to split it into pieces. 

I did some tests with different Delphi versions, with and without
unicode. And beside finding some bugs in XE's string-db code I think I
know a solution for all problems regarding strings and unicode.

But let me thing it out, write tests, write code and test if it works
indeed. 

On Thu, 2011-02-24 at 08:02 +0100, LacaK wrote:
> Hi,
> I am writting here to discuss bug
> http://bugs.freepascal.org/view.php?id=17268
> (I do not want reopen bug and writte there because I am not sure about
> my arguments)
> 
> IMHO root of problem is in different definition of TFieldDef.Size and
> TField.Size
> Documentation says, that
> 1. TFieldDef.Size should be "For string and byte fields, Size is the
> number of bytes reserved " 
> [?]http://docwiki.embarcadero.com/VCL/en/DB.TFieldDef.Size

I don't care what the documentation of others say. It is about how we
implemented it and how we document things. 

I've read the documentation and did some tests. The documentation is
simply wrong here. The behavior changed with different versions of
Delphi, the documentation is never adapted.

As I wrote in the bug-report, TFieldDef.Size should contain the number
of bytes to hold the string, without the leading 0.

Note that with the unicode-versions of Delphi have in TFieldDef.Size the
amount of characters * 4 in TFieldDef.Size. For WideStrings and normal
strings.

I'm thinking about adding a property to T(Buf)Dataset in which you can
specify if you want to use unicode-length strings, or 'normal' lengths.
So that we can add backwards-compatilibity, which Delphi doesn't.

> 2. TField.Size should be "the maximum number of characters in the
> string" (for ftString and I guess that also for ftWideString)
> [?]http://docwiki.embarcadero.com/VCL/en/DB.TField.Size
> [?]http://docwiki.embarcadero.com/VCL/en/DB.TStringField.Size

That's simply true. Also for widestrings. (tested)

> And now let's go look into fields.inc:
> 1. Function TFieldDef.CreateField copies Result.Size:=FSize (so
> TField.Size:=TFieldDef.Size)
> 2. function TStringField.GetDataSize returns for ftString ... Size+1
> 3. function TWideStringField.GetDataSize returns for ftWideString ...
> (Size + 1) * 2

Those functions have to be adapted, yes. I have the changes here but I
need to fine-tune the tests, clean up etc. 

> So also here we can see, that FieldDef.Size is expected to be number
> of characters not bytes.

This was the case in the past. And it didn't lead to problems as
widestrings were never used properly. (That's why no-one bothered how it
was implemented for widestrings)

> So IMHO logical conclusion will be say, that TFieldDef.Size for string
> fields has same menaing as Field.Size, so it is number of characters
> (so documentation is wrong in this case ... also fast test in Delphi
> shows, that FieldDef.Size=Field.Size=number of characters)

No, our implementation is wrong and we have to adapt our code. That way
it is also possible to fix the unicode-problems you've reported.

ps: if you want to know what the Delphi-XE-bug is. when using the
TClientDataset, for widestrings, TFieldDef.Size is four times the amount
of characters (ok), but when you get TWideSTringField.Datasize the
answer is two times TfieldDef.Size +2. So if you have a wide-string of
40 characters. TFieldDef.Size is 160, but TField.Datasize 322.
(obviously wrong)

Joost.

Joost.