[fpc-devel] Speed of TTestCase

[Michael Van Canneyt]

On Sat, 14 Jun 2025, Martin Frb via fpc-devel wrote:

> I have/had several tests that took very long to run.
>
> I have no figured out that between 50% and 80% of the time, went into 
> concatenating/formatting the message for "AssertEqual" (and related).
> Yes, some of my tests now run in less than 20% of the original time.
>
> That is
> - Any concatenation my code does before calling AssertEqual
> - The call to ComparisonMsg in AssertEqual
>
> So I started overloading AssertEqual
>
> class procedure TFastTestCase.AssertEquals(const AMessage: string; 
> Expected, Actual: integer);
> begin
>   if Expected = Actual then exit;
>   inherited AssertEquals(AMessage, Expected, Actual);
> end;
>
> class procedure TFastTestCase.AssertEquals(const AMessage: string;
>   const AFormatArgs: array of const; Expected, Actual: integer);
> begin
>   if Expected = Actual then exit;
>   inherited AssertEquals(Format(AMessage, AFormatArgs), Expected, Actual);
> end;
>
> * Yes, I am aware that skips counting the amount of calls. (needs 
> "Inc(AssertCount)" in case of early exit)
> * The 2nd version eliminates any call to format in my own code.
>
>
> I wonder if it is worth adding such changes to the official testcase.

I have heaps and heaps of testcases, but I've never felt the need to investigate
the speed of the Assert* calls.

Database access and HTTP mocking is much more time-intensive in the majority
of tests I do.

>
> ------------------
>
> Or maybe it is just the way I write tests?
>
> E.g. I have a test, that computes lists, it then does
>
>   for i := 0 to count do
>     AssertEquals(format('...', [...]), AExp[i], ATestRes[i]);
>
> Additionally those lists are tested in a loop, iterating through big 
> amounts of control parameters for each test-list to be generated.
>
> So I do end up with millions and millions of calls to AssertEqual.

Millions of AssertEquals calls ? 
I'm surprised that you are surprised it takes so long? :-)

If you present a patch to avoid constructing the message in case the test is
OK, that's certainly fine for me.

But then please do it so the comparison is done only once, i.e. something like:

if aExpected=aValue then
   Inc(AssertCount)
else
   Fail(ComparisonMsg(aMessage, aExpected,aValue),CallerAddress);

Comparing long strings also takes some time...

Michael.