[fpc-pascal] Unit testing for RTL packages

[Sven Barth]

Am 24.01.2021 um 20:58 schrieb Noel Duffy via fpc-pascal:
> On 24/01/21 11:53 pm, Sven Barth via fpc-pascal wrote:
>> Am 24.01.2021 um 08:37 schrieb Noel Duffy via fpc-pascal:
>>> How should one go about adding unit tests to a package in the RTL? 
>>> In many packages, for instance fcl-db, there is a tests folder with 
>>> fpcunit units and programs in them, and they're referenced in 
>>> fpmake.pp as example programs, but these do not appear to be built 
>>> or run during a standard build of the RTL. Separately, there are 
>>> tests for some packages in the top-level tests/test folder, which do 
>>> appear to be run with
>>>     make full TEST_FPC=path_to_your_compiler
>>>
>>> None of these latter tests use fpcunit.
>>>
>>> Do any of the fpcunit tests get run as part of a test cycle?
>>
>> Historically the tests for packages reside in 
>> tests/test/packages/<package>. If you need to add a new package you 
>> need to add it to the TESTPACKAGESDIRS variable in tests/Makefile.fpc 
>> and regenerate the Makefile.(*)
>>
>> Two, three years ago I also added the ability to have the tests 
>> directly in packages/<package>/tests (it started with the Rtti unit). 
>> If you add a new package this way you need to add it to the 
>> TESTPACKAGESDIRECTDIRS variable in tests/Makefile.fpc and regenerate 
>> the Makefile.(*)
>>
>> In *any* case you have to keep in mind the following:
>> - for a test to be picked up by the testsuite they need to start with 
>> a "t" and end in ".pp" (this means that you should prefix units that 
>> you don't want the testsuite to compile independantly with a "u" 
>> instead of a "t" or suffix them with ".pas" instead of ".pp")
>> - if it's a program it should return exit code 0 on success and 1 on 
>> error (libraries and units are not executed, they are only checkey 
>> for correct compilation)
>> - all rtl/packages units are available (which means that you can use 
>> fcl-fpcunit for example, though you should use the 
>> "consoletestrunner" (see also 
>> packages/rtl-objpas/tests/testrunner.rtlobjpas.pp))
>>
>> (*) invoke trunks' fpcmake utility (located in 
>> utils/fpcm/bin/<cpu>-<os> after a complete build) with the parameter 
>> "-Tall" inside the tests directory
>
> Thanks for the detailed reply. I will try this out later today. I'm 
> not actually adding a new package, just making changes to fcl-net, and 
> my tests already use consoletestrunner, so it should be 
> straightforward. (I know, famous last words!)

I meant "new package" in the sense of "not yet part of the list of 
tested packages", cause not every package is.

Regards,
Sven