[fpc-devel] FPC test suite and ALGLIB

[Jonas Maebe]

On 11 Dec 2009, at 10:42, Sergey Bochkanov wrote:

> Hello, Jonas.
>
>> Thanks, I've added them to svn. I cursorily tested them and they at
>> least work under Mac OS X/x86_64 and Mac OS X/PowerPC.
>
> Is  it  possible  to  test  them  under  other  platforms? SPARC, ARM,
> PowerPC64?

The tests are part of the nightly test suite now, so they are tested  
on all platforms for which nightly tests are performed. They seem to  
have worked fine on all properly tested platforms (ARM is not one of  
those though).

> However,  there  is  a  problem  you  should  know  about. ALGLIB uses
> non-deterministic  unit  tests. I.e.  test sets are generated randomly
> (instead  of  being  built-in, pre-generated). This approach have both
> benefits  and  drawbacks.

At the very least, the value of randseed should probably be written  
out on startup so that the test can be reproduced under the same  
circumstances if it fails.

> So we have two options:
> 1. to use only the most stable units
> 2.  to  give  me limited access to SVN so I may run these tests myself
> and study their results.
>
> I  have  enough  time  on my hands so I can help you with tests if you
> want.

Everyone has read-only access to the tests in svn, since they're part  
of the regular fpc svn repository: http://www.freepascal.org/develop.var#svn

To run the testsuite, first perform a "make clean all" in the FPC  
directory, then change to the tests directory and run

make FPC=compiler_to_compile_testsuite_utilities TEST_FPC=/full/path/ 
to/fpc/compiler/ppc386 TEST_OPT="-O2 -Cppentiumm" clean all digest

(replace ppc386 with ppcx64 in case of x86_64 platforms, and the  
contents of TEST_OPT with whatever options you want to use to compile  
the tests). Afterwards, you an find a list of failed tests in output/ 
$fulltarget/faillist, and output/$fulltarget/longlong contains the  
full compiler/test output).

This will always do a full testsuite run though (which takes quite a  
while to complete, even on a fast machine). There's currently no way  
to only execute a subset of the testsuite. The alglib tests are in  
tests/test/alglib.

If you want, I can give you svn write access to tests/test/alglib so  
you can easily add/change the tests there though.


Jonas