[fpc-pascal] New FPCUnit db results writer: testdbwriter: request for... testing

[Reinier Olislagers]

Hi list,

I've been working on a listener for FPCUnit that writes test results to
a database.

I intend to use it to run e.g. database tests on my Jenkins CI server
whenever a new build is done. Therefore, the db is quite normalized in
order to limit used space.
I've included a flattening view that denormalizes the output into a grid
suitable for human consumption.

The unit:
- stores results in a database, specified by db
type/server/name/user/password in testdbwriter.ini
- the testing program/test runner can override these connection settings
- can create a Firebird embedded database and use it if Firebird
driver+empty hostname is chosen, or as a fallback if no db info is
specified at all
- stores CPU+OS model (in the format FPC detects them) by default into
the test run table; can be overridden (e.g. to not store them)
- saves application name (can be overridden) and allows adding a comment
field into the test run table
- optionally stores a revision id/application version field per test
run. Useful for doing regression testing
- stores test suite names and test results, as well as any
exception/error messages (e.g. reason for test failure).
- can run in principle against any sqldb RDBMS as it uses simple SQL

SQL included for PostgreSQL and Firebird server to create the db; should
be easily adaptable to other environments.

Simple example included in examples directory.

I've also included a dbtestframework workalike (dbtests2db) with units
that runs the FPC 2.6.1 r22717 database test framework; only plug in
database.ini for the database test connection and testdbwriter.ini for
the storage option (can be the same server, but different db) and go.

I have tested it on Firebird, Firebird embedded and PostgreSQL.

Test results, bug reports, improvement suggestions and patches are all
welcome.

Next up for me is to integrate this into my Jenkins server and see how
it works. Based on this heavier testing, some improvements may be apparent.

Also, more queries/views will need to be defined in order to get
statistics and regression tests.

Finally, while the database is ready for multi-level/nested test suites,
the code is not yet. It simply uses the lowest test suite name as the
name. While that is sufficient for the database tests, it will need to
be improved as well.

Download:
https://bitbucket.org/reiniero/testdbwriter
download tab

Thanks very much,
Reinier