[fpc-devel] FPCUnit problem - do we fix or rewrite?

[Michael Van Canneyt]

On Tue, 19 May 2009, Graeme Geldenhuys wrote:

> Hi,
>
> I use FPCUnit extensively at work and in open source projects like
> tiOPF, but I hit a bit of a snag - actually a bug in FPCUnit which
> causes many tests to fail for the wrong reasons. Here is one such case
> - in the tiOPF project many database tests fail do to "Setup Once"
> issues: http://opensoft.homeip.net/tiopf/fpcunit/index.html
> Michael van Canneyt has also hit this bug before, but now I'm not sure
> what exactly to do about it.
>
> The bug has to do with running a "setup" or "teardown" only once. Yes
> FPCUnit has the TestDecorator which adds this support, but if you have
> a nested test hierarchy, the running once falls over. When doing unit
> tests that share DB connections this issues becomes a major pain.
>
> Michael had a more in-depth look at the issue and told me that the
> problem lies in the heart of FPCUnit's design and is not very easy to
> solve. So much so, that he recommends it is better to simply rewrite
> FPCUnit than try and patch the existing code. I have not taken such a
> close look to the issue, but had to modify my test hierarchy to get
> around the problem (which should really not have been needed in the
> first place).
>
> Michael, do you still feel the same way about this issue?  Dean Zobec,
> are you still around? FPCUnit was your baby to start with. Do you have
> any comments?

I still feel the same way.

> I'm at cross-roads now. Do I try and solve the problem using the
> current code, or do I collaborate with Michael and Dean (and anybody
> else interested) and start a rewrite? If a rewrite is the preferred
> option, I have a few questions

For me, a rewrite is the preferred option.

>  * how will this affect all the existing FPCUnit users?

It should not affect thim, this should be a dogma...
After the fix, the decorator stuff should work 'as designed', but
the rest should remain the same.

>  * will (or must) the new design have a similar API so our existing
> test suites will continue to run.

All code should run as-is.

>  * Do we make it more DUnit compatible? I already created many DUnit
> compatibility methods, but some things are not possible.

We'll have to look at this on a per-feature basis.

>  * DUnit2 (hosted in the tiOPF code repository and written by the
> late Peter McNab) has some major improvements over the existing DUnit.
> Do we incorporate/port that, or merge the DUnit2 code into a new
> testing framework that is FPC and Delphi compatible?

I'm not very comfortable taking over someone else's code, and as
Marco points out, there may be a license issue.

>  * Obviously a timeframe is also important
>  * Do we use Interfaces (like DUnit and DUnit2) or do we use abstract
> classes like FPCunit currently does.

I'm afraid I have seriously bad experiences with interfaces in FPC, 
and so would not like to see them used.

Michael.