[fpc-devel] The future of fpmake

[Marco van de Voort]

In our previous episode, Jonas Maebe said:

> > However I consider this a transitional problem only.  We are still in
> > the raw plumbing and discovery phase.  In time, I assume some
> > autogeneration helper functionality will emerge to make it at least
> > (way) more bearable.
> 
> That's all I'm asking for. However, auto-generation will exclude  
> directly modifying fpmake.pp (just like you currently can't safely  
> directly edit the Makefiles).

It remains FPC source :-) The autogenerated part can be in an .inc file, and
included from fpmake.pp

Don't forget to include that .inc in the non-source files section though
(exact details obviously not worked out yet) :-)

> > OTOH the trouble should not be exaggerated. I did the entire packages/
> > hierarchy in a few hrs spread over 2 days, and that was without any
> > previous fpmake experience at all, and communicating with Joost.
> 
> There's still a difference between initial conversions and maintenance  
> afterwards. The first time there are no potential older changes you  
> might accidentally overwrite.

True. Still I don't think it is any different from updating makefile.fpc
with autogenerated unit data. In both cases it is including a block of text
into the rest of the file. And at least fpcmake has the .inc workaround
possibility.

Provided that you can actually generate the content of course. But in
straight cases (files compiled/accessed don't depend on target) this should
be generatable from the output of the compilation of a buildunit.

And the buildunit can be fed with find . -name "*.pp |awk print ...  just
like the makefile.fpc can.

> > New packages are not added that often,
> 
> At least the package I referred to (univint) tends to change  
> significantly over time. The Mac OS X 10.6 SDK split a bunch of  
> headers and removed others compared to previous versions (most C  
> programmers use umbrella headers, so they don't notice that either).  

Maybe more an apache solution? univint105,univint106 etc, and only install
one?  (see bits below before you comment on multiple editing)

Speaking of which, I've been thinking about this, and maybe we should
installed most packages one level deeper (e.g. units/$fpctarget/pkgs),
except for versioned packages like httpd/apache. This way the pkgs patch can
be recursively included, and the versioned ones explicitely, with way less
chance on duplicate files in the searchpath.

> As Carbon gets phased out, more and more things will probably change.  
> It's not really feasible to manually go through the current list of  
> dependencies and to update them where necessary. Otoh, replacing them  
> with a newly auto-generated list currently risks removing code that  
> someone else added to manually to fix something.

There is a "law of conservation of trouble" (wet van behoud van ellende)
going on here. Splitting packages, means doing fixing multiple times,
supporting multiple versions with one set means complicated packaging and
building. Take your pick, trouble each way

But IMHO this is a problem and a tradeoff of this package. The complications
of this package are not the problem of the build system. With any
generalized systems there are winners and losers.

Still, univint _DOES_ seem to meet the criteria to generate fpmake data from
buildunit compilation output.  winunits-jedi is another such case, and
probably more OS header sets.

> > The trouble is that makefile.fpc does not contain all information.  It
> > is not a closed system, and has barely enough info to build, and depends
> > on the autobuilding capacity of the compiler.  Something that can't be
> > used for packaging and other features.
> 
> That's why I mentioned "or any other structured format".

Not generalizable. So only for packages where files used and
dependancies on units are not dependant on target. There you can parse
compiler output of a buildunit. (and that could be considered the structured
format, the uses line can be generated using a simple script)

> > customization of the template.
> >
> > I can't exactly remember if this also handled the include situation.  
> > Afaik
> > first the units were done, and later includefiles were added.
> 
> Unless it's fully automatic so that the current set of dependencies  

The information that must be fixated is quite complicated. This means a
fully automatic way is very, very complicated.

> can just be replaced with a new set of generated ones, that's not  
> really doable for univint, I think.

As said univint (still) falls into the trivial case.

If it gets more complicated, you need source level scanning. IOW parsing
uses and {$i } directives, but somehow follow all nodes of $ifdef
statements.  Afaik the current fcl-passrc doesn't allow this, since it
handles preprocessing like the compiler, only evaluating the TRUE parts. 

That program should then also get info on include dirs etc. It would be a
nice exercise in tree building :-)

But I hope you realize that I make this last bit up as I go. I doubt we ever
will earn back such an investment.  Damage control with scripting/generation
for often used scenarios, and hard labour in the rest is the only real
solution.