[fpc-pascal] sorting and merging array of records

[waldo kitty]

On 1/11/2012 20:35, Tomas Hajny wrote:
> On 11 Jan 12, at 17:46, waldo kitty wrote:
>   .
>   .
>> 1. right now the compare is working on the catalog number (TTLERec.catnbr) and
>> with duplicates:=FALSE there are no duplicates... however, i need to be able to
>> choose which record to keep when there is a duplicate catnbr but the epoch
>> (TTLERec.epoch) is different... right now it is throwing out all but the
>> first... how can i tell it how to decide which one to throw away? i saw earlier
>> that one of the parent objects has neat functions like AtPut which would easily
>> allow me to overwrite an existing record that's too old... i just don't know if
>> i can use that in the middle of an insert or a search or just exactly where i
>> would even put code to do this...
>
> The best solution is probably overriding the Insert method to fit
> your slightly modified logic. Copy the default implementation of
> TSortedCollection.Insert as found in objects.pp in FPC source tree
> and modify it according your needs (if Search returns true, perform
> the additional check for the epoch and depending on the result either
> use AtInsert (equally to the default implementation) or Dispose the
> previously stored record at position I ("I" returned by Search, "At
> (I)" used to access the previously stored record) and then use AtPut.

i've been mulling this over for a few hours and i'm wondering if overriding the 
insert method or simply doing the lofic in my importation routine would be 
best... right now my code blindly inserts the record but i could do a search for 
a matching "key" (catnbr by keyof ??) and then if i find a match, look at the 
epoch of the two records, one in vars and the other in the collection... if the 
one in the vars is newer, then do an AtPut to overwrite the one currently 
stored... otherwise, just go on to read the next file record into the vars and 
check it from there...

without looking at the "code to copy" if i want to override the insert method, 
it almost seems that there's a bug if it just throws away the record we're 
trying to insert... it would seem that if the code locates a "duplicate" record, 
it would properly dispose of unwanted data... unless i perform the previously 
mentioned logic checks manually to catch this, i don't see this happening... the 
number of records left in memory at the end of the program's execution is the 
same number as those not added because they are "dupes"...

>> 2. something else i'm running into is with duplicates:=FALSE, there's a whole
>> bucket load of records that are not disposed of when i wipe out the
>> collection... heaptrc hollers right nasty to me about'em on exit... i can only
>> assume that these are the duplicates but i don't understand why they are still
>> hanging around if insert or add threw them away already :/
>
> If you already had them in the collection, they're not added again.
> You only dispose records added to the collection at the end, but
> these are lost this way. You can also sort this out in the overridden
> Insert method (if you don't want to use the newly created record,
> dispose it).

as above, i don't know that they are /in/ the collection... i'm (currently) 
simply calling the insert method and leaving the work up to it... if it should 
be handling this gracefully, it isn't... at least not in a way that heaptrc likes ;)

>> [TRIM]
>>>> data^.catnbr := Copy(data^.satdata[1],3,5);
>>>> data^.epoch := Real_Value(data^.satdata[1],19,14);
>>>> inc(sat_cnt);
>>>> aTLECollection^.insert(data);
>>>> dispose(data);
>>>
>>> Don't do this! You'll free the memory you allocated for your record. The
>>> collection will only contain a pointer to this data! (Many of the rules I
>>> mentioned for T(FP)List apply here as well)
>>
>> uh? for some reason i thought that the insert was copying the data to a place
>> and then setting the pointer to there for the collection... i tried with and
>> without the dispose(data) but it still looked the same... of course i was tired
>> and might not have been looking at the right writeln debug output...
>   .
>   .
>
> No, Insert doesn't do any copying by default. In your current code,
> the copying is performed by calling the PTLERec constructor (in
> "New(PTLERec, Init(...") but if the pointer isn't inserted, it is
> thrown away currently.

thrown away by the default object insert code? why doesn't it properly wipe it 
out since it is tossing the pointer into the bitbucket?

> If you use the overridden Insert method as
> suggested above, you can Dispose it from within the Insert call if
> you don't need to insert it.

understood... and with an eye on the above discussion about doing the logic 
myself using the available routines, i could also take care of it there by 
deciding to AtPut, or, insert or dispose instead of blindly performing an insert 
and hoping for the best??

i'll probably have broken my code by the time you read this...  but i'll very 
likely be attempting to implement the logic in my Input_Satellite_List routine 
;) OB-)

i just gotta figure out the way that the search and such work as in if they are 
returning pointers to the record or the record data itself... pointers are still 
very alien to me :? :/