[fpc-pascal] sorting and merging array of records

[waldo kitty]

On 1/10/2012 13:13, Sven Barth wrote:
> On 10.01.2012 18:15, waldo kitty wrote:
>> On 1/10/2012 05:39, Sven Barth wrote:
>>>
>>> // somewhere else:
>>> aList.Sort(@SatDataCompare);
>>>
>>> What I wrote here might not be perfect, but it should give you a start.
>>
>> thank you very much! it has definitely given me some ideas... one thing
>> that concerns me, though, is that i'm using FPC 2.4.5 console mode...
>> this fgl unit is available in this FPC for this line?
>>
>> > type
>> > TSatDataList = specialize TFPGList<three_line_data>;
>
> Yes, this feature is available in 2.4.5 (though I would nevertheless suggest to
> you to update to the newly released 2.6.0 :) )

yes, i've got to figure out how to do that... my windows' installations are 
pulled from SVN following the procedure on one of the wiki pages but i have a 
.bat file that is scripted for this process... the only problem is that i 
generally have to start clean if i want to update fpc because something goes 
wrong during the make but i don't recall what... i do something almost exactly 
the same with lazarus (from the same page) but i manually compile it from within 
lazarus instead of trying from the command line... i'm rather scared to mess up 
what is currently a working installation :/

>> the other thing i'm trying to figure out is if i can easily and speedily
>> pull portions of the satdata fields for the sort and duplicate
>> functions... i'm thinking that it would be faster to expand the record
>> into each of its actual fields and fill them during import... i think
>> i'd rather copy(x,y,z) to individual record field during import rather
>> than pulling the needed fields during the sorting and merging phases...
>
> Yes, that might indeed be faster or better.

i implemented it by adding two additional fields to the record and filling that 
data as above... then i use those two fields in the dupe finding and comparison 
flow... i don't know if it works due to an error i just wrote about in another 
message in this thread... i'm hoping you or someone else has an idea where to 
look as i'm completely lost :(

> You could also try to experiment with TFPGMap if your satellite data has a
> unique key. You specialize it like the following:
>
> type
> TYourMap = specialize TFPGMap<YourKeyType, YourKeyData>;
>
> Where YourKeyType is the type of your key (e.g. String, Integer, etc) and
> YourKeyData is your data record. For more info I'd advice you to take a look at
> the class itself and its parent class TFPSMap, because I haven't used that class
> yet.

i think i'll leave that alone for now ;)

>> and one more trip through the raft of data can adjust the social name
>> for the satellite :)
>>
>
> Can't you adjust that when reading the data as well?

yes but it is still a loop of several hundred or thousand comparisons for each 
item in the list... i've not updated the XRF (crossreference) files in almost a 
year and there have been a lot of new birds launched... the XRF files currently 
number 37 files (of up to 1000 entries each) containing 15194 entries in 
total... not all satellites are represented because their common names are fine 
(by my standards) or they are not flying any more... the only thing this list is 
really for is to fix up the common names to what i want to use... for example 
satellite 04321 has two common names... AO-05 and Australis-OSCAR-5... amateur 
radio folk likely prefer AO-05 whereas myself and others prefer the OSCAR 
notation... it makes it easier for grouping tasks since not all OSCAR units are 
the same for the first characters... AO-06 is AMSAT-OSCAR-6 so you can see that 
the 'A' means different things... there's all kinds of common names, too... 
debris for certain ones (hundreds of entries for the chinese satellite that was 
destroyed) with a more informative name, rocket body parts, despin weights, 
shields, numerous covers, etc... [/rambling talk]

>> i'm trying to figure out how to also work out a method of deleting TLEs
>> of birds who are down... another list to process and walk through :P
>
> Are they listed in another file?

they probably will be... the XRF files have two leading spaces (i didn't design 
the format) and i could probably hijack the first space to put an 'X' if the 
satellite is no longer flying... then i could use you suggestion below to 
eliminate those entries from the list...

> If so then I suggest the following: read the file and for each record read you
> search in the list for that satellite and do a "YourList.Delete(Index)" where
> "Index" is your for-loop's counter variable (if a satellite is contained more
> than once in your list I'd suggest you to use a downwards counting loop from
> "YourList.Count - 1 downto 0" otherwise you'll get "List index out of
> bounds"-exceptions if you access the last elements (depending on how many you
> deleted). Otherwise you can cancel the loop using "Break" again.

i'll try to remember this and will likely do this during the common name 
adjustment phase... another field in the record for flying or not and then skip 
those not flying during the output file writing ;)