[fpc-devel] What do you think about TSQLScript using 'TextReader'?
silvioprog at gmail.com
Sun Jun 14 19:29:25 CEST 2015
A few weeks ago, I took a look at the TSQLScript (that IMHO is a very nice
class!) source, and I noticed that it seems slow for processing big files.
I did some tests with this class. My SQL file: 90 MB, with CREATE TABLE,
CREATE PROCEDURE, CREATE TRIGGER, ALTER TABLE, several INSERT INTO and
more. Using the current TSQLScript implementation, the execution of this
file is very slow, and the CPU/memory usage is extremely large(*), but
after make some changes in the original TSQLScript source code, adapting it
to use this new class (recently applied in FCL), I got a significant
difference, ie, the CPU/memory usage now is amazing less, increasing the
performance +-80%. =) The answer to the significant gain is simple: using a
textreader, the TSQLScript class read the SQL (from string, stream or file)
by demanding (line by line, and using a low level byte reading too),
instead of loading an entire file to a buffer in a stringlist.
So, after this explanation, what do you think about TSQLScript using
textreader? I can send a patch showing this idea as a real implementation,
of course, keeping the 'Script' as a stringlist, just adding a new
property, called 'Source' and declared as 'TTextReader'.
An observation: I suspect it has a small bug at this line: the
'Directive' variable could break the loop skipping for the next line after
its assignment. OK, if it is a bug, it is very easy to fix.
(*) In a specific case on a Linux server, the kernel killed my executable
while importing my SQL file, see more details here.
My public projects - github.com/silvioprog
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the fpc-devel