<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-GB link="#0563C1" vlink="#954F72"><div class=WordSection1><p class=MsoNormal>Hi everyone,<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Firstly, I realise this is an old subject, but I still can’t find any easy answer.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Really, the question is simple: Is there ANY way of TSQLQuery NOT reading the entire result set into memory?<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>This is really killing me. I’ve got quite a lot of web code using sqldb and it works really well. The classes for connection, transaction, query etc. are perfect for general use and make it so easy to write database applications. I find it excellent that I only need to change the connection class in my code to use a different database.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>For web code (mine included), fetching a few dozen, hundred or even a few thousand rows is no real problem. The sqldb classes are *<b>very</b>* fast and very easy to use.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>The problem comes when I need to write some apps to either analyse or move a lot of data. Initially all I need to do is analyse some data by simply reading through rows, doing some calculations and other analysis. For even 1 million rows this is very fast (10 seconds using MySQL) so no major problems. However, it does use quite a lot of memory.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Further use of this simply explodes. I have tables for analysis (and later extracting/moving) that are over 40 million rows. This currently is simply not possible in fpc, at least using the excellent sqldb classes. After blowing 16GB of RAM+swap it takes a while to recover.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>To be fair I’ve tried looking through the code and see how it’s implemented, but I can’t see any easy way of avoiding the memory buffering of the entire result set.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Apart from re-implementing a lot of classes or going down to the driver level I can’t currently see any easy way round this. I know there is some old discussion on this, but I’m kind of hoping that there may have been a few changes or something recently that I’ve missed.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Any help appreciated.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Regards,<o:p></o:p></p><p class=MsoNormal>Gary.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p></div></body></html>