[fpc-pascal] FreePascal Windows - Force files to write to disk

[James Richters]

I am still having this issue.  I've managed to narrow down what is happening
some.   The problem is my data is not actually being written all the way to
disk.   I now have a repeatable proven method to reproduce the issue.  Here
is the sequence.

In my freepascal console application, I create the file
Assign file
Rewrite file
Writeln file 
Writeln file
Writeln file ...
Flush file
Close file

I then immediately read back the file, and all is good.  I even look at the
file with notepad++ to see that the data is correct.... and it is fine.

If I then power off the test computer (not doing a correct windows shutdown,
this simulates an unattended power failure) when it boots back up, my file
has the correct date and time and the correct length, hovever the contents
of the file is a string of $00 with no other characters, no carriage returns
and no linefeeds, however the number of $00s is EXACTLY how many bytes the
file should have been.  It's like the file was allocated but data not
actually written to disk yet... but I thought Close (file) was supposed to
do that.    I think I am dealing with some kind of disk caching going on
here, but I'm not sure how to force my files to commit all the way to disk.
I should note that these systems are all using Samsung SSDs in them, perhaps
there is some SSD weirdness going on?     I also notice it's not just this
one file that is affected, but EVERY SINGLE FILE I create with freepascal,
this is the only one I notice because the other ones are all work files that
end up being re-created every time I run my program, but I checked them all
and they are all exactly the same..  correct lengths but all data is a
string of 00s   

I'm thinking of just disabling all write behind caching in windows and on
the SSD if I can figure it out,  it's not needed anyway anymore because SSDs
are way more than fast enough, and I would rather wait for my data to be
written NOW instead of it NOT getting written later!!    The problem with
solving this by disabling write caching is that I can never be sure that
this was done.  So I would like to find a real solution that would force my
files all the way to disk on my own without depending on the system being
configured a certain way.   I really need these tiny files to survive power
failures.

Thanks for any advice on how to do this

James



-----Original Message-----
From: fpc-pascal-bounces at lists.freepascal.org
[mailto:fpc-pascal-bounces at lists.freepascal.org] On Behalf Of James Richters
Sent: Friday, February 24, 2017 1:53 PM
To: 'FPC-Pascal users discussions' <fpc-pascal at lists.freepascal.org>
Subject: Re: [fpc-pascal] FreePascal Windows - Force files to write to disk

Thanks for the advice everyone.   It is very much appreciated.   I have been
working on a combination of ideas here.  Currently I am doing an MD5sum of a
string created from all my variables concatenated together and writing that
as the second to the last line of the file, I also am writing some fixed
text at the beginning and end of the file as suggested so I can check for
variables being overwritten in memory, great idea!   I am then reading the
file back to verify it has written properly, making a backup, checking that,
and also using ioresults as below. I will probably rename the file as well,
and exclude it from virus checkers.  When I read the file I can verify the
MD5sum and if that test fails, I'll just automatically try the backup, If
that also fails the MD5sum I'll notify the user.  

Thanks again for all the help

James
-----Original Message-----
From: fpc-pascal-bounces at lists.freepascal.org
[mailto:fpc-pascal-bounces at lists.freepascal.org] On Behalf Of Klaus Hartnegg
Sent: Thursday, February 23, 2017 6:33 AM
To: fpc-pascal at lists.freepascal.org
Subject: Re: [fpc-pascal] FreePascal Windows - Force files to write to disk

Am 21.02.2017 um 22:12 schrieb James Richters:
>  Assign(BitFile,'BitSave.pax');
    {$I-}
>  ReWrite(BitFile);
    if ioresult <> 0 then
       writeln ('error opening pax file');
>  WriteLn(BitFile,XADJ:1:8);
>  WriteLn(BitFile,YADJ:1:8);
>  WriteLn(BitFile,ZADJ:1:8);
>  WriteLn(BitFile,WADJ:1:8);
>  WriteLn(BitFile,AADJ:1:8);
>  WriteLn(BitFile,TADJ:1:8);
>  WriteLn(BitFile,VADJ:1:8);
>  WriteLn(BitFile,UADJ:1:8);
>  WriteLn(BitFile,CurrentTool);
>  WriteLn(Bitfile,P_Value[4]);
    writeln (bitfile, 'EOF');
    if ioresult <> 0 then
       writeln ('error writing pax file');
>  Close(BitFile);
    if ioresult <> 0 then
       writeln ('error closing pax file');
    {$I+}

This writes something at the end of the file that does not come from a
variable. Next time the error happens, you can check if that is present. 
Then your variables were overwritten in memory. Also it checks for errors.
And I would disable antivirus or at least tell it to not scan this file.
_______________________________________________
fpc-pascal maillist  -  fpc-pascal at lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

_______________________________________________
fpc-pascal maillist  -  fpc-pascal at lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal