[fpc-pascal] Heavy heap fragmentation issue

Martok listbox at martoks-place.de
Sun Jun 2 20:49:24 CEST 2019


>> can you break it down into a test program and post?
> 
> I should probably put the project on Github anyway, it's rather small, and it
> shouldn't have any dependencies. Give me an hour to clean it up a bit.
This should be good enough. It's been only used by like four people so the UI is
a bit of a mess, but you'll only need one button to see the problem.

So, here are the steps to reproduce:
- Get the code from
  <https://github.com/martok/ImageHash/tree/bug-fpcmm-fragment>
- Compile (originally written with FPC trunk >= late 2017, best use
  current trunk; LCL version shouldn't matter much)
- The default mode puts the exe in bin_i386, right next to the DLLs
- Run
- Provide data (~10 3MB JPEGs should do, or just paste one 10 times), either:
  - paste the full path to a directory containing a number of images
    in the top-center memo (no, there's no SelectDirectory dialog)
  - put the images in a folder 'data', that gets auto-added in this version
- Click 'Read'
- Wait for it to find visually similar images in your input
- Compare memory as reported by task manager or ProcExp to HeapStatus
  that is printed in the bottom-left log memo.

Note that the number of threads used (defaults to HostCPUs-1) doesn't matter
(can be changed down to 1), so it's not a concurrent alloc problem. You can also
apply the attached patch to even run the comparator in serial, but that doesn't
change anything either.

-- 
Regards,
Martok

-------------- next part --------------
diff --git a/uFrmMain.pas b/uFrmMain.pas
index a4425da..8d89f14 100644
--- a/uFrmMain.pas
+++ b/uFrmMain.pas
@@ -253,6 +253,7 @@ begin
     SetLength(loaders, seThreads.Value);
     t1:= GetTickCount64;
     btnRecompare.Click;
+    fClassifier.Suspend;
     lbStatus.Caption:= 'Setup loader...';
     for i:= 0 to High(loaders) do begin
       loaders[i]:= TImageHashThread.Create;
@@ -286,6 +287,7 @@ begin
       loaders[0].Start;
     end;
     WaitForMultipleThreads(@loaders[0], length(loaders), @Application.ProcessMessages, @fAbortFlag);
+    fClassifier.Resume;
     WaitForMultipleThreads(@fClassifier, 1, @Application.ProcessMessages, @fAbortFlag);
     t2:= GetTickCount64;
     meLog.Lines.Add('time:  %dms',[t2-t1]);


More information about the fpc-pascal mailing list