[fpc-devel] overflow error in Fpopendir

[Seth Grover]

I am debugging an issue with FPC 3.0 on Linux x86_64 which I believe I have
tracked down to Fpopendir rtl/linux/ossysc.inc:

-----------------------------------------------------------
function Fpopendir(dirname : pchar): pdir;  [public, alias :
'FPC_SYSC_OPENDIR'];

var
  fd:integer;
  st:stat;
  ptr:pdir;

begin
  Fpopendir:=nil;
  if Fpstat(dirname,st)<0 then
   exit;
{ Is it a dir ? }
  if not((st.st_mode and $f000)=$4000)then
   begin
     errno:=ESysENOTDIR;
     exit
   end;
{ Open it}
  fd:=Fpopen(dirname,O_RDONLY,438);
  if fd<0 then
   exit;
  new(ptr);
  if ptr=nil then
   exit;
  new(ptr^.dd_buf);
  if ptr^.dd_buf=nil then
   exit;
  ptr^.dd_fd:=fd;
  ptr^.dd_loc:=0;
  ptr^.dd_size:=0;
  ptr^.dd_nextoff:=0;
  ptr^.dd_max:=sizeof(ptr^.dd_buf^);
  Fpopendir:=ptr;
end;
-----------------------------------------------------------

In this process I have a LOT of open file handles. The system on which it
runs uses "ulimit -n 64000" to set the maximum number of open descriptors
to 64,000. However, once I get above 32768 file handles Fpopendir starts
failing, and causing me to leak directory descriptors. I didn't have debug
compiled down into the RTL, but looking at the machine code I can see
what's happening:

Fpopen is returning a good directory descriptor, for example, 51345.
However, the comparison "fd<0" is evaluating to true, which causes the
routine to error out.

Looking at the machine code:

   0x00007ffff7038106 <+118>:   callq  0x7ffff7037ee0
<SYSTEM_$$_FPOPEN$PCHAR$LONGINT$LONGINT$$LONGINT>
   0x00007ffff703810b <+123>:   mov    %eax,%ebx
   0x00007ffff703810d <+125>:   cmp    $0x0,%bx
   0x00007ffff7038112 <+130>:   jl     0x7ffff703816b
<SYSTEM_$$_FPOPENDIR$PCHAR$$PDIR+219>

At +123 I can see that $eax/$ebx contains the correct descriptor, 51345.
However, when I print $bx in the debugger, it looks like a negative number,
as if it had overflowed a signed 2-byte integer, so the function errors
out. This causes the file descriptor to be leaked as well, since no
reference to is is retained in the pdir record.

I'm a little bit confused at this, as fd is defined as an "integer," which
I thought was going to always be a 32-bit signed value on this system.
However, the FPC documentation tells me that this can be changed based on
compiler switches, so maybe somewhere up in one of the other .inc files
there is some flag making "integer" behave as a signed 16-byte number?

I am inclined just to patch my copy of the FPC rtl and change the
definition of fd from "integer" to "longint," but I wanted to get your take
on it first.

Thanks,

SG

--
Seth Grover

Be kind to all of your neighbors
Because they're just like you.
And you're nothing special
Unless they are too.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freepascal.org/pipermail/fpc-devel/attachments/20160630/a58c8c13/attachment.html>