[fpc-pascal] TProcess read buffer size
Ryan Joseph
genericptr at gmail.com
Fri Jul 16 16:59:32 CEST 2021
> On Jul 15, 2021, at 8:33 PM, Dennis Lee Bieber via fpc-pascal <fpc-pascal at lists.freepascal.org> wrote:
>
> https://flylib.com/books/en/3.126.1.111/1/
>
> """
> Only writes below PIPE_BUF bytes in size are guaranteed to be atomic.
> PIPE_BUF is 512 bytes on Mac OS X. The fpathconf() system call can be used
> to retrieve the value of PIPE_BUF given a pipe descriptor.
> """
>
> Granted you seem to be reading -- but the other end may be what is
> controlling the size of each packet (and may somehow even have set the pipe
> overall size to just one "buffer").
>
> Unless you can examine the writing side, or source code of your read
> method...
Yes I think you're right.
~$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 256
pipe size (512 bytes, -p) 1
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 2784
virtual memory (kbytes, -v) unlimited
It's just a PHP script that dumps 64k chunks but if the pipe buffer is 512b then I'm stuck. Is there a lower level library I can use to set the pipe size? I was curious if this is a bottle neck for me. Currently it takes about 7 seconds to read a 600MB file from one process to another, which is insanely slow.
Regards,
Ryan Joseph
More information about the fpc-pascal
mailing list