[fpc-pascal] File Enumeration speed

[Michael Van Canneyt]

On Sat, 28 Jul 2012, SteveG wrote:

> I am enumerating thru large numbers of files on my disk, and find I cant come 
> close with findfirst / findnext to matching the speed of cmd line apps 
> available in linux :eg ls / du

A regular ls only does a getdents() call.

FindFirst/FindNext does a getdents, but then additionally, per file in the result, a stat() call.

> I have a fairly tight file search function, and dont see how to gain more 
> speed
>
> Would anybody know what the limiting factors would be ?

The number of calls to stat() to get extended file information.

I suspect that if  you do a ls -l, it will be as slow as findfirst/findnext, because it does then 3 calls per file:

from strace ls -l /etc I get:

lstat("/etc/odbc.ini", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
lgetxattr("/etc/odbc.ini", "security.selinux", 0x14de920, 255) = -1 ENODATA (No data available)
getxattr("/etc/odbc.ini", "system.posix_acl_access", 0x0, 0) = -1 EOPNOTSUPP (Operation not supported)

If you want speedier operation, and have enough file information with the name, you can simply do a getdents().

> does the operating system keep an index somewhere ?

Normally not (at least other than the regular disc cache).

Michael.