[RFC] dirstate walk speedup

[Bryan O'Sullivan]

This patch speeds up dirstate walk performance substantially.

Here are some timings on a current kernel tree (22529 files), from my 
laptop running 2.6.22.  Numbers courtesy of "hg --time st".

TREE		COLD	SPEEDUP		WARM	SPEEDUP
		CACHE			CACHE

crew		4.62	 0%		0.90	 0%
fallback	4.38	 5%		0.78	13%
normal		4.18	10%		0.59	35%

The patch simply replaces calls to os.listdir with calls to 
osutil.listdir, which stats directory entries as soon as it does a readdir.

There are two versions of the osutil module: a pure Python version, 
which is still faster than the current code that calls os.listdir 
directly, and a C extension module that does a few extra tricks: it 
doesn't call lstat at all if it can avoid it, and it calls fstatat 
instead of lstat if available, reducing dentry lookups.

	<b
-------------- next part --------------
A non-text attachment was scrubbed...
Name: listdir-speedup.patch
Type: text/x-patch
Size: 16715 bytes
Desc: not available
Url : http://selenic.com/pipermail/mercurial-devel/attachments/20070927/4e6f60ce/attachment.bin