hg 1.1 - cvsps memory consumption

[Frank A. Kingswood]

Arne Babenhauserheide wrote:
> When I tried to convert the main repository, I ran into another problem 
> though: My system ran out of memory and once cvsps couldn't get more memory, 
> it grew extremely slow while still in rlog gathering mode. 
>
> Virtual memory consumption was at 2.3 GiB, reserved memory at 1.4 to 1.7 GiB, 
> and I had to stop it after 3 days to be able to work with my computer again (I 
> only have 2GiB of physical memory). 
>
> Is there room for reducing the memory footprint, or is it already optimized to 
> the maximum (so that it just needs to manage that much data)? 
>   
I'm sorry you bumped into memory issues, I saw this is a very large 
repository. Some commits go back 15 years!

Most of the data in memory are the filename and log message strings, and 
they are shared between duplicate copies. Further savings would be very 
hard to come by.

I guess you could tackle one top level directory at a time, and then 
merge the resulting repositories, but that would potentially split commits.

CVS rlog has a date option which is not used except to speed up 
incremental conversions. You could hack the source quite easily to do an 
partial conversion. See hgext/convert/cvsps.py, around line 180. It 
already sets a starting date for the incremental conversion, you could 
add code for setting an end date.

Frank

-- 
------------------------------------------------------------------------
Frank A. Kingswood                      frank at kingswood-consulting.co.uk
Cambridge, United Kingdom                               +44-7943-217 571