Strategies for push/merge problem?

Matt Mackall mpm at selenic.com
Mon Jul 14 11:38:55 CDT 2008


On Mon, 2008-07-14 at 11:45 -0400, Doug Philips wrote:
> On or about 2008-07-14 10:24:13, Roman Kennke inquired:
> > As soon as there are more than 1 developers working on the code, you basically have to always do the push (fail) & pull & merge & commit & push (again) dance, which is a major pain.
> 
> Why not simplify it to:
> % hg fetch
> % hg push
> 
> Of course that is assuming a very CVS point of view that you don't need to bother to check/test the result of the fetch. Perhaps if the person doing this is getting emails from the central repo, they can know that things won't break...

The only answer that actually scales is: don't use push.

This is not an accident of Mercurial's design, folks, Mercurial just
makes it more visible. It's a reality of asymmetrical nature of push and
pull. Anyone who's ever tried to push a string should not be surprised
by this.

Imagine we have a "repo" that contains a single file, with a single
integer (a counter, say). It doesn't even have to be a distributed
Mercurial repo. If we have n people trying to update the counter
themselves, they'll all have to first get the counter, update it, try to
put it back, and if it's changed in the mean time, they have to start
over. The more people that are trying to do this, the harder it gets. We
can try throwing locks at the problem, and now instead of people
fetching and updating and comparing, people simply wait. When many
people are trying to update the counter, they all spend most of their
time waiting. And if the locking isn't first-come-first-serve, some may
wait forever. And they're trying to do something a lot simpler and more
predictable than a merge! Merging is not the problem! The problem is
uncoordinated clients trying to "put" their update to a central
location.

If instead we have one person who's job it is to pull, he can simply
poll each user in turn and get their counter update and move on, without
ever having any conflicts, even if there are thousands of users. More
importantly, no one ever needs to wait.

ps: The Linux kernel used to do something very similar to the scenario
above when counting network packets, etc. It all worked fine until
someone showed up with a multi-million dollar machine with 1024 CPUs and
networking interfaces. With all 1024 CPUs trying to update the same
counter, not much actual work got done. Switching to a pull model
naturally made the problem vanish.

-- 
Mathematics is the supreme nostalgia of our time.



More information about the Mercurial mailing list