Tags in forests. My solution(s).

[Roman Kennke]

Hi Doug,

> On Friday, June 06, 2008, at 03:32PM, "Roman Kennke" indited:
> >Using snapshots in their own repository, the situation is much better
> >than using hg tag with respect to automatic handling of everything. The
> >autobuild processes only have to push to this small repository that
> >holds the snapshots. There's still a good chance that things need to be
> >merged, but it is almost guaranteed that this can be done without manual
> >intervention.
> 
> I'm still confused... what "things" are at risk of merge conflicts here?

I'm not saying merge conflicts. I cannot think of any merge conflicts,
unless a developer tries to set a tag that the autobuilder also sets, in
which case it actually should fail. But it still needs merging. And this
implies that at some point, the autobuilder processes have to run a
series of commands on one repository, for which I cannot guarantee that
they arrive in the right order. Suppose tagging works like hg add
tag-xyz && hg commit on the tags repository (in which case there is
really no need to merge, because we have no pull operation here). Now
what happens when 2 or more processes are doing this. There's a chance
(because hg doesn't know hg lock and hg unlock) that it ends up with hg
add tag-xyz1 && hg add tag-xyz2 && hg commit && hg commit. This might
infact still work correctly (except that the second commit is a no-op),
but I have a _very_ bad feeling about such an attempt.

> Are you saying that more than one automated process would be trying to create
> "tags" with the same name? I had assumed from your post and comments that
> each tag (of the 50) was unique...

Yes, this is not likely to be a problem.

> >... Either way, I'd have to perform multiple commands on the
> >master repository. Now imagine what happens when multiple autobuild
> >processes try to do this concurrently. Yes, I could implement some kind
> >of locking. Yes, I could implement a kind of service, that does all this
> >from only one process, but both of these solutions add more complexity
> >to the whole process, but I'd rather prefer to decrease complexity. And
> >this is exactly what the tagging-by-cloning approach does for me.
> 
> Whoa, why would you have to pull anything anywhere?
> Aren't the tags just meta-data that lives in your own separate meta-data repo?

Yeah, but I still have to either clone that, and push or changes back,
or work directly on this metadata repository as outlined above. Working
directly on the repo isn't super-fault-prove because of a lack of
locking, and pushing/pulling requires merging at some point (albeit
automatically possible), and also requires multiple operations per
process, which has the same problem with sequencing as the approach
described above.

> Right now, if I read your post and previous messages correctly, you are making
> the tags by fclone-ing into a single, well-known, centralized place.
> If that place were instead a repo in which only the auto-builders commited
> their tag updates, and from which everyone else (if they cared, and I'm not sure
> from your description if anyone but you does care) could pull?

I'm not understanding your proposal correctly. My primary problem is
that it really doesn't make sense to let multiple automatic processes
push/pull to one single repository. With the tagging-as-cloning approach
it basically works like this:
1. The autobuilder fclones the dev-master to the tag location, thus
creating a tag, let's call it xyz.
2. Then the autobuilder clones from that 'tag' clone to a temporary
location on its machine, and builds the software
3. When the build is successful, the tag is renamed to xyz-ok, otherwise
it is simply rm -rf 'ed.

I haven't found a way that is very reliable using the hg tag or hg fsnap
approach. With snapshots it is much better, but there are still too many
commands involved for my taste, and synchronization is practically not
existent in Mercurial, this calls for trouble.

> My primary assumption here is that each autobuilder has exclusive control over
> its tag(s)... maybe that assumption isn't true?

It is true. This is not the problem, the problem. The problem is that
several processes will have to modify a repository OR have to pull/push
stuff, both of which is error prone.

/Roman

-- 
Dipl.-Inform. (FH) Roman Kennke, Software Engineer, http://kennke.org
aicas Allerton Interworks Computer Automated Systems GmbH
Haid-und-Neu-Straße 18 * D-76131 Karlsruhe * Germany
http://www.aicas.com   * Tel: +49-721-663 968-0
USt-Id: DE216375633, Handelsregister HRB 109481, AG Karlsruhe
Geschäftsführer: Dr. James J. Hunt