internal fragmentation

a personal journal of hacking, science, and technology

Extending the Mercurial protocol with pushkey and bookmarks

Mon, 28 Jun 2010 14:17 by mpm in mercurial (link)

Historically, Mercurial can send and receive precisely one type of data when pushing and pulling: changesets. As just about everything that you’d ever want to push and pull is recorded in a project’s history (including tags and branches), this works out very cleanly and simply.

But recently (ok, 2008), we’ve borrowed a concept from Git for lightweight markers that aren’t part of history and leave no mark when they’re added, changed, or removed. Git calls these ‘branches’, though Mercurial already has a notion of branches that’s much more in keeping with the permanent markers used in other systems (and Mercurial’s general philosophy of permanent history), so Mercurial has named these ‘bookmarks‘. Like real bookmarks, Mercurial bookmarks mark your place, but can be moved around or removed without altering the book.

The only problem was that there was no way to push and pull these things in the protocol, so sharing your bookmarks with other users was tedious. And as sometimes happens with open source, we had a plan for a solution, but the implementor became too busy with other things for quite a while. Fortunately, extending Mercurial’s protocol isn’t terribly difficult so a couple weeks ago, I sat down to do it myself. Here’s how I did it.

First, I introduced a generic notion called “pushkeys”. Pushkeys are key/value pairs that can be pushed to the server and listed back. There are multiple namespaces and any service or extension can register a namespace. There’s also a special “namespaces” namespace that shows the registered namespaces. To set a key, you have to also send along the old value to avoid races. The whole of is about 30 lines long and is introduced in this changeset.

Next we add the pushkey capability and pushkey/listkey command support to the local repository, the ssh client and server, and the http client and server. Finally, the debugpushkey lets us test the new support from the command line:

$ hg debugpushkey ~/hg namespaces
$ hg debugpushkey ~/hg bookmarks
test-bookmark    ff2704a8ded37fbebd8b6eb5ec733731d725da8a
$ hg debugpushkey ~/hg bookmarks test-bookmark ff2704a8ded37fbebd8b6eb5ec733731d725da8a ''
$ hg debugpushkey ~/hg bookmarks

The above shows off the bookmark namespace service for pushkeys, which is added in the next changeset. Now all that remains is to add client support for bookmarks. This has a few different pieces:

  • default push and pull behavior: update bookmarks that are already present on both sides
  • push and pull -B: copy over specific bookmarks while pulling
  • in and out -B: list unique bookmarks on the client or server

These changes can be found starting here. The improved bookmark support will show up in Mercurial 1.6, due to be released on July 1st, but you can download a copy for testing today. Note that you’ll also need bookmark support enabled on your server (services like Google Code and Bitbucket may take a bit to catch up!).

There’s more to be done with bookmarks support, and there are a lot of other interesting things that can be done pretty easily with the pushkey protocol (for instance, those horrible advisory locks CVS folks always think they want). I’m sure we’ll be seeing some interesting ideas appear here soon.


  1. I’m so glad you took the time to implement this… 🙂

    There are so many things that can be done with this — I think I’ll use it for adding cryptographic signatures to changesets after the fact.

    Comment by Martin Geisler — Mon, 28 Jun 2010 @ 15:13

  2. It would be a great idea to have a possibility to pull via this protocol only some metadata about the repo – how many revisions are there, list of tags/branches/bookmarks/heads – so that UI clients (like Eclipse) can allow users to select WHAT to pull before loading the entire repository with gigabytes of the data to the local machine.

    Comment by Andrei Loskutov — Mon, 28 Jun 2010 @ 17:28

  3. Thanks for taking over Matt. I’m sorry I was too busy to implement it.

    Comment by dsp — Mon, 28 Jun 2010 @ 18:18

  4. […] 1.6 released! New Features:  Extending the Mercurial protocol with pushkey and bookmarks var flattr_wp_ver = '0.9.11'; var flattr_uid = '20870'; var flattr_url = […]

    Pingback by Random Links #219 | YASDW - yet another software developer weblog — Mon, 5 Jul 2010 @ 15:44

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.