internal fragmentation

There's a new book out by a pair of architects that's triggering a lot of clueless pop science news articles with their claim that the ecological impact of a dog is higher than an SUV.

But none of their methodology actually stands up to any scrutiny (never mind their numbers). Their argument is that feeding a dog (as measured in calories) requires more hectares of land to sustain than an SUV (as measured in joules). By looking at the amount of land theoretically required to produce x calories of food and comparing that to the amount of land to theoretically produce y joules of energy, they arrive at a ratio. Simple enough?

The basic problem is that oversimplified theory and practice don't agree. First, most dogs' diets consist almost entirely of food that would otherwise be waste product, as it's not fit for human consumption. So taking a measure for land productivity for human food production and using that as a proxy for a dog's eco-footprint is basically meaningless. Their actual food impact is so small it's hard to measure. Also note that dogs are "built" from renewable materials with very low upfront energy costs.

In contrast, SUVs aren't built from renewable materials and don't run on renewable fuels so their footprint can't be measured. There are no solar-powered SUVs and the energy efficiency of biodiesel is extremely low (and thought to be negative in many cases). And if we started running all the SUVs in the world on today's biodiesel, there would be no land left to grow food (there are already food riots being caused by ethanol demand). So the effective eco-footprint of an SUV is tremendously higher than the theoretical fraction of a hectares per year they quote.

Compare these two meaningless numbers and you get a result that is orders of magnitude out of touch with reality. If your instinct tells you that a 15kg dog chasing a ball around the yard and basking in the sun uses less of the world's resources than an 2500kg SUV roaring down a freeway or sitting in traffic, you're right.

Spoke with our legal counsel from SFLC this morning and determined that we should probably not hold up the release process any further. So the 1.4 feature freeze will be this Saturday, followed by a release on November 16th. This release will still be under the old GPLv2 only license, but will have SVN conversion support enabled.

The relicensing process is mostly about having all our ducks in a row to make distributors and co-developers happy. We're 98% done, but the end game is hard due to people that can't be located. I expect we'll have this wrapped up in time for the 1.5 release on March 1st.

The best way to help make the 1.4 release a success is of course to run the latest development version from the main repository and report any problems you encounter. There are also nightly builds available for Windows users. We generally keep our development tip end-user ready at all times (and do all our own day to day work with it), so testing tip is pretty safe.

I spent most of October arguing with people about licenses, talking with lawyers, and tracking down contributors for our big relicensing effort but we haven't quite finished yet so I'm letting our scheduled Nov 1st date slide. If it's not sorted out in a couple weeks, we'll cut a release anyway.

A lot of the work that I hoped would happen for 1.4 didn't get done, but we've been getting a steady stream of nice usability improvements. The one I'm most excited about is the new summary command, which combines the output of a bunch of commands in one place, giving you a quick view of what's going on in your repo:

$ hg sum --remote
parent: 9665:1de5ebfa5585 tip
 walkchangerevs: drop ui arg
branch: default
commit: 15 unknown (clean)
update: (current)
remote: 1 or more incoming

You can follow me at mpmselenic.

There's some fuss being made over OS X 10.6 using units derived from powers of 10 rather than powers of 2 to report drive size. But I think it's high time we joined the rest of technical world in the 20th century and adopted the metric system.

First off, note that there are hundreds, probably thousands of technical domains where the kilo prefix means 1000. Even in the domain of computer science, it is usually assumed to mean 1000. Consider: megahertz, kilobaud, gigabit, megapixel, MIPS, gigaflops. In fact, the only place where power of two nomenclature can be taken for granted is in the size of memory devices, where the wiring of address lines dictate that only powers of two are possible. Claiming that disk sizes should be measured the same as RAM is invoking a false consistency: RAM is the odd one out and always has been.

Further, there's essentially no reason for anyone but electrical engineers and kernel developers directly concerned with the geometry of RAM to have any exposure to the powers of two issue at all. No one writing a web app should care, no one writing a word processor should care, and certainly no one using a word processor should have to care. For the vast majority of computer users there is no upside to measuring things in units of 1024, only confusion.

Finally pushed a .1 release out the door. We had a surprising number of performance regressions related to the cleanups in 1.3, so everyone is encouraged to upgrade.

In the future, we're going to have to find a way to get more real users on big projects to test pre-release code, which means: easily available nightly builds.

In keeping with our time-based release plan, Mercurial 1.3 is out the door. I'm pretty pleased with it.

Exciting things in the queue for 1.4 including an improved wire protocol and possibly a new hashing algorithm. And hopefully support for a couple non-native subrepository types.

My grandfather passed away earlier today. Thinking back, it amazes me the degree to which he set the foundation for my later pursuits and hobbies: astronomy, physics, mathematics, electronics, computing.

He was a researcher for many decades at Bell Labs. He was intimately involved with the Apollo program, not to mention countless other efforts ranging from sonar to the invention of videoconferencing. He was there for the invention of the transistor.

He was colorblind (a trait he passed to me), but memorized the colorblindness eye charts in attempt to qualify for the astronaut program. He taught me the names of the constellations. And he took me to see Star Wars in the theater.

He also introduced me to Unix via a home-built VT52 terminal that dialed into a server at Bell Labs, the birthplace of Unix. I spent countless hours exploring; I retrieved the Amulet of Yendor at age 7. Not long after that, he sent me a first edition copy of The C Programming Language, also created by his coworkers. I quickly read it cover to cover, despite having no access to a C compiler. Thirty years later, I'm hacking the Linux kernel in C.

I saw him for the last time a couple weeks ago. I showed him the bash shell prompt on my GPhone, a machine thousands of times more capable and portable than the machines he introduced me to, and running a version of Unix I'd helped develop.

Looks like Mercurial: The Definitive Guide has finally hit paper. Looks like it's set to ship to coincide with Mercurial's planned 1.3 release on July 1st.

Before I caught the free software fever, I spent most of a decade working on laser and imaging-based measurement. And I had the fortune of spending that time working with an incredibly talented group at a small company called CyberOptics. At every place I've worked since, I've always been a little disappointed that the local talent didn't quite measure up to their level.

The founder of that company was Steve Case, a professor of optics turned entrepreneur. Today I learned he died in a solo plane crash. He was a good guy.

Also worth mentioning there's now video for the recent Google I/O talk on adapting Mercurial to BigTable. A pretty decent introduction to both Mercurial internals and their BigTable technology (see Hbase from the Hadoop project for an open alternative).

With less than a week before Mercurial's 1.3 code freeze, I've pushed two interesting new features. The first is the share extension which can be thought of as an even lighter-weight local clone. The core support for this has been in place for years now (since 0.9.2), we'd just never gotten around to making it available as a command.

Also, Mercurial now has support for subrepositories which is a way to automatically pull other projects such as libraries into your working directory as part of a larger project. Commits made at the top level repository will recursively record the state of the subrepos, effectively allowing atomic commits across multiple projects.

I've also managed to design the code in such a way that it should be quite easy to extend this to support subrepositories from other systems like SVN or git.

There's a good summary of my smem tool and presentation at LWN today.

LWN has a good write-up of my ELC kernel development panel comments.

Yesterday, I heard the OpenOffice.org project is running a pilot program for migrating to Mercurial.

And today I hear Google is adding Mercurial support to Google Code. They've apparently implemented a Mercurial storage backend based on BigTable to take advantage of their cluster infrastructure.

I got my first Rubik's Cube some time around 1980, shortly after they were released in the US, and spent countless hours trying to solve it. Eventually, I stumbled on a solution that was satisfactory for a kid my age: take it apart and put it back together.

Many years passed without me thinking much about the Cube until I read a paper about computerized searches to find the minimum number of moves to solve the cube. Not long after, I discovered the Mirror Cube and the Void Cube and that reawakened some toy lust a mild obsession with actually properly solving the Cube from scratch.

And late last night, I happened upon a classic Cube at the local pharmacy.

Here's roughly how I went about solving the cube:

• Solve the top layer (~1 minute)
• Solve the middle layer (~5 minutes)
• Solve the bottom layer (~3 hours)

The bottom layer is obviously the hard part. My strategy was to find various patterns of moves that scrambled the bottom while keeping the top layers intact and figure out how to do useful things with them. These aren't too tricky to find, but making sense of them is.

This part pretty much requires paper. First, write down a series of moves in a useful notation (eg F' for turn front face counter-clockwise). Then record the pattern of cubes on the bottom before and after applying the sequence. Now it's possible to work out what the combination of moves is doing in the abstract. For instance, this sequence flips the north and west edges, rotates the northwest and southeast corners clockwise, and the southwest corner counter-clockwise.

The things that can be wrong with the last face include: edges rotated, edges out of position, corners rotated, and corners out of position. Most of the simple patterns will change all of these at once. But combining sequences (and their mirror images) will let you cancel out some the unwanted changes in each sequence to narrow in on the solution.

That's one childhood dream down, now to become a writer and an astronaut.

A voice search button just appeared on the main page after the software update on my G1. Handles things like 'mercurial' and 'salem witch trial' and 'error 10053' pretty well but interprets 'Matt Mackall' as 'mathematical'.

Just finished a talk a bit ago at ELC 2009 on my new smem tool, which gives more meaningful measurements of memory usage of Linux's shared virtual memory system than the numbers reported by traditional tools like top, ps, free, etc. My slides are here.

Looks like Guido van Rossum, Python's Benevolent Dictator For Life, has chosen Mercurial for future Python development. More here.

Bryan O'Sullivan has finally announced that his excellent Mercurial book has been picked up by O'Reilly. Naturally, it will continue to be available under an open license.

I've just announced that Mercurial will be dropping support for Python2.3 with our 1.3 release in July. I've been rather reluctant to do this on the general principle that applications (and programming languages in particular!) should be stable enough to be usable over, say, 5 years. But it's now gotten to the point where Py2.3 hasn't been available in Debian for two whole releases (in other words, forever), so I can't actually test against it without building my own copy of Python.

On the upside, this means we'll get to start using a small handful of nice features added in Python 2.4. Namely decorators, sorted(), and generator expressions.

There's a lot of buzz going around about these videos, but really, you shouldn't miss them. Especially this one.

A quick aside about Linux and Youtube: I've had mixed results with swfdec for in-browser viewing, so I often use clive to download videos and view them outside my browser (and nicely buffered). I find "xsel | clive" is quite handy to grab a link I've copied to the clipboard.

The Mario Kartel is pleased to announce Puzzle Quest 4: Your Princess Is In Another Castle.

Dropped in on the first official meeting of the Twin Cities Makers. Seems like a fun group, even if they like to use webforums. Also, it took about an hour and a half of free-ranging gabble before we actually settled down to some business.

My local green-oriented coffee shop has started putting out candles at dinner time. It struck me as odd that they still thought burning things for light was green, so I've decided to do the math on their power efficiency and carbon impact.

A typical votive candle is about 2 ounces of paraffin and burns for about 15 hours. Its brightness is approximately 1 candela or 12 lumens and consumes power (in the form of paraffin fuel) at a rate of about 40W. That's one or two (or 15!) laptops' worth of power per candle.

Paraffin (like most fuels) is nearly all carbon by weight and when burned it generates more than its weight in carbon dioxide (12 atomic units of carbon from the candle + 32 atomic units of oxygen from the air per CO2 molecule). About .2kg total or about 13g per hour. Not much, but put a couple dozen of these out every night for a year, and you're beginning to talk about a substantial amount of carbon.

Now let's compare this to a 14W compact fluorescent bulb (as bright as a 75W conventional bulb). This generates about 840 lumens of light. That's equivalent to 70 candles. If we hooked this up to a dirty old coal-fired power plant (generating 1050g of CO2 per kWh), we'd generate about 14.7g of CO2 per hour - almost the same as our candle! If instead we used wind power (readily available in Minnesota) and include the amortized impact of building the wind turbines (something like 22g/kWh worst case), that drops to less than .3 grams per hour.

In other words, the global warming impact of candles is on the order of 60 to way more than 3000 times worse than fluorescent lighting for the same amount of light.

This doesn't consider heat though. In the winter, a candle is quite an efficient heat source as almost all of its energy is converted to heat. So we end up with 13g of CO2 per hour per candle to get 40 watts of heat. That same heat from an electric heater powered by our coal worst-case would result in 42 grams of CO2/hour. But with wind power, it'd be less than 1 gram of CO2 per hour in the worst case.

In the summer, there's no upside: candle heat battles against air conditioning. A single candle produces enough heat to cancel out more than 1% of a so-called "ton" of refrigeration (12000 BTUs / hour).

Lastly, we should look at production cost and impact. Light bulbs of any variety almost certainly win, simply because they have lifespans of thousands of hours and are so much brighter. It would take something like 45000 candles to match both the brightness and lifespan of our typical CFL bulb.

So wish for a greener tomorrow and blow out those candles.

Mercurial 1.2 went out the door a bit ago. It's the first release following our time-based release plan. All in all, it went pretty well, though I did have to delay a few days so that we could get some test builds made. Hopefully by 1.3 (due July 1st), test builds for most platforms will be automated.

Not electronics, but another thing I've been experimenting with. This is a granola recipe reduced to its bare minimum for ease of preparation.

Preheat oven to 200°F
In a large pot, mix 1/4 cup each of sugar, cooking oil and maple syrup
Add 1/2tsp salt (optional)
Heat on stove until bubbly
Reduce heat and fold in 4 cups rolled oats
Place pot in oven for 1 hour

If you're the type of person who likes to ruin things with raisins (and we are, but we ate them already), add 1/2 cup of 'em when you're finished.

• Twin Cities Maker group meeting March 11th
• Details for using Mercurial over Freenet
• Neutrino data suggest the nearby SN1987A may be an instance of the currently theoretical quark star, a middle ground between neutron stars and black holes
• Eric Drexler blogs about the National Space Society's proposal to focus on visiting Near-Earth Objects
• Wired has one of the better backgrounders on the current financial crisis I've seen.
  It assumed that correlation was a constant rather than something mercurial.

Mercurial has now entered a code freeze in preparation for next week's 1.2 release. This is going to be be a few days later than hoped as there were some issue with getting Windows test releases built that needed sorting out. Ahh, Windows..

Some students sat down across from me today at the coffee shop and one of them managed to break her flash drive while trying to transfer some files. Some panic ensued, as the drive apparently contained the only copy of her class project. After watching much fiddling, I offered to take a look at it. The connector was quite loose. I pried it open and as I suspected all the connector pins were pulled right off the board.

I tried aligning the pins and applying pressure across them to get the drive to mount, but no joy. Its LED blinked, but Linux wasn't detecting a new USB device. Stiffer measures were called for. I took it home and fired up the soldering iron.

I could tell that simply resoldering the connectors wasn't going to work, otherwise my first attempt would have succeeded. Copper pads had been pulled off and I was going to need to make some brand new connections. I pulled out some 30 gauge wire-wrap wire and proceeded to strip, flux, and tin all the leads. Hopefully I wouldn't melt or burn anything else on this board in the process.

All hooked up but again: no joy. Not even a flashing LED. So I unsolder everything, wick up as much solder as I can, take some close-up photos and poke around with a multimeter and determine that three of four pads are no longer connected to the controller chip. The copper they pulled up had broken the trace to the pad.

This was going to be tricky. I was going to have to solder wires directly onto the controller chip's legs, which were even smaller than the thinnest wire I have, not to mention much smaller than my cheapo soldering iron. I call Angela to let her know the prognosis was not good.

It was pretty hard to see even in macro photos which pins the broken pads were leading to so I found a datasheet for the flash controller (Phison PS2231) on the net and double checked. Vcc (pad 1) to rightmost pin, and data in and out to pins 5 and 6. Right next to each other and crossing. Fun fun fun!

I set about refluxing and retinning all of my leads and putting a bit of flux on the chip leads as well. Then I soldered the leads on. Vcc was easiest - I set the lead against the outside edge of the first leg and touched the iron above the lead end for about a second and it stuck. Next, I touched the third lead to the solder pad at the bottom of leg five and again briefly heated it to get another fragile connection. Lead two I attached to the upper shoulder of pin 6 to get maximal distance from the previous connection.

I carefully maneuvered my laptop into position to plug it in without disturbing the fragile connections.. and it worked! Drive mounted properly and all files recovered. Not bad for a software guy.

Then I got to explain to an art student what Linux was and what the difference was between megabytes and gigabytes.

Phillip Burgess has a clever approach to interfacing with external 2C devices from your laptop: just plug them into your VGA port. The DDC channel that the monitor uses to report its available display modes is just a subset of I2C. Even if you have no idea what I'm talking about, be sure to check out his fabulous compatibility chart.

This suggests another super-cheap method for programming AVRs. Use one AVR as an I2C slave, and allow the master (ie the laptop) to send down commands to drive its four remaining pins, which are attached to the appropriate pins on the device to be programmed. This is significantly easier than USB and available on most machines, though not without some software compatibility issues mentioned above.

Today's hack is an in-circuit programmer for breadboards. All my photos of my breadboard thus far have been cluttered with the programming wires. This arachnid-looking contraption lets me easily remove all that clutter. See my note on programmers from yesterday for the pin-out. Here's the spider cable with its victim:

An IC test clip would also do the job, of course, but I don't have any.

Method 1: The simplest method is the so-called 'DAPA' programmer, which directly controls the programming pins via a PC parallel port. For this to work, you must have a 'real' parallel port, and not a USB adapter.