About encoding manipulation in Mercurial

[Matt Mackall]

On Tue, May 12, 2009 at 01:18:25PM +0800, LAO Zhongcheng wrote:
> 
> Matt Mackall wrote:
> >On Mon, May 11, 2009 at 04:38:27PM +0800, LAO Zhongcheng wrote:
> >  
> >>Hi,
> >>I'm a newbie in Python.
> >>
> >>Is the file mercurial/changelog.py the core API of Mercurial.
> >>I found there are some callings to fromlocal
> >>in the line listed below of changelog#add method.
> >>208 user, desc = util.fromlocal(user), util.fromlocal(desc)
> >>
> >>I suppose that if a Mercurial extension is going to call this method,
> >>it has to pass a username and description in default locale
> >>(system locale or HGENCODING environment).
> >>Am I right?
> >>    
> >
> >Correct. However, you should -not- try to coerce user or desc to local
> >encoding if for some reason they're not in local encoding. Instead,
> >you should temporarily change util._encoding.
> >
> >  
> Do you mean that before I change util._encoding to UTF-8 before calling 
> the API,
> then calls it, and restore it to the old one at the end?
> I don't think it is a good idea for practise.
> 
> I expect Mercurial provides a much easier way like:
> 1) Enforing any call to core API to be in UTF-8. Outside the core API
> (command line front-end / extension), use a wrapping implementation if 
> you accept user input
> (either editor or command line) and performs a conversion on your own.

This is counterproductive when the vast majority of Mercurial
-intentionally avoids any translation of strings-. There are only a
very small set of things that we ever transcode: username, commit
message, and branch and tag names. I suggest you do a count of all the
users of fromlocal and tolocal in the Mercurial source.

> 2) Provides an argument easy to switch between UTF-8 and local encoding 
> assumptions.
> 
> This will be helpful when you implement an extension being encoding-aware.

It's not clear that this is a common (or even desirable) thing. There
are currently only two such extensions shipped with Mercurial: convert
and highlight.

-- 
Mathematics is the supreme nostalgia of our time.