New patch series:
intro:   http://selenic.com/pipermail/mercurial-devel/2008-August/007774.html
patch 1: http://selenic.com/pipermail/mercurial-devel/2008-August/007775.html
patch 2: http://selenic.com/pipermail/mercurial-devel/2008-August/007776.html

just adding a pointer to Jesse's last version of his patch he posted on 2008-06-30
(just for reference):
http://selenic.com/pipermail/mercurial-devel/2008-June/006983.html

See also http://selenic.com/pipermail/mercurial-devel/2008-August/007494.html
(latest patch series does not work any more)

Yet another patch series ["introduce fncache repo layout (rev 2)"] which would
solve Issue839 and Issue793:

intro: http://selenic.com/pipermail/mercurial-devel/2008-July/007333.html
patch 1: http://selenic.com/pipermail/mercurial-devel/2008-July/007335.html
patch 2: http://selenic.com/pipermail/mercurial-devel/2008-July/007334.html
patch 3: http://selenic.com/pipermail/mercurial-devel/2008-July/007336.html
patch 4: http://selenic.com/pipermail/mercurial-devel/2008-July/007337.html
patch 5: http://selenic.com/pipermail/mercurial-devel/2008-July/007338.html
patch 6: http://selenic.com/pipermail/mercurial-devel/2008-July/007339.html

Yet another patch series:

http://selenic.com/pipermail/mercurial-devel/2008-July/007206.html
http://selenic.com/pipermail/mercurial-devel/2008-July/007205.html
http://selenic.com/pipermail/mercurial-devel/2008-July/007207.html

would solve this issue and works fine.

Yet another patch was posted here:
http://selenic.com/pipermail/mercurial-devel/2008-July/007155.html
("filenamelog repository layout").

I've made a proof of concept patch that uses the longpath Windows file \\?\.. API.
I can clone and checkout long and reserved paths on windows (example repo:
http://www.cadifra.com/cgi-bin/repos/long2/)

I may update this patch for a while at
http://www.cadifra.com/cgi-bin/repos/hg-longpath/file/tip/longpath.patch.

Problem with this patch is, that explorer can't copy nor delete such
repo directories. As a workaround, robocopy could be used for copy
and I made a small python script for deleting long path trees
(available at http://www.cadifra.com/cgi-bin/repos/winrm/ ).

See also Issue793

Isn't this a critical bug? For us it's a showstopper. We can't even delete or
move the offending files because they're still in the revision history.

I have met the same problem trying to use Mercurial as an adhoc remote backup 
system. It is very easy to reproduce: try to clone a repository which contains 
the Documents And Settings directory, it generally has  deeply nested folder 
structure. Mercurial expands this folder structure even more by escaping the 
upper case chars. 

I suggest that the files stored in the repository have their names hashed 
instead of escaped (MD5?). To avoid too many files to clutter one directory, 
make two levels of folders after the first two chars of the hash.

Ie: .hg/store/a/b/abba72727727.i

Also:
- .hgignore should be case insensitive
- add a switch to continue in case of locked file.

Micke: if you can't open some of the files in the checkout with Notepad, clearly
the project structure is too deep to begin with. Very likely something fixable
on the NetBeans side (I don't know enough about the WSDL support to comment in
detail).

Since the Netbeans project has included support for Mercurial in the GUI we are
looking at moving our sources from CVS to Mercurial. For one of our projects
created in Netbeans we are having this problem. 

I created a repository on a linux server by converting a CVS checkout with "hg
convert" and published it with hgwebdir.cgi. When I try to clone it on my
windows workstation (Windows XP) I get the error "abort: The filename or
extension is too long". The filename hg clone complains about is:

D:\hg_clones\minsta_orderkvantitet\.hg\store\data/_min_order/src/conf/xml-resources/web-service-references/_min_order_service/wsdl/xxxxxxxxxxxxx.xxxxxx.xxxx.xxxxx__8080/_min_order_service_service/_min_order_service/____container__publishing__subctx/_m_e_t_a-_i_n_f/wsdl/_min_order_service_service__schema1.xsd.i
(311 chars)

I tried the workaround to set usestore = False in Mercurial.ini but I still get
the error but the file name has shortened a little:

D:\hg_clones\minsta_orderkvantitet\.hg\data/MinOrder/src/conf/xml-resources/web-service-references/MinOrderService/wsdl/xxxxxxxxxxxxx.xxxxxx.xxxx.xxxxx_8080/MinOrderServiceService/MinOrderService/__container_publishing_subctx/META-INF/wsdl/MinOrderServiceService_schema1.xsd.i
(276 chars)

I have tried both the Mercurial 1.0 binary installer and to install from sources
using python 2.5.2 and mingw32. I get the error in both cases. According to
http://bugs.python.org/issue542314 it should be possible to use longer filenames
in python 2.5 by using the prefix "\\?\".

For us this is a showstopper since it is our developer tool (Netbeans) that
creates this deep directory structure if one use web services so we cannot do
anything about it. The only thing we can do is to clone to a directory with an
extremely short path, for example "D:\m", but that feels like a strange
workaround since both the OS and Netbeans supports these kind of directory
structures and it limits our choice where we can store our cloned repositories.
This workaround also is dependent of the names of the included web services so
if a web service with an extremly long name is included the workaround doesn't
work anymore.

Inside my CVS checkout I can navigate to the deepest directory with explorer or
in a command window but I can't open the *.xsd file (total filename length = 271
chars) in notepad and the context menu is missing most of the alternatives
including delete. Still Netbeans obviously can create and open the file and also
commit the file to CVS. The issue with files that cannot be used by "normal"
programs is up to the user to handle. If the user has tools that creates these
kind of directory structures Mercurial should be able to handle them.

Interesting indeed. Using Vista, I made a very long path - just repeat 'mkdir
0123456789' and 'cd 0123456789' until you get an error. 'mkdir morestuff' from
cmd.exe at this point fails. Yet using Python 2.5.1, I was able to

os.mkdir('\\\\?\\' + os.getcwd() + '\\morestuff')

and successfully create a subdir. Back in cmd.exe I could even cd into this
subdir, and open it in the Explorer, yet it was still impossible to create a new
text document in this folder using the context menu. (Trying to prepend \\?\ to
the path in the address bar had no effect.)

Furthermore, creating slightly longer directory names, or files using
open(...,'w').close(), did not work. It is worth noting some quite serious
caveats in

http://msdn2.microsoft.com/en-us/library/aa365247(VS.85).aspx

such as

"Relative paths are limited to MAX_PATH characters.

When using the API to create a directory, the specified path cannot be so long
that you cannot not append an 8.3 file name (MAX_PATH - 12).

The shell and the file system have different requirements. It is possible to
create a path with the API that the shell UI cannot handle."

In fact, it seems that the \\?\ trick only actually "buys" you a few more
characters, and is not a solution at all.

Window supports filenames up to 32K characters.  See
http://msdn2.microsoft.com/en-us/library/aa363858.aspx and
http://msdn2.microsoft.com/en-us/library/aa365247(VS.85).aspx for info.

Python appears to support this as of 2.5*.  Maybe support for people using 2.5
or later could be added by prefixing "\\?\" to the path (I believe).

*http://mail.python.org/pipermail/python-bugs-list/2007-March/037810.html

I have made an additional change to ensure that a new Hg can work on an
unchanged disk copy of a repo using long paths made by an old Hg; if a long file
already exists, it is just used. I will send the revised patch to the list.

In fact it seems I was testing SSH cloning with a corrupt repository. Starting
over with a fresh repository, clone, push, and pull seem to work OK in the four
combinations of patched or unpatched local and remote Hg. The format of the
local clone depends on the local Hg.

I also checked and bundle/unbundle seem to be fully interoperable.

An unrelated bug (#876) was masking the true cause of the SSH clone failure,
which I will now try to investigate.

Attaching a working patch which lets me clone and play with a repository in a
dir on XP to which I could not otherwise clone. Note that it adds a new storage
requirement if you actually create any long files (I could see no way around
this), so from a repo using this flag, you need the patched Hg to do a local
clone or a clone via static-http. My tests show that it is interoperable with
unpatched Hg over http protocol, since the long name munging is done on the
remote side and should not affect the wire protocol. No success yet with SSH
cloning of repos with long names - I get a remote error

abort: received changelog group is empty
remote:   File "mercurial/sshserver.py", line 116, in do_changegroup
remote:   File "/tmp/hg/mercurial/util.py", line 1421, in read
remote:     for chunk in self.iter:
remote:   File "mercurial/localrepo.py", line 1863, in gengroup

which I do not yet understand.

For the usestore workaround from issue584 to work, you have to apply it globally
before cloning.

A Mercurial local repository storage escapes capital letters for safety on
Windows, which is good and necessary. However certain file paths which are
already too long in the checkout become even longer in the local repository,
making it impossible to use the repository on Windows, which seems to have about
a 260 character maximum. (For example, a tarball of the repository cannot even
be unpacked on Windows.)

The exact maximum is of course dependent on the length of the path of the root
directory of the repository, but I would say that above around 180 characters
for an in-repository path is dangerous.

Example encountered when trying to create a Hg repo based on an existing source
tree (in CVS, but not using convert extension, just an import of trunk):

web.core.syntax/test/qa-functional/data/goldenfiles/org/netbeans/test/web/core/syntax/J2EECompletionTest/J2EECompletionTest_src_java_org_completion_TestAdvancedAnnotation_java.pass

(180 chars) becomes

.hg/store/data/web.core.syntax/test/qa-functional/data/goldenfiles/org/netbeans/test/web/core/syntax/_j2_e_e_completion_test/_j2_e_e_completion_test__src__java__org__completion___test_advanced_annotation__java.pass.i

(216 chars) which is bound to cause troubles even where CVS worked.

Obviously it would be wise for the developer to reduce the length of the actual
file path. But I think Hg should also have some cutoff for the length of paths
it will create in the repository, say 150 by default but configurable. It
already does some name mangling for case sensitivity, so why not truncate some
path components to make the final path shorter?

(My understanding, which could be mistaken, is that paths in the .hg/store/data/
directory are arbitrary, since the manifest anyway maps real file paths in the
checkout to storage paths; and that Hg just uses storage paths derived from the
real paths as a debugging aid. It is not obvious from code comments whether
localrepo.py automatically manages conflicts between storage paths, but probably
not - '_' is escaped to '__' I guess to prevent such conflicts. In such a case,
simple truncation would be unsafe. But you could still truncate paths safely
using e.g. an unabbreviated SHA-1 hash for 40 chars plus most of the normal
filename for the remaining chars. I guess this would live in
util.py#_buildencodefun?)

I have seen issue #584. I have tried

---%<--- .hg/hgrc
[format]
usestore=0
---%<---

though it does not seem to have any effect (hg 0.9.5). Anyway the Windows case
mangling is necessary for safety (my team has encountered the analogous problems
with CVS a number of times in the past) and I would not want this to be turned off.