Bug 6875 - "unknown exception" using `hg split` or `hg commit --interactive`
Summary: "unknown exception" using `hg split` or `hg commit --interactive`
Status: CONFIRMED
Alias: None
Product: Mercurial
Classification: Unclassified
Component: Mercurial (show other bugs)
Version: 6.7.2
Hardware: PC Linux
: wish bug
Assignee: Bugzilla
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-04-29 01:16 UTC by errael
Modified: 2024-05-14 17:33 UTC (History)
2 users (show)

See Also:
Python Version: ---


Attachments
The file to edit created by "hg split" after the "e" sub command. (697 bytes, patch)
2024-05-14 17:28 UTC, errael
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description errael 2024-04-29 01:16 UTC
This is on linux, working with a clone of a githug repo; probably developed on windows.
Wondering if this could be a line ending issue.
To experiment, since `hg split` failed, tried `hg amend --extract; hg commit -i`, same thing.

Entered command
```
$ hg split xscreen.cpp
```

`y` for the first hunk
then `e` for the second
```
record change 2/9 to 'xscreen.cpp'?
(enter ? for help) [Ynesfdaq?] e
```

When it brings up the editor, gvim, there is the message
```
"/tmp/hg-editor-xxx.diff" [CR missing][dos]53L, 1799B
File has 39 DOS-style endings out of 53 lines.
```

In the editor, there were only `+` lines. Deleted some of them,
wrote the file, exited the editor, crash.
```
** unknown exception encountered, please report by visiting
** https://mercurial-scm.org/wiki/BugTracker
** Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
** Mercurial Distributed SCM (version 6.7.2)
** Extensions loaded: convert, evolve 11.1.3, extdiff, fsmonitor, graphlog, hggit 1.1.1 (dulwich 0.21.7), histedit, purge, rebase, share, topic 1.1.3, transplant
Traceback (most recent call last):
  File "/home/err/.venv/bin/hg", line 59, in <module>
    dispatch.run()
  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/dispatch.py", line 142, in run
    status = dispatch(req)
  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/dispatch.py", line 231, in dispatch
    status = _rundispatch(req)
  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/dispatch.py", line 275, in _rundispatch
    ret = _runcatch(req) or 0
  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/dispatch.py", line 456, in _runcatch
    return _callcatch(ui, _runcatchfunc)
  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/dispatch.py", line 466, in _callcatch
    return scmutil.callcatch(ui, func)
  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/scmutil.py", line 152, in callcatch
    return func()
  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/dispatch.py", line 446, in _runcatchfunc
    return _dispatch(req)
  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/dispatch.py", line 1271, in _dispatch
    return runcommand(
  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/dispatch.py", line 904, in runcommand
    ret = _runcommand(ui, options, cmd, d)
  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/dispatch.py", line 1283, in _runcommand
    return cmdfunc()
  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/dispatch.py", line 1269, in <lambda>
    d = lambda: util.checksignature(func)(ui, *args, **strcmdopt)
  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/util.py", line 1878, in check
    return func(*args, **kwargs)
  File "/home/err/.venv/lib/python3.10/site-packages/hgext3rd/evolve/cmdrewrite.py", line 1169, in cmdsplit
    cmdutil.dorecord(ui, repo, commands.commit, b'commit',
  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/cmdutil.py", line 687, in dorecord
    return commit(ui, repo, func, pats, opts)
  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/cmdutil.py", line 2953, in commit
    return commitfunc(ui, repo, message, matcher, opts)
  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/cmdutil.py", line 527, in _record
    chunks, newopts = filterfn(ui, original_headers, match)
  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/cmdutil.py", line 435, in recordfilter
    newchunks, newopts = filterchunks(
  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/cmdutil.py", line 421, in filterchunks
    return patch.filterpatch(ui, originalhunks, match, operation)
  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/patch.py", line 1335, in filterpatch
    applied[newhunk.filename()].append(newhunk)
KeyError: b'xscreen.cpp\r'
```
Comment 1 errael 2024-04-29 01:17 UTC
$ hg version -v
Mercurial Distributed SCM (version 6.7.2)

Enabled extensions:

  convert     internal  
  evolve      external  11.1.3
  extdiff     internal  
  fsmonitor   internal  
  graphlog    internal  
  hggit       external  1.1.1 (dulwich 0.21.7)
  histedit    internal  
  purge       internal  
  rebase      internal  
  share       internal  
  topic       external  1.1.3
  transplant  internal
Comment 2 Anton Shestakov 2024-05-03 11:54 UTC
This indeed looks like hg got confused at some point by line endings... But not sure when it started exactly. If you have a consistent way to reproduce this issue, you might want to take a stab at debugging it.

This:

  File "/home/err/.venv/lib/python3.10/site-packages/mercurial/patch.py", line 1335, in filterpatch
    applied[newhunk.filename()].append(newhunk)
KeyError: b'xscreen.cpp\r'

hints that the `applied` dictionary has file names as keys (without '\r' or '\n' I strongly suspect). But then at some point a diff header uses '\r\n' line endings and so the regular expression that is used to parse the header (i.e. header.diffgit_re in patch.py) sees \r as part of the useful string somehow and you get the error about the filename.

I could be wrong, but I think diffs generated for internal use by Mercurial shouldn't contain '\r's? Maybe that's where the root of this problem lies, but I cannot reproduce this locally.
Comment 3 errael 2024-05-04 03:31 UTC
@ Anton Shestakov

> If you have a consistent way to reproduce this issue, 
> you might want to take a stab at debugging it.

I've never tried debugging mercurial. And I'm a pretty
casual python user. There's
.../site-packages/mercurial
but I'd rather not start making modifications in there.

Any suggestions on setting up a debug environment?
Comment 4 Anton Shestakov 2024-05-06 12:18 UTC
The easiest way to set up a separate instance of mercurial would be to use virtualenv and `pip install mercurial hg-evolve hg-git`. But no idea how easy it would be to debug this problem, considering it's a git repo and hg-git does some monkey-patching of hg code, so there might be some surprises there when you try to look at it.

Perhaps you could point me to the repo on github and give some steps to reproduce the problem?
Comment 5 errael 2024-05-07 03:41 UTC
I'll play around with setting up a virtualenv. I currently use one (over a year old) and I've been wanting to set up a second to update python version. But debugging this issue does seem a bit much as a first mercurial debug.

Here's steps to reproduce

Running on linux. The github repo is small. Using gvim (vim).

- hg clone git+https://github.com/CruiserOne/Astrolog.git
- cd Astrolog
- gvim astrolog.cpp
        vim reports: "astrolog.cpp" [dos] 3290L, 99163B
- Add some lines to the file
- :wq - write the file, vim preserves the dos line endings
- hg ci -m 'add a few lines'
- $ hg diff -c .
    diff --git a/astrolog.cpp b/astrolog.cpp
    --- a/astrolog.cpp
    +++ b/astrolog.cpp
    @@ -53,6 +53,10 @@
     
     #include "astrolog.h"
     
    +one
    +two
    +three
    +four
     
     /*
     ******************************************************************************
- hg split astrolog.cpp
- e
    command to hg split to bring up the editor, gvim again, see message from vim
        "/tmp/hg-editor-xxx.dif [CR missing][dos] 24L, 697B
        File has 10 DOS-style line endings out of 24 lines
NOTE that mercurial has created a file with both kinds of line endings
- in the editor, delete two lines: "+two" and "+three"
- :wq   - write the file and quit the editor

OBSERVE the CRASH
    ...
    KeyError: b'astrolog.cpp\r'
Comment 6 Anton Shestakov 2024-05-10 12:23 UTC
I've started looking at this, and can reproduce. Although the result of editing the patch is not the same for me: if I edit the patch, then it simply fails to apply. I think this is caused by crecord interface interpreting the edited patch in a potentially wrong way (maybe dropping the '\r's).

What happens if you use text-only interface to do the split? I suspect it will actually work. Try this:

  hg split --config ui.interface=text astrolog.cpp
Comment 7 errael 2024-05-14 17:28 UTC
Created attachment 2167 [details]
The file to edit created by "hg split" after the "e" sub command.

I'm assuming the attachment won't be modified. But just in case, the 24 line file is made up like
- 4 NL lines
- 10 DOS lines
- 10 NL lines
Comment 8 errael 2024-05-14 17:32 UTC
> What happens if you use text-only interface to do the split? I suspect it will
> actually work. Try this:
> 
>   hg split --config ui.interface=text astrolog.cpp

Doing this is identical. Mercurial still brings up "gvim". I may have some config/environment variable that it interfering. But I have no idea what/why.
Comment 9 errael 2024-05-14 17:33 UTC
Oh, forgot to mention, I attached the file from /tmp that mercurial produces for editing.