Issue219

Title need a way to export all outgoing changes as a single diff
Priority feature Status chatting
Superseder Nosy List jglick, kupfer, movement, tonfa
Assigned To Topics

Created on 2006-04-26.16:13:37 by movement, last changed 2008-08-11.09:17:35 by djc.

Files
File name Uploaded Type Edit Remove
exportonecs.patch vok, 2006-05-22.09:23:43 text/plain
Messages
msg5228 (view) Author: jglick Date: 2008-02-12.20:52:48
Issue28 seems to be closely related to this, though with a different emphasis:
creating a consolidated patch of outgoing changes for review, not with the
intention of creating a real changeset out of it. Perhaps a single command could
satisfy both needs?
msg5200 (view) Author: djc Date: 2008-02-12.09:19:38
We have hg out -p now. However, it might be more useful if that uses export-like
patches (e.g. with hg headers).
msg2043 (view) Author: movement Date: 2006-09-27.15:27:13
Only if you're using mq, which is not suitable for all circumstances...
msg2040 (view) Author: tonfa Date: 2006-09-27.14:51:13
Does mq with git diffs solves this ? (qfold in particular)

regards,

Benoit
msg1409 (view) Author: movement Date: 2006-05-23.14:52:54
I meant I had a /file/ called 'tags' (i.e. a ctags file), not mercurial tags.
Somehow it got 'hg add'ed.

> It takes the working dir revision as second revision

Would it be possible to make it take an explicit range like other tools?
msg1408 (view) Author: vok Date: 2006-05-23.11:26:46
> I had a 'tags' file created, and it ended up in the changeset!
Just did run a test with a tag file. You are right, it does not makes sense
to inherit all the tags for revision not even existing. I have a look at this.
msg1407 (view) Author: vok Date: 2006-05-23.06:34:03
> I had a 'tags' file created, and it ended up in the changeset!
Is that a problem? If you use e.g. push -r on that changeset revision local 
tags should not be pushed.

> Also, somehow I ended up with the changeset committed as tip, even though
> I didn't use -k. And I can't rollback, so my repo is essentially corrupt
> again...
Some error interupted the process, is it related to the traceback you showed?
Another way to get rid of the temporary changset is the mq strip command.
The repo should not be currupted.

> hg  exportonecs 10068
> ** unknown exception encountered, details follow
> ...
> commands.revert(ui,repo,**revert_opts)
> ...
> opts['include'] = [os.path.join(cwd, i) for i in opts['include']]
Looks like I don't handle the include option when calling revert.
I check on that.

> PS why can't I specify a changeset range instead of just rev:tip?
It takes the working dir revision as second revision, might be tip or 
what ever you like.
msg1406 (view) Author: movement Date: 2006-05-23.00:06:47
now I'm getting:

hg  exportonecs 10068
** unknown exception encountered, details follow
** report bug details to mercurial@selenic.com
** Mercurial Distributed SCM (version a37fe97b7650)
Traceback (most recent call last):
  File "/usr/bin/hg", line 12, in ?
    commands.run()
  File "/usr/lib/python2.3/site-packages/mercurial/commands.py", line 3203, in run
    sys.exit(dispatch(sys.argv[1:]))
  File "/usr/lib/python2.3/site-packages/mercurial/commands.py", line 3365, in
dispatch
    return d()
  File "/usr/lib/python2.3/site-packages/mercurial/commands.py", line 3340, in
<lambda>
    d = lambda: func(u, repo, *args, **cmdoptions)
  File "/usr/lib/python2.3/site-packages/hgext/exportonecs.py", line 117, in
exportonecs
    commands.revert(ui,repo,**revert_opts)
  File "/usr/lib/python2.3/site-packages/mercurial/commands.py", line 2295, in
revert
    for src, abs, rel, exact in walk(repo, pats, opts, badmatch=mf.has_key):
  File "/usr/lib/python2.3/site-packages/mercurial/commands.py", line 61, in walk
    files, matchfn, results = makewalk(repo, pats, opts, node, head, badmatch)
  File "/usr/lib/python2.3/site-packages/mercurial/commands.py", line 52, in
makewalk
    files, matchfn, anypats = matchpats(repo, pats, opts, head)
  File "/usr/lib/python2.3/site-packages/mercurial/commands.py", line 45, in
matchpats
    opts['include'] = [os.path.join(cwd, i) for i in opts['include']]

PS why can't I specify a changeset range instead of just rev:tip?
msg1405 (view) Author: movement Date: 2006-05-22.23:59:38
I found a more serious problem: I had a 'tags' file created, and it
ended up in the changeset!

Also, somehow I ended up with the changeset committed as tip, even though
I didn't use -k. And I can't rollback, so my repo is essentially corrupt again...

(Another problem with not using -k: if I want an editor for the changeset,
there's no easy way to redirect the diff to a file).
msg1403 (view) Author: movement Date: 2006-05-22.19:21:17
Seems to work nicely on quick tests, with one problem I found:
trying to cancel an export via exiting commit message editor
without changing the text still continues.
msg1402 (view) Author: movement Date: 2006-05-22.19:13:50
> It fits my workflow.

I don't see a good reason to differ from other commands that commit, though.
Differences are bad. After all, you have -d and -u for your workflow.

> push the new changeset using the -r option

Great. I still think -k should be the default, though.

I'll try out your extension...
msg1401 (view) Author: vok Date: 2006-05-22.19:01:36
> > By default date and user name are taken from the last changeset.
> Why aren't they taken in the exact same way as doing a normal commit?
It fits my workflow. If user, date and the change message are not
changing the same changeset is created every time I run the exports.
I understand that this may be different for other. I can make it an option.

> No, I saw -k, but I wasn't clear that'd give me what I want
I would have guessed the  -k option is what you want.

> then I could push/pull that new tip, hg export it, or whatever.
What is not working today is the bundeling of only one head, which is what
I do when using the bundle option.

> i.e. what happens if I push?
If you use -k you should be able to push the new changeset using the -r option

> hg diff > patch
> ....
The diff format does not support binaries. The bundle files are more general.
msg1400 (view) Author: movement Date: 2006-05-22.17:11:02
No, I saw -k, but I wasn't clear that'd give me what I want i.e. what happens
if I push? If it gives the layout I gave earlier, I think it should be the
default. If not, I'd like to know what you do end up with...
msg1399 (view) Author: cboos Date: 2006-05-22.16:52:00
John Levon wrote:
> John Levon <levon@movementarian.org> added the comment:
> ...
> I'm a bit worried about all these things (backout etc.) modifying my checked
> out state. Would it be possible/feasible for these commands to all work on
> a 'secret' checked out tree?
>   

Better yet, what about a kind of "in memory" pseudo working dir?

This would allow for other kinds of check-ins not going through the 
working dir.
For example, "hg import" could be implemented like that, and this would 
make it possible to support the following scenario which was discussed 
some time ago:
  hg diff > patch
  (editing the patch, e.g. discard some diffs in order to postpone them)
  hg import patch
  hg update
  hg diff
  (shows the remaining diffs)

> This extension sounds like it works differently to how I'd expected. In
> particular, I'd like to have ended up with:
> ...
>
> then I could push/pull that new tip, hg export it, or whatever. Is that hard
> to do?
>   

Perhaps you've missed the '-k' option?

-- Christian
msg1398 (view) Author: movement Date: 2006-05-22.16:26:40
> By default date and user name are taken from the last changeset.

Why aren't they taken in the exact same way as doing a normal commit?

> the user is left with an unexpected state.

I'm a bit worried about all these things (backout etc.) modifying my checked
out state. Would it be possible/feasible for these commands to all work on
a 'secret' checked out tree?

This extension sounds like it works differently to how I'd expected. In
particular, I'd like to have ended up with:

    rev X
      |\
      | \
      |  | change A
      |  | change B
      |  | change C
      |  | change D
      |  . (old head)
      |
      . changesA-D (new tip)

then I could push/pull that new tip, hg export it, or whatever. Is that hard
to do?
      |  | change A

      |  | change A
msg1393 (view) Author: vok Date: 2006-05-22.09:23:43
I created an extension (exportonecs) that supports issue 219. Below
a description on what it does. Comments are wellcome.

/Volker

extension exportonecs: exports multiple changesets as one changeset 

Creates one changeset covering the changes between 
a given revision and the working dir revision. 
The output is either export (the default) or bundle format

If option -b FILE is used, the output is a bundle file in 
changegroup format. The file location must be outside the
repository, since the working dir is updated few times.

The command creates the new changeset by use of a 
standard commit. The commit options m,l,d,u work as
described in the commit command. By default date and
user name are taken from the last changeset.

After exporting, the created changeset is removed and the
original working dir state is restored. If option -k is
given the new changeset is kept e.g. for mailing it via the
patchbomb (email) extension.

Short commings:
The extension works by creating an additional changeset which is
removed by doing an rollback. If during the process an error occurs
the user is left with an unexpected state. A sane state can be
manualy restored with 'hg rollback' and 'hg up -C' in that case.

The extension takes the following command flow:
  hg up -C fromnode
  hg revert -r tonode
  hg commit [options] -A
  hg bundle tip or hg export tip
optional:
  hg rollback
  hg up -C tonode
msg1093 (view) Author: Omnifarious Date: 2006-04-28.00:52:42
On Thu, Apr 27, 2006 at 02:43:35PM -0700, Bryan O'Sullivan wrote:
> On Thu, 2006-04-27 at 10:55 -0700, Mike Kupfer wrote:
> 
> > Okay, but now I'm wondering just what OpenSolaris should be gearing up
> > to use.
> 
> I think that you should try using mq and see if it fits your needs,
> since it already exists and is quite wonderful.  However, it has a bit
> of a learning curve, and is profoundly different in concept from a
> normal mode of using an SCM.

If we could come up with some sort of diff format that could handle
binary diffs and copy/delete operations, it's possible the defficiency
of mq not handling those things could be fixed.

Have fun (if at all possible),
-- 
"It does me no injury for my neighbor to say there are twenty gods or no God.
It neither picks my pocket nor breaks my leg."  --- Thomas Jefferson
"Go to Heaven for the climate, Hell for the company."  -- Mark Twain
-- Eric Hopper (http://www.omnifarious.org/~hopper) --
msg1091 (view) Author: bos Date: 2006-04-27.21:39:15
On Thu, 2006-04-27 at 10:55 -0700, Mike Kupfer wrote:

> Okay, but now I'm wondering just what OpenSolaris should be gearing up
> to use.

I think that you should try using mq and see if it fits your needs,
since it already exists and is quite wonderful.  However, it has a bit
of a learning curve, and is profoundly different in concept from a
normal mode of using an SCM.

> Should we instead focus on helping make changeset squishing happen along
> with rename-with-history?

It might make sense to do that in parallel.  Mercurial has a number of
other users who would like to be able to compress many changesets into a
smaller number before pushing them, for whom an extension that made this
possible would make sense as it would for you.

See answers to question 4.5 in
http://www.selenic.com/mercurial/wiki/index.cgi/UserSurvey for an
indication of this.

If it gets written, this shouldn't be core functionality, though; I
think that an extension is the appropriate place for it.  The actual
coding might involve moving a bit of mq functionality into the core, so
that both mq and "hg squish" (or whatever it's to be called) can use it.

	<b
msg1089 (view) Author: movement Date: 2006-04-27.18:48:52
It makes sense to keep intermediate changesets forever in a project
branch. (I.e. the project gate will have all the individual commits
to that repo, but when pushing upstream, will only push one changeset).

Pushing all the changes upwards means, afaics, a significant number of
merge changesets etc. which would significantly clutter history.

For 'individual' branches, i.e. bug/RFE commits, I can't see any real
purpose to keeping all of history.
msg1088 (view) Author: mpm Date: 2006-04-27.18:41:19
On Thu, Apr 27, 2006 at 11:06:49AM -0700, Danek Duvall wrote:
> On Thu, Apr 27, 2006 at 10:55:19AM -0700, Mike Kupfer wrote:
> 
> > Okay, but now I'm wondering just what OpenSolaris should be gearing up
> > to use.  We don't want to see intermediate changesets in the "gate"
> > repository, so I've been assuming that we'll be making heavy use of mq.
> > Should we instead focus on helping make changeset squishing happen along
> > with rename-with-history?
> 
> We probably should play with both and see which model fits our needs
> better.  It could be that smaller gates -- individual bugfixes and the like
> -- can use changeset squishing and project gates could be managed with mq
> in mind (and individuals contributing to the project gate could use
> squishing in their own workspaces).  Indeed, the functionalities may be
> orthogonal in a similar fashion for anyone using mercurial.

I'd strongly encourage you to consider keeping the intermediate
changesets (they can contain valuable history) and instead look at
tools to collapse diffs when you're reviewing a branch. I believe
hg view can already do this.
msg1087 (view) Author: dduvall Date: 2006-04-27.18:03:31
On Thu, Apr 27, 2006 at 10:55:19AM -0700, Mike Kupfer wrote:

> Okay, but now I'm wondering just what OpenSolaris should be gearing up
> to use.  We don't want to see intermediate changesets in the "gate"
> repository, so I've been assuming that we'll be making heavy use of mq.
> Should we instead focus on helping make changeset squishing happen along
> with rename-with-history?

We probably should play with both and see which model fits our needs
better.  It could be that smaller gates -- individual bugfixes and the like
-- can use changeset squishing and project gates could be managed with mq
in mind (and individuals contributing to the project gate could use
squishing in their own workspaces).  Indeed, the functionalities may be
orthogonal in a similar fashion for anyone using mercurial.

Danek
msg1086 (view) Author: kupfer Date: 2006-04-27.17:52:04
>>>>> "bos" == Bryan O'Sullivan <bos@serpentine.com> writes:

bos> What you get if you use mq isn't quite the same thing as you get if
bos> you have squishing of changesets explicitly supported.

Okay, but now I'm wondering just what OpenSolaris should be gearing up
to use.  We don't want to see intermediate changesets in the "gate"
repository, so I've been assuming that we'll be making heavy use of mq.
Should we instead focus on helping make changeset squishing happen along
with rename-with-history?

mike
msg1084 (view) Author: bos Date: 2006-04-27.05:10:04
On Wed, 2006-04-26 at 21:50 -0700, Mike Kupfer wrote:
> >>>>> "John" == John Levon <mercurial-bugs@selenic.com> writes:
> 
> John> More generally, we need to be able to collapse many deltas into a
> John> single one for pushing back:
> 
> I'm confused.  I thought that's what Mercurial Queues is for.  What am I
> missing?

What you get if you use mq isn't quite the same thing as you get if you
have squishing of changesets explicitly supported.

For example, mq doesn't try to detect renamed files (and doing so by
computing similarity of files in patches is notoriously unreliable
anyway), whereas it would be simple to take the endpoints of a series of
renames and turn them into a single rename when compressing changesets.

Also, mq doesn't see other "meta" operations, such as changes of
executable bits.

Finally, it is in some respects easier to just commit a pile of changes
willy nilly and worry about compressing them after the fact.  Using mq
forces you to impose discipline on yourself, as you have to think about
which patch each change is going into, and refreshing a patch will lose
the earlier version of the patch unless you're managing patches in a
repo and committed the earlier version.

None of these points is a criticism of mq; it's just quite a different
development model from "plain hg", and it places different constraints
on you.  In fact, many of these constraints (particularly having to
think at every point about which patch an edit should end up in) can be
argued to result in better code coming out the other end.

	<b
msg1083 (view) Author: kupfer Date: 2006-04-27.04:47:38
>>>>> "John" == John Levon <mercurial-bugs@selenic.com> writes:

John> More generally, we need to be able to collapse many deltas into a
John> single one for pushing back:

I'm confused.  I thought that's what Mercurial Queues is for.  What am I
missing?

mike
msg1079 (view) Author: movement Date: 2006-04-26.16:28:45
Also: this should be allowed repeatedly, and it should use the tip
last pulled from as the default base (so I don't have to have an
up to date tree in order to collapse).
msg1077 (view) Author: movement Date: 2006-04-26.16:15:39
To clarify, whilst this seems possible with ThomasAH's method, it's distinctly
non obvious. A simple command would be much nicer.
msg1076 (view) Author: movement Date: 2006-04-26.16:13:35
Often I'll be making a number of changes (multiple local commits), but
wanting to produce a diff via hg export to send upstream, rather than
push/pull.

Today's example: I forgot to hg add some files when I committed, and since
I did a pull inbetween, I couldn't hg revert. What I wanted to be able to
do was to hg commit the newly added files, then create a single diff (one
changeset comment, one diff) in 'hg export' format. Also discussed here:

http://www.selenic.com/pipermail/mercurial/2005-December/005979.html

More generally, we need to be able to collapse many deltas into a single
one for pushing back:


<ThomasAH> movement: though this is not so complicated with Mercurial 0.8.1:  
hg co -C first && hg revert -r last && hg ci -m 'amalgamated first to last' &&
hg export

or instead of export:

<ThomasAH> movement: Of course,  hg push -r tip target

<movement> ThomasAH: cool. didn't work last time I tried.
<movement> you had to pull instead
<mpm> Doesn't work over ssh.
History
Date User Action Args
2008-08-11 09:17:35djcsetnosy: - djc
2008-02-12 20:52:48jglicksetnosy: + jglick
messages: + msg5228
2008-02-12 09:19:38djcsetnosy: + djc
messages: + msg5200
2006-09-27 15:27:14movementsetmessages: + msg2043
2006-09-27 14:51:13tonfasetnosy: + tonfa, - mercurial
messages: + msg2040
2006-05-23 14:53:02movementsetnosy: mercurial, movement, kupfer
messages: + msg1409
2006-05-23 11:26:57voksetnosy: mercurial, movement, kupfer
messages: + msg1408
2006-05-23 06:34:10voksetnosy: mercurial, movement, kupfer
messages: + msg1407
2006-05-23 00:06:48movementsetnosy: mercurial, movement, kupfer
messages: + msg1406
2006-05-22 23:59:47movementsetnosy: mercurial, movement, kupfer
messages: + msg1405
2006-05-22 19:21:17movementsetnosy: mercurial, movement, kupfer
messages: + msg1403
2006-05-22 19:13:50movementsetnosy: mercurial, movement, kupfer
messages: + msg1402
2006-05-22 19:01:58voksetnosy: mercurial, movement, kupfer
messages: + msg1401
2006-05-22 17:11:02movementsetnosy: mercurial, movement, kupfer
messages: + msg1400
2006-05-22 16:52:03cboossetnosy: mercurial, movement, kupfer
messages: + msg1399
2006-05-22 16:26:41movementsetnosy: mercurial, movement, kupfer
messages: + msg1398
2006-05-22 09:23:57voksetfiles: + exportonecs.patch
nosy: + mercurial
messages: + msg1393
2006-04-28 00:52:43Omnifarioussetmessages: + msg1093
2006-04-27 21:39:21bossetmessages: + msg1091
title: need a way to export all outgoing changes as a single diff -> need a way to export all outgoing changes as a single diff
2006-04-27 18:48:52movementsetmessages: + msg1089
2006-04-27 18:41:21mpmsetmessages: + msg1088
2006-04-27 18:03:32dduvallsetmessages: + msg1087
title: need a way to export all outgoing changes as a single diff -> need a way to export all outgoing changes as a single diff
2006-04-27 17:52:10kupfersetmessages: + msg1086
2006-04-27 05:10:12bossetmessages: + msg1084
2006-04-27 04:47:44kupfersetmessages: + msg1083
2006-04-27 04:45:53kupfersetnosy: + kupfer
2006-04-26 16:28:45movementsetmessages: + msg1079
2006-04-26 16:15:39movementsetstatus: unread -> chatting
messages: + msg1077
2006-04-26 16:13:37movementcreate