Bidirectional sync

See also ticket #1.

Contents

• Better (maybe even right) recipes for two-way sync
  • Zooko's recipe
    • Internally Consistent State
    • Subroutine CLEANUP: clean up any intermediate state
    • Subroutine A: translate the next patch from SVN to darcs
    • Subroutine B: translate the next patch from darcs to SVN
    • Algorithm C: complete sync
    • Incomplete Design
    • Limitations
    • Proof!
  • Rednaxela's recipe
• Status
  • The typical case
  • Another use case
  • Can tailor be of any help?

Better (maybe even right) recipes for two-way sync

The development of Tailor started right from this need, and if it's still a little flaky it's because the need fade away. Luckily other people keeps hinting about proper solution to the problem (as #1 witness)

Zooko's recipe

Let there be two Tailor directories, once which translates patches from svn to darcs (called "SD" from here on) and one which translates patches from darcs to svn (called "DS" from here on). Each of these Tailor directories contains an svn working dir subdirectory (called "SDs" or "DSs") and a darcs repo subdirectory (called "SDd" or "DSd").

In addition there is (of course) the SVN repo "SO" and an official darcs repo "DO".

Internally Consistent State

Let the internally consistent condition be such that the contents of all four of the subdirectories SDs, SDd, DSs, and DSd are up-to-date with each other, and furthermore that SDs and DSs are each a subset of SO (i.e., svn diff would show no changes) and SDd and DSd are each a subset of DO (i.e. darcs push DO would have no effect).

Subroutine CLEANUP: clean up any intermediate state

Make sure that any intermediate state left behind from Subroutine A or B completing, raising MergeConflict, or crashing (i.e. being killed in mid-step) gets cleaned up and that we return to the Internally Consistent State if possible, or else raise an error if impossible. CLEANUP has to be run at the beginning of every run of the two-way-sync algorithm, since we never know if the previous run crashed and left things in an inconsistent state.

1. For each patch in SDd which has the special line as the last line of the log entry, run darcs obliterate to obliterate that patch from SDd.
2. darcs pull -a DSd in SDd to pull all darcs patches from DSd into SDd.
3. darcs push -a DO in SDd. If the push fails (which will happen if it would cause a conflict in DO) then raise MergeConflict.
4. svn commit in DSs. If the commit fails (which will happen if it would cause a conflict in SO) then raise MergeConflict.

Subroutine A: translate the next patch from SVN to darcs

1. Run CLEANUP()
2. Use Tailor in the SD directory to translate the next patch from SVN to darcs. This will update SDs to that revision and add a new patch into SDd. If there are no more untranslated svn patches in SO then raise StopIteration.

Subroutine B: translate the next patch from darcs to SVN

1. Run CLEANUP()
2. Use Tailor in the DS directory to translate the next patch from darcs to svn. This will pull the next patch from DO into DSd and create a new revision in DSs and attempt to commit it to SO. The newly created svn revision will have a special line appended to its log entry saying "This patch originally recorded in darcs and translated into svn by Tailor brand patch wrangler.". If there are no more untranslated patches in DO then raise StopIteration. If the attempt to commit the new patch to SO fails (which it will if there is a merge conflict) then raise MergeConflict.

Algorithm C: complete sync

Run Subroutine A and B until they both raise StopIteration. I haven't figured out in which order it is best to run them. My current intuition is that it is best to run Subroutine A until it raises StopIteration, then run Subroutine B one time, then run Subroutine A against until it raises StopIteration again, then run Subroutine B again one time, etc. My main motivation is that this means any potential merge conflicts happen in darcsworld rather than in svnworld, which should reduce the number of merge conflicts and make the inevitable merge conflicts easier to clean up. But see Incomplete Design below.

Incomplete Design

What does the user do in response to MergeConflict in order to get back into the Internally Consistent State?

Limitations

Darcs patches which were originally svn patches created by svn commit to SO and then transferred into darcs world by this algorithm can have the SVN revision number inserted into their patch name (e.g. by using the following two Tailor options:

patch-name-format = [r%(revision)s] %(firstlogline)s
remove-first-log-line = True

However, darcs patches originally created by darcs record and then transferred into svn world by this algorithm will not have the svn revision number in their patch name in darcs world, although of course they will get a revision number assigned to them when translated into svn world. Too bad that there isn't a way to assign revision numbers to the original darcs-world patch without too many sync/conflict issues cropping up. (Couldn't we use an svn revision property "tailor:from-darcs" or something?)

Proof!

Snippet from Jan 30, 2006 chat on #tailor, where zooko said:

Hi!
I have two way sync working.
There are many uglinesses to my version./
1.  it uses some old snapshot of tailor the old version
2.  It runs "darcs revert --all" after each darcs pull
3.  It runs the shell command "yes | darcs obliterate --patch="$PATTERN"" after every svn commit ....
Probably other uglinesses too, but it works.
https://yumyum.zooko.com:19144/pub/
https://yumyum.zooko.com:19144/pub/two-way-sync/
As you can see, it uses the old tailor.info files...
#2 above -- the darcs revert --all -- is necessary because you can't set "--allow-conflicts" on pull, only on apply, and tailor gets darcs patches by pulling them.

Rednaxela's recipe

See this attachment for recipe that is being worked out between CVS and BaazarNG

Status

Unfortunately I wasn't able to keep up the good work I started many months ago in a dusty branch of Tailor...

With Zooko's recipe as a possible goal, I started hacking the ConfigurationFile format to allow an easier switch of the source and target backends, simply by tagging a project as ''two-ways'' in its section. This is something that shall be kept, augmented with a more general way of "linking" projects and repositories each other (so that it becomes easier to share and reuse common settings).

The pending (read never finished) part of changes were

• a different way of splitting the repositories classes than that landed in master repo
• first reluctant steps toward moving a few peculiarities from the DualWorkingDir to the SourceRepository and TargetRepository abstract and concrete subclasses
• the highest level of Zooko's recipe, based on the new ability of going ''one'' patch at a time

I'm gonna trash most of these changes.

Given that my own need of a TwoWaySync dropped quite drastically once I felt in love with Darcs, maybe I don't think often enough about the issue to focus it well enough, so I'll write a few notes here.

The typical case

A developer wants to contribute to a well known software which is being developed within a master Subversion repository. He will spend his time mostly on little fixes here and there, and developing a bit more intricated part. He doesn't want to step on main developers feet, so he would like to present clean and well formed patches. Normally, he'd create a branch in the Subversion repository, maybe named after his own name.

Luckily, he lurked something about Darcs, and starts using it doing the most basic form of TwoWaySync:

1. Check out a current working dir:

   $ svn co http://svn.server/project/trunk project
   

2. Put everything under darcs too:

   $ cd project
   $ darcs init
   $ darcs add --recursive *
   $ darcs record -a -m "My own hacking start point, from revision r1234"
   

3. Tag this initial state:

   $ darcs tag initial

4. Until satisfied

   1. Keep up-to-date:

      $ svn update
      

   2. Hack around, resolving eventual conflicts, maybe even amending already recorded Darcs patches

   3. Record new patches in Darcs, or unrecord and redo some previous ones

5. Commit the whole work to the master repository:

   $ darcs changes --summary --from-tag=initial > /tmp/clog
   $ svn ci -F /tmp/clog .
   

Another use case

Another guy develops his products extending and customizing a set of free-software components. He does not appreciate the fact that each one uses a different kind of version control, so he seeks a way of staying as neutral as possible, keeping together his own patches and those coming down from upstream, possibly collaborating with them on some aspects and thus sending now and then a few fixes upstream.

This is harder, because:

1. In this context, obviously the "branches" must survive for a potentially long time
2. Not all local patches are sent to the other side
3. Learning and dealing with different types of VC may not be so funny

Can tailor be of any help?

... to be continued