source: tailor/README @ 506

About
=====

Tailor is a tool to migrate changesets between CVS, Subversion_,
darcs_, ArX_, monotone_, Codeville_, Mercurial_ and `Baazar-NG`_ [#]_
repositories.

This script makes it easier to keep the upstream changes merged in
a branch of a product, storing needed information such as the upstream
URI and revision in special properties on the branched directory.

The following ascii-art illustrates the usual scenario::

                           +------------+            +------------+
  +--------------+         | Immutable  |            | Working    |
  | Upstream CVS |-------->| darcs      |----------->| darcs      |
  | repository   | tailor  | repository | darcs pull | repository |
  +--------------+         +------------+            +------------+
                                                           |^         
                                                           ||
                                                           ||
                                                           v|
                                                          User

Ideally you should be able to swap and replace "CVS server" and "darcs
repository" with any combination of the supported systems.

A more convoluted setup shows how brave people are using it to get a
two way sync::

  +----------+        +--------+        +--------+       +---------+
  |          | -----> | hybrid | darcs  |        | ----> |   my    |
  | upstream | tailor |  CVS   | -----> | master | darcs | working |
  |   CVS    | <----- | darcs  | <----- | darcs  | <---- |  darcs  |
  |          |        |  sync  | tailor |        |       |         |
  +----------+        +--------+        +--------+       +---------+
               (cron)            (cron)


.. [#] ArX, Monotone, Codeville, Mercurial and Baazar-NG systems may be
       used only as the `target` backend, since the `source` support
       isn't coded yet. Contributions on these backends will be very
       appreciated, since I do not use them enough to figure out the
       best way to get pending changes and build tailor ChangeSets out
       of them.

.. _subversion: http://subversion.tigris.org/
.. _darcs: http://www.darcs.net/
.. _codeville: http://www.codeville.org/
.. _baazar-ng: http://www.bazaar-ng.org/
.. _mercurial: http://www.selenic.com/mercurial/
.. _arx: http://www.nongnu.org/arx/


Installation
============

tailor is written in Python, and thus Python must be installed on
your system to use it.  It has been successfully used with Python 2.3
and 2.4.

Since it relies on external tools to do the real work such as `cvs`,
`darcs` and `svn`, they need to be installed as well, although only
those you will actually use.

Make tailor executable::

 $ chmod +x tailor

You can either run tailor where it is currently located, or move it
along with the vcpx directory to a location in your PATH.

There's even a standard setup.py that you may use to install the
script using Python conventional distutils.


Examples
========

1. Bootstrap a new tailored project, starting at upstream revision 10::

    $ tailor -b -s svn -R http://svn.server/path/to/svnrepo \
      --module /Product/trunk -r 10 --subdir Product ~/darcs/MyProduct 

2. Bootstrap a new product, fetching its whole CVS repository and
   storing under SVN

   a. First create a SVN repository and extract a working copy out of
      it. This is needed also when using a CVS or a Monotone target::

       $ cd $HOME
       $ svnadmin create --fs-type fsfs svn_repo
       $ svn co file://$HOME/svn_repo svnwc

   b. Then run tailor once to set up the repository::

       $ cd $HOME/svnwc
       $ tailor --source-kind cvs --target-kind svn --bootstrap \
                --repository :pserver:cvs.zope.org:/cvs-repository \
                --module CMF/CMFCore --revision INITIAL cmfcore

   c. Finally run it again without --bootstrap to bring the repository
      in sync with latest CVS version::

      $ tailor -D -v $HOME/svnwc/cmfcore

   .. note:: In the step a) above, you may want to install an
             appropriate hook in the repository to enable the
             propset command to operate on unversioned properties,
             as described in the `svn manual`__. Then you can
             specify '--use-svn-propset' option, and tailor will
             put the original author and timestamp in the proper
             svn metadata instead of appending them to the changelog.

             Other than the annoying repository manual intervention,
             this thread__ and this other__ explain why using
             ``-r{DATE}`` may produce strange results with this setup.

__ http://svn.haxx.se/users/archive-2005-07/0605.shtml
__ http://svn.haxx.se/users/archive-2005-03/0596.shtml


3. Showing each command bootstrap a new DARCS repos in
   "~/darcs/cmftopic" under which the upstream module will be
   extracted as "CMFTopic" (ie, the last component of the module
   name)::

    $ tailor -D -b -R :pserver:anonymous@cvs.zope.org:/cvs-repository/ \
                -m CMF/CMFTopic ~/darcs/cmftopic
              
4. Merge upstream changes since last update/bootstrap::

    $ tailor ~/svnwc/MyProduct

5. Migrate a whole SVN repository, starting from the beginning::

    $ tailor -b -s svn -R svn+ssh://caia/tmp/svn/repo --module / \
      --subdir testsvn --revision 1 testdir

   .. warning:: When using --revision with a SVN source repository, be
                sure that the path your are tracking was there at that
                revision, otherwise svn will exit with an error.

6. Bootstrap from a CVS branch, starting from a given point in time::

    $ tailor -b -s cvs -R :pserver:anonymous@server.org:/cvsroot/ \
                --revision 'unstable-branch 2001-12-25 23:26:48 UTC' \
                --module something new-darcs-repository

7. Bootstrap from a CVS branch, starting from the beginning of time::

    $ tailor.py -b -s cvs -R :pserver:anonymous@server.org:/cvsroot/ \
                --revision 'unstable-branch INITIAL' \
                --module something new-darcs-repository

8. Bootstrap a new SVN repository with tailor's master one. This
   demonstrate the new --target-repository feature::

    $ tailor -D -v -b -R http://nautilus.homeip.net/~lele/projects/tailor \
            -s darcs -t svn --target-repository file:///tmp/svnrepo \
            --target-module / /tmp/wc
    $ svn info file:///tmp/svnrepo [Status 1]
    $ svnadmin create --fs-type fsfs /tmp/svnrepo [Ok]
    $ svn co --quiet file:///tmp/svnrepo/ /tmp/wc [Ok]
    /tmp/wc $ darcs get --partial --verbose http://.../tailor tailor [Ok]
    /tmp/wc/tailor $ darcs changes --last 1 --xml-output [Ok]
    /tmp/wc $ svn add --quiet --no-auto-props --non-recursive tailor [Ok]
    ...

9. Given the configuration file shown below in `Config file format`_,
   the following command::

    $ tailor --configfile example.tailor --bootstrap

   is equivalent to this one::

    $ tailor --configfile example.tailor --bootstrap tailor

   in that they bootstrap respectively the default project(s) or
   the ones specified on the command line (and in this case there
   is just a single default project, tailor).

   This one instead::

    $ tailor --configfile example.tailor tailor tailor-reverse

   executes an update on both projects.


Resolving conflicts
===================

Should one of the replayed changes generate any conflict, tailor
will prompt the user to correct them. This is done after the upstream
patch has been applied and before the final commit on the target
system, so that manually tweaking the conflict can produce a clean
patch.


Shortcomings
============

Tailor currently suffers of the following reported problems:

a) It does not handle "empty" CVS checkouts, in other words you cannot
   bootstrap a project that has nothing in its CVS upstream
   repository, or from a point in time where this condition was true.

b) It's completely unsupported under Windows, evenif it now uses
   2.4's subprocess_ that seems able to hide Windows crazyness...

c) Monotone, Codeville and Baazar-NG are (currently) only supported as
   *target*

d) Specifying ``--subdir .`` may not work, in particular when dealing
   with remote CVS repositories (it does when the CVS repository is
   on local machine).
   
This list will always be incomplete, but I'll do my best to keep it
short :-)

.. _subprocess: http://www.lysator.liu.se/~astrand/popen5/


Config file format
==================

When your project is composed by multiple upstream modules, it is
easier to collect such information in a single file. This is done by
specifying the `--configfile` option with a file name as argument. In
this case, tailor will read the above information from a
standard Python ConfigParser file. 

For example::

    [DEFAULT]
    verbose = Yes
    target-module = None
    projects = tailor

    [tailor]
    root = /tmp/n9
    source = darcs:tailor
    target = svn:tailor
    refill-changelogs = Yes
    state-file = tailor.state

    [tailor-reverse]
    root = /tmp/n9
    source = svn:tailor
    target = darcs:tailor
    refill-changelogs = Yes
    state-file = reverse.state
    
    [svn:tailor]
    repository = file:///tmp/testtai
    module = /project1

    [darcs:tailor]
    repository = /home/lele/WiP/cvsync

.. hint:: If you execute the examples above with ``--verbose --debug``
          **and without** ``--configfile``, tailor will write a
          template config filled with the values specified by the
          other options. You can redirect 


Using a Python script as configuration file
-------------------------------------------

Instead of executing ``tailor --configfile project.tailor.conf``
you can prepend the following signature to the config itself::

  #!/usr/bin/env /path/to/tailor

Giving execute mode to it will permit the launch of the tailor
process by running the config script directly::

  $ ./project.tailor.conf

When a config file is signed in this way [#]_, either you pass it as
argument to ``--configfile`` or executed as above, tailor will
actually execute it as a full fledged Python script, that may define
functions that alter the behaviour of tailor itself.

A common usage of this functionality is to define so called `hooks`,
sequences of functions that are executed at particular points in
the tailorization process.

For example::

    #!/usr/bin/env /home/lele/WiP/tailor/tailor
    """
    [project]
    source = svn:repository
    target = darcs:repository
    before-commit = transform_crlf_to_lf
    ...
    """

    def transform_crlf_to_lf(context, changeset):
        for e in changeset.entries:
            ....

.. [#] Tailor does actually read just the first two bytes from the
       file, and compare them with "#!", so you are free to choose
       whatever syntax works in your environment.


State file
----------

The state file stores two things: the last upstream revision that
has been applied to the tree, and a sequence of pending (not yet
applied) changesets, that may be empty. In the latter case, tailor
will fetch latest changes from the upstream repository.

Interactive sessions
--------------------

Tailor offers an alternative way of driving its activity by using an
interactive session, bringing the configuration more similar to a
script.

Although this is still a work-in-progress, it's quickly becoming my
preferred usage, as its functionality cover the underlying tailor
features.

For example, this is the script I'm using to test it::

    cd /tmp
    state_file tailor.state
    source_kind darcs
    target_kind svn
    source_repository /home/lele/WiP/cvsync
    sub_directory cvsync
    refill_changelogs no
    patch_name_format [%(module)s] Changeset %(revision)s by %(author)s

that, as you can see, specifies the context needed by tailor to do its
work. With the following command line::

    $ tailor --verbose --interactive tailor.cmds
    Welcome to the Tailor interactive session: you can issue several commands
    with the usual `readline` facilities. With "help" you'll get a list of
    available commands.

    Current directory: /tmp
    Current state file: /tmp/tailor.state
    Current source kind: darcs
    Current target kind: svn
    Current source repository: /home/lele/WiP/cvsync
    Sub directory: cvsync
    Refill changelogs: False
    Patch name format: [%(module)s] Changeset %(revision)s by %(author)s
    tailor $ help
    
    Documented commands (type help <topic>):
    ========================================
    EOF                print_executed_commands  state_file
    bootstrap          refill_changelogs        sub_directory
    cd                 remove_first_log_line    target_kind
    current_directory  save                     target_module
    exit               source_kind              target_repository
    logfile            source_module
    patch_name_format  source_repository

    Undocumented commands:
    ======================
    help

    tailor $ help bootstrap
    
        Usage: bootstrap [revision]

        Checkout the initial upstream revision, by default HEAD (or
        specified by argument), then import the subtree into the
        target repository.

As another example, consider the following commands::

    cd /tmp/wc
    state_file tailor.state
    source_kind cvs
    target_kind darcs
    source_repository :pserver:anonymous@cvs.sourceforge.net:/cvsroot/buildbot
    source_module buildbot
    refill_changelogs false
    patch_name_format %(firstlogline)s
    print_executed_commands true
    bootstrap 2004/01/01 20:05:24
    update ask

Assuming ``/tmp/wc`` exists, executing them will first bootstrap a new
darcs repository with a snapshot at the given date of the upstream
sources, then will proceed with the update, asking confirmation about
any single changeset::

    ...
    Collected 586 upstream changesets
    Changeset 2004-01-07 00:42:07 by warner:
    * buildbot/changes/mail.py (parseFreshCVSMail): ....
    * test/test_mailparse.py (Test1.testMsg9): test for same

    Apply [Y/n/v/h/q]? h
    y: yes, apply it and keep going
    n: no, skip the current changeset
    v: view more detailed information
    q: do not apply the current changeset and stop iterating

    Apply [Y/n/v/h/q]? v
    Revision: 2004-01-07 00:42:07 by warner
    Date: 2004-01-07 00:42:07
    Author: warner
    Modified: ChangeLog,buildbot/changes/mail.py,test/test_mailparse.py
    Added: test/mail/msg9
    Log: * buildbot/changes/mail.py (parseFreshCVSMail): ...
    * test/test_mailparse.py (Test1.testMsg9): test for same
    
    Apply [Y/n/v/h/q]?

        
Further help
============

See the output of ``tailor -h`` for some further tips.  There's
also a `wiki page`_ that may give you some other hints.  The
development of Tailor is mainly driven by user requests at this point,
and the preferred comunication media is the dedicated `mailing list`_
[#]_.

.. _wiki page:
   http://www.darcs.net/DarcsWiki/Tailor

.. _mailing list:
   http://lists.zooko.com/mailman/listinfo/tailor
   
I will be more than happy to answer any doubt, question or suggestion
you may have on it. I'm usually hanging as "lelit" on the IRC channel
devoted to darcs on the `freenode.net` network. Do not hesitate to
contact me either by email or chatting there.

.. [#] I wish to say a big `Thank you` to `Zooko <zooko@zooko.com>`_,
       for hosting the ML and for supporting Tailor in several ways,
       from suggestions to bug reporting and fixing.


Author
======

Lele Gaifax <lele@nautilus.homeip.net>


Monotone support
----------------

Monotone_ support was kindly contributed by `Markus Schiltknecht
<markus@bluegap.ch>`_ and further developed by `rghetta
<birrachiara@tin.it>`_.  Since I'm not currently using monotone (so
little time, so many VCSs...) I'm not in position to test it out
properly, but I'll do my best to keep it in sync, maybe with your
support :-)

.. _monotone: http://www.venge.net/monotone/


License
=======

Tailor is distribuited under the `GNU General Public License`__.

__ http://www.fsf.org/licensing/licenses/gpl.html


About this document
===================

This document and most of the internal documention use the
reStructuredText format so that it can be easily converted into other
formats, such as HTML.  For more information about this, please see:

  http://docutils.sourceforge.net/rst.html


.. vim:ft=rest
.. Local Variables:
.. mode: rst
.. End: