Tailor 1.0 ########## .. contents:: About ===== Tailor is a tool to migrate changesets between ArX_, `Bazaar-NG`_ CVS_, Codeville_, Darcs_, Mercurial_, Monotone_, Tla_ and Subversion_ [#]_ repositories. This script makes it easier to keep the upstream changes merged in a branch of a product, storing needed information such as the upstream URI and revision in special properties on the branched directory. The following ascii-art illustrates the usual scenario:: +------------+ +------------+ +--------------+ | Immutable | | Working | | Upstream CVS |-------->| darcs |----------->| darcs | | repository | tailor | repository | darcs pull | repository | +--------------+ +------------+ +------------+ |^ || || v| User Ideally you should be able to swap and replace "CVS server" and "darcs repository" with any combination of the supported systems. A more convoluted setup shows how brave people are using it to get a two way sync:: +----------+ +--------+ +--------+ +---------+ | | -----> | hybrid | darcs | | ----> | my | | upstream | tailor | CVS | -----> | master | darcs | working | | CVS | <----- | darcs | <----- | darcs | <---- | darcs | | | | sync | tailor | | | | +----------+ +--------+ +--------+ +---------+ (cron) (cron) .. [#] ArX, Monotone, Codeville, Mercurial and Baazar-NG systems may be used only as the `target` backend, since the `source` support isn't coded yet. Contributions on these backends will be very appreciated, since I do not use them enough to figure out the best way to get pending changes and build tailor ChangeSets out of them. To the opposite, Tla is supported only as a source system. .. _arx: http://www.nongnu.org/arx/ .. _bazaar-ng: http://www.bazaar-ng.org/ .. _codeville: http://www.codeville.org/ .. _cvs: http://www.nongnu.org/cvs/ .. _darcs: http://www.darcs.net/ .. _mercurial: http://www.selenic.com/mercurial/ .. _monotone: http://www.venge.net/monotone/ .. _subversion: http://subversion.tigris.org/ .. _tla: http://www.gnuarch.org/arch/index.html Installation ============ tailor is written in Python, and thus Python must be installed on your system to use it. It has been successfully used with Python 2.3 and 2.4. Since it relies on external tools to do the real work such as `cvs`, `darcs` [#]_ and `svn`, they need to be installed as well, although only those you will actually use. Make tailor executable:: $ chmod +x tailor You can either run tailor where it is currently located, or move it along with the vcpx directory to a location in your PATH. There's even a standard setup.py that you may use to install the script using Python conventional distutils. .. [#] Darcs 1.0.2 is too old, 1.0.3 is good, 1.0.4 (the fourth release candidate is under final testing) is recommended since it's faster in most operations! Testing ======= You can run the test suite with the following command line:: $ tailor test -v that runs all the tests in sequence, or:: $ tailor test -v TailorTest to trigger just a subset of them. Operation ========= tailor needs now a configuration file that collects the various bits of information it needs to do it's job. The simplest way of starting out a new configuration is by omitting the --configfile command line option, and specifying the other as needed plus --verbose: in this situation, tailor will print out an equivalent configuration that you can redirect to a file, that you later will pass as --configfile. Examples -------- 1. Bootstrap a new tailored project, starting at upstream revision 10 a. First create a config file:: $ tailor --verbose -s svn -R http://svn.server/path/to/svnrepo \ --module /Product/trunk -r 10 --subdir Product \ ~/darcs/MyProduct > myproject.tailor b. Modify it as you like (mostly adjusting root-directories and the like):: $ emacs myproject.tailor c. Run tailor on it:: $ tailor --configfile myproject.tailor 2. Bootstrap a new product, fetching its whole CVS repository and storing under SVN a. First create a config file:: $ tailor --verbose --source-kind cvs --target-kind svn \ --repository :pserver:cvs.zope.org:/cvs-repository \ --module CMF/CMFCore --revision INITIAL \ --target-repository file:///some/where/svnrepo \ --target-module / cmfcore > cmfcore.tailor b. Modify it as you like (mostly adjusting root-directories and the like):: $ emacs cmfcore.tailor c. Run tailor on it once, to bootstrap the project:: $ tailor -D -v --configfile cmfcore.tailor If the target repository is on the local filesystem (ie, it starts with ``file:///``) and it does not exist, tailor creates a new empty Subversion repository at the specified location. .. note:: Before step d) below, you may want to install an appropriate hook in the repository to enable the propset command to operate on unversioned properties, as described in the `svn manual`__. Then you can specify '--use-svn-propset' option, and tailor will put the original author and timestamp in the proper svn metadata instead of appending them to the changelog. Other than the annoying repository manual intervention, this thread__ and this other__ explain why using ``-r{DATE}`` may produce strange results with this setup. d. Run tailor again and again, to sync up with latest changes:: $ tailor -D -v --configfile myproject.tailor __ http://svnbook.red-bean.com/en/1.0/ch05s02.html#svn-ch-5-sect-2.1 __ http://svn.haxx.se/users/archive-2005-07/0605.shtml __ http://svn.haxx.se/users/archive-2005-03/0596.shtml 3. Given the configuration file shown below in `Config file format`_, the following command:: $ tailor --configfile example.tailor is equivalent to this one:: $ tailor --configfile example.tailor tailor in that they operate respectively on the default project(s) or the ones specified on the command line (and in this case there is just a single default project, tailor). This one instead:: $ tailor --configfile example.tailor tailor tailor-reverse operates on both projects. CVS start-revision ------------------ With CVS, you can specify a particular *point in time* specifying a `start-revision` with a timestamp like ``2001-12-25 23:26:48 UTC``. To specify also a particular `branch`, prepend it before the timestamp, as in ``unstable-branch 2001-12-25 23:26:48 UTC``. Resolving conflicts =================== Should one of the replayed changes generate any conflict, tailor will prompt the user to correct them. This is done after the upstream patch has been applied and before the final commit on the target system, so that manually tweaking the conflict can produce a clean patch. Shortcomings ============ Tailor currently suffers of the following reported problems: a) It does not handle "empty" CVS checkouts, in other words you cannot bootstrap a project that has nothing in its CVS upstream repository, or from a point in time where this condition was true. b) It's completely unsupported under Windows, evenif it now uses 2.4's subprocess_ that seems able to hide Windows crazyness... c) ArX, Monotone, Codeville and Baazar-NG are (currently) only supported as *target*; Tla only as *source*. d) Specifying ``--subdir .`` may not work, in particular when dealing with remote CVS repositories (it does when the CVS repository is on local machine). This list will always be incomplete, but I'll do my best to keep it short :-) .. _subprocess: http://www.lysator.liu.se/~astrand/popen5/ Config file format ================== When your project is composed by multiple upstream modules, it is easier to collect such information in a single file. This is done by specifying the `--configfile` option with a file name as argument. In this case, tailor will read the above information from a standard Python ConfigParser file. For example:: [DEFAULT] verbose = True projects = tailor [tailor] root-directory = /tmp/n9 source = darcs:tailor target = svn:tailor dont-refill-changelogs = True state-file = tailor.state [tailor-reverse] root-directory = /tmp/n9 source = svn:tailor target = darcs:tailor state-file = reverse.state [svn:tailor] repository = file:///tmp/testtai module = /project1 subdir = svnside [darcs:tailor] repository = ~/WiP/cvsync subdir = darcside The configuration may hold one or more `projects`_ and two or more `repositories`_: project names do not contains colons ":", repository names must and the first part of the name before the colon specify the kind of the repository. So, the above example contains two projects, one that goes from `darcs` to `subversion`, the other in the opposite direction. The ``[DEFAULT]`` section contains the default values, that will be used when a specific setting is missing from the particular section. You can specify on which project tailor should operate by giving its name on the command line, even more than one. When not explicitly given, tailor will look at ``projects`` in the ``[DEFAULT]`` section, and if its missing it will loop over all projects in the configuration. The following simpler config just go in one direction, for a single project, so no need neither for ``[DEFAULT].projects`` nor command line arguments:: [pxlib] source = cvs:pxlib target = hg:pxlib root-directory = ~/mypxlib start-revision = INITIAL subdir = pxlib [cvs:pxlib] repository = :pserver:anonymous@cvs.sf.net:/cvsroot/pxlib module = pxlib [hg:pxlib] hg-command = /usr/local/bin/hg This will use a single directory, ``pxlib`` to contain both the source and the target system. If you prefer keeping the separated, you just need to specify a different directory for each repository, as in:: [pxlib] source = cvs:pxlib target = hg:pxlib root-directory = ~/mypxlib start-revision = INITIAL [cvs:pxlib] repository = :pserver:anonymous@cvs.sf.net:/cvsroot/pxlib module = pxlib subdir = original [hg:pxlib] hg-command = /usr/local/bin/hg subdir = migrated This will extract upstream CVS sources into ``~/mypxlib/original``, and create a new Mercurial repository in ``~/mypxlib/migrated``. Configuration sections ---------------------- Projects ~~~~~~~~ A project is identified by a section whose name does not contain any colon (":") character, and configured with the following values: root-directory This is where all the fun will happen: this directory will contain the source and the target working copy, and usually the state and the log file. It support the conventional `~user` to indicate user's home directory. subdir This is the subdirectory, relative to the `root-directory`, where tailor will extract the source working copy. It may be '.' for some backend kinds. state-file Name of the state file needed to store tailor last activity. source The source repository: a repository name is something like "darcs:somename", that will be loaded from the homonymous section in the configuration. target The counterpart of `source`, the repository that will receive the changes coming from there. Non mandatory options: verbose Print the commands as they are executed. debug Print also their output. before-commit This is a function name, or a sequence of function names enclosed by brackets, that will be executed on each changeset just before it get replayed on the target system: this may be used to perform any kind of alteration on the content of the changeset, or to skip some of them. after-commit This is a function name, or a sequence of function names enclosed by brackets, that will be executed on each changeset just after the commit on the target system: this may be used for example to create a tag. subdir The name of the subdirectory, under ``root-directory``, that will contain the source and target repositories/working directories. Repositories ~~~~~~~~~~~~ All the section whose name contains at least one colon character denote a repository. A single repository may be shared by zero, one or more projects. The first part of the name up to the first colon indicates the `kind` of the repository, one of ``arx``, ``bzr``, ``bzrng``, ``cdv``, ``cvs``, ``darcs``, ``hg``, ``monotone``, ``svn`` and ``tla``. When a repository is used as a `source`, it must indicate its origin with ``repository``, and for some backends also a ``module``, but are not required when it's a target system, even if some backend may use the information to create the target repository (like ``svn`` backend does). Each repository may then impose a particular external binary to be used, as done in the example above for ``hg``. Actually only one of them, `bzrng`, accepts an option ``python-path``, to indicate where the BazaarNG Python library is located. A repository may also have it's own ``subdir``: when the `source` and `target` repositories use different subdirectories, tailor uses ``rsync`` to copy the changes between the two after each applied changeset. Using a Python script as configuration file ------------------------------------------- Instead of executing ``tailor --configfile project.tailor.conf`` you can prepend the following signature to the config itself:: #!/usr/bin/env /path/to/tailor Giving execute mode to it will permit the launch of the tailor process by running the config script directly:: $ ./project.tailor.conf When a config file is signed in this way [#]_, either you pass it as argument to ``--configfile`` or executed as above, tailor will actually execute it as a full fledged Python script, that may define functions that alter the behaviour of tailor itself. A common usage of this functionality is to define so called `hooks`, sequences of functions that are executed at particular points in the tailorization process. Just to illustrate the functionality, consider the following example:: #!/usr/bin/env tailor """ [DEFAULT] debug = False verbose = True [project] target = bzrng:target root-directory = /tmp/prova state-file = tailor.state source = darcs:source before-commit = before after-commit = after start-revision = Almost arbitrarily tagging this as version 0.8 [bzrng:target] python-path = /opt/src/bzr.dev subdir = bzrside [darcs:source] repository = /home/lele/WiP/cvsync subdir = darcside """ def before(wd, changeset): print "BEFORE", changeset changeset.author = "LELE" return changeset def after(wd, changeset): print "AFTER", changeset With the above in a `script` called say ``tester``, just doing:: $ chmod 755 tester $ ./tester will migrate the history from a darcs repository to a bazaar-ng one, forcing the author to a well-known name :-) .. [#] Tailor does actually read just the first two bytes from the file, and compare them with "#!", so you are free to choose whatever syntax works in your environment. State file ---------- The state file stores two things: the last upstream revision that has been applied to the tree, and a sequence of pending (not yet applied) changesets, that may be empty. In the latter case, tailor will fetch latest changes from the upstream repository. Further help ============ See the output of ``tailor -h`` for some further tips. There's also a `wiki page`_ that may give you some other hints. The development of Tailor is mainly driven by user requests at this point, and the preferred comunication media is the dedicated `mailing list`_ [#]_. .. _wiki page: http://www.darcs.net/DarcsWiki/Tailor .. _mailing list: http://lists.zooko.com/mailman/listinfo/tailor I will be more than happy to answer any doubt, question or suggestion you may have on it. I'm usually hanging as "lelit" on the IRC channel devoted to darcs on the `freenode.net` network. Do not hesitate to contact me either by email or chatting there. .. [#] I wish to say a big `Thank you` to `Zooko `_, for hosting the ML and for supporting Tailor in several ways, from suggestions to bug reporting and fixing. Authors ======= Lele Gaifax Since I'm not currently using all the supported systems (so little time, so many VCSs...) I'm not in position to test them out properly, but I'll do my best to keep them in sync, maybe with your support :-) ArX support ----------- ArX_ support was contributed by `Walter Landry `_. Bazaar-NG support ----------------- `Bazaar-NG`_ support was contributed by `Johan Rydberg `_. Monotone support ---------------- Monotone_ support was kindly contributed by `Markus Schiltknecht `_ and further developed by `rghetta `_. Tla support ----------- Tla_ support was contributed by `Robin Farine `_. License ======= Tailor is distribuited under the `GNU General Public License`__. __ http://www.fsf.org/licensing/licenses/gpl.html About this document =================== This document and most of the internal documention use the reStructuredText format so that it can be easily converted into other formats, such as HTML. For more information about this, please see: http://docutils.sourceforge.net/rst.html .. vim:ft=rest .. Local Variables: .. mode: rst .. End: