parent dirs ordered after child dirs in large svn commit

[pistos]

As per

 http://lists.zooko.com/pipermail/tailor/2006-January/000310.html

CVS source to Subversion target. When doing a large svn commit, and the list of files must be split up across multiple commit commands, parent subdirs are being commited later than children subdirs, causing this error:

'/webdev/sandbox/beta/images/headers' is not under version control and is not part of the commit, yet its child
'/webdev/sandbox/beta/images/headers/welcome.gif' is part of the commit

[pistos]

If I do the commit manually (outside of tailor), but using just a parent dir (not a full file list), and then continue the tailor processing, it seems to work.

[lele]

Anybody saw this problem recently? I guess it's been fixed by recent changes to the abstract target _replayChangeset().

[fullermd]

I see this (at least, the same effect) currently in a CVS->bzr conversion. Tested on both 0.9.23 release, and a fresh 'darcs get'. This isn't a particularly large commit (12 or 13 files), but the directories end up at the end of the 'Added:' list, and so it bombs out in the bzrlib when it tries to add a file before its parent directory.

[lele]

This is hopefully fixed by [1157]. The problem wasn't related to command line splits, but rather by the way CVS (and HG?) alters the changeset introducing fake ADDs on new directories.

[fullermd]

With the latest changes, I still get the error, only now instead of the directory ending up at the end of the 'adding' list, it doesn't appear at all.

I can reproduce this with a simple CVS repo, created with the following steps:

1. Touch a file 'foo' and import as the first revision.
2. Create a dir 'bar' with a file 'baz' in it, and commit as the second revision.
3. Run tailor over it.

It will try, when converting the second commit, to add just 'bar/baz', but not 'bar/'.

[lele]

Johan Rydberg offered this ugly but working quick fix:  http://www.ginandtonique.org/jrydberg/bzr.tailor.patch.1

Hopefully I'll find the time to check what's going on.

[licquia]

I've written a test which duplicates the bug for me. It's at  http://bzr.licquia.org/tailor-add-order-fix (that's a bzr repository; sorry, but I couldn't wrap my head around darcs).

This is a real pain for an important project of mine, and the hack patch doesn't quite fix it (it makes the opposite error, sometimes trying to add a directory twice). So, if you don't get the time to find it, perhaps I'll have a fix soon. Any hints would be appreciated. :-)

FWIW, it seems that the CVS log parser is not picking up when directories and files in them are added in the same changeset, and the code to do the fake adds isn't noticing the missing directory add.

[lele]

I will happily merge your tests, even if that'll require some bzr attempts. Unfortunately this cannot happen in the following days, but most probably during the weekend. Sorry about taking so long.

This evening I'll try to get a closer view of the tests.

[licquia]

Hey, no problem. I understand how it is.

I think I may have a fix. It's in two parts.

_willApplyChangeset() in cvsps.py creates needed directories in CVS, but doesn't add them to the changeset. So I deleted the entire function (which seems to do nothing else), and moved directory creation into _applyChangeset(). There was a case where updates are changed into adds where this is done; I used that as a model.

That gets the directory add into the changeset, but after the file it's trying to add (which is the original bug). So, I made it possible to insert entries into a Changeset, and used that when adding directories so they precede the files they contain.

It passes one of my tests. I test both cvs and cvsps, and for some reason, the cvsps test has decided to throw EmptyRepositoryFoolsMe? now (where it didn't before). I'm going to see if I can figure that out.

All results so far are in my public bzr repo mentioned above.

[lele]

This should be fixed by [1246]: thanks Jeff for providing a reference fix, I slightly edited your changes.