art taylor

I have finally come to the conclusion that Subversion is evil, and a general step backwards in software configuration management.

I had the same opinion prior to using it, but now, after a week of dorking with it, I can say it has lived up to its goal of being a better cvs than cvs. That can be compared to a goal of being a better Titanic than the Titanic.

I am very branch happy, and this is where I started itching with Subversion before I even dug into the manual. cvs is not renowned for being friendly with people who like branches, and I've spent a lot of time faking branches with tags in cvs.

I would prefer to use darcs, which is what I use at home, or bazaar-ng, but honestly, there just isn't the tools support for either of those that is available for Subversion, and training devs on the scm-du-jour is not billable activity. There's a slick IDEA plugin for Subversion, and along with maven 2's idea:idea plugin, you can have a version-controlled, integrated, happy project up in about ninety seconds. Not counting the time it takes IDEA to start up, read all the jar contents, and become usable, of course.

So I've been spending a lot of time trying to figure out why I can't grok branching on Subversion, and I really think it's just that I have this mental block. I don't require a lot, and I don't require anything that isn't available via Perforce, bzr, darcs, etc.

I've given up and accepted that I'll have to adhere to Subversion's fake branch and tag structure. I've created scripts to do the svn mkdir $PROJECT_NAME; svn mkdir $PROJECT_NAME/trunk; svn mkdir $PROJECT_NAME/tags; svn mkdir $PROJECT_NAME/branches; svn commit -m "created stupid base directory structure."

I've also created scripts that sort of do the merges from the trunk to a branch, and vice versa. I can't believe was a pain that is to do, because you really do have to think of them as independent file trees.

Imagine doing branching on a non-versioning filesystem using a weak diff program, and you're just about there. It required a little too much work to figure out "ok, I've branched at this revision, and I've made these changes, and I rebased/integrated/merged down at this point, then what changes do I need to make to push things back to trunk?" When the manual involves manual examination of log output to figure this out, you know it's a JV tool.

This is a very simplified workflow demonstrating a typical arrangement with two developers. With the series of events (assume time increases downward) illustrated here, everything is more or less ok until the owner of Branch 1 merges down the changes from t1.1, integrating the changes made by the owner of Branch 2. This merge should go fairly smoothly, but what's incredibly odd is Subversion's behavior after this.

If the owner of Branch 1 makes subsequent changes and then tries to merge b1.4 back up to the Trunk, there is no automated method to do this. All of the manuals seem to miss the fact that there are changes in b1.1 and b1.2 that do not exist in the Trunk, and advise that you manipulate the diffs manually, because there is no way to guarantee idempotent operations.

I foresee trying to use Subversion just as long as I have 3-5 developers to worry about, where we can create a lot of tiny branches, and commit them to the trunk. Then there will be a period of chaos when everyone works right on the trunk, and at that point, we'll be able to justify the licensing cost for a real tool, whether it be Perforce, Clearcase, or whatever.

All developers working on a single line of code is the dumbest idea in the entire universe, and I say that having been a Peugeot owner. What happens is that the less-disciplined developers hold back a month's worth of code (hopefully they're at least doing daily updates) and then check it all in at once at the end. If their drive blows up, or any of a number of data integrity issues arise, that's $50k+ down the drain (40 hours x 4 weeks x $200+/hr + $20k in salary, benefits, office space, etc.). Otherwise, they're making a lot of tiny checkins, and that's nice, but you're left with a tree in a non-coherent, incomplete, non-releasable state.

Sure, that feature your cave-dweller is working on may be a make-or-break feature for your client despite the fact that only a moron would be working in isolation for a month. However, that stuff happens, and if you need to demonstrate progress on short notice, it's far easier to build off the trunk and know you'll have a single functional unit than wonder which pieces are stubbed out, not working, or just plain spaghetti.

The best you can hope for in this situation is that developers will use their own version control for their branches. "Branch 1" will be in bzr on some guy's Windows box and "Branch 2" will be in darcs on some guy's Mac. They'll both be snapshots of the Trunk, and hopefully everything will just work out.

As with programming languages and development environments, I love that most people are happy working with stone tools, and if we want to live in the 21st century, we have to wear hair shirts.

Sometimes it's good to think twice, but those who know me know I just consider that a waste of time. Measure once, cut thirty times, that's my motto.

I was pondering the Subversion problem I was having last week, and realized that I could solve the problem with some disgusting and brittle hackery.

According to the doco, Subversion repository copies are very inexpensive, basically being copy-on-write links, like those short-lived CD-based overlay filesystems.¹ Tags, therefore, were especially lightweight, with an O(n) disk footprint based on the number of files in the "tag". Remember, these aren't tags, but rather, someone's answer to "How would we implement tags if we only had a filesystem?" It's the equivalent of a symbolic link farm, but more efficient.

This, combined with the retarded svn merge concept (which they admit now to be nothing more than "diff and apply") allowed me to cobble something together, using the second-nastiest shell scripting I've ever done. Witness the following diagram.

It's basically the same as the previous diagram, with one modification. As Branch 1 is created, a snapshot is made of Trunk at the same time. I might pick a name like [branch]-latest-trunk-merge-point but it could be anything.

As the developer working on Branch 2 (reduced in detail as it is here merely for illustrative purposes) merges up, he creates Trunk version t1.1, necessitating a merge down to Branch 1. Just doing a merge would create all kinds of havoc, possibly overwriting or deleting files added or modified on Branch 1. Of course, one could just svn revert, but it's the well-known pain in the butt that the Subversion developers just wave away with "look at svn log and see where things happened, then use the revision numbers!"

With our trailing Trunk tag, though, we can take the snapshot taken at the last merge-down point (in this case, branch creation) and diff it against the current HEAD of Trunk (yes, I realize the cranio-posterial reference here is not coincidental), and apply that to Branch 1.

Here's a sticky bit, though. Assuming all is well and there are no conflicts (or even if there are), we can't just do a straight svn commit. We'd leave the Branch 1 Trunk snapshot back in an inappropriate state, which would cause us to include older changes (which cannot be guaranteed to apply idempotently) in the next merge down. So we have to use a "special" commit. (I am aware of the euphemisms for "retarded" that are popping up in the context of my solution.) This commit-from-trunk-merge needs to do an svn commit, and, if successful, delete the old tag and recreate it from the HEAD of Trunk.

When it's time to merge Branch 1 up to Trunk, another diff is taken, pulling just the changes to take the most recent version of Trunk to Branch 1, version b1.4.

All of this relies on a bit of discipline, which complicated the script significantly. I had to use the first form of the svn merge command (svn merge svn+ssh://svn.example.com/repos/tags/Branch2-trunk-tag svn+ssh://svn.example.com/repos/tags/trunk branches/Branch2) for performance's sake, relying on the remote svnserve process. I had to check to make sure that the working copies were all in sync with the repository before attempting merges in either direction, although I suspect svn would have caught me with this anyway. (Developers should be merging down from Trunk frequently anyway, so I don't feel bad about this.) I have to make sure all the developers use the correct merge and commit statements to keep the repository intact, and I will probably have to resort to locking the repository during these operations, a thing I don't really look forward to doing.

I think I can support about 4-6 developers using this mechanism, more if they understand version control and branching. Beyond that, and I think I'm going to have to be very aggressive in breaking up projects into multiple modules and befouling the world with yet another SOA.

The underlying problem is that, really, you can apply the diff of any two sets of files to any other set of files, as long as the gnu diff can be applied to whatever target you have in mind. Sure, this is sweet and powerful in some fanciful situation, but for the common case of a branching project, it's like a gun with a hairtrigger and a secret firecracker set in the handle to give you a surprise.

1 Holy crap, this didn't die.