New topics: Your Pet, IOU, Baby IQ, The Poisons, Birther II, Games, Future Power

Welcome to the Tech Space!

Check out our Hardware

Skip to end of metadata
Go to start of metadata

I am already using SVN, a centralized source code repository, as my source code control system, and I am quite happy with it. I have taken the time to go through a lot of my old projects and put them in SVN. And for some projects where I had sources for multiple versions, I went through the effort to check those versions in chronologically in order to recreate some history for some of the projects. I have one large repo for all my projects, regardless of client.

However a client recently requested to regularly receive a copy of all their projects in a way that a competent developer could easily rebuild all the projects. It was what if I "get hit by a bus" worry. Not so easy to give them a copy of my whole repo, and unwind all the other projects from it. Or at least I think so, I don't really know, haven't tried. The thought of having to give the client copies of all the tools I used, environment setups, describing it all for a neophyte developer, etc., it is a bit daunting. I have already setup development environment in a virtual machine that can be handed to them, but the client doesn't really understand the benefits of that approach.

Another client is working with mercurial, which is a distributed version control system. The neat thing about distributed VCS is that everyone developing gets their own copy of the repo. There can still be a central repo that everyone updates. But a developer will start working on a mercurial project by checking out his own clone of the repo. He can play to his hearts content, taking advantage of version control of local changes, to do and undo, whatever is needed, without disturbing the central repository. Later, when it is time to checkin some changes, he can push changes from his local copy to the central repo.

This distributed source code control system was a new concept to me, but I quickly saw this as an answer to my first client's need. I could just periodically clone my central repo and give them a copy of the clone. I could maintain the canonical central repo for my own needs. Someday, if they want to fork, they got their full history with them. If I keep my central repository private, I can give them a clone when they pay their bill of exactly the version they just paid for. Everyone can be protected without a lot of extra work.

With everything in a central repo like Subversion, and just checking out the current version of everything, it is easy to understand. Sometimes at milestones I checkin binaries to my central SVN, so if I want later to get a runable copy of my project, its ght there. Since I dont check in binaries real often, I dont think about how much they cause the overall size of the repository to grow.

But with Mercurial, the issue of binaries is something to think about. Mercurial pulls down a complete history of changes for everything, literally a clone of the central repository. If there were multiple copies of binaries there, this could get very large. So there are a lot of discussions online about relative sizes of the different repository solutions. I've read a lot about the pros and cons of the various solutions.

I've decided that the best way to address these issues is to maintain a variety of solutions:

  • continue to maintain a subversion repository for project binaries (although perhaps just a regular file store would do as well)
  • maintain a regular file share for binaries of tools, third party libraries, and their installers
  • use mercurial for project source code

Keeping project source code separate from the rest makes it easy and cheap to pull down multiple clones of the project source. Tools and libraries are the kinds of things that are probably installed once, and put in central locations, and dont need to weight down the source code checkout.

Some of the things I need to figure out:

  • How to allow subversion and mercurial to work with the same set of source code files in the same directory
  • How to setup a mercurial server

An alternative to getting mercurial and subversion happy with the same files in the same directory:

  • "I'd use my own SCM. I'd get the files from Python's Subversion server, then check them right back in to my own SCM, then use that as usual during development." link

Here is how to implement this:

  1. Check out sources to the read-only Subversion tree. This is original source tree without changes, often called "trunk" or "mainline"
  2. Check that tree in to Mercurial. This tree will always be read-only. You only pull changes from this tree, changes are never pushed into this tree
  3. Clone that tree to a second directory. This is the tree where you do your work, your working directory.
  4. When there are updates to the Subversion tree, update your read-only tree and pull the changes into your work tree.
  5. When it's time to make your patch, diff your work tree to the read-only tree.

Like a waterfall: Subversion -> to read-only-mercurial -> to working directory

  • 1. Get the sources from subversion
    • Example subversion commandline to get trunk:
      % svn co http://svn.python.org/projects/python/branches/p3yk trunk
    • 1A. Turn the directory into a mercurial respository
      •  % hg init trunk 
    • 1B. check in the files, without subversion metadata, and without other generated files
      • create trunk/.hgignore with these contents:
        syntax: glob
        .svn\
        *.o
        
        # Windows build output
        *.obj
        *.pch
        *.idb
        *.pdb
        *.exp
        *.exe
        *.lib
        *.dll
        *.suo
        *.ncb
        *.ilk
        *.sbr
        *.res
        ~BuildLog.htm
        BuildLog.htm
        
        # Python-specific build output
        *.pyd
        *.pyc
        *.pyo
        PC\python_nt_d.h
        PC\pythonnt_rc_d.h
        PC\python_nt.h
        PC\pythonnt_rc.h
        
    • 1C. Check the tree into mercurial:
      % cd trunk
      % hg add
      
    • 1D. Create the branch (which is just a clone of this first repository!)
      In Mercurial, a branch is a repository. Nothing more or less. A repository is a branch. Repeat the soothing mantra.
      Source
      • % hg branch larry@hastings.org:py3k:trunk
    • 1E. Checkin to mercurial
      % hg ci

Resources:

Source code management alternatives:

  • zip backups - "hillbilly SCM" link
  • Git
  • Centralized version control systems
    • Sourcesafe
    • Perforce
    • Subversion
      • it's branch support sucks link
  • Distributed Version Control Systems
Labels:
None
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.