Component-based development with Plastic SCM

In this article, we'll discuss Plastic SCM's support for an increasingly common scenario: managing a development project that consists of multiple independent software components. This methodology is becoming almost inevitable, given the reality that development is often distributed across multiple locations -- maybe multiple continents! Given the availability of high-quality open source code at little or no cost, a project's success can hinge on the ability to manage multiple, mutually-independent development streams.

1. What is a Component?

From an SCM perspective, a component is a set of development artifacts that evolve on the same schedule. For example: source code for a shared library that is used by several of a product's subsystems, accompanying multimedia files, and the library runtime. It has nothing to do with how code is organized into functions, methods, and classes. Instead, it has to do with branching and labeling policies, and with release calendars.

Geography might also be a factor. If data-transmission latencies are significant for your organization -- given the distance and volume of data involved -- it makes sense to regard some parts of the system as separate components, to be updated occasionally, rather than continually.

2. The Scenario

Let's look at a simple scenario, which we'll use throughout this article to illustrate Plastic SCM's component-oriented features. Let's say a development project has been using a single Plastic SCM repository, called myproj, which stores a typical directory structure:

For simplicity, let's suppose that developers work on the /main branch, so that their workspaces use this selector:

repository "myproj"
path "/"
br "/main"
co "/main"

Now, suppose that the project leader has contracted another development organization to provide a test suite for the project. That team uses a separate repository, myproj_test (and probably, a different repository server with a different set of users):

 

It quickly becomes clear that in order for the test suite to work optimally (or at all!), data from both repositories must be assembled into a single directory hierarchy. In particular, it is decided that:

The contents of the myproj_test repository must appear to be located within myproj, as a subdirectory named test.

3. The Solution

Plastic SCM's workspace configuration language makes it easy to combine data from multiple repositories into a single directory hierarchy. Borrowing from traditional Unix/Linux file system terminology, a workspace can mount a repository at any pathname. More precisely:

  • The workspace mounts one particular repository, termed the base repository, at the workspace's root directory. This is accomplished with a rule in the workspace's selector that begins like this:

repository "repository-name"

... or equivalently:

repository "repository-name" mount "/"

  • Any number of other repositories can be mounted as sub-repositories at locations within the base repository, using rules in the workspace's selector that begin like this:

repository "sub-repository-name" mount "pathname-within-base-repository"

For such a rule to work, the "mount point" must exist within the base repository, as a directory item. It's not enough to create a directory as a private object within the workspace - the mount-point directory must be placed under source control. (To eliminate possible confusion, make sure the mount-point directory is empty. Its contents, if any, will not be accessible in a workspace that mounts another repository at that point.)

Here's a step-by-step "recipe" for fulfilling our scenario's requirement to combine data from two repositories.

  1. Start with a workspace that is configured to use the /main branch of the myproj repository:

    repository "myproj"
    path "/"
    br "/main"
    co "/main"
  2. Go to the root directory of the workspace, and create a subdirectory named test.
  3. Place subdirectory test under source control.
  4. Modify the workspace's selector, to mount repository myproj_test on the test subdirectory:

repository "myproj_test" mount "/test"
path "/"
br "/main"
co "/main"

repository "myproj"
path "/"
br "/main"
co "/main"

Note that you must work "from the bottom up" - the base repository is configured by the last rule; the rules that configure one or more sub-repositories must occur earlier in the selector.

The workspace now combines data from the two repositories into a single directory hierarchy:

 

3.1. Shared Library Example

As we mentioned earlier, a set of one or more shared libraries makes a typical component. A repository named libraries might have two subdirectories, so that it looks like this when loaded into its own workspace:

The overall product might have multiple subsystems, each in its own repository. You can configure a workspace to load both a subsystem's repository and the common libraries repository. For example, this selector:

repository "libraries" mount "/lib"
path "/"
br "/main"
co "/main"

repository "salad_prefs"
path "/"
br "/main"
co "/main"

... combines the salad_prefs and libraries repositories:

Similarly, this selector:

repository "libraries" mount "/regexp/libs"
path "/"
br "/main"
co "/main"

repository "salad_search"
path "/"
br "/main"
co "/main"

... combines the salad_search and libraries repositories:

Note that these two workspaces mount libraries as a sub-repository at different levels of the directory hierarchy.

4. Using a Multiple-Repository Workspace

We've seen that a workspace can combine data from multiple "component" repositories into a single directory hierarchy. This makes it easy to assemble all the data required for system builds, global code analyses, system tests, and so on. But the workspace can also be used for ongoing development.

For the most part, day-to-day operations (such as Checkout and Checkin) work the same way as in a single-repository workspace. But it's important to understand that each repository retains its separate identity in all Plastic SCM operations. For example:

  • The Items view includes a Repository column, indicating each item's "home".
  • Each repository has its own sequence of changesets. A single Checkin command might involve 3 items from repository myproj and 2 items from repository myproj_test. This command would create a separate changeset in each repository (with the same comment string, but not necessarily the same changeset number).
  • Each repository has its own set of labels. Repository myproj might have labels BL01, BL02, ... and myproj_test might have WEEK.1, WEEK.2, ... If you want all of a workspace's repositories to have the same set of labels, the Label view's Create new label command makes it easy to implement this policy:

Likewise, invoking the Apply label to workspace contents command on the base repository applies the label to its sub-repositories, also -in those sub-repositories where the label exists.

  • Each repository has its own set of branches. When you invoke the BranchExplorer, you are prompted to indicate which repository's branch hierarchy you wish to view.

5. Configuring a Multiple-Repository Workspace

So far, we've combined multiple repositories into a single directory hierarchy, but haven't fully shown how they are independent components. It's easy- just modify the workspace's selector. Here are some examples:

  • Incorporate the contents of the myproj_test repository on a read-only basis:

  • Use the configuration of repository myproj that is labeled BL04, along with the configuration of repository myproj_test that is labeled WEEK.3 (both configurations will be read-only):

Instead of editing the selector manually to insert the label clauses, you can use the Labels view, which shows the labels for all the repositories used by the workspace:

  • Work on the task428 branch in repository myproj, and on the 2011.011 branch in repository myproj_test:

As with a label configuration, you can set a branch configuration using point-and-click. Use the Branches view:

6. Components in a Single-Repository Workspace

Your organization might place a high value on the simplicity of maintaining a single central repository. In this case, you can still "componentize" the source base to a large extent - although not quite to the same degree, and not with the same ease-of-use, that using multiple repositories makes possible.

Following the same scenario as in Section 2 above: if the entire source base (including the test subtree) is stored in a single repository, you can still give the test subtree a measure of independence. The key is to use multiple "path" rules in the workspace's selector:

repository "repository-name"

path "/test"
configuration clauses

path "/"
configuration clauses

As above, configuration rules must be ordered "from the bottom up" - the path rule for the root directory comes last.

Here's how you can configure this single-repository workspace to produce the same results described in the preceding section:

  • Make the test subtree read-only:

repository "yourproj"
path "/test"
br "/main"
path "/"
br "/main"
co "/main"

  • In the test subtree, use the revisions labeled WEEK.3; in the rest of the repository, use the revisions labeled BL04:

repository "yourproj"
path "/test"
label "WEEK.3"
path "/"
label "BL04"

Note that applying the labels is not as easy here as in the multiple-repository scenario: the Plastic SCM GUI does not have a command for applying label WEEK.3 to the test subtree only; and it does have a way to apply label BL04 to the entire repository except for the test subtree. The Plastic SCM CLI and a good scripting tool can handle this situation easily!

  • Work on the 2011.011 branch in the test subtree, and on the task428 branch in the rest of the repository:

repository "yourproj"
path "/test"
smartbranch "/main/2011.001"
path "/"
smartbranch "/main/task428"

In this situation, a single Checkin command that involves 2 items from the test subtree and 3 items from the rest of the repository creates two separate changesets: one for the items on the 2011.011 branch, and another for the items on the task428 branch.

7. Loading a Subtree of a Repository into a Workspace

In this note, we've described how to make an entire repository into a sub-repository of another one (from the perspective of a particular workspace). We'll end by describing another feature that, in a sense, provides the opposite capability: loading not an entire repository, but just a subtree, into a workspace.

Suppose repository common has this directory hierarchy:

If you just want to work on some 3D icons, it doesn't make sense to load the entire repository into a workspace: why load all those files you won't be using (right now, anyway), and why have to navigate through all those directory levels? Instead, you can use a root specifier in the repository rule:

repository "common" root "/data/gui/animation/3D/icons"
path "/"
br "/main"
co "/main"

This selector causes a workspace to be loaded with the contents of the icons subdirectory only: