The current guide explains the Plastic SCM distributed system capabilities: a general description of the different distributed and replication scenarios supported, followed by detailed explanations.

Plastic SCM distributed capabilities enable setting up different servers for multi-site development support, being able to both replicate and reconcile changes made on replicated branches.

This document will give a general overview of the system’s distributed capabilities.

Plastic SCM has the ability to create different multi-site scenarios ranging from single-server to fully distributed deployments.

Figure 1. Deployment scenarios supported

As Figure 1 shows Plastic SCM can be configured to work in a single server mode, which is the default mode on installation and the conventional mode available on all SCM products.

The next step has been called classic multisite in which several servers exist, one for each development location, and contents are replicated among them. The basic rule at this distribution stage is that branch mastership is kept by only one site at a time: if a branch is modified at one site, the other ones won’t modify it until the branch is replicated again. In many systems this behaviour is encouraged by the software itself, preventing simultaneous changes in a master/slave relationship. Plastic is not restricted to work on this mode. Several deployment strategies will find this configuration possibility useful, but Plastic won’t have any restriction to perform changes in the distributed copies because it doesn’t implement any master/slave functionality, although it can be simulated by the system administrator if necessary using both permissions and a clear replication policy.

Full multisite support is almost identical to the previous distribution stage with only one difference: all the SCM servers can modify their branches at any time. Changes can be reconciled back later on if the same branch is modified more than once at different locations.

Full distribution is exactly the same as full multisite, but on this deployment scenario each developer has his own SCM server. There’s only one restriction imposed to systems working on this mode: servers must be light enough to run on non-dedicated workstations and even laptops. Plastic SCM servers can be easily configured to work in this mode, introducing full disconnected support. A developer can take his laptop home and continue working as if he were at the office and reconcile his work when he’s back at the office.

2. Branch replication

The main distributed operation is replication. By means of this operation repositories can be distributed on several machines. The replication unit in Plastic SCM is the branch. Users specify a branch to be replicated from a given source repository to a destination one, and then the revisions on the branch plus their labels, links, attributes, changesets and so on will be replicated from the source to the destination repository.

Figure 2. Branch replication between two locations

Figure 2 shows two repository servers at two different locations. The server at location 2 has replicated the branch main at location 1. Then it has created another two branches which have been later on replicated into location 1.

As you can see from the figure, distributed repositories don’t have to be exact clones. They share replicated branches and their contents but the entire repositories don’t have to be identical, they can evolve separately sharing some common branches.

3. Distributed scenarios

There are several possible distributed scenarios with Plastic SCM. They will be explained in detail in this topic.

3.1. Multi-site replication with mastership policy

In this scenario two or more servers are used in replication. Servers will normally run at different locations to enable geographically distributed teams to work together on the same project. A server at each location will solve the problem of slow or unreliable internet connections between sites.

Figure 3. Multi-site readonly

The Figure 3 shows both a deployment diagram and a detailed view of the branching strategy. This set up resembles classic multi-site replication as implemented by many master/slave based products. In Plastic this scenario is just one possibility and it will be used to explain replication.

The two sites, location 1 and location 2, will have their own servers. Both sites will be working on the same code-base, so developers will need to check-in changes at any time. The chosen strategy would be the following one:

Both servers will have an exact replicated copy of the main branch, containing the latest baseline.
The new baselines will be generated only in one server at a time, so they will be implementing a sort of mastership behaviour. Let’s assume new baselines will be created on server 01.
Developers at both locations will follow the branch per task pattern. Branches will be created using the latest baseline as starting point. The main branch won’t be modified in parallel at the two sites.
Periodically, for instance twice a week depending on the amount of work finished, all the task-branches created at site 2 will be replicated into site 1. All branches will be integrated on main, tested, and a new baseline will be created. Alternative integration branches could exist at each site in order to ease the integration process.
Once the release is finished the main branch will be replicated from site 1 to site 2 and a new development iteration will begin.

Figure 3 shows how branches are replicated from site 2 to site 1 in its lower area.

Figure 4 and Figure 5 describe the previous scenario step by step. They show how the main branch is first replicated from location 1 into location 2, how the newly created release 58 is then available to the two development groups.

Then the groups start working on and creating task branches independently from each other, but starting at a well-known point: release 58.

Once the iteration is finished, branches task1012, task1013 and task1030 created at location 2 are replicated to location 1 to be integrated.

Once the integration is finished, the branch main will be replicated again to location 2, so that the development group there can continue working with the latest approved baseline.

Note that both repositories are not identical after the development iteration finishes, but the content on the main branches, considering they’re being modified at only one site, is exactly the same.

3.2. Multi-site replication with shared mastership

The deployment required for this scenario is exactly the same as in the mastership case. The difference will be on the way in which the replicated branch evolves. Now developers will make simultaneous changes to the replicated branch and Plastic will have to help reconciling these changes together.

Figure 6 depitcs the situation: on two different sites users are working against the same branch which has been replicated from location 1 to location 2. Then both groups perform changes directly on their copy of the main branch.

Figure 6. Multi-site replication with changes performed in parallel

Changes are drawn as performed directly on the main branch, but sites could also be using the branch per task pattern and directly integrating their own work on main.

When site 1 (or viceversa) requests changes made at site 2 using replication, the newly created changesets on branch main at site 2 can’t be directly plugged into the main branch at site 1, because intermediate changes have been made. Then Plastic creates a fetch branch containing all the replicated changes. The branch can be integrated on the main branch using regular merge commands.

3.3. Pure distributed scenario

At a pure distributed scenario there isn’t a central server, but each developer runs his own server containing his own repositories.

This strategy can be fully implemented with Plastic configuring a server on each developer’s workstation.

This fully distributed scenario can be adopted by any company, although they would normally prefer to count on a central copy. On distributed development there will always be a master server, not necessarily due to software restrictitions but to some sort of meritocracy, as it happens at open source projects. So it is usually better to explicity decide which one will be the computer containing the well-known stable releases. Obviously there will be more than one satisfying this requisite, but it is better for simplicity to exactly determine which will be the master one at any time.

In corporate scenarios this pure distributed ability can be tuned to support a mixed scenario:

Onsite developers continue using a regular client/server configuration when working against a central server.
The central server plays the role of master copy.
Developers working at a different location (at home, for instance) have their own repository server which they can keep in synch regularly.
Developers working on laptops can also run their own servers and then implement fully disconnected support.

Alternatively all developer’s workstations could run Plastic SCM servers. This is totally supported but the system. Using or not this capability will depend on the organization itself, developers’ skills and the amount of administrative burden required.

Figure 7 depicts the concepts described above.

Figure 7. Fully distributed scenario in detail

4. How replication works

So far general distributed system’s behaviour has been introduced. This topic will explain in detail how Plastic SCM replicates revisions between branches on different repositories and how fetch branches are created to resolve conflicts.

4.1. Replication at item level

The diagrams and samples introduced above focused on overall branch behaviour. But, what happens at the item level? Figure 8 details a replication sample studying what happens at the item level.

Figure 8. Replication at item level. Step by step explanation.

The sample focuses on a file named /src/main.cpp at the branch /main/fix. The branch is replicated from repository A at Location A to repository B at Location B. Note that the figure specifies the Plastic command needed in order to run a replication.

At step 1 the only existing revision for main.cpp at branch /main/fix in location A is copied to location B. Now both repositories have the same content regarding main.cpp at the given branch.

Step 2 shows how the file is modified at rep A: two new revisions are created.

At step 3 the developer at location B runs once more the same replication command. The two new revisions created at rep A are now copied into rep B.

During replication Plastic first fetches the revisions at the branch specified by the user (starting at the last previously replicated changeset if any). Then it will try to push the revisions on the destination repository. To do so, Plastic first checks whether the fetched revisions can be actually linked on destination. In the case in the sample Plastic checks which revision at rep B is the parent of the revision 1 being replicated. It finds that the parent is revision 0, and that revision 0 doesn’t have any other child on /main/fix branch. So Plastic decides to link revision 1 to the existing revision 0.

At step 4 the developer at rep B makes a new change starting from the latest replicated revision of main.cpp.

At step 5 the developer at rep A replicates /main/fix at rep B. The newly created revision 3 gets copied and correctly placed on his repository.

Note that the sample from Figure 8 shows only one change at a time on the branch, so no conflicts can happen. While following this strategy the two replicated branches will continue being exact clones on replication.

Figure 9 shows a more complex scenario. Both locations start with the same configuration: three revisions of main.cpp at branch /main.

At step 2 the two repositories evolve in parallel when the developers introduce new changes on main.cpp.

At step 3 the user at rep A tries to replicate changes from rep B. Now Plastic can’t directly link revisions 3 and 4, created at rep B to revision 2 because a new revision 3 has also been created at the branch.

Note that internally Plastic SCM identifies each object by a GUID (Globally Unique Identifier[1]. So don’t get confused by the revision numbers shown in the sample. Despite of the fact that now there are two revisions with revno = 3 in the sample, Plastic knows they’re not the same.

If revision 3 at repA didn’t exist, then Plastic would have placed revision 3 and 4 from rep B just hanging from the existing revision 2. But now it can’t do that. So what Plastic actually does is creating a fetch branch to place the replicated revisions. Note that the revision 3 and 4 coming from /main at rep B are now revisions 0 and 1 at a fetch branch, but the former revision 3 is linked to its real parent: revision 2.

Once the fetch branch has been created, the user can run a regular merge operation to get the changes made at the two repositories integrated together.

Figure 9. Replication of items modified at two locations

4.2. When are fetch branches created?

Fetch branches are created on replication when the revisions which are being synchronized can’t be placed on the original branch because the branch evolved in parallel at the two sites.

At most one fetch branch is created on each replication operation.

If several items are involved on a replication operation and only one of them requires a fetch branch (the others could be placed on the original branch because no change has been performed in parallel), Plastic will place all the replicated revisions for all items on the fetch branch. This eases the later integration process, because otherwise revisions replicated together would be placed on different branches.

4.3. Related objects replication

Links, attributes and labels are replicated together with branches and their revisions. Labels and attributes are related to a certain revision, so it is clear they’ll be replicated together with their corresponding objects but, what will happen with links?

Links actually relate two revisions together, so they’ll be always fetched but only pushed to the destination repository when both the source and destination objects exist.

Figure 10 shows a detailed example in which a branch is replicated. There’s a merge link between the fix and the main branches that is not replicated because fix branch doesn’t exist on destination.

When the fix branch is also replicated, then the link is copied and correctly placed on the repB.

Figure 10. Link replication explained

The same happens for parent links which are the links relating a revision with its parent. Check the following sample at Figure 11 to check how links to parent revisions are also replicated and correctly set when the right branches are replicated.

Figure 11. Parent link replication

5. Replication modes

There are two replication modes available:

Direct server to server replication: a Plastic client will tell the destination server to replicate a branch from a source server. Servers will communicate to replicate data.
Package based replication: Plastic client connects with the source server and creates a replication package. The package will be imported later on by the destination server.

Figure 12 depicts the two available replication modes.

Figure 12. Replication modes

The package based replication introduces the ability to keep in synch servers which are not allowed to connect directly due to security restrictions.

6. Replication from the command line

All the replication scenarios and possibilities described can be set up with a single Plastic command: replicate.

cm replicate srcbranch destinationrepos

Where src branch is a branch spec identifying the branch to be replicated and its repository, and destination repos is the repository where the branch is going to be replicated.

6.1. Direct server replication

Suppose you want to replicate the branch main at repository code at server london:8084 to repository code_clone at bangalore:7070. The command would be:

cm replicate br:/main@rep:code@repserver:london:8084 rep:code_clone@repserver:bangalore:7070

6.2. Replication packages

To replicate branches using packages the first step will be creating a replication package, and then importing the package into another server.

Suppose you have to create a replication package for the main branch at repository code at server box:8084.

cm replicate br:/main@rep:code@repserver:box:8084 –package=box.pk

The previous command will generate a package named box.pk with all the content of the main branch.

Later on the package will be imported at the repository server berlin:7070.

cm replicate rep:code@repserver:berlin:7070 –import=box.pk

7. Authentication

During replication different servers have to communicate with each other.This means that servers running different authentication modes will have to exchange data.

To do so the replication system is able to set up different authentication options.

7.1. Setting up authentication modes

Figure 13 shows a tipycal scenario with a client and two servers. All the involved Plastic components are configured to work in LDAP and they share the same LDAP credential, so no translation is required.

Note that authentication happens at two levels:

The client needs to be authenticated in order to connect to the destination server. In the figure the destination server is berlin.
Then berlin will need to connect to server london to retrieve information about the branch to be replicated (main in the sample).

If both servers were not using the same authentication mechanism or not authenticating against the same LDAP authority, step 2 would fail.

Figure 13. Distributed system authentication diagram

Figure 14 shows a scenario in which the server at london is configured to use user/password authentication. In this case a command like the one specified at the top of the figure will fail because authentication between servers won’t work at step 2.

Figure 14. Specifying credential to replicate between servers

To solve this problem the replication system has the ability to specify authentication credentials to be used between servers. In the sample the client can specify to the server berlin a user and password to communicate with server london.

Figure 15 shows two different ways to specify authentication credentials when using user/password at the source server.

Figure 15. Two different ways to specify authentication credentials

The first option is actually specifying the mode plus the user and password (for UP) at the command line.

The second one uses an authentication file, which is useful when authentication credentials are going to be used repeteadly. As the figure shows, an authentication file is a simple text file containing two lines:

The authentication working mode: UPWorkingMode, LDAPWorkingMode, NameWorkingMode, ADWorkingMode or NameIDWorkingMode
Specific authentication data for the authentication mode. The data specified in the second column is exactly the same the client generates at client.conf configuration file under the key SecurityConfig. So a way to create an authfile is manually editing the file, specifying the working mode on the first line and copying the right data from a correctly configured (using the configuration wizard, for instance) client.conf file.

Suppose now that replication must happen in the opposite direction, from berlin to london as Figure 16 shows. The parameters to connect to a LDAP server (in this case an Active Directory acessed through LDAP) are specified. Normally in LDAP an authentication file will be used to ease the process.

Figure 16. UP server communicating with LDAP one

Note: If replication is performed through replication packages the client needs to be able to connect to the source or destination servers, depending on whether it is performing an export or import operation.

7.2. Translating users and groups

When replication is performed between servers with different security modes, authentication is not the only issue. User and group identifications have to be translated between the different security modes.

The sample at Figure 17 tries to replicate from a user/password authentication mode to a LDAP based one. The user list at the UP node stores plain names but the user list at the LDAP server stores SIDs. When the owner of a certain revision being replicated needs to be copied from repA to repB, a user or group will be taken from the user list at repA and introduced into the list at repB. If a name coming from repA is directly inserted on the list at repB, there will be a prlbem later on when the server at berlin tries to resolve the LDAP identifier because it will find an invalid one.

So in order to solve the problem translation will be needed.

Figure 17. User and group translation on replication

The Plastic replication system supports three different translation modes:

Copy mode: it is the default behaviour. The security IDs are just copied between repositories on replication. It is only valid when the servers hosting the different repositories involved work in the same authentication mode.
Name mode: translation between security identifiers is done based on name. In the sample at Figure 17 suppose user daniel has to be translated by name from repA to repB. At repB the Plastic server will try to locate a user with name daniel and will introduce its LDAP SID into the table if required.
Translation table: it also performs a translation based on name, but driven by a table. The table, specified by the user, tells the destination server how to match names: it tells how a source user or group name has to be converted into a destination name. Figure 18 explains how a translation table is built and how it can translate between different authentication modes.

Figure 18. Translation table explained

Note: a translation table is just a plain text file with two names per line separated by a semi-colon “;”. The first name indicates the user or group to be translated (source) and the one on the right the destination one.

8. Replication from the graphical interface

Replication can be used from both the command line and the Plastic Graphical User Interface (GUI) tool. All the possible actions are located in a submenu under the branch options, because replication is primarily related to branches. This topic will describe how to perform the most common replication actions from the GUI.

8.1. Replication actions

From the GUI replication and distributed collaboration has been organized in the following actions:

1. Branch actions:

a. Push the selected branch

b. Pop the selected branch

c. Pop a remote branch

2. Package actions:

a. Create a replication package from the current branch

b. Create a replication package from a branch

c. Import a replication package

The Figure 19 depicts the different available operations. From the command line all the operations are issued from a single command, but the GUI makes a distinction between push (move changes from your server to a destination) and pop (bring changes from a remote repository to yours) actions.

Figure 19. GUI replication actions

8.2. Replication menu

As it was mentioned before, all replication actions can be accessed from the branch menu (check Figure 20).

The options push this branch, pop this branch and create replication package from this branch are related to the branch currently selected in the branch view. The other options: pop remote branch, create replication package and import replication package are generic replication actions which are not constrained to the current branch but are located under the branch menu to keep all the replication options together.

Figure 20. GUI branch menu with replication options

8.3. Pushing your changes to a remote repository

Whenever you want to push your changes to a remote repository, select push this branch on the branch menu. Pushing your changes means sending the changes made on the selected branch to a remote repository.

Figure 21. Pushing changes

If the branch already exists on the destination repository the changes will be synchronized (potentially creating a fetch branch on destination that will need a merge). If the branch doesn’t exist on the destination repository a new branch will be created (identified by the same GUID used on the source repository).

Check Figure 21 for a detailed explanation.

8.4. Synchronizing your branch with remote changes

Once you’ve pushed your branch to a different repository, the branch can be modified remotely. At a certain point in time you’ll be interested on retrieving the changes made remotely into your branch. In order to do that you’ve to use the pop this branch action from the replication branch menu.

The dialog depicted on Figure 22 is very similar to the one used to push changes but this time your server is located on the right as destination of the operation.

Figure 22. Pop changes from a remote branch

When you pop changes from a remote branch a fetch branch can be potentially created if there are conflicts with the files or directories being replicated.

8.5. Importing a remote branch

Another common scenario during replication is importing a branch from a remote repository into yours in order to start making changes or create child branches from it.

In order to perform the import use the pop remote branch option. The dialog show at Figure 23 will be displayed. Notice that this time you can choose both the source server, repository and branch, and also the destination repository on your server.

Figure 23. Pop a remote branch

8.6. Managing remote authentication

As it was described on chapter 7, different Plastic servers can be using different authentication modes. By default when you try to connect to a remote server you’ll be using your current profile (the configuration used to connect to your server). But sometimes the default profile won’t be valid on the remote server.

In order to configure Plastic to be able to connect with a remote server with different authentication mode, use the advanced options button on the replication dialog. It will pop up a dialog like the one on Figure 24.

Figure 24. Advanced options dialog

The dialog shows the profile currently selected (the default one on the screenshot) and also the translation mode (refer to chapter 7 for more information) and the optional translation table.

You can have different authentication profiles created from previous replication operations, and you can list them or create new ones pressing the browse button located on the right of the remote server configuration profile edit box.

It will display a dialog like the one on Figure 25 which will allow you to select, edit, create or remove a profile.

Figure 25. Profile selection dialog

Note: the replication dialog will try to choose a profile automatically each time you change the server. It will look for the most suitable profile based on the server information provided.

8.7. Running the replication process

So far all the steps have been focused on setting up the replication process. Once the operation is correctly configured, press the replicate button and you’ll enter the replication progress dialog as explained on Figure 26.

The replication operation is divided into three main states: fetch metadata, push metadata and transfer revision data. The first one happens on the source server, the second one on the destination one and the third one involves the two servers as data is transferred from the source to the destination.

At any point in time the operation can be cancelled pressing the cancel button.

When the replication operation finishes a summary is displayed containing detailed information about the number of objects created.

If a fetch branch is created on the process, a message box will notify it.

Figure 26. Replication operation progress

8.8. Creating a package

A replication package can be created from a branch on your repository or from any branch on any server you can connect to. In order to create a package from the selected branch on the branch view, click on create replication package from this branch. If you want to create a package from any remote branch, click on create replication package on the replication menu.

Figure 27. Create replication package from the GUI

Figure 27 shows the package creation dialog. It will generate a replication package from the selected branch which will contain all data and metadata from the branch. It can be used to replicate between servers when no direct connection is available.

8.9. Importing a replication package

From the replication menu select import replication package and select a package file to be imported. The dialog is shown at Figure 28.

Figure 28. Import a replication package