Performance, scalability and network optimizations

Performance results of Plastic SCM

How does Plastic compare to other version control systems in pure speed?

We frequently benchmark Plastic against other key version control systems known for its high performance and continuously improve in key operations such us add, checkin and update.
We’ve compared Plastic, Git and a very well-known commercial version control system extensively used in the game industry. We’ve tested with 3 types of codebases: small, medium and large.

The tests benchmark a full add+checkin (adding the entire code base to the version control) and also a ‘clean update’ (downloading the entire codebase to a clean directory).
While certainly adding a big codebase of 140GB is not something a team will do on a daily basis, it shows how Plastic can handle the scenario. When the data to be checked in is really large, disk and network IO tend to be the bottleneck, so probably the ‘medium’ scenario is the most relevant since even in large repositories the team members will more often do ‘medium’ checkins (or even ‘small’) than large ones.

All the tests have been performed with Plastic SCM 5.4 using the ‘filesystem blob storage’ which saves big blobs (bigger than a certain configurable size) on the filesystem instead of the database. Plastic uses a SQL Server database backend.

When comparing to Git it is important to highlight that Plastic is configured in client/server mode, so all data transfer has to be sent through the network.

SMALL - 45,925 files 2,946 directories – 601 MB
The code for the SMALL repo is the Linux kernel

Plastic is 7.3 times faster than ‘commercial version control’ doing add + checkin

Plastic is 4 times faster than Git – consider that Git is checking in locally while Plastic is sending data through the network from client to server

MEDIUM - 221,141 files 13,247 directories – 8.08 GB
The code for the MEDIUM repo is created as follows: linux + mono + openoffice + alienarena + war.dat

This scenario is specially relevant because even if game teams use LARGE repositories, most of the time they’ll be doing ‘medium’ checkins (it is not so common to checkin 142Gb as it is a few Gb). In these cases is where Plastic shines since it excels managing the metadata and IO speed is not yet the limit factor.

Plastic is 6 times faster than ‘commercial version control’ doing add + checkin

Plastic is 3.3 times faster than Git – consider that Git is checking in locally while Plastic is sending data through the network from client to server.

LARGE - 360,399 files 22,572 directories – 142 GB
The code for the LARGE repo is created as follows: mediumWks + mono (again) + 4 x example data containing big binaries (from a real game)

Plastic is 3 times faster than the 'other commercial version control'.

Plastic needs 35 minutes less to complete a add/ci than Git -> 1.7 times faster.

Note: the Git test was repeated several times because we found occasional issues (out of memory).

It is important to highlight that:

With a checkin of 142 Gb most of the time is spent in network data transfer and writing data to disk. Disk and network IO set the limits in these scenarios.
While a LARGE repo is common in the gaming industry, adding 142 Gb is not the most common scenario. It is more common to do updates or checkins of smaller sizes *inside* the LARGE repo. In these scenarios the way to handle the metadata is more relevant than the IO limits, and Plastic clearly beats competitors.

Test environment:

Both client and server:

CPU: 4CPU, 14 ECU (x64) / Intel Xeon CPU E5-2670 v2 @ 2.50GHz

RAM: 30.5 GB

Network: 1Gbps

OS: Windows Server 2012
Disk IO:

Server Machine: 64.5MB/sec Read speed (HDTune)

Client Machine: 145 MB/sec Read speed (HDTune)

Write speed: 195Mb/s (cm iostats)