We frequently benchmark Plastic against other key version control systems known for its high performance and continuously improve in key operations such us add, checkin and update.
We’ve compared Plastic, Git and a very well-known commercial version control system extensively used in the game industry. We’ve tested with 3 types of codebases: small, medium and large.
The tests benchmark a full add+checkin (adding the entire code base to the version control) and also a ‘clean update’ (downloading the entire codebase to a clean directory).
While certainly adding a big codebase of 140GB is not something a team will do on a daily basis, it shows how Plastic can handle the scenario. When the data to be checked in is really large, disk and network IO tend to be the bottleneck, so probably the ‘medium’ scenario is the most relevant since even in large repositories the team members will more often do ‘medium’ checkins (or even ‘small’) than large ones.
All the tests have been performed with Plastic SCM 5.4 using the ‘filesystem blob storage’ which saves big blobs (bigger than a certain configurable size) on the filesystem instead of the database. Plastic uses a SQL Server database backend.
When comparing to Git it is important to highlight that Plastic is configured in client/server mode, so all data transfer has to be sent through the network.
Plastic is 7.3 times faster than ‘commercial version control’ doing add + checkin
Plastic is 4 times faster than Git – consider that Git is checking in locally while Plastic is sending data through the network from client to server
This scenario is specially relevant because even if game teams use LARGE repositories, most of the time they’ll be doing ‘medium’ checkins (it is not so common to checkin 142Gb as it is a few Gb). In these cases is where Plastic shines since it excels managing the metadata and IO speed is not yet the limit factor.
Plastic is 6 times faster than ‘commercial version control’ doing add + checkin
Plastic is 3.3 times faster than Git – consider that Git is checking in locally while Plastic is sending data through the network from client to server.
Plastic is 3 times faster than the ‘other commercial version control’.
Plastic needs 35 minutes less to complete a add/ci than Git -> 1.7 times faster.
Note: the Git test was repeated several times because we found occasional issues (out of memory).
CPU: 4CPU, 14 ECU (x64) / Intel Xeon CPU E5-2670 v2 @ 2.50GHz
RAM: 30.5 GB
OS: Windows Server 2012
Server Machine: 64.5MB/sec Read speed (HDTune)
Client Machine: 145 MB/sec Read speed (HDTune)
Write speed: 195Mb/s (cm iostats)