Plastic SCM Database backend migration from SQLite to MySQL

When you install Plastic SCM in linux it comes with SQlite embedded database backend. This is the backend used out of the box when installing Plastic SCM on linux, it runs well for evaluation of the system without real heavy operations and having many users connecting to the system, this embedded Database solution may be lacking some of the features of a real database server which can handle more threads, connections etc.
Plastic SCM can easilly be configured to interface and use a variaty of database backends, for an updated supported database list visit the info center at: http://www.plasticscm.com/features/sql-backends.aspx

In this article we will go through step by step to configure an already installed Plastic SCM server "which runs with the default SQLite" to MySQL.

NOTE: Make sure that all your users have cheked in their work, in other words, no checkouts, since checkouts are not replicated.

System used:

• Ubutnu server - 10.04
• Plastic SCM version 3.0.187.19
• MySQL 5.1.x

The setup:

Plastic SCM and MySQL will be installed on the same machine. (can also run on different machines) We will run two instances of the Plastic servers, one with the SQlite we will call it Plastic_SQLite and other with the MySQL we will call it Plastic_Mysql and will replicate from Plastic_SQLite to Plastic_MySQL.

MySQL installation and setup:

This chapter is handled here for convenience of the user, and have little to do with Plastic SCM, this is by no means a manual for MySQL and there are tons of information on the www. We will show the basic steps needed to have MySQl setup correctly with Plastic since many times when the users escalates issues regarding Plastic SCM cannot show repositories etc. it is related to the Database backend not set up correctly.

On Ubuntu server 10.04:

1. Open a terminal connection, or ssh into your ubuntu server.
2. Run sudo apt-get update
3. Run sudo apt-get install mysql-server-5.1
4. Installation of MySQL server will start.
5. Follow the screen-shots below.

6. You would probably want to create a MySQL user that is needed to be used with Plastic SCM server other than the default MySQL root user.
   1. Log in into MySQL with the root user and password that you set up above: mysql -uroot -p
   2. Create a MySQL user called plastic with password called password4plastic, execute the below commands within the MySQL shell.

mysql>CREATE USER 'plastic'@'localhost' IDENTIFIED BY 'password4plastic';
mysql>GRANT ALL PRIVILEGES ON *.* TO 'plastic'@'localhost' WITH GRANT OPTION;
mysql>flush tables;
mysql>FLUSH PRIVILEGES;

3. Test the newly created MySQl user by logging in Mysql with that user. mysql -uplastic -p

Setting up Plastic SCM with MySQL:

7. Your Plastic SCM is already installed and working with SQlite through a configuration file called db.conf placed in ../plasticscm/server directory Your database files where all your code are in the SQLite files in the ../plasticscm/server/
8. The files have .sqlite extension and looks like this:
   rep_1.plastic.sqlite
   rep_3.plastic.sqlite
   repositories.plastic.sqlite

   The amount of how many rep_X.plastic.sqlite files etc. depends on how many repositories you have created in Plastic and it is directly related.

9. Stop the plastic server
   1. cd /opt/plasticscm/server
   2. sudo ./plasticsd stop
10. Copy recursively the directory /opt/plasticscm/server/* to /opt/plasticscm/server_SQLite
    1. sudo cp -R /opt/plasticscm/server /opt/plasticscm/server_SQLite
    2. /opt/plasticscm/server_SQLite will be the secondary running instance of your plastic server called Plastic_SQLite

10. Rename /opt/plasticscm/server/db.conf to /opt/plasticscm/db.conf.sqlite
    1. mv /opt/plasticscm/server/db.conf /opt/plasticscm/db.conf.sqliteconf
11. Now setup this instance of Plastic SCM server with MySQL by using the guidelines of this technical article.
    1. http://www.plasticscm.com/infocenter/technical-articles/kb-how-to-configure-plastic-scm-with-mysql.aspx
    2. Delete the logfile ../plasticscm/server/loader.log.txt a new one will be created when the Plastic SCM server starts.
    3. cd /opt/plasticscm/server
    4. sudo rm loader.log.txt
    5. Rename the /opt/plasticscm/server/*.sqlite files to /opt/plasticscm/server/*.sqlite.bck
    6. sudo rename -v 's/\.sqlite$/\.sqlitebck/' *.sqlite
    7. sudo ./plasticsd start

12. The main plastic server in /opt/plasticscm/server is running and listening on the default TCP port 8084, we will start the other instance of the plastic server in /opt/plasticscm/server_SQLite so we can replicate your data from SQLite to MySQL, a different TCP listening port for Plastic_SQLite instance is needed.
    1. Run the second instance listening on a free tcp port on your machine, I use for this example TCP port 8090
    2. Adjust the script /opt/plasticscm/server_SQLite/plasticd to reflect the path for this server instance
    3. testervm@UbuntuD-1004-VM:/opt/plasticscm/server_SQLite$ sudo ./plasticd --port=8090 --daemon &

13. Now we have two plastic server instances running on the same machine with this setup
    1. Plastic_MySQL: Plastic scm server running with MySQL database backend, and listening on port TCP 8084
    2. Plastic_SQLite: Plastic scm server running with embedded SQLite database backend, and listening on port TCP 8090

3. test connectivity with the client to both servers with the cm lrep command.

cm lrep localhost:8084
cm lrep localhost:8090

Replicating from Plastic_SQLite to Plastic_MySQL

A note of caution: I assume that your new Plastic_MySQL has not been used, meaning, you dont have data checkedin (commited) there that you dont want to overwrite! for the sake of double checking, although the Plastic server has created the default repository for you upon server start, go ahead and create another repository for the replication destination. the command to create a new repository in Plastic server and to check that the repository is created on that server.are:

cm makerepository localhost:8084 REPLICA_REPO
cm lrep localhost:8084

14. There are a couple of options to do this replication, and Plastic SCM comes with replication tools.
    1. Replication must be from top-down branches, meaning, first you replicate branch /main then /main/child01 then /main/child01/child001 and so on. A screen-dump of a simple branch structure is illustrated below.

2. The embedded CLI replication command is "cm replicate", and if you type "cm help replicate" you will have access to help example and how to run the command.
3. If you have already a lot of branches on the source repository, you can prepare a script to perform the replication through the command line shell, see Related articles at the buttom of the page for some examples.
4. We can also share with you an internal program that replicates all the branches on a source repository to a specified destination repository, the tool called replikate.exe and you can get it here Download it, and extract the zip file in /opt/plastictools/
5. If you run sudo ./replikate without parameters, you'll find how to use the program, but the basic usage is as follows: /Path_to_replikate_directory$ sudo ./replikate "name_of_src_rep@src_server_name:src_server_port "name_of_dst_rep@dst_server_name:dst_server_port"

Example:

sudo ./replikate "default@localhost:8090" "REPLICA_REPO@localhost:8084"

A log file "replikate.log.txt" will be created in the Replikate.exe program directory. To view the log file do a

less replikate.log.txt

6. Verifying with Plastic client that you succesfully replicated all the branches.

cm find "branches on repository 'default@localhost:8090'" --format={name} --nototal >/tmp/srcrepo.txt
cm find "branches on repository 'REPLICA_REPO@localhost:8084'" --format={name} --nototal >/tmp/dstrepo.txt
diff -as /tmp/srcrepo.txt /tmp/dstrepo.txt

The above lines performs a simple verification to verify if the replication has replicated all thge branches.

Wrapping up:

1. After finishing replicating all your data above, Kill the Plastic_SQLite instance Identify the PID of the Plastic instance you want to terminate: sudo ps -ef |grep -i plast* sudo kill XXX
2. You can choose to delete the directory for your secondary Plastic SCM server /opt/plasticscm/Plastic_SQLite and its content, but maybe it is more desired to keep it for testing, failover etc., in Codice Software we dont like to delete things!, but it is completly your choice.

Related articles:

http://codicesoftware.blogspot.com/2009/03/setting-up-fallback-server-using.html
http://www.plasticscm.com/infocenter/technical-articles/kb-how-to-configure-plastic-scm-with-mysql.aspx

 

Written by: Miller Auker - 2011

Download

Download Plastic SCM