Release Notes

All platforms - Proxy: It is now able to monitor free space on disk and clean up the cache to avoid running out of disk.

You'll see something as follows in the log when it runs out of disk:

INFO  Daemon - Disk is running out of space. 5.78 Gb available. Will delete 1.15 Gb from cache
INFO  Daemon - Cleaning up 1.15 Gb (9576 entries)
INFO  Daemon - Cleaned up 1.15 Gb (9576 entries) in 2578 ms

And, if there is enough room (twice the size of the cache) then something as follows:

INFO  Daemon - Disk still has enough space: 19.53 Gb available.

All platforms - Plastic Proxy: Implemented a disk cache limit.

Now the proxy can limit the size of its on-disk cache.

== How to configure ==

Very easy: last use the new MaxCacheSizeInGb in the plasticcached.conf file.

Example:

<PlasticCacheConf>
        <MaxCacheSizeInGb>20.5</MaxCacheSizeInGb>
</PlasticCacheConf>

And the Proxy will make sure its cache size doesn't exceed 20.5 Gb.

A value of 0 means the clean is disabled.

== How it works ==

The Proxy maintains a LRU (Least Recently Used list) and updates it on every data access.

Every 10 minutes, it checks if the total cache size is bigger than the value configured. If the maximum size is exceeded, the Proxy retrieves the least recently used entries and removes them from disk.

Every 3 minutes the Proxy saves the LRU info to disk. The LRU data is written to proxy-data-path/proxy-lru.dat. The data is also written on shutdown.

On startup and then every 8 hours, the Proxy checks if it needs to walk the entire cache directory to rebuild the LRU and recover from possible inconsistencies. While the check runs every 8 hours, a rebuild will only happen once every 3 days. The date of the last rebuild is stored under proxy-data-path/proxy-last-cache-walk.dat.

proxy-data-path is the directory where you configured the proxy to store the cache.

For existing Proxy installations, the new version will rebuild the cache on the fly.

== Some logs ==

During the startup of the Proxy you'll see:

INFO  LRU - Walking the disk to build the LRU (least recently used). c:\plastic\proxydata
INFO  LRU - Last LRU rebuild from disk happened less than 3 days ago. Skipping. 11/04/2019 16:50:00
INFO  Daemon - Load LRU info from c:\plastic\proxydata\proxy-lru.dat
INFO  Daemon - LRU info loaded in 656 ms. 107299 entries. 5.89 Gb total in cache

You can see how the Proxy skips the rebuild and how it loaded 107k entries to the LRU.

Then, every 3 minutes, and during shutdown:

INFO  Daemon - Save LRU to c:\plastic\proxydata\proxy-lru.dat
INFO  Daemon - LRU info saved in 312 ms. 107299 entries. 5.89 Gb total in cache

And during a clean up:

INFO  Daemon - Cleaning up 0.69 Gb (17564 entries) of cached data to enforce the 5.20 Gb limit
INFO  Daemon - Cleaned up 0.69 (17564 entries) Gb in 5110 ms
INFO  Daemon - Save LRU to c:\plastic\proxydata\proxy-lru.dat
INFO  Daemon - LRU info saved in 265 ms. 89735 entries. 5.20 Gb total in cache

DevOps: ConflictsBot was useless with MergeRules enabled (they are available since release 8.0.16.3433).

From now on, ConflictsBot will be able to report merge conflicts again at earlier stages. (The fix is to skip the MergeRules check in the underlying dry-run merge operation the ConflictsBot runs).

All platforms - Plastic, Command line client: in the debug log for an update, the metadata download and processing times were incorrectly calculated. This has been fixed.

All platforms - Checkin & Update: Multi-thread checkin and update enabled by default now.

Plastic can be super-fast, but historically we were to shy enabling the full power by default. But, smart defaults are key for usability, because nobody wants to dig into config files. So, that's what we are doing now.

== Checkin ==

Plastic uses a pipeline to checkin as fast as possible. There are 3 phases:

* Read from disk.

* Compress data.

* Send data.

By default, each phase uses one thread, so the checkin is always multi-thread.

But, using fast disks (typically SSD and beyond), everything can be done faster using more threads in each phase.

The new defaults are:

* Read from disk. Controlled by UploadReadPoolSize in client.conf = 2

* Compress data. Configured by UploadCompressionSize in client.conf = 2

* Send data. Configured by Upload_SendDataThreadCount in client.conf = 3

Remark: The old defaults are written to client.conf on all installations out there, so this version will consider 1 as not configured, and will set the new values. In case you really want to force 1 thread, set these 3 values to zero. Yes, it is confusing, but we need to bypass the defaults, we'll remove this weird thing in one year or so when everyone is upgraded.

== Update ==

Update uses 1 thread to download and write. The setting is controlled by UploadReadPoolSize in client.conf and was set to zero by default. The default will be now 2.

All platforms – Proxy Server, Client: Added support to cache data from Plastic Cloud.

This is one of the long-awaited features for the Proxy: the ability to cache data from the Cloud.

And it is finally there. No changes required, just install a proxy (and your client too if you use encrypted data in the Cloud) and start caching data.

All platforms - Server: Replica data transfer can be 3 times faster now on fast networks.

In our tests, pushing 18Gb from a repo in Denmark to one in Oregon, went down from 30 minutes to 9. This is quite an speed up!

Screenshot

The key is using the multi-thread capabilities to make sure Plastic reaches the maximum network potential.

To enable multiple threads, set the ReplicaDataTransferThreads in server.conf. The actual number will depend on your environment. In ours, we saw an incredible speed up going from 1 (the default) to 8.

The process distributes the size of data to send evenly among the available threads, and that's the key to achieve a consistent improvement.

WARNING: do not use this with SQL backends, it is only safe to use with Jet.

REMARK: push/pull to Cloud was already multi-thread and is not affected by this task.

Here is sample output before and after:

Before: 2180860 ms = 36.34 min

>cm push /main@unity-game@localhost:8087 mirror4@34.xxx.xxx.xxx:8087 --timer
[##############################################################]
/ Finished OK
Fetching metadata               [##############################]
Pushing metadata                [##############################]
Introducing data                [##############################]
2180860 ms

After: 534750 ms = 8.9 min

>cm push /main@unity-game@localhost:8087 mirror3@34.xxx.xxx.xxx:8087 --timer
[##############################################################]
\ Finished OK
Fetching metadata               [##############################]
Pushing metadata                [##############################]
Introducing data                [##############################]
534750 ms

All platforms - Plastic, Gluon: changes to the diff window comparison method {"Ignore EOL", "Ignore whitespaces", "Ignore EOL and whitespaces", "Recognize all"} are now persisted.

It used to be a bit annoying that when you changed the comparison method, that change would be forgotten as soon as you closed the diff, and you would have to change it again next time. That happened because we initialise the diff view with the global setting from Preferences -> Diff and merge. Now, changing the option in the diff window updates the global setting, so next time you open a diff you get your previously selected option.

Windows - Plastic: after failing to save an attribute value change, due to permissions for example, the GUI would be left displaying the wrong value. it now updated to show the correct attribute value in this case.