Release Notes

All platforms - Proxy: It is now able to monitor free space on disk and clean up the cache to avoid running out of disk.

You'll see something as follows in the log when it runs out of disk:

INFO  Daemon - Disk is running out of space. 5.78 Gb available. Will delete 1.15 Gb from cache
INFO  Daemon - Cleaning up 1.15 Gb (9576 entries)
INFO  Daemon - Cleaned up 1.15 Gb (9576 entries) in 2578 ms

And, if there is enough room (twice the size of the cache) then something as follows:

INFO  Daemon - Disk still has enough space: 19.53 Gb available.

All platforms - Plastic Proxy: Implemented a disk cache limit.

Now the proxy can limit the size of its on-disk cache.

== How to configure ==

Very easy: last use the new MaxCacheSizeInGb in the plasticcached.conf file.

Example:

<PlasticCacheConf>
        <MaxCacheSizeInGb>20.5</MaxCacheSizeInGb>
</PlasticCacheConf>

And the Proxy will make sure its cache size doesn't exceed 20.5 Gb.

A value of 0 means the clean is disabled.

== How it works ==

The Proxy maintains an LRU (Least Recently Used list) and updates it on every data access.

Every 10 minutes, it checks if the total cache size is bigger than the value configured. If the maximum size is exceeded, the Proxy retrieves the least recently used entries and removes them from disk.

Every 3 minutes the Proxy saves the LRU info to disk, to proxy-data-path/proxy-lru.dat. The Proxy also writes data on shutdown.

On startup and then every 8 hours, the Proxy checks if it needs to walk the entire cache directory to rebuild the LRU and recover from possible inconsistencies. While the check runs every 8 hours, a rebuild only happens once every 3 days. The Proxy stores the date of the last rebuild under proxy-data-path/proxy-last-cache-walk.dat.

proxy-data-path

is the directory where you configured the proxy to store the cache. For existing Proxy installations, the new version will rebuild the cache on the fly. == Some logs == During the startup of the Proxy you'll see:

INFO  LRU - Walking the disk to build the LRU (least recently used). c:\plastic\proxydata
INFO  LRU - Last LRU rebuild from disk happened less than 3 days ago. Skipping. 11/04/2019 16:50:00
INFO  Daemon - Load LRU info from c:\plastic\proxydata\proxy-lru.dat
INFO  Daemon - LRU info loaded in 656 ms. 107299 entries. 5.89 Gb total in cache

You can see how the Proxy skips the rebuild and how it loaded 107k entries to the LRU.

Then, every 3 minutes, and during shutdown:

INFO  Daemon - Save LRU to c:\plastic\proxydata\proxy-lru.dat
INFO  Daemon - LRU info saved in 312 ms. 107299 entries. 5.89 Gb total in cache

And during a clean up:

INFO  Daemon - Cleaning up 0.69 Gb (17564 entries) of cached data to enforce the 5.20 Gb limit
INFO  Daemon - Cleaned up 0.69 (17564 entries) Gb in 5110 ms
INFO  Daemon - Save LRU to c:\plastic\proxydata\proxy-lru.dat
INFO  Daemon - LRU info saved in 265 ms. 89735 entries. 5.20 Gb total in cache

DevOps: ConflictsBot was useless with MergeRules enabled (they are available since release 8.0.16.3433).

Now, ConflictsBot can report merge conflicts again at earlier stages!

The fix is skipping the MergeRules check in the underlying dry-run merge operation the ConflictsBot runs.

All platforms - Plastic, Command line client: Metadata download and processing times in the update debug log were not correct. This is now fixed.

All platforms - Checkin & Update: Multi-thread checkin and update enabled by default now.

Plastic can be super-fast, but historically we were shy enabling the full power by default. But smart defaults are key for usability, because nobody wants to dig into config files. So, that's what we are doing now.

== Checkin ==

Plastic uses a pipeline to checkin as fast as possible. There are 3 phases:

* Read from disk.

* Compress data.

* Send data.

By default, each phase uses one thread, so the checkin is always multi-thread.

But when using fast disks (typically SSD and beyond) everything can be faster using more threads in each phase.

The new defaults are as follows. These configuration keys belong at client.conf file:

* Read from disk. Controlled by UploadReadPoolSize. Default value is 2.

* Compress data. Configured by UploadCompressionSize. Default value is 2.

* Send data. Configured by Upload_SendDataThreadCount. Default value is 3.

Remark: The old defaults are written to client.conf on all installations out there, so this version will consider 1 as not configured, and will set the new values. In case you really want to force 1 thread, set these 3 values to zero. Yes, it is confusing, but we need to bypass the defaults, we'll remove this weird thing in one year or so when everyone is upgraded.

== Update ==

Update uses 1 thread to download and write. The setting is controlled by UploadReadPoolSize in client.conf and was zero by default. The default will be now 2.

All platforms – Proxy Server, Client: Added support to cache data from Plastic Cloud.

This is one of the long-awaited features for the Proxy: the ability to cache data from the Cloud.

And it is finally here! No changes required, just install a Proxy (and your client too if you use encrypted data in the Cloud) and start caching data.

All platforms - Server: Replica data transfer can be 3 times faster now on fast networks.

In our tests, pushing 18Gb from a repo in Denmark to one in Oregon, went down from 30 minutes to 9. This is quite a speed up!

The key is using the multi-thread capabilities to make sure Plastic reaches the maximum network potential.

To enable multiple threads, set the ReplicaDataTransferThreads key in server.conf to the number of threads you wish to use.

The actual number will depend on your environment. In ours, we saw an incredible speed up going from 1 (the default) to 8.

The process distributes the size of data to send evenly among the available threads, and that's the key to achieve a consistent improvement.

WARNING: do not use this with SQL backends, it is only safe to use with Jet.

REMARK: push/pull to Cloud was already multi-thread and is not affected by this task.

Here is sample output before and after:

Before: 2180860 ms = 36.34 min

>cm push /main@unity-game@localhost:8087 mirror4@34.xxx.xxx.xxx:8087 --timer
[##############################################################]
/ Finished OK
Fetching metadata               [##############################]
Pushing metadata                [##############################]
Introducing data                [##############################]
2180860 ms

After: 534750 ms = 8.9 min

>cm push /main@unity-game@localhost:8087 mirror3@34.xxx.xxx.xxx:8087 --timer
[##############################################################]
\ Finished OK
Fetching metadata               [##############################]
Pushing metadata                [##############################]
Introducing data                [##############################]
534750 ms

All platforms - Plastic, Gluon: changes to the diff window comparison method {"Ignore EOL", "Ignore whitespaces", "Ignore EOL and whitespaces", "Recognize all"} now persist.

Before this change, you lost comparison method changes when closing the diff - you would need to change them again next time. That happened because the diff view loaded the global setting from Preferences -> Diff and merge. Now, changing the option in the diff window updates those global settings, so the next time you open a diff you get your previously selected option.

Windows - Plastic: The GUI displayed the wrong value for an attribute if it failed to save a value change (due to lack of permissions, for example). In this case, now the GUI recovers and shows the right value for the attribute, along with the error message.