View on GitHub

Quelea

A Haskell library for declarative programming over eventually consistent store (Cassandra).

Download this project as a .zip file Download this project as a tar.gz file

LWW GC Benchmark

Quelea’s programming model encourages programmers to define operations on a replicated data object as folds over effects accumulated over the object. A straightforward implementation, which lets effects to simply accumulate over time would not scale. Quelea implements effect summarization (or, garbage collection), which is the key to ensure that the programming model remains practical and competitive to alternative models. The aim of this benchmark is to facilitate experiments to demonstrate the impact of garbage collection. We use Last-Writer-Wins register operations for this purpose.

Building the benchmark

Navigate to ~/git/Codeec/tests/LWW and run make LWW_1key. It produces one binary named LWW_1key.

Running the benchmark

LWW_1key implements three garbage collection policies - No_GC, GC_Mem_Only, and GC_Full, each corresponding to a different curve on Fig. 9(d) of the paper. We conduct three experiments with the binary, each lasting for 1 minute and employing different garbage collection strategy. For 1 minute, clients keep making write and read requests (with an inter-request delay of 1ms (1000 μs)) to a single LWW register object. For each GC strategy, we then measure how many client requests have been served under 1 minute.

No GC

Corresponds to Quelea with no effect summarization. The command to run is:

./LWW_1key --kind Daemon --numRounds 10000 --numThreads 2 --measureLatency --gcSetting No_GC --terminateAfter 60

lww-gc-ngc

Ignore the Unconfigured column family errors, if any; these are generated by outstanding requests after 60s timout has been reached, and Cassandra column families have been dropped. However, if there are any ZMQError messages, please refer to the troubleshooting guide for a quick fix.

GC in memory only

This is the case where Quelea does garbage collection only in the memory, but not on the disk. The command to run is:

./LWW_1key --kind Daemon --numRounds 10000 --numThreads 2 --measureLatency --gcSetting GC_Mem_Only --terminateAfter 60

lww-gc-mgc

Full GC

Corresponds to Quelea with effect summarization in both memory and disk. The command to run is:

./LWW_1key --kind Daemon --numRounds 10000 --numThreads 2 --measureLatency --gcSetting GC_Full --terminateAfter 60

lww-gc-fgc

Observations

As sample runs demonstrate, you should be able to observe that Quelea with effect summarization is orders of magnitude more scalable than one without.