Performance

VM Performance Test

In this guide, we'll demonstrate how to evaluate the performance of ZebClient in different deploy modes. This is useful for those looking to gauge ZebClient's performance during a trial period or for verifying the correct functioning of their infrastructure.

What does the Test Measure?

This test measures the performance of sequential write and read operations directly to the underlying storage, bypassing the OS cache. It evaluates the throughput and latency of the storage system when faced with a specific workload, which consists of 16 concurrent threads each writing 64KB blocks. The data being written follows a specific pattern and is 50% compressible. The test runs for a duration of 5 minutes, providing insights into the sustained write capabilities of the storage under test.

You can reproduce this test and customize it for your needs by consulting fio documentation to simulate your load requirements.

How do I Test my Cluster?

First, define your cluster needs, please refer to Deployment modes for more information about available architectures.

For testing purposes, we suggest using the following formula to define your optimal load:

• Number of jobs --numjobs = number of physical cores

• Filesize --size = RAM/(Number of physical cores)

• Ramp up time --ramp_time=120 (2 minutes of ramp-up time exclude any performance "spikes" that might affect the results)

Single-Agent Performance

Those tests were performed on cluster built from:

Deployment mode: direct

1x Standard L32s v3 Linux (redhat 8.8)

1x db: Standard D16s v4 Linux (ubuntu 20.04)

4x Azure Blob Storage Performance: Standard, StorageV2 (general purpose v2)

Follow these steps:

1. ssh to agent and cd /fs(default mount point).

2. Run the write test fio --name=64kseqwrite --rw=write --direct=1 --ioengine=libaio --bs=64k --numjobs=16 --size=4G --time_based=1 --runtime=300 --group_reporting --buffer_compress_percentage=50 --buffer_pattern=0xdeadbeef

3. Flush system cache between each runsysctl vm.drop_caches=3

4. Run the read test fio --name=64kseqread --rw=read --direct=1 --ioengine=libaio --bs=64k --numjobs=16 --size=4G --time_based=1 --runtime=300 --group_reporting --buffer_compress_percentage=50 --buffer_pattern=0xdeadbeef

• Total write IOPS: 40.1k

• Total write bandwidth: 2627 MB/s

• Total read IOPS: 73.1k

• Total read BW: 4789 MB/s

Single Agent performance across data compressability spectrum

Multi-Agent Cluster Performance:

Those tests were performed on cluster built from:

Deployment mode: direct 6x Standard L32s v3 Linux (Redhat 8.8)

1x db: Standard D16s v4 Linux (Ubuntu 20.04)

4x Azure Blob Storage Performance: Standard, StorageV2 (general purpose v2)

Running the FIO Test from Every Agent

If you would like to run the same test across multiple agents you can run the same command on each agent Make sure you are operating either in different directories like /fs/directory1, /fs/directory2 ... /fs/directory3 or use a unique name instead of --name=64kseqread parameter, for example --name=64kseqread_agent1.

• Total write IOPS: 173.6k

• Total write bandwidth: 11449 MB/s

• Total read IOPS: 350.8k

• Total read BW: 22699 MB/s

Multi Agent performance across data compressability spectrum

Conclusion

The ZebClient file system shows heightened performance with increasing data compressibility. For files 50% compressible or more, there's a notable uptick in both READ and WRITE speeds. Conversely, for non-compressible data, it's advantageous to avoid compression, preventing unnecessary CPU consumption by the compression algorithm. In essence, using compression for suitable data can optimize throughput, while bypassing it for non-compressible data conserves resources.