Deployment Modes and Tuning

General Deployment Tuning

  • Compression - Data compression is enabled by default in all default deployment modes. If data is less compressible than 50% a performance test should be done to evaluate if compression is benificial to the use-case.

  • Cache - Default cache size is optimal for most use-cacses. Tune down the cache size for small Application nodes with low RAM. Minimum RAM size for an Application node using a default deploy is 12GB

  • k+m - If Tiered mode deploy gives significant better performance than Converged for the tested use-case, contact Zebware for a support to tune the Converged deploy.

  • Scale-out - For large scale-out clusters concider using Tiered or Direct as the first choice of deployment mode.

Deployment Modes

ZebClient offers three distinct cluster deployment modes, enabling users to tailor their ZebClient setup and optimize costs. Each mode is meticulously designed to ensure optimal node utilization and deliver peak performance.

Tiered Mode

In the tiered mode, ZebClient deploys one or multiple tiers of the Acceleration Layer on dedicated nodes. This setup ensures unparalleled I/O performance and scalability, bound only by the underlying infrastructure. Pros: Peerless and scalable performance across all metrics. Cons: While the costs are competitive, they are the highest among the three deployment modes.

Converged Mode

In the converged mode, ZebClient introduces the Acceleration Layer on the application nodes, integrating it alongside customer applications. This mode permits users to allocate a portion of the CPU's capabilities to the Acceleration Layer for the cluster agents. This is achieved with a modest increase in CPU consumption. Pros: Superior IOPS performance relative to direct mode. Moderate infrastructure overhead, facilitating cost optimization. Cons: Marginally elevated CPU consumption due to ZebClient processes on the nodes. Augmented network I/O demand by the ZebClient system.

Direct Mode

In direct mode, ZebClient operates with a direct connection to Cold Object Storage. This mode bypasses the acceleration layer, which allows users to reduce infrastructure costs by accessing data straight from the Cloud Object Storage. Pros: Simple to maintain and manage. Ideal for sequential I/O operations on compressible data. Cons: IOPS performance is constrained by the Cloud Object Storage's capabilities.

Summary

  • In a Tiered deploy of ZebClient the agent is using an acceleration tier for warm data where the warm data resides on fast NVMe disks and Cold data on S3 or Blob stoarge.

  • For a Converged deploy, ZebClient agent and server are running togheter on the same node having warm tier data on fast SSD/NVMe disk and cold data on S3 or Blob storage

  • The Direct deploy of ZebClient do not have an acceleration layer for the Warm data and ZebClient agent directly stores and reads data from S3 or Blob storage

The Tiered Deploy

General - Tiered

Use the Tiered deploy for maximum performance with high troughput and operations per second (ops) workloads. Scaling out the Acceleration tier to larger k+m will increase the read throughput but write throughput will be capped on each VM by the instance network bandwidth. All application nodes will operate in "stateless" mode and can be scaled both up and out without restrictions.

Default configuration:

# Application VM/node
   No VMs: 1 - 12
   OS: Ubuntu
   vCPUs: 8 - 32
   RAM: 64 - 256 GB

# ZebClient Agent 
   k+m = 3+1
   cachesize = 5 GB (RAM) # Hot read cache
   compression = lz4

# ZebClient acceleration nodes
   No VMs: 4
   server vm: Standard_L8s_v3
   OS: Ubuntu

# Cold object storage
   object storage: Blob

# Metadata node
   No VMs: 1
   metadata vm: Standard_D4s_v2
   disk: Premium SSD v2
   disk size: 32 GB

# Metadata server
   KeyDB (Redis)

The Converged Deploy

General - Converged

Converged or "ZebClient hybrid converged" is a costeffective alternative to Tiered deploy where no extra acceleration VM instances are deployed in a separate tier. Converged deploy use cases are the same as for a Tiered deploy but the ZebClient server nodes in the cluster are "stateful" and needs to be handled with a graceful shutdown. Converged deploy delivers high performance for troughput and operations per second (ops) workloads. Scaling out the number of "ZebClient hybrid converged" to larger k+m will increase the read throughput but write throughput will be capped on each VM by the instance network bandwidth. All non-converged application nodes will operate in "stateless" mode and can be scaled both up and out without restrictions. All converged application nodes will operate in "stateful" mode and needs to be handled gracefully up and down. The Converged VMs must have a large high performance disk onboard. If there is a need for local NVMe performance also for the onboard application (scratch files) this NVMe disk can be shared to reduce costs.

Default configuration:

# Application VM/node
   No VMs: 1 - 12
   OS: Ubuntu
   vCPUs: 8 - 32
   RAM: 64 - 256 GB

# ZebClient Agent 
   k+m = 1+1
   cachesize = 5 GB (RAM) # Hot read cache
   compression = lz4

# ZebClient acceleration nodes
   No VMs: 2
   server vm: Standard_L8s_v3 # Or uniform with Application nodes
   OS: Ubuntu

# Cold object storage
   object storage: Blob

# Metadata node
   No VMs: 1
   metadata vm: Standard_D4s_v2
   disk: Premium SSD v2
   disk size: 32 GB

# Metadata server
   KeyDB (Redis)

The Direct Deploy

General - Direct

Direct is a very costeffective alternative where no acceleration VM instances are used. Direct deploy use cases are where a reduced performance for operations per second (ops) is acceptable and the use-case is manly throughput bond. Scaling up and out the number of application nodes will deliver scalability with high cluster throughput togheter with a "stateless" operation mode.

Default configuration:

# Application VM/node
   No VMs: 1 - 12
   OS: Ubuntu
   vCPUs: 8 - 32
   RAM: 64 - 256 GB

# ZebClient Agent 
   k+m = 1+0
   cachesize = 5 GB (RAM) # Hot read cache
   compression = lz4

# Cold object storage
   object storage: Blob

# Metadata node
   No VMs: 1
   metadata vm: Standard_D4s_v2
   disk: Premium SSD v2
   disk size: 32 GB

# Metadata server
   KeyDB (Redis)

Last updated