Product Components
Last updated
Last updated
The diagram below depicts ZebClient's high-level architecture, which comprises a compute layer and acceleration storage and storage layers. This topic describes each layer.
The Application Layer is where ZebClient agents deliver data as close as possible to the consuming application. It supports two modes of access:
Filesystem: The ZebClient file system can be mounted to the compute node in a POSIX-compatible manner, which provides the data as a mount point directly in the node context.
Kubernetes: With the Kubernetes CSI Driver, ZebClient provides the file system directly to nodes in the Kubernetes pod.
The ZebClient agent is a distributed data engine that manages the marshaling of data from and to the acceleration nodes, manages the caching of data and metadata close to the application, and executes the erasure code used to shard the data. If direct mode is used where no acceleration tier is deployed, the ZebClient agent also manages the placement of data in object storage. Please refer to Data Striping for how this erasure coding stripes the data across the Acceleration Nodes.
The acceleration layer runs ZebClient acceleration engines. Acceleration engines are High-Performance Compute clusters that run data management and metadata management workloads. Each cluster is isolated and can be installed either on-premise, in a private cloud, or in the customer account of a public cloud infrastructure. Within each cluster, acceleration nodes store data and metadata. The engine loads data from the storage layer into the acceleration layer at runtime based on the cluster configuration.
A benefit of the decoupled storage and acceleration architecture is that multiple acceleration engines can be assigned to the same storage. This allows for granular control over which hardware is assigned to which tasks, the sharding configuration employed, the sizing of the hardware used, and the overall size of the acceleration layer cache. Each engine can have a different configuration and size depending on the workload. Engines work in parallel, and unison against a central metadata database to provide the features such as POSIX compliance.
The storage layer within ZebClient runs on any S3-compatible or on Blob storage. After you ingest data into ZebClient, this is where the data and metadata associated with a database are saved. When you ingest data, you use a ZebClient agent, which erasure codes the data and prepares the relevant metadata ready for use by the compute layer. The data is erasure coded, indexed, and optionally compressed, to support highly efficient pruning ready for delivery acceleration.
The services layer provides management for all features of the ZebClient Cluster It gives IT departments the tools they require to manage all aspects of their ZebClient environment. Its most important functions are:
Cluster Management: Handles the adding, removing, and configuration of cluster nodes.
Security: Handles authentication to the cluster.
Data Governance: Handles management of the data held within the ZebClient cluster.
Monitoring: Provides access to the health of the cluster
External data is any source for the data that will be stored in the ZebClient cluster. This can be any type of data such as:
Transactional systems: These are systems used to manage day-to-day business operations, such as ERP, CRM, and e-commerce systems.
Log files: Log files generated by applications and systems can provide valuable information about system performance, usage patterns, and error conditions.
Social media: Social media platforms generate a large amount of data that can be analyzed to gain insights into consumer behavior, opinions, and preferences.
IoT devices: Internet of Things (IoT) devices, such as sensors and smart devices, generate massive amounts of data that can be used for predictive maintenance, asset tracking, and other applications.
Structured data: Relational databases, such as Oracle and MySQL, contain structured data that can be valuable for data warehousing and business intelligence purposes.
Unstructured data: Office data, Photos, semistructured data, possible to perform ML and Advanced analytics operations onto
