Data Framework
To help your company amaze customers with technically advanced products.
- Supports faster time-to-market for products using advanced data processing algorithms and artificial intelligence.
- Built by developers for developers.
- Integrated with knowledge base.
What data framework is?
It is a data utility tool for building applications powered by Large Language Models (LLMs). It simplifies connecting LLMs with your own data, a process known as data augmentation, by providing tools for data ingestion, indexing, and querying. Its primary use case is creating Retrieval-Augmented Generation (RAG) applications, like chatbots that can answer questions based on your specific documents and knowledge bases.
Deployment models
Extravaganza Data Framework in Cloud
To start the journey with provided solution please register private or company account for cloud services, accept terms and conditions of services usage and freely use the provided solutions.
Compliant with LlamaIndex API.
Extravaganza Data Framework Cloud On-Prem
Cloud On-Prem brings Data Framework features directly to your infrastructure. Extravaganza Cloud On-Prem enables you to deploy the complete solution with your own environment to meet your company internal needs.
This solution is built for organizations that require enhanced data privacy, strict compliance adherence, and complete infrastructure control. Whether you’re managing critical infrastructure, operating under stringent regulations like GDPR or HIPAA, or simply need the flexibility to customize your environment, Extravaganza Data Framework Cloud On-Prem delivers enterprise-grade solution without compromising on security or control.
Compliant with LlamaIndex API.
Before You Begin
Ensure you have the following ready before starting the installation:
Required:
- Kubernetes cluster configured
- Helm (version 3.12 or newer with OCI Configuration)
- Kubectl with access to Kubernetes cluster
Optional:
- CPU architecture amd64 with AVX2/AVX512 instructions support is recommended
Minimal System Requirements:
| Component | Details |
|---|---|
| Kubernetes | Version 1.33 or newer |
| TLS certificate | Single certificate for all API endpoints. Must be trusted by all connecting entities. |
| Data Framework services | 2 machines with 4-8 CPU cores each, 8-16 GiB memory, GPU acceleration is not required but always welcome (with Nvidia CUDA 13.2 compatible hardware or Apple Metal) |
| NVMe/SSD storage | Storage for uploaded documents, 256 GiB for PVCs (cloud services are not ephemeral) |
| Third-party Services | 2 machines with 4 CPU cores each, 160 GiB NVMe/SSD storage each for PVCs |
Required Components
Third-Party Services
All these components must be installed prior to the Data Framework cloud services:
| Component | Version | Purpose | Notes |
|---|---|---|---|
| MariaDB | 12.x | Main metadata database | |
| Kafka broker | 4.x | Broker for internal inter-service communication |
Step by Step Guide to Install
Key functions
- Data Connection: Allows you to connect to various data sources, including structured, unstructured, and semi-structured data from PDFs, databases, and more.
- Data Indexing: It organizes and stores your data in an efficient format, like a vector store, or graph store, to make it easy for the LLM to retrieve relevant information quickly.
- Data Querying: It enables you to query your data through the LLM, which combines the user’s query with the relevant data to generate an augmented and context-aware answer.
- RAG Pipeline: It provides a framework to build production-ready RAG pipelines, which retrieve relevant data and feed it to the LLM to improve the accuracy and relevance of its responses.
Use cases
- Conversational Chatbots: Creating chatbots that can answer questions using a company’s internal documents and knowledge bases.
- Semantic Search: Building search engines that can understand and respond to natural language queries against large datasets.
- Knowledge Agents: Developing intelligent agents that can process and reason over complex information to perform tasks.
- Data Augmentation: Enriching public LLMs with private data to create applications tailored for specific domains.
API Reference
Data framework API is compliant with LlamaIndex API:
See the original LlamaIndex API reference