Glide Introduction
Welcome to the documentation of Glide - The Cloud Native LLM Gateway.
What is Glide?
Glide is a lightweight, blazing-fast cloud-native open-source LLM gateway that delivers high-performance LLMOps in one single binary. It provides a simplified way to build application resilience, reduce latency, and manage API keys, among many other features. Currently, Glide is under development, and the documentation is updated frequently.
Why Glide?
The True Gateway
Focus on your business logic and application while letting Glide handle common LLMOps problems like fallbacks, cost control, routing, caching, etc.
Blazing-Fast & Cloud-Native
High performance and scalability are our priority, so we don’t slow you down and scale out as you grow.
Resilient
Glide is designed with resiliency and failures in mind, so your application can achieve high availability working with external models and survive the toughest real-world situations.
Configurable
A Glide cluster can serve multiple applications with totally different needs.
No Vendor Lock-Ins
Switch your model providers seamlessly for your application without a need to modify them.
Open Source
We welcome all kinds of feedback and contributions in Glide. Let’s make it work in the best possible way for you!
Glide Features
Unified API
Change your model providers without a need to modify your applications.
High Performance
Written in Golang with the best practices of software development in a cloud-native environment in mind.
Flexible Routing
Route flexibly across several models with various strategies like the least latency, weighted round-robin, round-robin, priority-based routing. All strategies include low-latency fallback mechanisms to ensure resiliency.
Support Popular LLM Providers
Glide supports popular LLM providers.
Centralized Configuration and API Key Management
Manage your configurations and API keys centrally.
Production-ready Observability
Always know what’s going on with the heart of your system via logs, metrics, and traces. OpenTelemetry is natively supported (coming soon).
Glide Installation
Docker
Get started with Glide in a few minutes with Docker (recommended).
Command Line Interface (CLI)
Install Glide locally into your developer machine or bare metal setup via CLI (MacOS, Linux).
Kubernetes
Install Glide into a Kubernetes cluster.
Glide License
Glide is provided under the Apache 2.0 open source license.
Developed with ❤️ by Roma, Max, and the rest of the EinStack Community.
Frequently Asked Questions (FAQs)
Q: What is the main purpose of Glide?
A: Glide is designed to be a lightweight, blazing-fast cloud-native open-source LLM gateway that delivers high-performance LLMOps.
Q: How can Glide help in managing API keys?
A: Glide provides centralized configuration and API key management, allowing you to manage your API keys easily and efficiently.
Q: Can Glide handle fallbacks and cost control?
A: Yes, Glide can handle common LLMOps problems like fallbacks, cost control, routing, caching, etc.
Q: Is Glide open source?
A: Yes, Glide is an open-source project. We welcome all kinds of feedback and contributions.
Q: Which programming language is Glide written in?
A: Glide is written in Golang, following the best practices of software development in a cloud-native environment.
