Avian: AI Inference for Enterprise

Avian is a generative AI platform used by teams at companies like eBay, Salesforce, and Boeing to run inference on state-of-the-art language models. It offers a fast, open-source LLM API designed for enterprise-grade performance.

Key Features

• Speed and Performance: Avian boasts impressive processing speeds, achieving 142 tokens per second with the Meta Llama 3.1 405B Instruct model. This speed is powered by Nvidia H200 SXM hardware, ensuring reliability and efficiency.
• Cost-Effectiveness: At $3 per million tokens, Avian offers a competitive pricing model, significantly reducing the cost of AI inference compared to other solutions.
• Open-Source Foundation: Avian leverages open-source models, providing transparency and control over the AI infrastructure. It's built on Meta's Llama 3.1 405B model.
• OpenAI Compatibility: Avian's API is designed to be OpenAI-compatible, making integration into existing workflows seamless. Switching from OpenAI requires only a simple base URL change.
• Native Tool Calling: Avian supports native tool calling, enabling seamless integration with external tools and APIs for enhanced capabilities.
• Streaming Capabilities: The platform offers efficient streaming for real-time responses, ideal for interactive applications.
• Privacy and Security: Avian prioritizes privacy and security, operating with secure, SOC/2 approved Open Source Foundation language models on Microsoft Azure. Live queries are used, ensuring no data is stored, and compliance with GDPR, CCPA, and SOC/2 is maintained.
• Data Connectors: Avian provides data connectors for various sources, including spreadsheets, databases, and popular platforms like Shopify, LinkedIn Ads, and Google Analytics.
• Built-in Functionality: Avian includes built-in features like RAG (Retrieval Augmented Generation), internet access, tool calling, and code interpreter.

Model Performance

Avian's use of the Llama 3.1 405B model results in superior natural language understanding, excellent performance on complex reasoning tasks, high accuracy in knowledge-based queries, and a competitive edge in human evaluation tests.

Ease of Use

Getting started with Avian is quick and easy. Setup takes only a minute, and the OpenAI-compatible API simplifies integration.

Comparisons

While Avian's performance rivals that of OpenAI, it offers a significant cost advantage and the benefit of using open-source models. The specific performance differences will vary depending on the task and model used, but Avian consistently demonstrates competitive or superior results in many benchmarks.

Conclusion

Avian provides a powerful, private, and secure solution for enterprise-grade AI inference. Its speed, cost-effectiveness, and OpenAI compatibility make it a compelling alternative for businesses seeking to leverage the power of state-of-the-art language models.