Overview
Three jobs that used to be separate now sit in the same codebase: serving a REST API, serving a trained model, and serving a large language model. Each has its own established tooling, its own conventions, and its own idea of what a request looks like, so a team doing all three ends up maintaining three architectures that happen to run next to each other.
Flama is an open-source Python framework that treats them as one problem. It is built on the Asynchronous Server Gateway Interface, and offers a type-driven, async-first programming model that covers REST API development, predictive model serving and generative inference in a single architecture.
This paper is the reference description of it: the architecture, the programming model through worked examples, and a comparison against existing web frameworks, model-serving platforms and LLM inference engines.
The architecture
The framework is organised around seven subsystems. A component-based dependency injection system resolves handler parameters from their type annotations at startup, which is what makes the programming model type-driven rather than convention-driven. A pluggable schema layer puts Pydantic, Marshmallow and Typesystem behind a single adapter, so the choice of validation library stops being an architectural commitment.
An automatic CRUD generator turns a SQLAlchemy table and a schema class into REST endpoints, backed by the Repository and Unit of Work patterns. A Rust-accelerated core, compiled through Maturin, handles routing, JSON encoding, compression and parsing. A Model Context Protocol module turns any application into an MCP server over JSON-RPC 2.0.
The remaining two subsystems are what separate this from a general-purpose web framework, and they are worth taking one at a time.
Serving models
The first is a portable binary format, .flm, which packages a trained model together with its metadata. It covers models from scikit-learn, TensorFlow, PyTorch and Hugging Face Transformers, which means the artefact rather than the training framework becomes the unit of deployment.
The consequence is zero-code deployment: a packaged model is served without writing a handler for it. The command-line interface runs applications and also serves, packages and inspects models directly, so the same artefact can be examined and deployed from one place.
LLM services
The second is a multi-backend LLM server, running vLLM on Linux with CUDA or MLX on Apple Silicon. The same server exposes four wire protocols through a shared codec: OpenAI, Anthropic, Ollama, and a native streaming dialect.
That is the part that makes the unification concrete rather than nominal. A client written against any of those four protocols talks to the same deployment, so the inference backend and the client contract become independent choices.
What comes built in
Around those subsystems the framework ships JWT authentication, two pagination strategies, background tasks in either threads or processes, WebSocket endpoints, and streaming over both Server-Sent Events and NDJSON.
OpenAPI 3.2.0 documents are generated from handler signatures rather than maintained alongside them, which follows from the same decision as everything else here: the type annotations are the source of truth, and the rest is derived from them.
Why it matters
Most of the research on this site ends in a model, and a model that cannot be deployed is a result rather than a tool. Flama is the piece that closes that gap, and it is the framework the rest of our work is served through.
It is open source, and this paper is the citable description of what it does and how it is built.
Citation
José A. Perdiguero López, Miguel A. Durán-Olivencia. Flama: a Python framework for development and deployment of production-ready APIs, machine learning, and LLM services. arXiv, 2026. https://arxiv.org/abs/2608.18733