Heroku Managed Inference and Agents

Heroku AI provides access to to top models and built-in tools for agents.

mia - tools mia - overview
Install Add-on

Managed Inference

Managed Inference and Agents simplifies AI integration by providing access to powerful foundation models, including text, embedding, and diffusion models. Easily attach model resources to your Heroku app, and the add-on will automatically configure environment variables, enabling seamless API calls. Invoke models using the CLI plug-in or with API endpoints.

Agents

Extend Agents with tools that allow Large Language Models (LLMs) to execute actions within Heroku’s trusted environment. Deploy autonomous agents that can call APIs, run code, or interact with your app through tools like code_exec, http, or custom ones. Move from prototyping to production with optimized inference latency and minimal infrastructure management.

Model Context protocol

The Model Context Protocol (MCP) is an open standard that helps you extend Agents by connecting large language models to tools, services, and data sources. You can bring your own custom tools by deploying them as a heroku app and registering them by attaching the addon. Access all you mcp servers through a single toolkit.

Use Cases

Text Generation Use models like Claude-Sonnet to generate text, write code, or chat intelligently. Retrieval-Augmented Generation (RAG) Bring your own data to power LLMs with up-to-date, domain-specific knowledge. Personalize User Experiences: Leverage agents to deliver tailored content, recommendations, or support. Data Analysis and Business Intelligence: Deploy agents that can analyze large datasets, identify trends, generate reports, and provide actionable insights.

Metered Billing

For those customers paying by credit card, Heroku Managed Inference and Agents uses metered billing, as set forth in the Plans & Pricing tables below For enterprise customers, your usage of Heroku Managed Inference and Agents will consume your General Add-on Credits and/or Data Add-on Credits as set forth in the Plans & Pricing tables below.

Data protection

Heroku Managed Inference and Agents doesn’t store or log your prompts and completions. Heroku Managed Inference and Agents doesn’t use your prompts and completions to train any models and doesn’t distribute them to third parties for training.

Pricing & Availability

Region Filter

Get Started with Heroku Managed Inference

Access All Models with One Add-on

Recommended

Attach Heroku Managed Inference in Standard Mode to get instant access to all supported Managed Inference models through a single add-on.

Pro: Access to 20+ AI models
Pro: Access to new models immediately when they become available
Pro: Switch models without redeploying
Pro: Pay only for what you use
Pro: One inference key for all models
Pro: Available in US and EU regions

Attach Standard Model

Recommended

Attach the Standard Model to get access to all available AI models through a single add-on. Switch between models at any time by changing the model parameter in your API calls.

Attach on the Dashboard

Attach Standard Model (opens in new tab)

Or, Attach using the CLI

$ heroku addons:create heroku-inference:standard -a $APP_NAME

The Standard Model is available to apps in all regions. Override the region by adding the --region flag.

View Documentation (opens in new tab)

Attach an Individual Model

Select a model to attach to your application.

Attach on the Dashboard

Attach Amazon Rerank 1.0 (opens in new tab)

Or, Attach using the CLI

$ heroku addons:create heroku-inference:amazon-rerank-1-0 -a $APP_NAME

This model is available to apps in all regions. Override the region by adding the --region flag.

View Documentation (opens in new tab)

Documentation