> ## Documentation Index
> Fetch the complete documentation index at: https://portkey-docs-chore-bedrock-service-tier-support.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# October

**Portkey in October 🎃 🪔**

October was packed with treats (no tricks!) for Portkey. As we celebrate Halloween and Diwali, we're lighting up your AI infrastructure with some exciting updates. Let's dive in!

### Executive Summary

|                          |                                                                                                                                                                                                                                                                                                                                                                  |
| :----------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Guardrails GA Release    | Production-ready guardrails to enforce LLM behavior in real-time, with support for PII detection, moderation, and more — are now generally available. [(*Docs*)](/product/guardrails)                                                                                                                                                                            |
| Enterprise Momentum      | Refreshed Portkey's [enterprise offering](https://portkey.ai/docs/product/enterprise-offering) with enhanced security features, and support for [AWS Assume Role Auth](/product/ai-gateway/virtual-keys/bedrock-amazon-assumed-role). Also [onboarded](https://x.com/PortkeyAI/status/1841292805454643393) one of the world's largest tech companies to Portkey. |
| Provider Ecosystem       | Added 7 new providers including [vLLM](/integrations/llms/vllm), [Triton](/integrations/llms/triton), [Lambda Labs](/integrations/llms/lambda), and more.                                                                                                                                                                                                        |
| Image Generation         | Added support for Stable Diffusion v3 and Google Imagen.                                                                                                                                                                                                                                                                                                         |
| Integrations             | Added [MindsDB](/integrations/libraries/mindsdb), [ToolJet](/integrations/libraries/tooljet), [LibreChat](/integrations/libraries/librechat), and [OpenWebUI](/integrations/libraries/openwebui).                                                                                                                                                                |
| Prompt Caching           | Anthropic's prompt caching feature is now available directly in prompt playground. [(*Docs*)](/integrations/llms/anthropic/prompt-caching#prompt-templates-support)                                                                                                                                                                                              |
| .NET                     | You can now integrate Portkey with [your .NET app](/api-reference/inference-api/sdks/c-sharp)                                                                                                                                                                                                                                                                    |
| Agent Tooling Leadership | Portkey was recognized for providing [11 critical capabilities](https://x.com/PortkeyAI/status/1851596076488479001) for production-grade AI agents, leading the Agent Ops tooling benchmark.                                                                                                                                                                     |
| Featured Coverage        | Our DevOps for AI vision featured in the [People+AI Newsletter](https://sreeramsridhar.substack.com/p/building-the-devops-for-ai) and [Pulse2 publication](https://pulse2.com/portkey-profile-rohit-agarwal-interview/).                                                                                                                                         |

### Features

* **AWS Assume Role Support**: Enhanced Bedrock authentication for enterprise security [(*Docs*)](/product/ai-gateway/virtual-keys/bedrock-amazon-assumed-role)
* **User Management API**: New API to resend user invites [(*Docs*)](/api-reference/admin-api/control-plane/user-invites/resend-a-user-invite). Also updated the API specs for Prompt Completions [API](/api-reference/inference-api/prompts/prompt-completion), Prompt Render [API](/api-reference/inference-api/prompts/render), and Insert Log [API](/api-reference/admin-api/data-plane/logs/insert-a-log)
* **New OpenAI Param**: OpenAI's `max_completion_tokens` is now supported <Tooltip tip="This means you can send either the `max_completion_tokens` or `max_tokens` param in your requests and Portkey will handle both correctly">across all providers</Tooltip>
* **Caching**: Improved cost calculations for OpenAI & Azure OpenAI cached responses, and Anthropic's prompt caching feature is now available directly in prompt playground
* **Gemini Updates**: Added support for Gemini JSON mode and Controlled Generations along with Pydantic support
* **Bedrock**: Integrated Converse API for `/chat/completions`. [(*Docs*)](/integrations/llms/aws-bedrock#bedrock-converse-api)
* **Enterprise**: Refreshed Portkey's [enterprise offering](https://portkey.ai/docs/product/enterprise-offering) with enhanced security features.
* **C# (.NET) Support**: You can now integrate Portkey in your .NET apps using the OpenAI official library. [(*Docs*)](/api-reference/inference-api/sdks/c-sharp)

***

### Models & Providers

**7 New Providers**: Expanding your model hosting and deployment options.

<CardGroup cols={4}>
  <Card title="Lemonfox" href="/integrations/llms/lemon-fox" />

  <Card title="Lambda Labs" href="/integrations/llms/lambda" />

  <Card title="Dashscope" href="/integrations/llms/dashscope" />

  <Card title="Upstage" href="/integrations/llms/upstage" />

  <Card title="Nvidia Triton" href="/integrations/llms/triton" />

  <Card title="Github" />

  <Card title="vLLM" href="/integrations/llms/vllm" />
</CardGroup>

**2 Image Generation Models**: Strengthening our multimodal capabilities with next-gen image models.

<CardGroup cols={2}>
  <Card title="Stable Diffusion v3" href="/api-reference/inference-api/images/create-image">
    Now available across [Stability AI](/integrations/llms/stability-ai), [Fireworks](/integrations/llms/fireworks), [AWS Bedrock](/integrations/llms/aws-bedrock), and [Segmind](/integrations/llms/segmind)
  </Card>

  <Card title="Imagen on Google Vertex" href="/integrations/llms/vertex-ai#image-generation-models">
    Official support for Google's Imagen model through Vertex AI
  </Card>
</CardGroup>

**2 New LLMs**:

<CardGroup cols={2}>
  <Card title="Llama 3.2">
    Now integrated with [Fireworks](/integrations/llms/fireworks), [AWS Bedrock](/integrations/llms/aws-bedrock), [Groq](/integrations/llms/groq), and [Together AI](/integrations/llms/together-ai)
  </Card>

  <Card title="Vertex Embeddings" href="/integrations/llms/vertex-ai#text-embedding-models">
    Added support for both `English` and `Multilingual` embedding models from Google Vertex AI
  </Card>
</CardGroup>

***

### Integrations

**Model Management & Monitoring**: Enhance your AI infrastructure with enterprise-grade observability.

<CardGroup cols={2}>
  <Card title="LibreChat" href="https://github.com/timmanik/librechat-for-portkey">
    You can now track costs per user on your LibreChat instance by forwarding unique user IDs from LibreChat to Portkey - thanks to [Tim](https://www.linkedin.com/in/tim-manik/)'s contribution!
  </Card>

  <Card title="OpenWebUI" href="/integrations/libraries/openwebui">
    Portkey is the only plugin you’ll need for model management, cost tracking, observability, metadata logging, and more for your Open WebUI instance.
  </Card>
</CardGroup>

**Data & App Integration**: Connect your existing tools and databases to LLMs.

<CardGroup cols={2}>
  <Card title="MindsDB" href="/integrations/libraries/mindsdb">
    Connect your databases, vector stores, and apps to 250+ LLMs with enterprise-grade monitoring and reliability built-in.
  </Card>

  <Card title="ToolJet" href="/integrations/libraries/tooljet">
    Add AI-powered capabilities such as chat completions and automations into your ToolJet apps easily.
  </Card>
</CardGroup>

***

### Guardrails

The guardrails feature is now **generally available** - it brings production-ready content filtering and response validation to your LLM apps.

**Updated Content Safety Guardrails:**

<CardGroup cols={2}>
  <Card title="PII Detection" href="/product/guardrails/list-of-guardrail-checks#pro-llm-guardrails">
    Detect sensitive personal information in user messages
  </Card>

  <Card title="Content Moderation" href="/product/guardrails/list-of-guardrail-checks#pro-llm-guardrails">
    Automated content filtering and moderation
  </Card>
</CardGroup>

**Updated Guardrails to Ensure Response Quality:**

<CardGroup cols={2}>
  <Card title="Language Detection" href="/product/guardrails/list-of-guardrail-checks#pro-llm-guardrails">
    Automatically detect and validate response languages
  </Card>

  <Card title="Gibberish Detection" href="/product/guardrails/list-of-guardrail-checks#pro-llm-guardrails">
    Filter out nonsensical or low-quality responses
  </Card>
</CardGroup>

**And More!**

<CardGroup cols={2}>
  <Card title="Custom Webhooks" href="/integrations/guardrails/bring-your-own-guardrails">
    Metadata sent to the Portkey API will now be automatically forwarded to your custom webhook endpoint.
  </Card>

  <Card title="Lowercase Detection" href="/product/guardrails/list-of-guardrail-checks#portkeys-default-guardrail-checks">
    Check if the given string is lowercase or not.
  </Card>
</CardGroup>

***

### Resources

**Quick Implementation Guides:**

* [Guide to Prompt Caching](https://x.com/PortkeyAI/status/1843209780627997089): Learn how to optimize your LLM costs
* [Production Apps with Vercel](https://x.com/PortkeyAI/status/1844675148609204615): Learn how to build prod-ready apps using Vercel AI SDK
* [OpenAI Swarm Cheat Sheet](https://x.com/jumbld/status/1846909380354064526): Learn how OpenAI's new Swarm framework really works

**Technical Deep Dives for Production Deployments:**

<CardGroup cols={2}>
  <Card title="OpenAI Swarm + Portkey" href="https://www.youtube.com/watch?v=9qLkAEJol9A">
    Build and secure multi-agent AI systems using OpenAI Swarm and Portkey
  </Card>

  <Card title="RAG with Observability" href="https://github.com/Portkey-AI/gateway/blob/main/cookbook/use-cases/Contextual%20Embeddings%20Guide%20Anthropic%2C%20Cohere%2C%20Voyage.ipynb">
    Enhanced version of Anthropic's RAG Cookbook with unified API and monitoring
  </Card>
</CardGroup>

**Latest insights on AI infrastructure and tooling**:

* [Automated Prompt Engineering](https://portkey.ai/blog/what-is-automated-prompt-engineering/): Scale your prompt engineering workflow
* [OpenAI's Prompt Caching](https://portkey.ai/blog/openais-prompt-caching-a-deep-dive/): Optimize costs and performance
* [Complete Prompt Engineering Guide](https://portkey.ai/blog/the-complete-guide-to-prompt-engineering/): Best practices and patterns
* [OpenTelemetry Guide](https://portkey.ai/blog/the-developers-guide-to-opentelemetry-a-real-time-journey-into-observability/): Real-time observability for AI systems

Check out more technical content on our [Blog →](https://portkey.ai/blog).

***

### Fixes

**Model & Provider Enhancements**

Fixed core provider issues and improved reliability:

* Enhanced streaming transformer for Perplexity
* Fixed response transformation for Ollama
* ⭐️ Added missing logprob mapping for Azure OpenAI (Thanks [Avishkar](https://www.linkedin.com/in/avishkar-gupta/)!)
* Fixed token counting for Vertex embeddings (now using tokens instead of characters)
* Added support for Bedrock cross-region model IDs with pricing
* Fixed media file handling for Vertex AI & Gemini

**Default Models**

We've also reset the <Tooltip tip="These are the models called by default when you do not pass a specific model slug in your request">default model options</Tooltip> for the following providers:

* **Fireworks**: `accounts/fireworks/models/llama-v3p1-405b-instruct`
* **Together AI**: `meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo`
* **Gemini**: `gemini-1.5-pro`

**Dev Ex Improvements**

* Added support for `anthropic-beta` and `anthropic-version` headers in the Portkey API
* In Portkey SDK, the Portkey API key is now optional when you're calling the self-hosted Gateway
* Enhanced support for custom provider headers in SDK

***

### Community Updates

**Upcoming Events**

<CardGroup cols={1}>
  <Card title="LLMs in Prod Dinner Singapore" href="https://lu.ma/llms-in-prod-dinner">
    Join top tech leaders for a closed-door dinner around OpenAI Dev Day. [Register here](https://lu.ma/llms-in-prod-dinner)
  </Card>
</CardGroup>

**Service Reliability**

When OpenAI users were hitting usage limits earlier this month, [Portkey users remained unaffected](https://x.com/PortkeyAI/status/1841172271076954588) thanks to our built-in reliability features.

**Industry Recognition**

* Our DevOps for AI vision was featured in the [People+AI Newsletter](https://sreeramsridhar.substack.com/p/building-the-devops-for-ai) and [Pulse2 publication](https://pulse2.com/portkey-profile-rohit-agarwal-interview/).
* Portkey was recognized for providing [11 critical capabilities](https://x.com/PortkeyAI/status/1851596076488479001) for production-grade AI agents.

**Recent Events**

We co-sponsored the [TED AI Hackathon](https://x.com/PortkeyAI/status/1847733473529999377)! Thanks to everyone who participated and built amazing projects.

***

### Support

<CardGroup cols={2}>
  <Card title="Bug Report" icon="bug" href="https://github.com/Portkey-AI/gateway/issues">
    Found a bug or have a feature request? Open an issue on our GitHub repository.
  </Card>

  <Card title="Join Portkey Discord" icon="discord" href="https://portkey.wiki/community">
    Collaborate with Industry Practitioners and get 24x7 support.
  </Card>
</CardGroup>