> ## Documentation Index
> Fetch the complete documentation index at: https://portkey-docs-chore-bedrock-service-tier-support.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# November

**Portkey in November ❄️**

<img width="150" src="https://mintcdn.com/portkey-docs-chore-bedrock-service-tier-support/QsafeLudo8ykASbv/images/changelog/netapp.jpeg?fit=max&auto=format&n=QsafeLudo8ykASbv&q=85&s=33f4303ccadf35a82c8e7e039f2e95be" data-path="images/changelog/netapp.jpeg" />

We [won](https://www.linkedin.com/posts/1rohitagarwal_we-just-won-the-best-growth-strategy-award-activity-7272134964110868480-_mvc/?utm_source=share\&utm_medium=member_desktop) the NetApp Excellerator Award, launched [prompt.new](https://prompt.new/) for faster development, added folder organization and AI suggestions for prompt templates, introduced multi-workspace analytics.

Plus, there's now support for OpenAI's Realtime API and much more. Let's dive in!

## Summary

| Area         | Key Updates                                                                                                                                                                                                                    |
| :----------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Platform     | • See multi-workspace analytics & logs on a single dashboard<br />• Support for Realtime API across OpenAI and Azure OpenAI<br />• More granular security & access control settings<br />• Organize your prompts in folders    |
| Integrations | • Route to AWS Sagemaker models through Portkey<br />• Support for xAI provider and Llama 3.3 & Gemini 2.0 Flash models<br />• New `strictOpenAiCompliance` flag on the Gateway                                                |
| Enterprise   | • Support for AWS STS with IMDS/IRSA auth<br />• Support for Azure Entra (formerly Active Directory) to manage Azure auth<br />• Set budget limits with periodic resets<br />• Support for any S3-compatible store for logging |
| Community    | • Won NetApp's Best Growth Strategy Award<br />• Hosted first Practitioners Dinner in Singapore<br />• Weekly AI Engineering Office Hours                                                                                      |

#### Enterprise Spotlight

**When API Gateways Don't Cut It**<br />
As AI infrastructure becomes increasingly critical for enterprises, technology leaders are choosing Portkey's AI Gateway for their AI operations.

<Frame caption="Akshay Darbari, Director of Platform Engineering at Premera Blue Cross">
  <img width="700" src="https://mintcdn.com/portkey-docs-chore-bedrock-service-tier-support/QsafeLudo8ykASbv/images/changelog/premera-testimonial.jpeg?fit=max&auto=format&n=QsafeLudo8ykASbv&q=85&s=1985e625feaa65de8319030dc185c3a6" data-path="images/changelog/premera-testimonial.jpeg" />
</Frame>

When Premera Blue Cross’ Director of Platform Engineering needed an AI Gateway, they chose Portkey. Why? Because traditional API gateways [weren’t built](https://portkey.ai/blog/ai-gateway-vs-api-gateway) for AI-first companies. Are you in the same boat? Schedule an [expert consultation here](https://calendly.com/portkey-ai/quick-consult).

***

## Platform

#### Prompt Management

* Type [prompt.new](https://prompt.new) in your browser to spin up a new prompt playground! [Try it now →](https://prompt.new)
* Organize your prompt templates with folders and subfolders:

<Frame>
  <iframe width="700" height="250" src="https://www.youtube.com/embed/Edn4UBkVZZk" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" />
</Frame>

* Use AI to write and improve your prompts - right inside the playground:

<Frame>
  <iframe width="700" height="250" src="https://www.youtube.com/embed/1oU1oY0q9Ok" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" />
</Frame>

* Add custom tags/labels like `staging`, `production` to any prompt version to track changes, and call them directly:

<Frame>
  <img width="200" src="https://mintcdn.com/portkey-docs-chore-bedrock-service-tier-support/QsafeLudo8ykASbv/images/changelog/prompt-labels.gif?s=7c3055bf17f1bf52a11fd47e05d7b2e8" data-path="images/changelog/prompt-labels.gif" />
</Frame>

<CodeGroup>
  ```ts @staging {2} theme={null}
  const promptCompletion = portkey.prompts.completions.create({
      promptID: "pp-article-xx@staging",
      variables: {"":""}
  })
  ```

  ```ts @dev {2} theme={null}
  const promptCompletion = portkey.prompts.completions.create({
      promptID: "pp-article-xx@dev",
      variables: {"":""}
  })
  ```

  ```ts @prod {2} theme={null}
  const promptCompletion = portkey.prompts.completions.create({
      promptID: "pp-article-xx@prod",
      variables: {"":""}
  })
  ```
</CodeGroup>

* Each response inside the playground now gives metrics to monitor LLM throughput and latency

<Frame>
  <img width="450" src="https://mintcdn.com/portkey-docs-chore-bedrock-service-tier-support/QsafeLudo8ykASbv/images/changelog/prompt-metrics.png?fit=max&auto=format&n=QsafeLudo8ykASbv&q=85&s=60daf03d0ed39abd65d37e3f43eca547" data-path="images/changelog/prompt-metrics.png" />
</Frame>

#### Analytics

**Org-wide Executive Reports**<br />

<Frame>
  <img src="https://mintcdn.com/portkey-docs-chore-bedrock-service-tier-support/QsafeLudo8ykASbv/images/changelog/multi-workspace%202.gif?s=b112c2bc13a02cf0ae5dc498ff581537" width="400" data-path="images/changelog/multi-workspace 2.gif" />
</Frame>

Monitor analytics and logs across all workspaces in your organization through a unified dashboard. This centralized view provides comprehensive insights into cost, performance, and accuracy metrics for your deployed AI applications.

* Track token usage patterns across requests & responses

<Frame>
  <img width="150" src="https://mintcdn.com/portkey-docs-chore-bedrock-service-tier-support/WFmeMRRtfA5ARhyP/images/changelog/tokens-request-response.png?fit=max&auto=format&n=WFmeMRRtfA5ARhyP&q=85&s=a0f3ea68f1819238583f5a1e80b8368f" data-path="images/changelog/tokens-request-response.png" />
</Frame>

* You can now filter logs and analytics with specific Portkey API keys. This is useful if you are tying a particular key to an internal user and want to see their usage!

#### Enterprise

We've strengthened our enterprise authentication capabilities with comprehensive cloud provider integrations.

* Expanded AWS authentication options, for adding your Bedrock models or Sagemaker deployments:
  * IMDS-based auth (recommended for AWS environments)
  * IRSA-based auth for Kubernetes workloads
  * Role-based auth for non-AWS environments
  * STS integration with assumed roles
* Also expanded the Azure Integration:
  * Azure Entra (formerly Active Directory)
  * Managed identity support
* Granular access permissions for API Keys and Virtual Keys across your organization
* Support for sending Azure `deploymentConfig` while making Virtual Keys through API. [Docs](/api-reference/admin-api/control-plane/virtual-keys/create-virtual-key)

<Frame>
  <img width="450" src="https://mintcdn.com/portkey-docs-chore-bedrock-service-tier-support/QsafeLudo8ykASbv/images/changelog/security-settings.png?fit=max&auto=format&n=QsafeLudo8ykASbv&q=85&s=93eb3306745134ff2b93a966d57fb7d1" data-path="images/changelog/security-settings.png" />
</Frame>

***

#### More Customer Love

Felipe & team are building [beconfident](https://beconfident.app/), and here's what they had to say about Portkey:

> "Now that we've seen positive results, we're going to move all our prompts to Portkey."

<img src="https://mintcdn.com/portkey-docs-chore-bedrock-service-tier-support/QsafeLudo8ykASbv/images/changelog/felipe-testimonial.jpeg?fit=max&auto=format&n=QsafeLudo8ykASbv&q=85&s=847e00f7465eee0f154bfd6bca2f8040" width="400" data-path="images/changelog/felipe-testimonial.jpeg" />

***

## Integrations

#### Providers

<CardGroup cols={2}>
  <Card title="AWS Sagemaker" href="/integrations/llms/aws-sagemaker">
    Add your Sagemaker deployments to Portkey easily
  </Card>

  <Card title="xAI" href="/integrations/llms/x-ai">
    Call Grok models through Portkey!
  </Card>

  <Card title="Ollama Tools" href="/integrations/llms/ollama">
    Tool calls are now supported on Ollama!
  </Card>

  <Card title="Vertex AI Controlled Generations" href="/integrations/llms/vertex-ai">
    The Controlled Generations (read: `Structured Outputs`) feature on Vertex AI is now supported!
  </Card>
</CardGroup>

#### Libraries

<CardGroup cols={2}>
  <Card title="OpenAI Swarm" href="/integrations/agents/openai-swarm">
    Complete observability for Swarm agents
  </Card>

  <Card title="Supabase" href="/integrations/libraries/supabase">
    Add LLM features to your Supabase apps
  </Card>

  <Card title="Semantic Kernel" href="/api-reference/inference-api/sdks/c-sharp#microsoft-semantic-kernel-example">
    Use Portkey in your Microsoft Semantic Kernel apps to easily observe your requests and make them reliable
  </Card>
</CardGroup>

#### Guardrails

<CardGroup cols={2}>
  <Card title="Pangea" href="/product/guardrails/pangea">
    Enhanced security with PII detection and content moderation
  </Card>
</CardGroup>

## Resources

Essential reading for your AI infrastructure:

* [What is an LLM Gateway?](https://portkey.ai/blog/what-is-an-llm-gateway/): Complete introduction
* [O1 Models Analysis](https://portkey.ai/blog/openai-o1-model-card-analysis/): Understanding OpenAI's latest
* [LLM Gateway Guide](https://portkey.ai/blog/build-vs-buy-llm-gateways/): Making infrastructure choices
* [Chat platform Comparison](https://portkey.ai/blog/librechat-vs-openwebui/): LibreChat vs OpenWebUI
* [AI vs API Gateway](https://portkey.ai/blog/ai-gateway-vs-api-gateway/): Key differences
* [FinOps for GenAI](https://portkey.ai/blog/finops-to-optimize-genai-costs/): Optimization strategies

## Community

<CardGroup cols={2}>
  <Card icon="youtube" title="Our Scaling Story" href="https://www.youtube.com/watch?v=9VbjUBze9Y0">
    Building our billion-request architecture
  </Card>
</CardGroup>

#### Office Hour

One thing we keep hearing from the Portkey community: you want to learn how other teams are solving production challenges and get the most out of the platform. Not through docs or tutorials, but through real conversations with fellow practitioners.

That's why we've started a new series of **AI Engineering Hours** since last week to bring the Portkey community together to discuss exactly this!

<Card title="Link to join the next office hour" href="/changelog/office-hour" />

#### Practitioners' Dinner

We [hosted](https://www.linkedin.com/posts/vrushank-vyas_coming-back-from-the-first-portkey-practitioners-activity-7267647180188860416-a5Fs/?utm_source=share\&utm_medium=member_desktop) some of Singapore's leading Gen AI engineers & leaders for a roundtable conversation - one profound insight emerged: Companies serious about Gen AI have realized it's as much a platform engineering challenge as it is an AI challenge.

Curious what we mean? Read the [meetup note here](https://www.linkedin.com/posts/vrushank-vyas_coming-back-from-the-first-portkey-practitioners-activity-7267647180188860416-a5Fs/?utm_source=share\&utm_medium=member_desktop).

## Improvements

#### Providers

* Gemini: Enhanced message and media handling
* Bedrock: Improved message formatting
* Vertex AI: Added Zod validation

#### SDK

* Stream support for assistant threads
* Enhanced Pydantic compatibility
* Fixed semantic cache behavior
* Resolved Python Httpx proxy issues

***

## Support

<CardGroup cols={2}>
  <Card title="Need Help?" icon="bug" href="https://github.com/Portkey-AI/gateway/issues">
    Open an issue on GitHub
  </Card>

  <Card title="Join Us" icon="discord" href="https://portkey.wiki/community">
    Get support in our Discord
  </Card>
</CardGroup>

Special thanks to [harupy](https://github.com/harupy) and [Ignacio Gleser](https://www.linkedin.com/in/ignacio-gleser-3499b33a/) for their contributions!