AWS Bedrock - Portkey Docs

Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including models hosted on AWS Bedrock. With Portkey, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through Model Catalog.

Portkey SDK Integration with AWS Bedrock

Portkey provides a consistent API to interact with models from various providers. To integrate Bedrock with Portkey:

1. Install the Portkey SDK

Add the Portkey SDK to your application to interact with Anthropic’s API through Portkey’s gateway.

NodeJS
Python

npm install --save portkey-ai

pip install portkey-ai

2. Initialize Portkey with the Bedrock Provider

There are two ways to integrate AWS Bedrock with Portkey:

AWS Access Key

Use your AWS Secret Access Key, AWS Access Key Id, and AWS Region to create your AI Provider on Portkey’s app.

Integration Guide

AWS Assumed Role

Take your AWS Assumed Role ARN and AWS Region to create your Bedrock provider.

Integration Guide

NodeJS SDK
Python SDK

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY", // defaults to process.env["PORTKEY_API_KEY"]
    provider:"@PROVIDER" // Your Bedrock Provider Slug
})

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
    provider="@PROVIDER"   # Replace with Your Bedrock Provider Slug
)

Using Bedrock Provider with AWS STS

If you’re using AWS Security Token Service, you can pass your aws_session_token along with the AI Provider slug:

NodeJS
Python

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY", // defaults to process.env["PORTKEY_API_KEY"]
    provider:"@PROVIDER" // Your Bedrock Provider Slug,
    aws_session_token: ""
})

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
    provider="@PROVIDER"   # Replace with your Provider Slug for Bedrock,
    aws_session_token=""
)

Not using Bedrock Provider from Model Catalog?

Check out this example on how you can directly use your AWS details to make a Bedrock request through Portkey.

3. Invoke Chat Completions with AWS bedrock

Use the Portkey instance to send requests to Anthropic. You can also override the provider slug directly in the API call if needed.

NodeJS SDK
Python SDK

const chatCompletion = await portkey.chat.completions.create({
    messages: [{ role: 'user', content: 'Say this is a test' }],
    model: 'us.anthropic.claude-3-7-sonnet-20250219-v1:0',
    max_tokens: 250 // Required field for Anthropic
});

console.log(chatCompletion.choices);

completion = portkey.chat.completions.create(
    messages= [{ "role": 'user', "content": 'Say this is a test' }],
    model= 'us.anthropic.claude-3-7-sonnet-20250219-v1:0',
    max_tokens=250 # Required field for Anthropic
)

print(completion.choices)

Using the /messages Route with Bedrock Models

Access Bedrock’s Claude models through Anthropic’s native /messages endpoint using Portkey’s SDK or Anthropic’s SDK.

This route only works with Claude models on Bedrock. For other models, use the standard OpenAI compliant endpoint.

Pass service_tier with Bedrock Claude models to use Bedrock service tiers. See Bedrock Service Tiers.

cURL
Python SDK
NodeJS SDK
Anthropic Python SDK
Anthropic TypeScript SDK

curl --location 'https://api.portkey.ai/v1/messages' \
--header 'x-portkey-provider: @your-bedrock-provider' \
--header 'Content-Type: application/json' \
--header 'x-portkey-api-key: YOUR_PORTKEY_API_KEY' \
--data '{
    "model": "us.anthropic.claude-3-7-sonnet-20250219-v1:0",
    "max_tokens": 250,
    "messages": [
        {
            "role": "user",
            "content": "Hello, Claude"
        }
    ]
}'

    Coming Soon!

    Coming Soon!

Anthropic Python SDK

  import anthropic

  client = anthropic.Anthropic(
      api_key="dummy", # we will use portkey's provider slug
      default_headers={"x-portkey-api-key": "YOUR_PORTKEY_API_KEY"},
      base_url="https://api.portkey.ai/v1"
  )
  message = client.messages.create(
      model="@your-provider-slug/your-model-name",
      max_tokens=250,
      messages=[
          {"role": "user", "content": "Hello, Claude"}
      ],
  )
  print(message.content)

Anthropic TS SDK

   import Anthropic from '@anthropic-ai/sdk';

   const anthropic = new Anthropic({
     apiKey: 'dummy', // we will use portkey's provider slug
     baseURL: "https://api.portkey.ai/v1",
     defaultHeaders: { "x-portkey-api-key": "YOUR_PORTKEY_API_KEY" }
   });

   const msg = await anthropic.messages.create({
     model: "@your-provider-slug/your-model-name",
     max_tokens: 1024,
     messages: [{ role: "user", content: "Hello, Claude" }],
   });
   console.log(msg);

Counting Tokens

Portkey supports the AWS Bedrock CountTokens API to estimate token usage before sending requests. Check out the count-tokens guide for more details.

Using Vision Models

Portkey’s multimodal Gateway fully supports Bedrock’s vision models anthropic.claude-3-sonnet, anthropic.claude-3-haiku, and anthropic.claude-3-opus For more info, check out this guide: Vision

Extended Thinking (Reasoning Models) (Beta)

The assistants thinking response is returned in the response_chunk.choices[0].delta.content_blocks array, not the response.choices[0].message.content string.

Models like us.anthropic.claude-3-7-sonnet-20250219-v1:0 support extended thinking. This is similar to openai thinking, but you get the model’s reasoning as it processes the request as well. Note that you will have to set strict_open_ai_compliance=False in the headers to use this feature.

Single turn conversation

from portkey_ai import Portkey

# Initialize the Portkey client
portkey = Portkey(
    api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
    provider="@PROVIDER",
    strict_openai_compliance=False
)

# Create the request
response = portkey.chat.completions.create(
  model="us.anthropic.claude-3-7-sonnet-20250219-v1:0",
  max_tokens=3000,
  thinking={
      "type": "enabled",
      "budget_tokens": 2030
  },
  stream=True,
  messages=[
      {
          "role": "user",
          "content": [
              {
                  "type": "text",
                  "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?"
              }
          ]
      }
  ]
)
print(response)
# in case of streaming responses you'd have to parse the response_chunk.choices[0].delta.content_blocks array
# response = portkey.chat.completions.create(
#   ...same config as above but with stream: true
# )
# for chunk in response:
#     if chunk.choices[0].delta:
#         content_blocks = chunk.choices[0].delta.get("content_blocks")
#         if content_blocks is not None:
#             for content_block in content_blocks:
#                 print(content_block)

import Portkey from 'portkey-ai';

// Initialize the Portkey client
const portkey = new Portkey({
  apiKey: "PORTKEY_API_KEY", // Replace with your Portkey API key
  provider:"@PROVIDER", // Add your bedrock's Provider slug from model catalog
  strictOpenAICompliance: false
});

// Generate a chat completion
async function getChatCompletionFunctions() {
    const response = await portkey.chat.completions.create({
      model: "us.anthropic.claude-3-7-sonnet-20250219-v1:0",
      max_tokens: 3000,
      thinking: {
          type: "enabled",
          budget_tokens: 2030
      },
      stream: true,
      messages: [
          {
              role: "user",
              content: [
                  {
                      type: "text",
                      text: "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?"
                  }
              ]
          }
      ]
    });
    console.log(response);
  // in case of streaming responses you'd have to parse the response_chunk.choices[0].delta.content_blocks array
  // const response = await portkey.chat.completions.create({
  //   ...same config as above but with stream: true
  // });
  // for await (const chunk of response) {
  //   if (chunk.choices[0].delta?.content_blocks) {
  //     for (const contentBlock of chunk.choices[0].delta.content_blocks) {
  //       console.log(contentBlock);
  //     }
  //   }
  // }
  }
// Call the function
getChatCompletionFunctions();

import OpenAI from 'openai'; // We're using the v4 SDK
import { PORTKEY_GATEWAY_URL, createHeaders } from 'portkey-ai'

const openai = new OpenAI({
  apiKey: 'dummy', // defaults to process.env["OPENAI_API_KEY"],
  baseURL: PORTKEY_GATEWAY_URL,
  defaultHeaders: createHeaders({
    provider: "@bedrock", // your portkey bedrock provider slug from model catalog
    apiKey: "PORTKEY_API_KEY", // defaults to process.env["PORTKEY_API_KEY"]
    strictOpenAICompliance: false
  })
});

// Generate a chat completion with streaming
async function getChatCompletionFunctions(){
  const response = await openai.chat.completions.create({
    model: "us.anthropic.claude-3-7-sonnet-20250219-v1:0",
    max_tokens: 3000,
    thinking: {
        type: "enabled",
        budget_tokens: 2030
    },
    stream: true,
    messages: [
        {
            role: "user",
            content: [
                {
                    type: "text",
                    text: "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?"
                }
            ]
        }
    ],
  });

  console.log(response)
  // in case of streaming responses you'd have to parse the response_chunk.choices[0].delta.content_blocks array
  // const response = await openai.chat.completions.create({
  //   ...same config as above but with stream: true
  // });
  // for await (const chunk of response) {
  //   if (chunk.choices[0].delta?.content_blocks) {
  //     for (const contentBlock of chunk.choices[0].delta.content_blocks) {
  //       console.log(contentBlock);
  //     }
  //   }
  // }
}
await getChatCompletionFunctions();

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

openai = OpenAI(
    api_key='dummt',
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=createHeaders(
        provider="@bedrock", # your bedrock provider slug from model catalog
        api_key="PORTKEY_API_KEY",
        strict_openai_compliance=False
    )
)


response = openai.chat.completions.create(
    model="us.anthropic.claude-3-7-sonnet-20250219-v1:0",
    max_tokens=3000,
    thinking={
        "type": "enabled",
        "budget_tokens": 2030
    },
    stream=True,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?"
                }
            ]
        }
    ]
)

print(response)

curl "https://api.portkey.ai/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: @your-bedrock-provider-slug" \
  -H "x-portkey-strict-openai-compliance: false" \
  -d '{
    "model": "us.anthropic.claude-3-7-sonnet-20250219-v1:0",
    "max_tokens": 3000,
    "thinking": {
      "type": "enabled",
      "budget_tokens": 2030
    },
    "stream": true,
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "when does the flight from new york to bengaluru land tomorrow, what time, what is its flight number, and what is its baggage belt?"
          }
        ]
      }
    ]
  }'

Multi turn conversation

from portkey_ai import Portkey

# Initialize the Portkey client
portkey = Portkey(
    api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
    provider="@PROVIDER",
    strict_openai_compliance=False
)

# Create the request
response = portkey.chat.completions.create(
  model="us.anthropic.claude-3-7-sonnet-20250219-v1:0",
  max_tokens=3000,
  thinking={
      "type": "enabled",
      "budget_tokens": 2030
  },
  stream=True,
  messages=[
      {
          "role": "user",
          "content": [
              {
                  "type": "text",
                  "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?"
              }
          ]
      },
      {
          "role": "assistant",
          "content": [
                  {
                      "type": "thinking",
                      "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.",
                      "signature": "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE="
                  }
          ]
      },
      {
          "role": "user",
          "content": "thanks that's good to know, how about to chennai?"
      }
  ]
)
print(response)

import Portkey from 'portkey-ai';

// Initialize the Portkey client
const portkey = new Portkey({
  apiKey: "PORTKEY_API_KEY", // Replace with your Portkey API key
  provider:"@PROVIDER", // Add your bedrock's Provider Slug from model catalog
  strictOpenAICompliance: false
});

// Generate a chat completion
async function getChatCompletionFunctions() {
    const response = await portkey.chat.completions.create({
      model: "us.anthropic.claude-3-7-sonnet-20250219-v1:0",
      max_tokens: 3000,
      thinking: {
          type: "enabled",
          budget_tokens: 2030
      },
      stream: true,
      messages: [
          {
              role: "user",
              content: [
                  {
                      type: "text",
                      text: "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?"
                  }
              ]
          },
          {
              role: "assistant",
              content: [
                      {
                          type: "thinking",
                          thinking: "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.",
                          signature: "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE="
                      }
              ]
          },
          {
              role: "user",
              content: "thanks that's good to know, how about to chennai?"
          }
      ]
    });
    console.log(response);
  }
// Call the function
getChatCompletionFunctions();

import OpenAI from 'openai'; // We're using the v4 SDK
import { PORTKEY_GATEWAY_URL, createHeaders } from 'portkey-ai'

const openai = new OpenAI({
  apiKey: 'dummy', // we are using provider from portkey
  baseURL: PORTKEY_GATEWAY_URL,
  defaultHeaders: createHeaders({
    provider: "@bedrock", // Replace with your bedrock provider slug from model catalog
    apiKey: "PORTKEY_API_KEY", // defaults to process.env["PORTKEY_API_KEY"]
    strictOpenAICompliance: false
  })
});

// Generate a chat completion with streaming
async function getChatCompletionFunctions(){
  const response = await openai.chat.completions.create({
    model: "us.anthropic.claude-3-7-sonnet-20250219-v1:0",
    max_tokens: 3000,
    thinking: {
        type: "enabled",
        budget_tokens: 2030
    },
    stream: true,
    messages: [
        {
            role: "user",
            content: [
                {
                    type: "text",
                    text: "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?"
                }
            ]
        },
        {
            role: "assistant",
            content: [
                    {
                        type: "thinking",
                        thinking: "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.",
                        signature: "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE="
                    }
            ]
        },
        {
            role: "user",
            content: "thanks that's good to know, how about to chennai?"
        }
    ],
  });

  console.log(response)

}
await getChatCompletionFunctions();

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

openai = OpenAI(
    api_key='dummy', # we are using provider from portkey
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=createHeaders(
        provider="@bedrock-slug", # Replace with your bedrock provider slug from model catalog
        api_key="PORTKEY_API_KEY",
        strict_openai_compliance=False
    )
)


response = openai.chat.completions.create(
    model="us.anthropic.claude-3-7-sonnet-20250219-v1:0",
    max_tokens=3000,
    thinking={
        "type": "enabled",
        "budget_tokens": 2030
    },
    stream=True,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?"
                }
            ]
        },
        {
            "role": "assistant",
            "content": [
                    {
                        "type": "thinking",
                        "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.",
                        signature: "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE="
                    }
            ]
        },
        {
            "role": "user",
            "content": "thanks that's good to know, how about to chennai?"
        }
    ]
)

print(response)

curl "https://api.portkey.ai/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: @your-bedrock-provider-slug" \
  -H "x-portkey-strict-openai-compliance: false" \
  -d '{
    "model": "us.anthropic.claude-3-7-sonnet-20250219-v1:0",
    "max_tokens": 3000,
    "thinking": {
      "type": "enabled",
      "budget_tokens": 2030
    },
    "stream": true,
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "when does the flight from baroda to bangalore land tomorrow, what time, what is its flight number, and what is its baggage belt?"
          }
        ]
      },
      {
        "role": "assistant",
        "content": [
                {
                    "type": "thinking",
                    "thinking": "The user is asking several questions about a flight from Baroda (also known as Vadodara) to Bangalore:\n1. When does the flight land tomorrow\n2. What time does it land\n3. What is the flight number\n4. What is the baggage belt number at the arrival airport\n\nTo properly answer these questions, I would need access to airline flight schedules and airport information systems. However, I don't have:\n- Real-time or scheduled flight information\n- Access to airport baggage claim allocation systems\n- Information about specific flights between these cities\n- The ability to look up tomorrow's specific flight schedules\n\nThis question requires current, specific flight information that I don't have access to. Instead of guessing or providing potentially incorrect information, I should explain this limitation and suggest ways the user could find this information.",
                    "signature": "EqoBCkgIARABGAIiQBVA7FBNLRtWarDSy9TAjwtOpcTSYHJ+2GYEoaorq3V+d3eapde04bvEfykD/66xZXjJ5yyqogJ8DEkNMotspRsSDKzuUJ9FKhSNt/3PdxoMaFZuH+1z1aLF8OeQIjCrA1+T2lsErrbgrve6eDWeMvP+1sqVqv/JcIn1jOmuzrPi2tNz5M0oqkOO9txJf7QqEPPw6RG3JLO2h7nV1BMN6wE="
                }
        ]
      },
      {
        "role": "user",
        "content": "thanks that's good to know, how about to chennai?"
      }
    ]
  }'

Inference Profiles

Inference profiles are a resource in Amazon Bedrock that define a model and one or more Regions to which the inference profile can route model invocation requests. To use inference profiles, your IAM role needs to additionally have the following permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "bedrock:GetInferenceProfile"
            ],
            "Resource": [
                "arn:aws:bedrock:*:*:inference-profile/*",
                "arn:aws:bedrock:*:*:application-inference-profile/*"
            ]
        }
    ]
}

This is a pre-requisite for using inference profiles, as the gateway needs to fetch the foundation model to process the request. For reference, see the following documentation: https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-prereq.html

Bedrock Guardrails

You can use Bedrock guardrails directly in your chat completions requests to add content filtering and safety measures. Guardrails help ensure that model responses adhere to your specific safety and content policies.

We recommend using guardrails through the Portkey UI for easier management and configuration. You can learn more about guardrails here.

Using Guardrails in Chat Completions

To enable guardrails, include the guardrailConfig parameter in your request:

NodeJS SDK
Python SDK
cURL

const chatCompletion = await portkey.chat.completions.create({
    messages: [{ role: 'user', content: 'Say this is a test' }],
    model: 'us.anthropic.claude-3-7-sonnet-20250219-v1:0',
    max_tokens: 250,
    guardrailConfig: {
        guardrailIdentifier: "your-guardrail-id",
        guardrailVersion: "DRAFT", // or specific version number
        trace: "enabled" // optional: "enabled" or "disabled"
    }
});

completion = portkey.chat.completions.create(
    messages=[{ "role": 'user', "content": 'Say this is a test' }],
    model='us.anthropic.claude-3-7-sonnet-20250219-v1:0',
    max_tokens=250,
    guardrail_config={  # Note: snake_case also supported
        "guardrailIdentifier": "your-guardrail-id",
        "guardrailVersion": "DRAFT",
        "trace": "enabled"
    }
)

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: @your-bedrock-provider" \
  -d '{
    "model": "us.anthropic.claude-3-7-sonnet-20250219-v1:0",
    "max_tokens": 250,
    "messages": [{"role": "user","content": "Say this is a test"}],
    "guardrailConfig": {
      "guardrailIdentifier": "your-guardrail-id",
      "guardrailVersion": "DRAFT",
      "trace": "enabled"
    }
  }'

Guardrail Configuration Parameters

Parameter	Type	Required	Description
`guardrailIdentifier`	string	Yes	The unique identifier of your Bedrock guardrail
`guardrailVersion`	string	Yes	Version of the guardrail (`"DRAFT"` for the latest draft version, or a specific version number)
`trace`	string	No	Controls trace generation (`"enabled"` or `"disabled"`)

Both guardrailConfig (camelCase) and guardrail_config (snake_case) parameter names are supported for compatibility.

When a guardrail is triggered, the response will include a guardrail_intervened stop reason. You can access detailed trace information if tracing is enabled.

Bedrock Converse API

Portkey uses the AWS Converse API internally for making chat completions requests. If you need to pass additional input fields or parameters like anthropic_beta, top_k, frequency_penalty etc. that are specific to a model, you can pass it with this key:

"additionalModelRequestFields": {
    "frequency_penalty": 0.4
}

If you require the model to respond with certain fields that are specific to a model, you need to pass this key:

"additionalModelResponseFieldPaths": [ "/stop_sequence" ]

Bedrock Service Tiers

Amazon Bedrock service tiers let you choose how Bedrock handles inference capacity for a request. Availability depends on your model, AWS Region, account access, and AWS quotas. Pass service_tier in the request body. Portkey forwards the required Bedrock service tier format for both Chat Completions and Claude /messages requests.

Supported Values

Value	Notes
`default`	Uses the default Bedrock service tier.
`priority`	Uses priority capacity when it is available for your model and account.
`flex`	Uses the Bedrock flex tier when the model supports it.
`reserved`	Uses reserved capacity when available.
`auto`, `scale`, `standard_only`	Compatibility values from OpenAI or Anthropic clients. Portkey maps these to Bedrock’s `default` tier.

AWS determines which tiers are available for each model, Region, and account. If a tier is not enabled for your AWS environment, Bedrock may reject the request.

Response Fields

Route	Field	Notes
Chat Completions	`service_tier`	Returns Bedrock’s tier value when Bedrock includes it in the response, such as `default`, `flex`, `priority`, or `reserved`.
`/messages`	`usage.service_tier`	Returns `standard` or `priority`, following Anthropic’s response shape. Portkey maps Bedrock `default` and `flex` to `standard`, and `priority` and `reserved` to `priority`.

The examples below use priority. Replace it with any tier supported for your model and AWS account.

const chatCompletion = await portkey.chat.completions.create({
    messages: [{ role: "user", content: "Say this is a test" }],
    model: "moonshot.kimi-k2-thinking",
    max_tokens: 250,
    service_tier: "priority"
})

completion = portkey.chat.completions.create(
    messages=[{ "role": "user", "content": "Say this is a test" }],
    model="moonshot.kimi-k2-thinking",
    max_tokens=250,
    service_tier="priority"
)

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: @your-bedrock-provider" \
  -d '{
    "model": "moonshot.kimi-k2-thinking",
    "max_tokens": 250,
    "messages": [{"role": "user", "content": "Say this is a test"}],
    "service_tier": "priority"
  }'

Managing AWS Bedrock Prompts

You can manage all prompts to AWS bedrock in the Prompt Library. All the current models of Anthropic are supported and you can easily start testing different prompts. Once you’re ready with your prompt, you can use the portkey.prompts.completions.create interface to use the prompt in your application.

Making Requests without using Portkey’s Model Catalog

If you do not want to add your AWS details to Portkey vault, you can also directly pass them while instantiating the Portkey client.

Mapping the Bedrock Details

Node SDK	Python SDK	REST Headers
awsAccessKeyId	aws_access_key_id	x-portkey-aws-access-key-id
awsSecretAccessKey	aws_secret_access_key	x-portkey-aws-secret-access-key
awsRegion	aws_region	x-portkey-aws-region
awsSessionToken	aws_session_token	x-portkey-aws-session-token

Example

NodeJS
Python
cURL

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    provider: "bedrock",
    awsAccessKeyId: "AWS_ACCESS_KEY_ID",
    awsSecretAccessKey: "AWS_SECRET_ACCESS_KEY",
    awsRegion: "us-east-1",
    awsSessionToken: "AWS_SESSION_TOKEN"
})

from portkey_ai import Portkey

client = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="bedrock",
    aws_access_key_id="",
    aws_secret_access_key="",
    aws_region="us-east-1",
    aws_session_token=""
)

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-provider: bedrock" \
  -H "x-portkey-aws-access-key-id: $AWS_ACCESS_KEY_ID" \
  -H "x-portkey-aws-secret-access-key: $AWS_SECRET_ACCESS_KEY" \
  -H "x-portkey-aws-region: $AWS_REGION" \
  -H "x-portkey-aws-session-token: $AWS_TOKEN" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user","content": "Hello!"}]
  }'

Using AWS PrivateLink for Bedrock [Self Hosted Enterprise]

Though using assumed role is in itself enough for enterprise security. You can additional configure AWS PrivateLink for Bedrock to ensure that your requests are not traversed outside your VPC.

Create a private link between the VPC you’ve deployed Portkey and AWS Bedrock (the endpoint is in most cases https://bedrock.{your_region}.amazonaws.com).
When configuring your integration on portkey, simply configure the custom host option to point to your VPC endpoint for the private link.

AWS GovCloud (US)

Integration is identical to standard Bedrock. Only the endpoint changes — set a Custom Host for your region.

Steps

In Portkey, create or edit your Bedrock provider.
Open “Advanced Options”.
Set “Custom Host” to your GovCloud Bedrock endpoint:
- https://bedrock.us-gov-east-1.amazonaws.com
- https://bedrock.us-gov-west-1.amazonaws.com
- and more similarly…
Save and use normally in SDKs and via the gateway.

Notes

For the complete list of Bedrock endpoints, see the official AWS reference: Amazon Bedrock endpoints and quotas.
For FIPS-compliant endpoints and additional compliance information, see: FIPS Endpoints by Service.

Supported Models

List of supported Amazon Bedrock model IDs

How to Find Your AWS Credentials

Navigate here in the AWS Management Console to obtain your AWS Access Key ID and AWS Secret Access Key.

In the console, you’ll find the ‘Access keys’ section. Click on ‘Create access key’.
Copy the Secret Access Key once it is generated, and you can view the Access Key ID along with it.

On the same page under the ‘Access keys’ section, where you created your Secret Access key, you will also find your Access Key ID.

And lastly, get Your AWS Region from the Home Page of AWS Bedrock as shown in the image below.

Next Steps

The complete list of features supported in the SDK are available on the link below.

SDK

You’ll find more information in the relevant sections:

​Portkey SDK Integration with AWS Bedrock

​1. Install the Portkey SDK

​2. Initialize Portkey with the Bedrock Provider

AWS Access Key

AWS Assumed Role

​Using Bedrock Provider with AWS STS

​Not using Bedrock Provider from Model Catalog?

​3. Invoke Chat Completions with AWS bedrock

​Using the /messages Route with Bedrock Models

Counting Tokens

​Using Vision Models

​Extended Thinking (Reasoning Models) (Beta)

​Single turn conversation

​Multi turn conversation

​Inference Profiles

​Bedrock Guardrails

​Using Guardrails in Chat Completions

​Guardrail Configuration Parameters

​Bedrock Converse API

​Bedrock Service Tiers

​Supported Values

​Response Fields

​Managing AWS Bedrock Prompts

​Making Requests without using Portkey’s Model Catalog

​Mapping the Bedrock Details

​Example

​Using AWS PrivateLink for Bedrock [Self Hosted Enterprise]

​AWS GovCloud (US)

​Steps

​Notes

​Supported Models

List of supported Amazon Bedrock model IDs

​How to Find Your AWS Credentials

​Next Steps

SDK

Portkey SDK Integration with AWS Bedrock

1. Install the Portkey SDK

2. Initialize Portkey with the Bedrock Provider

Using Bedrock Provider with AWS STS

Not using Bedrock Provider from Model Catalog?

3. Invoke Chat Completions with AWS bedrock

Using the /messages Route with Bedrock Models

Using Vision Models

Extended Thinking (Reasoning Models) (Beta)

Single turn conversation

Multi turn conversation

Inference Profiles

Bedrock Guardrails

Using Guardrails in Chat Completions

Guardrail Configuration Parameters

Bedrock Converse API

Bedrock Service Tiers

Supported Values

Response Fields

Managing AWS Bedrock Prompts

Making Requests without using Portkey’s Model Catalog

Mapping the Bedrock Details

Example

Using AWS PrivateLink for Bedrock [Self Hosted Enterprise]

AWS GovCloud (US)

Steps

Notes

Supported Models

How to Find Your AWS Credentials

Next Steps