Integrating GPT Actions with an AWS AppSync Cognito-Protected API

TL;DR

Many AWS customers have developed AppSync GraphQL APIs to power their sites and mobile apps. Based on the recent GPT store announcement, some are now looking to integrate such APIs with GPT Actions, to provide a chat-based experience to consume their tried-and-tested functionality.

In this post, we’ll explore how to achieve this. We’ll focus on using Amazon Cognito, the de facto standard for securing AppSync APIs, and explore how GPT Actions’ compatibility with OAuth2-protected APIs aligns with Cognito’s capabilities.

These are the main pitfalls that I encountered along the way:

• GPT Actions require an OpenAPI specification, which isn’t natively compatible with AppSync’s GraphQL. Solution: Use a generic OpenAPI schema and provide the GPT instructions containing the GraphQL schema and even some example queries.
• If you are using Amplify, your login screen likely isn’t compatible with OAuth2. Solution: Use Cognito’s built-in “Hosted UI”.
• GPT Actions require that OAuth endpoints and the API are under the same root domain. Solution: Use custom domains for AppSync, Cognito, and API Gateway.
• Your AppSync implementation may require specific claims available only in ID tokens, whereas standard OAuth processes use access tokens. Solution: Proxy the token endpoint to replace the access token with the ID token.
• Your Cognito User Pool may not generate ID tokens by default. Solution: Configure the ‘openid’ scope in the GPT settings.
• We get an “Undefined” in the OAuth redirect URL after updating the action. Solution: Reconfigure the action settings and re-enter the app client ID/secret.
• Cognito throws a misconfiguration error after updating the action. Solution: Update the Cognito app client callback URL to match the new GPT OAuth redirect URL each time the Action is changed.
• GPT may generate invalid requests because the entire GQL query is a single JSON string. Solution: Create a REST API layer over the GraphQL API for a clearer OpenAPI spec and simplified API to fit your GPT’s usage.
• GPT may generate invalid requests if any of your REST API path parameters (e.g., IDs) contain special characters. Solution: Escape the special characters in the REST API layer before returning the data to the GPT and unescape it when sending it back to AppSync.
• You may want a highly customized login screen, but the Cognito Hosted UI only allows some basic logo & CSS adjustments. Solution: This one’s hard — we would need to create a custom web app, reimplement the whole OAuth authorization code flow, and use the proprietary Cognito User Pool APIs. I recommend checking this AWS blog: Should I use the hosted UI or create a custom UI in Amazon Cognito?

Background

Let’s start by understanding the key components involved:

1. GPT Actions: Part of the OpenAI ecosystem, GPT Actions allow for the customization of GPT models to interact intelligently with various APIs. They can be tailored for specific use cases, enabling a GPT model to access third-party applications and perform functions based on the user’s needs.
2. AWS AppSync: A managed service from AWS that uses GraphQL, AppSync allows for the development of scalable applications by securely accessing, manipulating, and combining data from multiple sources.
3. Amazon Cognito: This is a comprehensive identity management service that offers user sign-up, sign-in, and access control. It’s widely used for securing APIs, including those made with AWS AppSync, and is compliant with OAuth2 for authentication.

Let’s anchor this discussion in a practical use case: a Q&A site. Imagine we had this site running before the broad availability of LLMs, and now we want to leverage it to build one or more GPT Actions. The primary goal is to allow a GPT Action to interact with the site’s API to perform functions like:

1. Browsing Latest Questions: The GPT can assist users in navigating through the most recent questions on the Q&A site, sorted by date.
2. Answering and Committing Answers: It enables the GPT to not only answer questions but also to commit these answers back to the Q&A site, contributing to the growing knowledge base.
3. Moderating Content: The GPT can help in moderating the questions posted on the site. It can identify and delete content that is rude, obnoxious, offensive, or otherwise reprehensible.

Prerequisites: Cognito & AppSync domains

GPT Actions require that OAuth endpoints used for logging the users in and the APIs to call are under the same root domain. If that is not the case for your AppSync API and Cognito User Pool yet, that’s the first this we need to solve.

Let’s look at how to set up the Cognito domain. It must be on a root domain that we control, such that we can place the AppSync API under that same domain as well. This is how you’d do it with AWS CDK:

    const domain = userPool.addDomain("CognitoDomain", {
      customDomain: {
        domainName: `qanda-auth.${this.props.domainName}`,
        certificate: Certificate.fromCertificateArn(
          this,
          "CognitoCertificate",
          this.props.certificateArn
        ),
      },
    });

    const hostedZone = HostedZone.fromLookup(this, "CognitoHostedZone", {
      domainName: this.props.domainName,
    });
    new CnameRecord(this, "CognitoCnameRecord", {
      zone: hostedZone,
      recordName: "qanda-auth",
      domainName: domain.cloudFrontDomainName,
    });

• First, we added a “Cognito Custom Domain”, which creates a CloudFront distribution behind the scenes to serve the hosted UI.
• Then we added a DNS record to alias our custom domain name to the CloudFront distribution domain name.

Placing the AppSync API under a custom domain follows the same basic structure:

    const api = new GraphqlApi(this, "Api", {
      domainName: {
        domainName: `qanda-api.${this.props.domainName}`,
        certificate: Certificate.fromCertificateArn(
          this,
          "AppSyncCertificate",
          this.props.certificateArn
        ),
      },
      // ...
    });

    const hostedZone = HostedZone.fromLookup(this, "AppSyncHostedZone", {
      domainName: this.props.domainName,
    });
    new CnameRecord(this, "AppSyncCnameRecord", {
      zone: hostedZone,
      recordName: "qanda-api",
      domainName: Fn.select(2, Fn.split('/', api.graphqlUrl)),
    });

Version 1: Using AppSync with Access Tokens

In a scenario where the AppSync API does not require custom token claims, integrating GPT Actions with AWS AppSync can be more straightforward, because the access token passed by the GPT is sufficient.

The first step is to set up a Cognito App Client for the GPT integration to facilitate authentication and token management. CDK example:

    userPool.addClient("AppClient", {
      authFlows: {
        userPassword: true,
        userSrp: true,
      },
      generateSecret: true,
      oAuth: {
        flows: {
          implicitCodeGrant: true,
          authorizationCodeGrant: true,
        },
        scopes: [OAuthScope.OPENID],
        callbackUrls: this.props.callbackUrls,
      },
    });

Once we have created this, copy the Client ID and Secret.

AWS Cognito App Client Information

Then, we can create the GPT and the Action. Since GraphQL’s schema definition language differs and is not directly compatible with OpenAPI, we have to use a generic OpenAPI schema. To ensure that the GPT can then use the API properly, we must configure instructions containing the GraphQL schema and even some example queries.

Configuration:

• Authentication = OAuth
  — Client ID = <the ID of the Cognito App Client>
  — Client Secret = <the secret for the Cognito App Client>
  — Authorization URL = https://<Cognito Custom Domain>/oauth2/authorize (“authorize” endpoint)
  — Token URL = https://<Cognito Custom Domain>/oauth2/token (“token” endpoint)
  — Scope = openid
  — Token Exchange Method = Basic authorization header
• Schema = the generic OpenAPI schema for GraphQL APIs. I decided to not include the “variables” parameter — it seemed to cause the GPT to fail to use the API more often.

Once the action is created, the main GPT editor page will show you a “Callback URL” at the bottom. We need to configure this URL as the callback URL on the Cognito App Client that we created before.

GPT Editor Page with Callback URL

Cognito App Client Settings

Once this is done, we can test the GPT to ensure that it works. Example. Here’s what the user experience looks like:

• First, after the user’s first message, they are asked to login to our API:

• Then, they are redirected to the Cognito Hosted UI:

• After logging in, they are redirected back to the ChatGPT UI, and the API is called:

• ChatGPT will ask users for confirmation before making “write” operations (i.e., anything except HTTP GET calls; all GraphQL calls are POST). To streamline the user experience and reduce the frequency of confirmation prompts, you can specify a custom property in the OpenAPI specification.

Version 2: Using AppSync with ID Tokens

If our API relies on custom claims and needs to be called with Cognito ID tokens, we need to “trick” the GPT to send the ID token to the AppSync API instead of the access token.

First, we need to understand how the OAuth authorization code flow used by the GPT works. Initially, when a user attempts to access a service, they are redirected to an authentication server (Cognito). Here, they log in, and the server then issues a unique authorization code to the application. This code is a temporary token and is exchanged by the application for an access token and an ID token, using the client credentials. The application can then use these tokens to make API requests on behalf of the user.

Hence one way of “tricking” the GPT into using the ID token is to intercept the exchange API call and swap the “access token” value for the “id token” one. We can do this by building a small REST API. As before, this API must have the same root domain as the Cognito Hosted UI and the AppSync API.

    const api = new LambdaRestApi(this, "LambdaRestApi", {
      handler,
      domainName: {
        certificate,
        domainName: `qanda-gpt-api.${props.domainName}`,
      },
    });

    const hostedZone = HostedZone.fromLookup(this, "ApiHostedZone", {
      domainName: props.domainName,
    });
    new ARecord(this, "ApiARecord", {
      zone: hostedZone,
      recordName: "qanda-gpt-api",
      target: RecordTarget.fromAlias(new ApiGateway(api)),
    });

This REST API can be powered by a single Lambda function, which just forwards requests to Cognito and manipulates the response:

async function handleTokenExchange(
  event: APIGatewayProxyEvent
): Promise<APIGatewayProxyResult> {
  const response = await axios.post(
    `https://${process.env.COGNITO_DOMAIN}/oauth2/token`,
    event.body,
    {
      headers: {
        Authorization: event.headers["Authorization"],
        "Content-Type": event.headers["content-type"],
      },
    }
  );
  const data = response.data ?? {};
  return {
    statusCode: response.status,
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ ...data, access_token: data.id_token }),
  };
}

Lastly, we must update the “Token URL” setting in the GPT Action. At the time of writing this blog, ChatGPT has a bug that requires us to re-enter the Client ID/Secret as well, otherwise the Action ends up being broken. Moreover, once we change the Action, its Callback URL is regenerated, so it needs to be reconfigured on the Cognito App Client.

Once these changes are done, the GPT should work as normal, while sending ID Tokens to our AppSync API.

Version 3: Using a thin wrapper REST API

While experimenting with the AppSync-powered Action, I observed occasional errors in the GPT’s handling of the GraphQL query. On each retry, it would make an even bigger mess of it, as it didn’t understand the actual mistake. As the user, I would give it “hints” on how to use the API and recover, but that’s utterly unacceptable in most user-facing apps.

I’ve found that GPTs are less prone to such issues when using a REST API — most likely because a REST API can be neatly described using an OpenAPI spec, covering every operation explicitly, whereas for GraphQL APIs we have just one generic “execute query” operation, in which the GPT will need to shoehorn all the different access patterns.

Building a REST API on top of a GraphQL API is trivial. There are even libraries out there that can do this for us. For our demo purposes, we can just extend the REST API that we used for exchanging the tokens to also handle this use case.

For example, this is how we’d proxy calls to list the questions:

async function listQuestions(
  event: APIGatewayProxyEvent
): Promise<APIGatewayProxyResult> {
  const today = new Date().toISOString().split("T")[0];
  const authHeader =
    event.headers.Authorization || event.headers.authorization || "";
  const questions = await callAppSync<{
    data: { listQuestions: { items: any[] } };
  }>(
    `query ListQuestions($date: String!, $limit: Int!) {
        listQuestions(date: $date, limit: $limit) {
        items {
            id
            content
            createdAt
            answers {
                content
            }
        }
        }
    }`,
    authHeader,
    {
      date: today,
      limit: 5,
    }
  );
  return {
    statusCode: 200,
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify(questions.data.listQuestions.items),
  };
}

A significant advantage of this approach is its ability to streamline the API, tailoring it specifically to the use cases required by our GPT. We can add default parameter values, compose API calls, map errors, reshape data structures, etc. to make the API more GPT-friendly.

Final Architecture and Code

This is the final architecture:

System Block Diagram

… and the complete AWS CDK code for this demo implementation is here: https://github.com/serban-petrescu/aws-proto-cognito-appsync-gpt

• Problem: GPT Actions require OAuth endpoints and the API under the same root domain.
  Solution: Use custom domains for AppSync, Cognito, and API Gateway.
  Handling Cognito Groups with ID Tokens:
• Problem: GPT requires ID tokens, but OAuth usually provides access tokens.
  Solution: Proxy the token endpoint to replace the access token with the ID token.
  Generating ID Tokens:
• Problem: Cognito doesn’t generate ID tokens by default.
  Solution: Configure the ‘openid’ scope in the GPT settings.
  Issue with OAuth Redirect After Action Update:
• Problem: “Undefined” in URL after updating the action.
  Solution: Reconfigure action authentication settings and re-enter Client ID/Secret.
  Cognito Misconfiguration Post-Update:
• Problem: Cognito claims misconfiguration after login redirect post-action update.
  Solution: Update Cognito app client callback URL to match the new GPT OAuth redirect URL each time the Action is changed.

In Plain English 🚀

Thank you for being a part of the In Plain English community! Before you go:

• Be sure to clap and follow the writer ️👏️️
• Follow us: X | LinkedIn | YouTube | Discord | Newsletter
• Visit our other platforms: Stackademic | CoFeed | Venture
• More content at PlainEnglish.io