All posts

Scaling FastAPI on AWS with Infrastructure as Code

I’m a big fan of FastAPI. In my experience, it is the easiest way to develop an API with Python. That‘s the reason why I picked it when I…

I’m a big fan of FastAPI. In my experience, it is the easiest way to develop an API with Python. That‘s the reason why I picked it when I needed to set up a simple API for a project I was doing, where I needed to serve some pre-calculated data. But I ran into an issue: I had no idea about the traffic volume I could expect. This made me wonder. How can you deploy FastAPI to AWS in a scalable way? This post will walk through the solution that I opted for and why, using Amazon Elastic Container Service and the Infrastructure as Code (IaC) that I used to make it happen.

image

The App

To deploy an app, we first have to make one.

Just a quick heads-up, if you already have a FastAPI application that you want to deploy — which is likely — you can skip this part and continue to the next section. This section is purely meant to create a template app to build off from.

Lately, I’ve been really into uv for Python project management (you can read all about it here). We start by initializing our project by running:

uv init fastapi-app-iac

Open up the newly created project in your IDE. This being a FastAPI app, let’s include it in our dependencies:

uv add "fastapi[standard]"

My personal preference is to separate the application code from other code like documentation, or infrastructure. To do this, we’ll create a new file at app/main.py that includes the basic FastAPI app code as mentioned in the docs.

from typing import Union

from fastapi import FastAPI

app = FastAPI()

@app.get("/")
def read_root():
    return {"Hello": "World"}

@app.get("/items/{item_id}")
def read_item(item_id: int, q: Union[str, None] = None):
    return {"item_id": item_id, "q": q}

We can then start the app by running

fastapi dev app/main.py

Hosting Strategy

Now that we have a working app; let’s deploy it on AWS. But, before we dive in, it is good to know that there are a lot of different ways to actually host and manage an API on AWS. From barebones on an EC2 instance, serverless with AWS Lambda, to fully managed using services like App Runner. Generally speaking, the more managed the deployment option, the more expensive it is, the less of a headache it will give you. Up to a certain point, of course.

A popular way of deploying simple APIs on AWS is to go the serverless route. Serverless is attractive because it scales to zero — you don’t pay for what you don’t use, and you do not need to worry about the underlaying server infrastructure and configuration. At first glance, this seems perfect. But then you get hit with the cold starts and the realization that you pay per execution. That is fine for low traffic applications, but becomes expensive quickly once your app scales.

On the other side of the spectrum you could opt to do everything yourself, and host your app on an EC2 instance. This could be the cheapest option, or even free, but also causes a lot of headaches. You now have to think about — and implement — horizontal and vertical scaling strategies to handle increased workloads. And how will you ensure application versions are synchronized? It sounds like a lot of work, because it is. In some cases, the savings of using EC2 can be completely diminished by the additional configuration and maintenance costs of the setup.

Instead of serverless or EC2, for this example we opt for a balanced approach: Elastic Container Services (ECS). ECS allows us to build, manage and deploy containerized applications. It allows us to define Docker containers as something called “Tasks,” to which we allocate resources like compute and memory, and scaling policies. Opposed to serverless, ECS cannot scale to zero, which means at least one Task will be active at all times. On the upside — this means no cold starts. Compared with EC2, there are more abstractions in place for application management, that comes with a slightly higher price per resource. In that sense, ECS sits between EC2 and serverless.

The deployment

AWS provides their own Cloud Development Kit (CDK) that supports Python. CDK is a library that offers a great interface to define, bundle and deploy your resources on AWS. If you are unfamiliar with CDK, I highly recommend you check out the documentation first.

There are a couple of steps we need to follow to deploy our app:

  1. Create a health endpoint

  2. Create a hosted zone in AWS, with certificate

  3. Installing the webserver

  4. Package the app with Docker

  5. Create the CDK Stack

  6. Deploy the app

1. Create a health endpoint

In order to deploy the app, AWS requires us to setup a health endpoint. This endpoint will be used by AWS to check if the API is live and working. Add the following route to your API:

@app.get("/health")
def read_root():
    return "success"

Note that it does not really matter what the route returns, as long as it carries a status code 200, and is publicly accessible.

2. Creating a hosted zone in AWS, with certificate

To host an API, we’ll need access to a domain. Domain management within AWS is located within the Route53 service, through so-called hosted zones. Create a hosted zone for your domain, and create a certificate as well using AWS Certificate Manager. If this is the first time working with Route53 and AWS Certificate Manager, I highly recommend you checkout this blog by Pankaj Makhijani. It goes into great detail on how you can set this up.

Save the hosted zone ID and the ID of the certificated. We need this to host our API in a minute.

3. Installing the webserver

To deploy our app, we first need to run FastAPI through a web server. If we read the official documentation about it:

“FastAPI uses a standard for building Python web frameworks and servers called ASGI. […] The main thing you need to run a FastAPI application in a remote server machine is an ASGI server program likeUvicorn*, this is the one that comes by default in the*fastapicommand.”

Make sure uvicorn is installed and added to your requirements.

4. Package the app with Docker

Now, we need to package the application with Docker so that we can deploy it to AWS. To do this, create the following Dockerfile in the app/ folder:

FROM python:3.10-alpine
RUN apk update
RUN apk add make automake gcc g++ subversion python3-dev

WORKDIR /src
COPY ./requirements.txt /src/requirements.txt

RUN pip install --no-cache-dir --upgrade -r /src/requirements.txt
COPY ./app /src/app
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "80"]

This Dockerfile extends the official Alpine Linux image that contains an installation of Python 3.10. If you need — or want — a different Python version, check the Dockerhub to see which versions are available. However, this post assumes you are building off of one of the -alpine images.

The Dockerfile then installs some required packages, install our requirements, copy our app to our working directory and start the app using the before mentioned uvicorn ASGI server program on port 80.

We’re now ready for the next step.

5. Creating the CDK stack

In order to host our API on ECS, we need a few resources:

  • Virtual Private Cloud (VPC) with security group and appropriate rules

  • An ECS service with a cluster, task definition, scaling policies, and an application load balancer

  • Resources required to link our application to our domain using SSL

The way I like to explain how to set up these resources and what happens, is to first show you the complete code, and then walk you through it step by step. The full stack code is below, and is located in .deployments/fastapi_stack.py to keep it separate from our app code:

import os
from constructs import Construct
from aws_cdk import (
  Duration,
  Stack,
  Tags,
  aws_certificatemanager as certificatemanager,
  aws_ec2 as ec2,
  aws_ecs as ecs,
  aws_ecs_patterns as ecs_patterns,
  aws_elasticloadbalancingv2 as elasticloadbalancingv2,
  aws_route53 as route53,
  aws_route53_targets as route53_targets,
)

STACK_NAME = "FastAPIStack"
ENVIRONMENT_NAME = os.environ.get("ENVIRONMENT")
AWS_ACCOUNT_ID = os.environ.get("AWS_ACCOUNT_ID")
AWS_REGION = os.environ.get("AWS_REGION")
URL = os.environ.get("URL")
HOSTED_ZONE_ID = os.environ.get("HOSTED_ZONE_ID")
CERTIFICATE_ID = os.environ.get("CERTIFICATE_ID")

class FastAPIStack(Stack):
    def __init__(self, scope: Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        tags = {
            "stack-name": STACK_NAME,
            "environment-name": ENVIRONMENT_NAME,
            "AppManagerCFNStackKey": STACK_NAME,
        }

        # Set global tags
        for tag, value in tags.items():
            Tags.of(scope).add(tag, value)

        # Create VPC
        self.vpc = ec2.Vpc(
          self,
          f"{STACK_NAME}-vpc",
          max_azs=2
        )

        sg = ec2.SecurityGroup(
            self,
            f"{STACK_NAME}-alb-sg",
            vpc=self.vpc,
            allow_all_outbound=True,
            security_group_name=f"{STACK_NAME}-alb-sg",
        )

        # This allows all ingress to the https port
        sg.add_ingress_rule(
            ec2.Peer.any_ipv4(),
            ec2.Port.tcp(443),
            "Allow access from the world",
        )

        # Create Fargate Cluster
        self.ecs_cluster = ecs.Cluster(
            self,
            f"{STACK_NAME}-ecs-cluster",
            vpc=self.vpc,
            container_insights=True,
        )

        # Define Docker image for the Service
        image = ecs_patterns.ApplicationLoadBalancedTaskImageOptions(
            image=ecs.ContainerImage.from_asset(
                directory="../app",
            )
        )

        # Create Fargate Service and ALB
        self.ecs_service = ecs_patterns.ApplicationLoadBalancedFargateService(
            self,
            f"{STACK_NAME}-fargate",
            cluster=self.ecs_cluster,
            cpu=512,
            memory_limit_mib=2048,
            task_image_options=image,
            security_groups=[sg],
            open_listener=True,
            load_balancer_name=f"{STACK_NAME}-fargate-alb",
        )

        # Set scaling policies
        scale_target = self.ecs_service.service.auto_scale_task_count(
            min_capacity=1, max_capacity=80
        )
        scale_target.scale_on_request_count(
            "scale-out-requests-threshold",
            requests_per_target=4,
            target_group=self.ecs_service.target_group,
            scale_out_cooldown=Duration.seconds(10),
            scale_in_cooldown=Duration.seconds(10),
        )

        # Configure healthcheck
        self.ecs_service.target_group.configure_health_check(
            enabled=True,
            path="/health",
        )

        # ALB Fargate Service automatically creates a listener
        self.ecs_service.listener.add_action(
            f"{id}-http-Listener-action",
            action=elasticloadbalancingv2.ListenerAction.redirect(
                port="443", protocol="HTTPS"
            ),
        )
        
        # Define certificate from ARN. Note: ARN is AWS environment dependent
        certificate_arn = f"arn:aws:acm:{AWS_REGION}:{AWS_ACCOUNT_ID}:certificate/{CERTIFICATE_ID}"
        certificate = certificatemanager.Certificate.from_certificate_arn(
            self,
            f"{STACK_NAME}-domain-cert",
            certificate_arn
        )

        # Add HTTPS listener to ALB
        self.ecs_service.load_balancer.add_listener(
            f"{STACK_NAME}-https-listener",
            default_target_groups=[self.ecs_service.target_group],
            open=True,
            port=443,
            ssl_policy=elasticloadbalancingv2.SslPolicy.RECOMMENDED,
            certificates=[certificate],
        )

        # Hosted zone
        hosted_zone = route53.HostedZone.from_hosted_zone_attributes(
            self,
            f"{STACK_NAME}-hosted-zone",
            hosted_zone_id=HOSTED_ZONE,
            zone_name=URL,
        )

        # Set alias record to the ALB
        self.arecord = route53.ARecord(
            self,
            f"{STACK_NAME}-alias-record",
            target=route53.RecordTarget.from_alias(
                route53_targets.LoadBalancerTarget(
                    self.ecs_service.load_balancer
                )
            ),
            zone=hosted_zone,
        )

This is what’s happening in the code above

**Environment Variables are set:**After our imports, we see we need a few environment variables to make this work

  • Stack name: Hard coded set to FastAPIStack, can be changed to your liking

  • Environment name: Retrieved from your environment variables, set to production for your production application

  • AWS Account ID: Can be retrieved from the console

  • AWS Region: The AWS region you want to deploy your app in, e.g. eu-west-1

  • URL of the API: e.g. api.example.com

  • Hosted Zone ID for the URL: The ID from step 2

  • Certificate ID: The other ID from step 2

Global Tags are added: The stack defines three global tags, that are assigned to every resource within the stack, automatically. 1) The name of the stack, 2) the environment of the stack, and 3) the AppManagerCFNStackKey to make sure the application is recognized in AWS Application Manager. Feel free to extend these tags to your wishes.

**VPC is created:**Now the Virtual Private Cloud, and the security group resources are created, attaching the relevant security groups.

**App is created:**The app consists of the task defined from the Dockerfile we created earlier, the cluster itself and the ApplicationLoadBalancedFargateService. This is an L3-construct that defines a Fargate service running on an ECS cluster fronted by an application load balancer through a single interface. See these docs to read everything about it. We also define our scaling policies here.

**App is connected to our URL:**We import the certificate into the stack, add a listener that redirects traffic to the secure 443 HTTPS port, and connect our app to our URL with an A-Record.

To adopt this stack, we need two more files:

.deployments/cdk.json:

{
  "app": "python3 cdk.py"
}

and .deployments/cdk.py

from aws_cdk import App
from fastapi_stack import FastAPIStack

app = App()
FastAPIStack(app, "FastAPIStack")
app.synth()

These files will point CDK to the Stack we just created.

It is good to know that this is a minimal setup. Feel free to explore all the options that these resources provide, and configure it to your liking.

6. Deploying the app

CDK needs to be configured before we can actually deploy the application. If you haven’t used CDK before, use the official documentation to do so before continuing.

Deploying the app can be done by running a few commands. This can be from your local machine, or in your CI/CD pipeline provider such as GitHub Actions, Bitbucket- or GitLab Pipelines, by running:

pip install -r app/requirements.txt
npm install -g aws-cdk
cd .deployments && cdk deploy  -o ../../cdk.out --require-approval never

An important note: make sure the required environment variables are available here.

These commands will first synthesize the stack, transforming it into CloudFormation templates, and then deploy the resources one by one. You can check the status of the deployment in the command line, or in the CloudFormation service.

Did you like this post? Be sure to follow me for more! Claps are also highly appreciated, as they let me know what you are interested in. Thank you for making it all the way to the end!