30 June, 2023

Implementing The Perxeive Servers In The Cloud As A Secure REST API Via AWS VPC And Fargate

The recently released version 2.0 of the Perxeive app in the Apple App Store required a complete rewrite of the backend servers to support the quantum leap in functionality. Having previously relied on AWS Lambdas sitting behind AWS Internet Gateway to deploy the rest API I needed a solution that overcame the latency problems that arise from the time for AWS Lambdas to initialise.

After researching a number of solutions I decided to go with AWS Fargate in a VPC to deliver both low latency and security. The REST API was delivered via an AWS Internet Gateway secured via AWS Cognito Auth.

The benefits of this approach included:

AWS Fargate enabled a continued serverless approach
AWS Fargate in a VPC provided low latency REST API calls
AWS VPC with an AWS Cognito auth secured AWS Internet Gateway delivered high levels of security to the REST API
AWS NatGateway enabled secure access from the private subnet of the VPC to the AWS DynamoDB NoSQL database
The AWS Network Load Balancer provided as much network scalability as would ever be needed
The AWS Application Load Balancer enabled highly scalable compute resources
The AWS Fargate compute resources were configurable by task
AWS CloudFormation template enabled the stack to be created efficiently

The image at the top of the page is the AWS CloudFormation designer representation of the full set of AWS components that are defined in the CloudFormation template for the stack. To put this into perspective, the template runs to 1841 lines of YAML. The learning curve to define the stack in the CloudFormation template was extremely steep. Whilst the AWS documentation is good, it is impossible to provide examples for all potential use cases and most of the use cases I required were not covered in the documents. This required a lot of trial and error, combined with combing through Google search results, GitHub repos and Stack Overflow solutions.

For anyone interested in trying a similar approach, I have included a quick guide to the core set up below.

Conclusion

Setting up this back end infrastructure proved highly challenging and at times frustrating. However, the final outcome provides an elegant solution. The latency experienced by app users is now negligible and with no downtime at the time of writing.

About The Author

I build businesses, both as independent startups and as new initiatives within large global companies. If you are looking to build a business and require leadership then please contact me via the About section of this website.

Guide To Setting Up AWS VPC With AWS Fargate

Create VPC

Setting up the Virtual Private Cloud (VPC)

The first step in implementing AWS Fargate serverless is setting up a new VPC. A VPC is essentially your own private network within the Amazon Web Services (AWS) cloud, allowing you to securely launch AWS resources into your own isolated network.

Creating a VPC involves defining public and private subnets for your Fargate tasks. Public subnets are accessible from the internet while private subnets are not. By separating these networks, we can control the flow of data and enhance security measures. For instance, sensitive workloads can be placed in private subnets and accessed through secure channels like VPN connections or AWS Direct Connect.

Here's a step-by-step guide on how to set up your VPC:

Create a new VPC: In the Amazon VPC console, click on "Start VPC Wizard" and select the first option, "VPC with Public and Private Subnets". Input your desired IP CIDR block (e.g., 10.0.0.0/16) and give it a name.
Create subnets: You'll be prompted to create at least one public and one private subnet. For the public subnet, ensure that "Auto-assign Public IP" is checked so that instances launched into this subnet receive public IP addresses.

Now that you have your VPC and subnets set up, let's proceed with configuring necessary network components such as NAT gateways, internet gateways, and route tables.

Configuring Network Components

To enable communication between instances in the private subnet and the internet (for software updates or accessing AWS services), we need to establish a few more components:

NAT Gateways: These allow instances in the private subnet to connect to the internet or other AWS services, but prevent the internet from initiating a connection with those instances. To create a NAT gateway, allocate an Elastic IP address and attach it to your NAT instance within the public subnet.
Internet Gateways: These enable communication between instances in the public subnet and the internet. Simply create an internet gateway and attach it to your VPC.
Route Tables: Route tables determine where network traffic is directed. For example, traffic destined for the internet gets routed through the internet gateway. Configure your route tables accordingly:
- Public subnet: Set a default route via the internet gateway.
- Private subnet: Set a route via the NAT gateway for instances to access the internet.

Lastly, don't forget about security groups! They act as virtual firewalls, controlling inbound and outbound traffic to your instances based on IP addresses, port ranges, and protocols. Configure them appropriately to restrict unwanted network traffic and maintain a secure environment for your Fargate tasks.

By following these steps, you'll have successfully created a VPC with public and private subnets for AWS Fargate serverless tasks.

Configure Networking for ECS

Before diving into the world of Fargate serverless computing, it's crucial to set up the right networking configurations in your Amazon Elastic Container Service (ECS) environment. In this section, we will discuss how to configure the necessary components within a newly created VPC for ECS: clusters, security groups, and IAM roles.

Amazon ECS Cluster

Firstly, you'll want to create an Amazon Elastic Container Service (ECS) cluster, which serves as the foundation for managing and running tasks and services within your VPC. This cluster essentially acts as a logical grouping of resources, allowing you to launch and maintain your containerized applications at scale. To create a new ECS cluster:

Navigate to the Amazon ECS console in your AWS Management Console.
Select 'Clusters' from the left navigation pane.
Click the 'Create Cluster' button.
Follow the on-screen instructions, selecting the desired network configuration options for your use case.

Once created, you'll be able to manage tasks and services within this cluster, providing a solid foundation for your serverless Fargate environment.

Security Groups and IAM Roles

To ensure secure communication between the ECS cluster, load balancers, worker nodes, and other allowed sources, you'll need to attach a security group with appropriate inbound traffic rules. This step is vital for maintaining your VPC's security posture and preventing unauthorized access or data breaches.

When creating the security group, consider allowing inbound traffic from necessary sources such as load balancers and worker nodes. Keeping the security group restrictive but still permitting required traffic will strike an optimal balance between functionality and security.

Additionally, you should create an IAM role for your ECS tasks with essential permissions such as Amazon Elastic Container Registry (ECR) access and logging capabilities. This IAM role allows your tasks to interact with other AWS services seamlessly, ensuring a smooth experience while adhering to the principle of least privilege. To create an IAM role:

Navigate to the Identity and Access Management (IAM) console in your AWS Management Console.
Click 'Roles' from the left navigation pane.
Select 'Create Role.'
Choose 'ECS Task Role' as the trusted entity type and follow the on-screen instructions, granting necessary permissions.

By following these guidelines for configuring networking within your ECS environment, you will lay a solid foundation to harness the power of AWS Fargate serverless computing with peace of mind knowing that your VPC is properly secured and ready for action.

Create Task Definition

The cornerstone of every successful AWS Fargate deployment is a well-crafted task definition. This essential blueprint outlines all the necessary components, from container images and resource requirements to networking configurations and IAM roles, that will dictate how your serverless application behaves within the VPC.

To create a task definition in Fargate, you'll need to specify container definitions—the individual units that define your Docker images and associated resource requirements. Ensuring each container is optimally configured is critical to achieving seamless orchestration, as Fargate will utilize these settings to allocate the necessary compute resources when scaling your application.

Next, it's time to dive into networking configurations. By setting up the awsvpc network mode, you gain unparalleled control over your Fargate task's networking settings, such as security groups, subnets, and elastic network interfaces (ENIs). This fine-grained configuration ensures that your serverless application can securely communicate with other resources within the VPC while maintaining network isolation when needed.

Finally, you must define an IAM task role to grant your Fargate task the necessary permissions. This crucial step guarantees that your application can interact with other AWS services, such as Amazon S3 or Amazon DynamoDB, without encountering access issues. By meticulously crafting each of these components, you lay a solid foundation for a secure and scalable serverless application in your VPC.

Create Service or Task

When it comes to deploying your applications in the world of serverless computing with AWS Fargate and Amazon ECS, you have two primary options to choose from based on your specific use case requirements. These choices include creating an Amazon ECS service, ideal for load balancing and scaling, or launching a standalone AWS Fargate task perfect for one-time execution of containerized workloads or ad-hoc scaling.

Amazon ECS Service

The first option available is creating an Amazon Elastic Container Service (ECS) service. This is the ideal solution if you require load balancing or scaling as it allows for efficient management of your containerized tasks. When deploying a service, simply specify your desired number of running tasks to ensure that the appropriate resources are continuously provisioned and available when required.

Additionally, integrating an ECS service with your existing application load balancer (ALB) will enable seamless distribution of incoming traffic amongst your designated tasks for optimal performance and availability. By taking advantage of these features offered through Amazon ECS services, you can effortlessly achieve a highly-available serverless infrastructure tailored to meet the demands of your dynamic workloads.

Accessing Tasks and Services

To ensure seamless access to your tasks and services running on Fargate within your Amazon Virtual Private Cloud (VPC), you need to configure specific ingress rules in your security groups and network settings. This process involves setting up the appropriate access policies, which will enable secure communication between your resources.

Firstly, it's essential to establish a solid understanding of your VPC's security groups, as they act as virtual firewalls for your network traffic. These groups regulate both inbound and outbound traffic based on predefined rules that you create. When implementing Fargate in your VPC, these rules determine which resources can access your tasks or services and under what conditions.

There are two primary methods for accessing tasks and services within a VPC:

Configuring ingress rules in security groups: This involves creating rules that permit traffic to enter the specific security group associated with your Fargate tasks or services. It's crucial to define these rules carefully, ensuring they only allow connections from trusted sources and for required protocols and ports.
Establishing private connections: In cases where a higher level of security is necessary, you can leverage AWS Direct Connect or Client VPN to create secure, private connections between your on-premises network and your VPC. This approach eliminates the need for traffic traversal over public networks, thereby reducing potential threats and enhancing data privacy.