Automating Infrastructure Deployment Using AWS CDK and CloudFormation
Overview
Automating infrastructure deployment is crucial for maintaining scalability, security, and efficiency in today’s dynamic cloud environments. Our client previously relied on manual processes to create infrastructure and deploy applications in AWS using IAM user credentials. Each time a new AWS account was created for their clients, they had to update and pass parameters manually, making the process tedious and error-prone.
Client Background
Our client is a technology-driven company that builds and deploys serverless applications using JavaScript frameworks like TypeScript, Node.js, Nx utility, and Python scripts while managing cloud infrastructure for its customers. They leverage AWS CloudFormation and CDK to define infrastructure as code (IaC) but faced operational bottlenecks due to manual parameter updates and IAM user-based deployments.
Challenges
- The client was using IAM user credentials to create and deploy infrastructure, which posed security risks.
- Each time a new AWS account was created, they had to manually update and pass required parameters during infrastructure deployment.
- There was no automation in place to track infrastructure changes across development, testing, and production environments.
- The manual deployment approach led to inconsistencies and increased operational overhead.
Our Approach
Step 1: Automating Deployment Pipelines
- We created a CI/CD pipeline using AWS CodePipeline to detect infrastructure changes from Bitbucket repository branches (dev, test, and master).
- Whenever a new change was made to a branch, the pipeline automatically cloned the repository, triggered CloudFormation updates using AWS CDK, and deployed the StackSet in the management account, ensuring the stacks were provisioned in the target AWS accounts.
Step 2: Scheduled Infrastructure Deployments
- A connection was established using the AWS CodeStar plugin in the main AWS account to monitor changes in the specified Bitbucket repository branches.
- When changes were detected, a schedule was created in Amazon Eventbridge to trigger another pipeline that was deployed using AWS CDK.
- This secondary pipeline then deployed infrastructure changes using stackset to the relevant AWS accounts under the client’s organization.
Step 3: Implementation of AWS CDK and Self-Managed StackSet
- We designed a self-managed StackSet that enabled centralized infrastructure management across multiple AWS accounts with enhanced capabilities to modify workflows.
- Parameters required for new AWS accounts were dynamically managed in AWS SSM Parameter Store, eliminating the need for manual intervention.
- Security was improved by eliminating IAM user credentials and leveraging AWS Identity Center (SSO) and IAM roles for deployments.
Outcome
- Fully Automated Infrastructure Deployment: The client no longer had to manually deploy infrastructure updates.
- Enhanced Security: Removed reliance on IAM user credentials, reducing security risks.
- Scalability: New AWS accounts were automatically provisioned with the necessary infrastructure, requiring minimal manual intervention.
- Operational Efficiency: The automated pipeline significantly reduced deployment time and errors, improving overall productivity.
- Consistency: Standardized infrastructure across all AWS accounts under the client’s organization.