We delivered an automated, auditable, single-click workflow that updates AMI IDs across CloudFormation stacks and reports progress back into CI. The solution cut manual touch points to one action and made the process observable and reliable.
Global Asset Management, Alternative Investments & Retirement Solutions.
Multibillion‑dollar financial services firm serving institutional and individual investors across credit, equity, real assets, and retirement strategies.
Amazon Web Service (AWS)
Repetitive, manual AMI updates across many CloudFormation stacks and accounts leading to slow patch adoption, configuration drift, API Gateway timeouts, and increased operational risk.
AWS DevOps & Cloud Automation Consulting to design and implement a single‑click, auditable AMI patching control plane integrated with Jenkins, API Gateway, Lambda, DynamoDB, and CloudFormation.
The client struggled to keep large server fleets consistently patched due to manual, error‑prone AMI updates across many CloudFormation stacks, accounts, and environments. AMI changes required repeated stack parameter edits, manual triggers, and ad‑hoc verification, which slowed patch adoption and increased operational risk.
TechAnek designed and delivered an automated AMI update control plane integrated with the client’s Jenkins CI, providing a single‑click workflow that discovers impacted stacks, orchestrates updates asynchronously, tracks job state in DynamoDB, and exposes real‑time status via a REST API. This solution significantly improved deployment reliability and reduced manual effort while preserving centralized control, auditability, and clear notifications.
The client faced four critical operational and security challenges that were inhibiting growth and increasing risk:
AMI parameters were spread across dozens of CloudFormation stacks and autoscaling groups in multiple accounts, making it difficult to consistently move entire environments to the latest patched images.
Engineers had to manually edit CloudFormation or SSM parameters and then remember to trigger stack updates, a process highly susceptible to human error. Wrong stacks were updated, some updates were never executed, and teams frequently assumed environments were current when they were not.
Expired or poorly tagged AMIs made it difficult to identify which workloads were still running on unpatched or unsupported images. This lack of clear visibility hindered risk assessment, slowed security response, and complicated compliance reporting for patch management.
Early attempts to automate discovery by scanning all stacks synchronously through API Gateway regularly hit 29‑second timeouts on large accounts. As AMI releases followed a weekly cadence, this unreliable process consumed valuable engineering time and prevented patching from keeping pace with business and security expectations.
Techanek designed a three-phase automated infrastructure deployment architecture built on native AWS services
A Jenkins Groovy pipeline acts as the primary interface for engineers, asking only for the target AMI ID and environment tag.
Once triggered, the pipeline sends a JSON payload to an API Gateway endpoint and then continuously polls for status, turning complex orchestration into a simple, repeatable, one‑click action.
AWS API Gateway exposes a hardened REST endpoint that validates requests and invokes the first Lambda function to begin discovery.
This layer enforces authentication, rate limits, and the 29‑second timeout boundary, while delegating long‑running work to asynchronous back‑end processing.
The first Lambda function scans autoscaling groups and CloudFormation stacks to identify those matching the supplied AMI name and tags.
It writes a job record with a unique job ID into DynamoDB, then selectively updates stack parameters or SSM Parameter Store values, re‑invoking itself asynchronously for large accounts to avoid timeout.
The second Lambda function reads the DynamoDB job record by job ID and returns live status of the campaign.
It reports stacks as pending, done, failed, or skipped, and surfaces CloudFormation events and concise failure reasons so engineers can quickly act on issues.
The second Lambda function reads the DynamoDB job record by job ID and returns live status of the campaign.
It reports stacks as pending, done, failed, or skipped, and surfaces CloudFormation events and concise failure reasons so engineers can quickly act on issues.
The Jenkins pipeline orchestrates the entire flow: it triggers the discovery Lambda, polls the status Lambda, and remains running until the job completes or errors out. At completion, Jenkins sends a human‑readable email summarizing succeeded, failed, and unchanged stacks, and a helper job can pre‑populate the latest AMI by OS family, further reducing console‑level manual work.
Jenkins pipelines (Groovy) for single‑click triggers, status polling, and user reporting
Amazon API Gateway
AWS Lambda
Amazon DynamoDB
AWS CloudFormation
Auto Scaling Groups
Terraform modules to provision API Gateway, Lambda functions, DynamoDB table, and associated IAM roles
The automation transformation delivered measurable impact across security, efficiency, and scalability dimensions:
AMI patching was inconsistent and slow, with many stacks still running on older or poorly tagged images, making it difficult to prove that environments were fully patched and secure.
A standardized, automated AMI update workflow now moves entire environments to the latest images through a single CI‑driven trigger, improving patch adoption speed by 70%.
Engineers spent significant time manually editing CloudFormation and SSM parameters, triggering stack updates, and validating changes across accounts, often repeating the same steps every week.
The new control plane reduced manual touch points to a single Jenkins action, cutting operational effort by 85% and making AMI updates a background, reliable process.
AMI rollouts were fragile, with some stacks updated and others left behind due to manual coordination, leading to partial deployments, configuration drift, and unpredictable behavior across environments.
The asynchronous, Lambda‑based orchestration now delivers 98% deployment reliability, with updates safely coordinated across dev, staging, and production without API Gateway timeouts.
Lack of consolidated status and error reporting made it hard to quickly identify failed AMI updates or understand where environments were out of sync.
Job‑level traceability, CloudFormation event capture, and automated email reports now provide a human‑readable audit trail for every AMI update run.
Single‑click initiation from Jenkins through to automated CloudFormation/SSM updates and reporting
Robust enumeration of stacks and autoscaling groups across accounts, with safe handling of large inventories via asynchronous re‑invocation
Persistent job records in DynamoDB with clear status transitions and metadata for each update campaign
On‑demand status queries that show pending, completed, failed, and skipped stacks together with succinct failure reasons and related CloudFormation events
Tight loop between Jenkins and the AWS control plane, including automated email notifications and optional helper jobs to fetch latest AMIs by OS family
Implementing end-to-end tracing with OpenTelemetry transformed how our client understood and managed their Azure AI Application. By capturing every step from prompt to response they gained clear visibility into the system, making it easier to fix issues, improve performance, and deliver more accurate answers. With trace data now flowing into Azure Application Insights, the team can quickly respond to feedback, ensure reliability and keep improving turning observability into a key driver of trust and continuous improvement.
