Enterprise AI Observability and Model Governance on Azure

Tag 1

Tag 2

Summary

A large financial services client with a multibillion dollar annual turnover faced repeated operational friction while keeping server fleets patched. Their application stacks are provisioned by CloudFormation and populated into autoscaling groups. AMI updates required repetitive manual edits to stack parameters, which created human errors and consumed developer time.

We delivered an automated, auditable, single-click workflow that updates AMI IDs across CloudFormation stacks and reports progress back into CI. The solution cut manual touch points to one action and made the process observable and reliable.

Client and Context

Our client is a leading global asset management firm known for its innovative approach to alternative investments and retirement solutions. Guided by values like pushing boundaries, creating opportunities and leading with integrity, they support both institutional and individual investors across credit, equity, real assets and retirement strategies.

As part of their digital transformation journey, the firm built an AI-powered Application on AWS to enhance customer engagement, showcasing their commitment to responsible, scalable technology and modern client experiences.

The Core Problem

Updating AMIs at scale looked simple in theory but in practice created several failure modes and inefficiencies.

Many CloudFormation stacks across accounts and environments. Each stack contained an AMI parameter. This required tracking and updating multiple independent parameters to move an environment to a new AMI.
Manual updates were error prone. Engineers sometimes updated the wrong stack, forgot to trigger the stack update after changing an SSM parameter, or assumed a stack was already current when it was not.
Detection was hard. Some AMIs were expired or untagged, making it difficult to tell whether a stack was referencing a new image.
Pace of change made the task repetitive. With patched AMIs released on a weekly cadence, the work consumed developer time that would be better spent on higher value items.
Timeouts and responsiveness. The initial approach to automation that queried all stacks synchronously ran into API Gateway timeouts when attempting to scan large accounts.

The client needed a reliable, auditable workflow that removed most manual steps while preserving control and notification.

Solution overview

We designed a lightweight control plane that integrates with the client CI system and the existing CloudFormation workflow. Key goals were minimal user interaction, clear status reporting and robust execution under account scale.

Components we delivered

Components we delivered