Recovering an Azure Migration a Major Consultancy Couldn't Finish

At a glance: Took over an Azure migration after one of the industry’s largest consultancies burned through its entire budget delivering an unproven, undocumented containerisation and an integration-only environment. We built a fully secured non-production and production environment from nothing to get it there.

Problem

We joined the migration in May. The contractor before us was one of the largest, most recognised consultancies in the industry, engaged to containerise the application and build out the integration environment. They burned through their entire budget doing just that, without ever properly proving the containerised application out, and without leaving any documentation behind.

The application was PCI DSS eligible, so there was no scope to cut corners on security or compliance to hit a date. Whatever was rebuilt had to be built properly the first time.

What we inherited was infrastructure that reached integration only. The entire non-production and production environments still needed to be built out and secured from nothing.

The client team, lacking the experience to work with the complex and undocumented Terraform module and supermodule structure left behind, had fallen into managing resources by hand directly in the Azure portal. The result was a mix of resources that were and weren’t under Infrastructure as Code, with no consistent pattern for which was which. Role assignments were being made manually by a single person, with no tracking of what had been granted or why. None of the architecture was documented, and there was no monitoring or alerting in place anywhere.

Secrets management was another gap. Credentials were being hardcoded into Git and, in other cases, added to Key Vault manually by hand, with no consistent, auditable way to get a secret from source to the service that needed it.

Behind the technical gap sat real financial and reputational pressure. The client was still running and paying for infrastructure on a legacy IaaS provider whose contract had already expired, while continuing to carry the reliability risk of staying on it. The migration was also the CIO’s own initiative, so his credibility, not just the project’s, was riding on getting it delivered.

Tech Stack

Azure - AKS, Key Vault, Application Gateway, virtual networks, private endpoints, Azure Container Registry, PostgreSQL infrastructure (DBAs handled the data migration itself once infrastructure was stood up), Bastion, VMs, Event Hubs (log shipping and security/audit events to Splunk SIEM)
Entra ID - OIDC, federated identity, and workload identity for CI/CD and in-cluster workload authentication
Terraform - infrastructure as code
GitLab - CI/CD pipelines, with dedicated private runners deployed into every region and subscription
ArgoCD - GitOps deployment of Helm charts for the development teams
Grafana - monitoring and alerting, integrated with Jira Service Management
Grafana Alloy - Prometheus-based metrics collection feeding Grafana directly
FluentBit - log collection and routing to Event Hub across multiple log types

What We Did

Rebuilt and secured the non-production and production environments from scratch, extending what the contractor had only reached at integration. Non-production was completed first and used to prove out every pattern, so production followed quickly, largely a case of pointing the same code at a different subscription and region.
Brought the environment fully under Terraform, resolving the mixed clickops/IaC state and removing manual resource management as the default way of working.
Replaced manual, single-person role assignment with a proper Entra ID group-based RBAC model, fully defined in code and auditable.
Produced the architecture documentation that had never existed under the previous contractor, covering how the Terraform module and supermodule structure was organised and a full walkthrough of how to build an environment from nothing through to a deployed application, giving the client’s own team, and anyone new, a real map of how things hung together.
Closed the security gaps standing between the environment and PCI DSS compliance: default-deny network policies, encryption in transit and at rest, and the access control and audit logging needed to pass assessment.
Built monitoring and alerting from nothing: Grafana dashboards, Alloy for metrics, FluentBit for log routing, Jira Service Management integration for alerting, and a runbook for every alert so a team new to the platform knew not just what had fired but what to do about it.
Delivered dedicated private GitLab runners into every region and subscription, and locked down private endpoints across the estate, meeting PCI’s private-networking requirement.
Built an ephemeral secrets pipeline that populates Key Vault directly, removing hardcoded credentials from Git and manual secret entry entirely, with no secret ever stored in the pipeline itself.
Diagnosed the actual root cause of a transaction-loss bug the contractor had shipped without ever finding: a legacy application, never tested under Kubernetes with multiple replicas against a representative workload (ISO 8583 over TCP from card terminals, not HTTP), had a health check that flapped across nodes and dropped in-flight transactions. The client had opted for a lift-and-shift rather than a full rearchitecture, so we confirmed the correct long-term fix (a dedicated L4 proxy in front of the app) and, against the time available, shipped a pragmatic interim mitigation instead: pinning the app to a single node with a hot spare in a second availability zone ready to take over on failure.
Resolved a log-parsing problem that had taken the client’s team three weeks without success, not for lack of ability but for lack of the senior technical support they should have had from the outset, and which was blocking the ops team’s own testing of their dashboards and alerts. The legacy application produced no standard JSON logs, so FluentBit needed three separate parsers to get logs into Splunk correctly. Solved in an afternoon.

Outcome

Delivered a production-ready platform by early July, with the application fully migrated and live by mid July, inside the contingency window and roughly two months ahead of the trajectory the project was on before we joined, which had been tracking towards a September go-live.
Eliminated the client’s ongoing legacy IaaS cost of approximately €30,000 a month.
Migration cutover ran without incident: every QA dry run came back clean beforehand, and go-live itself completed without a single issue. We were present from the start of the cutover through an additional week of standby afterward, in case anything needed catching.
Handed the platform and full documentation back to the client as business-as-usual, with a retainer arrangement in place for a few months afterward to cover anything beyond what their own team could handle.
The application was the business’s first in the cloud, so there was no existing support model to fall back on. We helped define a target operating model with clear RACI ownership, giving the client a support structure of their own rather than leaving them dependent on us.
Delivered without incident, on a compressed timeline, on a PCI-eligible application. The CIO’s own credibility had been riding on this migration, and it landed clean.

The client had already spent a significant sum with a global consultancy before we were ever brought in, for a fraction of what was actually needed to reach production. Worth asking what this would have looked like if we’d been engaged from day one.

Client Quotes

“We were constantly on the back foot with this project. The contracting house wanted to operate in a black box, which left the team that should have finished it scrambling, pushing our completion dates further and further away while the business demanded results. We were genuinely concerned this was going to be a failure, but Eiscir’s no-problem attitude and ability to drop in incredibly senior resources turned the whole project around.” — Project Manager

“Honestly, we were totally lost and demoralised before Eiscir came on board. We were worried this would be another contractor who’d leave us with a platform to maintain that we knew nothing about, but it became clear early on they were there to help us. Not only do we now understand what we have, we can confidently go on call and deal with anything that comes up.” — Project Engineer

“Hats off to Eiscir for the rapid turnaround. This was our second attempt to get one of our core applications into the cloud, so the internal pressure to get it done was extreme. You get the feeling they’ve seen it all before and just get on with it, and most importantly, delivered well before we were expecting. If anything, it was a surprise we were ready to go so quickly after they got involved.” — CTO

Get in Touch

If you’re facing similar challenges, reach out to us at hello@eiscir.com.