Cloud Orchestration Upgrade to Transform Retail Chain Operations

Executive Summary

A major retail chain grappled with a critical challenge – an outdated infrastructure impacting revenue, satisfaction, and competitiveness. The urgency to upgrade foundational legacy systems went beyond operational efficiency, it posed a direct threat to financial success and innovation capabilities. The outdated systems hindered adaptability, innovation, and customer expectations, translating into inefficiencies that impacted the bottom line and jeopardized customer satisfaction.

The complexity of the upgrade, involving over 600 Directed Acyclic Graphs (DAGs) representing tasks and dependencies, underscored the integral role these workflows played in revenue-generating processes. The team at Factspan recognized the urgency and broader implications, strategically introducing a custom code comparison framework and integrating crucial services like Google Cloud Storage. The solution became a lifeline for sustained success, securing the store’s position, ensuring customer satisfaction, and paving the way for future innovation.

About the Client

The client company has a long history in the retail industry and is a well-known reputable brand in the US. The company operates at numerous locations across the United States and has an online store, providing customers with convenient shopping options.

The client is recognized for its diverse product selection, competitive prices, and frequent sales events. Additionally, the company often collaborates with designers and brands to offer exclusive collections, further enhancing its appeal to shoppers.

Business Challenge

Confronted with a critical dilemma, the retail chain grappled with outdated Python and Airflow systems, exacerbating operational inefficiencies. The complexity of managing 600 Directed Acyclic Graphs (DAGs) on obsolete versions emerged as a substantial hurdle, directly impeding operational efficiency. This wasn’t solely a technical concern, it manifested as a formidable obstacle to the company’s financial stability.

The outdated systems posed a tangible risk, leading to significant revenue loss and jeopardizing the store’s competitive standing. The overarching challenge lay in navigating a landscape where inefficient workflows and antiquated systems converged, presenting a multifaceted threat to both revenue streams and customer satisfaction.

Our Solution

To overcome this challenge, the retail chain adopted a meticulous approach. They carefully identified the scope of DAGs and mapped out associated control-M jobs. Imagine navigating through a complex network of roads to find the optimal route. To smoothly transition from Python version 2 and Airflow version 1 to their advanced Version 2 counterparts, they deployed a sophisticated custom code comparison framework. This internal tool ensured that the existing Python and SQL code seamlessly adapted to the new Composer v2 environment.

Utilizing essential services and tools such as Google Cloud’s BigQuery (GCP BQ), Cloud Storage (GCS) buckets, Airflow, GitLab, and Control-M, the migration extended beyond mere code, encompassing vital files like SQL, JSON, and Shell scripts. This strategic use of tools, coupled with a bespoke framework, played a pivotal role in minimizing disruptions and streamlining the migration process. The result: the integrity of over 600 DAGs was maintained in the new and improved environment.

In essence, this wasn’t just an upgrade, it was a strategic move that aligned with the store’s commitment to excellence. The successful transition not only addressed immediate challenges but set the stage for a more efficient and innovative operational landscape.

Business Impact

• 98% upgrade success, minimal disruption
• 20% higher customer satisfaction, trust
• 30% improved operational efficiency, agility
• 15% cost savings, resource optimization