Downer T&I Transforms Data Landscape in 4 Months : A Case Study in Efficiency

4 months

Centralised data hub designed in
four months

ISO27001

ISO27001 compliance
achieved

About Downer

Downer is the leading provider of integrated services in Australia and New Zealand. Their Transport & Infrastructure (T&I) division delivers multi-disciplined solutions to customers across the transport sector. This includes capabilities such as road services, transport infrastructure, rail, light rail and buses, end-to-end transport solutions and asset management.

The Challenge

Downer T&I face challenges with disparate sources of truth, as multiple data solutions exist, leading to inconsistencies. The current infrastructure and technology do not have the ability to scale up, which makes efficient operations difficult. Additionally, the presence of various tools and technologies across Australia and New Zealand makes it hard to establish a unified data. The absence of a standard approach to data science also complicates efforts. Prolonged delivery times hinder team productivity, and the current tools are not future proof, which necessitated a comprehensive data transformation initiative.

Solution

Data Migration Approach

dataengine assessed and provided a data migration approach for two key operational projects.

Phase 1: Understanding the foundation by conducting a comprehensive assessment of current state to understand data formats, functionalities, and potential migration challenges.

Phase 2: Defined a detailed migration plan outlining and aligning to Azure and Databricks best practices, which included the following:

Clear activities: Defined data extraction, transformation, and loading steps.
Optimised resources: Strategic allocation of resources and tools.
Defined timelines: Assisted with a well-structured schedule with milestones to track progress.
Key technical risk, dependencies, and options
Provided target state recommendation and best practices for CICD, Databricks workspace, C# code migration to Azure.

Centralised Data Hub Design

dataengine designed a new T&I data hub that offers a centralised solution for managing the Transport and Infrastructure’s data in a data mesh strategy. Designed on secure and scalable Microsoft Azure cloud architecture, this hub will consolidate data from across the business unit and improve data accessibility. Data will be stored in a variety of formats, including raw, processed, and archived, with tools for easy discovery and management. Advanced data processing and transformation capabilities will enable in-depth analysis, while access controls and security measures safeguard sensitive information. The solution was compliant to internal architecture and security standards such as ISO27001.

This solution will not only improve data governance but also empower users to make data-driven decisions through tools like Power BI and Databricks notebooks. Additionally, a separate integration service ensures a secure testing environment for application pipelines. By centralising data storage, processing, and access, the T&I data hub paves the way for improved business performance and future data-driven initiatives.

The key features of T&I data domains are:

Operational analytics – Near real-time for quicker day-to-day decision-making.
Self-service capability – Create, revise, and enhance new insights based on the business needs and domain.
Data products – Shift from Excel to automated data pipelines, business Data Lake for data management and ability to create repeatable data products using modern tools.
Shared Operate Model – Shared operation and governance capability between IT and the business.

Operational Charter

The T&I Data Platform Charter serves as a critical foundation for the platform’s success. This document outlines the platform’s purpose, objectives, and scope, ensuring everyone involved is on the same page. More importantly, it clearly defines roles and responsibilities for key stakeholders such as Security, Group IT, and CloudOps. This promotes clear communication and avoids confusion during Downer’s ongoing transformation. The charter also extends to the operational aspects of the T&I data community, establishing clear lines of accountability for data governance and delivery of services by providing this roadmap, the charter fosters collaboration and minimises potential roadblocks, ensuring the T&I data platform functions smoothly and delivers ongoing value.

Outcome

T&I Leverages Data Engineering Expertise for Rapid Progress

By partnering with dataengine’s capabilities, T&I achieved significant progress within a complex operating model and delivered tangible results within Downer’s standards. Here’s a look at the accomplishments in just four months:

Defined Requirements: Established clear requirements for T&I’s data platforms, ensuring they meet specific needs.
Optimized Azure Tenancy: Aligned the existing Azure tenancy with best practices. This involved implementing Azure Management Groups and subscriptions for enhanced scalability and security.
Data Migration Strategy: Collaboratively defined a data migration approach to effectively move data from existing systems to the new platform.
Future-Proofed Foundation: Established a secure and scalable foundation for the data mesh strategy, ensuring the platform can adapt to future growth.
Standards-Aligned Architecture: Developed a High-Level Solution Architecture (HLSA) that adheres to Downer, Azure, and Databricks standards. This architecture received formal security compliance approval.
Collaborative Charter Definition: Facilitated five workshops to collaboratively define a charter outlining roles and responsibilities between IT and T&I teams.

These accomplishments demonstrate the effectiveness of dataengine’s expertise in accelerating T&I’s journey towards a streamlined and secure data platform.

Ready to Leverage Data?

Keen to explore your streamlined and secure data platform with us?