Description

• Design and build production-grade data pipelines spanning source ingestion, bronze landing, silver transformation, and gold-layer data product delivery within the squad's domain scope.
• Own pipeline KTLO (Keep the Lights On) for assigned data products, including monitoring, alerting, incident response, and ongoing reliability improvements.
• Implement data ingestion patterns for assigned source systems including batch file ingestion, API-based ingestion, and event-driven streaming (Kafka, Azure Event Hub) depending on squad scope.
• Apply medallion architecture (bronze, silver, gold) and Fabric IQ certification standards consistently across all data product builds.
• Collaborate with System Analysts to implement field-level transformations, business rule logic, and data quality checks as specified in product requirements documentation.
• Participate in and contribute to pipeline design reviews, ensuring solutions align with the organization's Databricks and Fabric engineering standards.
• Support the migration and deprecation of legacy platforms including SCOpsBI SQL Server and SAP boundary systems, following the organization's extract, validate, rebuild, cutover, and decommission pattern.
• Write and maintain comprehensive pipeline documentation including data lineage, transformation logic, SLA definitions, and dependency maps.
• Contribute to the organization's DevOps and engineering reliability practices including CI/CD pipeline setup, testing frameworks, and incident runbooks.
• Mentor and technically guide junior and mid-level data engineers within the squad.

REQUIRED QUALIFICATIONS

• 7-10 years of data engineering experience with a strong track record delivering production data pipelines in large enterprise environments.
• Expert proficiency in PySpark or Spark SQL for large-scale data transformation on distributed compute platforms.
• Hands-on experience with Databricks including Delta Lake, Unity Catalog, and workflow orchestration.
• Experience with Microsoft Fabric or Azure Synapse Analytics; familiarity with Fabric IQ and OneLake is a plus.
• Proficiency with pipeline orchestration tools such as Azure Data Factory, Databricks Workflows, or Apache Airflow.
• Solid SQL skills for data validation, transformation logic, and ad-hoc source system analysis.
• Experience building and maintaining ingestion pipelines from enterprise operational systems such as ERP, WMS, TMS, or comparable platforms.
• Strong understanding of data quality frameworks, including implementing checks, alerting on anomalies, and maintaining SLA-compliant pipeline health.
• Experience with DevOps practices for data pipelines including version control (Git), CI/CD, and automated testing.
• Ability to operate independently on complex technical problems with minimal oversight while maintaining clear technical documentation.

PREFERRED QUALIFICATIONS

• Experience with real-time streaming technologies including Kafka, Azure Event Hub, Delta Live Tables, or Spark Structured Streaming.
• Familiarity with Oracle, ERP ingestion patterns, or large-scale ERP migration programs. Background in supply chain, logistics, reverse logistics, or procurement data domains.
• Experience with legacy platform migration and decommission programs. Exposure to T-Mobile's Omni lakehouse platform or MagentaBuilt integrations.
• Python development skills beyond PySpark, including utility scripting and framework contributions

Key Skills

Education

Bachelor's degree