Description

Design and build enterprise-grade data solutions across the full data lifecycle.

This role is remote.

Responsibilities

• Design, build, and optimize ETL/ELT pipelines for ingestion, transformation, and publishing using Azure Data Factory, Spark, and Databricks.
• Implement end-to-end data architecture for batch and real-time processing using Azure and Microsoft Fabric.
• Develop scalable data processing frameworks using Databricks and Apache Spark.
• Define and enforce data engineering best practices, including CI/CD, versioning, testing, and monitoring.

Required Skills

• 12+ years of experience in data architecture and engineering.
• Strong hands-on experience with Azure Data Factory (ADF) for data orchestration.
• Proficiency in PySpark, Python, and Spark SQL.
• Experience designing data lakes, data warehouses, or Lakehouse architectures.
• Expertise in SQL and relational data modeling.
• Experience with CI/CD pipelines for data platforms, including Azure DevOps and GitHub Actions.
• Familiarity with Azure Synapse and Azure SQL.
• Solid understanding of distributed data processing and performance tuning.

Key Skills

Education

Bachelor's degree