Description

You will own the design and implementation of data infrastructure.

Responsibilities

• Architect and implement data pipelines using Databricks and Delta Lake for both batch and streaming data.
• Develop and maintain ETL/ELT workflows integrating varied data sources into cloud data lakes and warehouses.
• Implement data governance, security, and compliance standards across data flows.
• Automate workflows using Python scripting and CI/CD pipelines.
• Troubleshoot and resolve issues within data ingestion, transformation, and storage processes.

Required Skills

• 8+ years in data engineering or a related role.
• Expert-level Python programming for data processing and automation.
• Hands-on experience with Databricks and Delta Lake.
• Strong knowledge of Azure Data Factory (ADF), Data Lake, and Synapse or equivalent cloud platforms.
• Proficiency in SQL and database performance tuning.
• Familiarity with DevOps practices, CI/CD, and infrastructure-as-code tools.
• Experience with data migration and cloud-native architectures.

Key Skills

Education

Any Gradute