Description

You will design, build, and optimize large-scale ETL pipelines across Azure and AWS cloud environments.

This role is on-site.

Responsibilities

• Build and implement ETL processes using Databricks and PySpark.
• Design scalable data warehouse solutions using cluster and parallel architectures.
• Write and optimize SQL queries to enhance data workflow performance.
• Manage large data volumes using distributed RDBMS and high-scale architectures.
• Collaborate with onshore teams, business analysts, and engineers to align on requirements and best practices.

Required Skills

• 8+ years of proven experience in data engineering.
• Expertise in Databricks, Spark, and Python.
• Advanced SQL knowledge with strong database fundamentals, performance tuning, and schema design.
• Hands-on experience with RDBMS and distributed systems.
• Proficiency with Azure and AWS cloud platforms.
• Experience using Git and Azure DevOps.
• Ability to interpret and write custom shell scripts.

Preferred Skills

• Experience handling real-time data, Change Data Feed (CDF), and unstructured data.
• Understanding of CI/CD frameworks and Python scripting.

Key Skills

Education

Bachelor's or Master's degrees