Description

You design, develop, and maintain scalable ETL/ELT pipelines.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks and Apache Spark (PySpark).
• Lead and support migration of data pipelines and applications from on-premises environments to AWS.
• Build robust data ingestion frameworks for structured, semi-structured, and unstructured data sources.
• Write, optimize, and maintain complex SQL queries across RDBMS, data lakes, and federated data environments.
• Develop reusable frameworks and components to standardize data processing patterns.

Required Skills

• 12+ years of overall IT experience with a strong focus on data engineering.
• Hands-on expertise with Databricks SaaS, Python, and PySpark.
• Expert-level SQL skills, including writing and optimizing complex queries.
• Strong experience with AWS-based ETL services (AWS Glue, EC2, EMR, Amazon S3).
• Solid understanding of Data Lake and Lakehouse architectures.
• Proven experience deploying production code using Git and CI/CD pipelines.
• Experience working with RDBMS and large-scale analytical datasets.
• Ability to independently design and build ETL pipelines.
• Bachelor’s degree in Computer Science or equivalent professional experience.

Key Skills

Education

Bachelor's degree