Description

You will build and maintain scalable data pipelines using Apache Spark, Databricks, and AWS.

This role is on-site.

Responsibilities

• Develop and optimize ETL transformations and data pipelines using PySpark or Scala.
• Implement big data solutions on AWS, leveraging Delta Lake for data management.
• Perform performance tuning and ensure data quality across processing workflows.
• Utilize Databricks features including Workflows, Delta Live Tables, and Unity Catalog.
• Integrate data from multiple sources into modern data lake architectures.

Required Skills

• 5+ years of experience in big data engineering.
• Strong expertise in Apache Spark, Databricks, and PySpark or Scala.
• Hands-on experience with AWS cloud services.
• Proficiency in SQL, ETL processes, and Data Pipelines.
• Experience with Delta Lake and performance tuning.
• Strong understanding of data governance and data quality practices.

Preferred Skills

• Experience with CI/CD tools, Airflow, or Kafka.
• Familiarity with Agile practices and relevant certifications.

Key Skills

Education

Any Graduate