Description

You will build and maintain data pipelines to support analytics and machine learning workloads.

This role is on-site.

Responsibilities

• Design and maintain ETL/ELT workflows using Databricks.
• Develop scalable data pipelines using Python and Apache Spark.
• Optimize data transformation logic for performance and cost efficiency.
• Manage data models and schemas across SQL and NoSQL databases.
• Implement data quality, validation, and monitoring frameworks.

Required Skills

• 5+ years of experience in data engineering.
• Expert proficiency in Databricks (Delta Lake, Notebooks, Spark SQL, Jobs, Workflows).
• Strong programming skills in Python for data processing and automation.
• Deep knowledge of Apache Spark (RDDs/DataFrames, optimization, partitioning).
• Proficiency in SQL, query tuning, and schema design.
• Experience with cloud platforms: AWS, Azure, or GCP.
• Familiarity with ETL/ELT tools (ADF, Glue, Dataflow, or equivalent).
• Experience implementing CI/CD for data pipelines (Git, DevOps, Jenkins).

Key Skills

Education

Any Gradute