Description

You will design, build, and maintain ETL/ELT workflows using Databricks.

Responsibilities

• Develop scalable data pipelines using Python and Spark.
• Optimize data transformation logic for performance and cost efficiency.
• Develop and manage data models, schemas, and tables across SQL/NoSQL databases.
• Implement data quality, validation, and monitoring frameworks.
• Troubleshoot pipeline failures and continuously improve pipeline reliability.

Required Skills

• 5+ years of experience in data engineering.
• Mandatory experience with Databricks, including Delta Lake, Notebooks, Spark SQL, Jobs, and Workflows.
• Strong proficiency in Python for data processing, Spark APIs, and automation.
• Expertise in Database Engineering, including strong SQL, query tuning, and schema design.
• Hands-on experience with Apache Spark (RDDs/DataFrames, optimization, partitioning).
• Experience with at least one Cloud Platform: AWS, Azure, or GCP.
• Familiarity with ETL/ELT tools (ADF, Glue, Dataflow, or equivalent).
• Experience implementing CI/CD for data pipelines using Git, DevOps, or Jenkins.

Key Skills

Education

Any Gradute