Description

You will design, build, and optimize high-throughput batch data pipelines in a cloud environment.

Responsibilities

• Design and implement data lakehouse architectures using Databricks on Azure or AWS.
• Develop and maintain batch data processing pipelines using Databricks.
• Troubleshoot and debug complex, multi-threaded data processing applications.
• Manage build automation and configuration using Git and Jenkins.

Required Skills

• 5+ years of professional experience in data engineering.
• 3+ years hands-on expertise with Databricks for batch processing.
• Experience with Azure or AWS cloud platforms.
• Proficiency in Scala.
• Experience with Delta Lake and Synapse Analytics.
• Familiarity with Data Factory, Cosmos DB, and Event Hub.
• Experience with Git/Jenkins for CI/CD.
• Ability to collaborate effectively and work independently.

Preferred Skills

• Experience with Big Data solutions and NoSQL DBMSs.
• Knowledge of Agile methodologies.

Key Skills

Education

Any Graduate