Description

You will design, build, and maintain scalable data infrastructure and ETL pipelines on Databricks and AWS.

This role is on-site.

Responsibilities

• Design, develop, and maintain ETL pipelines for large-volume data processing while ensuring data quality.
• Implement and optimize data models and schemas using Delta Tables on the Databricks platform.
• Develop and maintain data warehousing solutions, including modeling and indexing.
• Troubleshoot and resolve data issues to ensure pipeline efficiency and reliability.
• Recommend solutions to improve data infrastructure based on emerging technology trends.

Required Skills

• 5+ years of experience as a Data Engineer or in a similar role.
• Strong proficiency in Python and Spark for scripting and data processing.
• Hands-on experience with the Databricks platform and Delta Tables.
• Proficiency in SQL and experience with MySQL.
• Experience with AWS services, including S3, RDS, and Glue.
• Solid understanding of ETL processes, data modeling, and data warehousing concepts.
• Ability to work independently or within a team structure.

Key Skills

Education

Any Graduate