Description

You will build and manage scalable data processing solutions using Azure Databricks. This role is on-site.

Responsibilities

• Design and implement scalable data processing solutions within Azure Databricks.
• Optimize data pipelines and workflows for performance and scalability.
• Manage ETL processes between RDBMS (such as Oracle) and Databricks or NoSQL databases.
• Ensure data quality and integrity during transformation and load processes.
• Set up performance monitoring and alerting for pipelines and data integrity.

Required Skills

• 5+ years of professional experience in data engineering.
• Proficiency in Python, PySpark, and Scala.
• Hands-on experience with MongoDB.
• Experience managing ETL workflows between Oracle, RDBMS, and other data stores.
• Working knowledge of Azure environments.
• Ability to troubleshoot and resolve data-related issues.
• Experience developing and maintaining data architecture and best practices.

Key Skills

Education

Any Gradute