Description

You will lead the architecture and implementation of Databricks-based ETL frameworks for large-scale enterprise systems on AWS.

Responsibilities

• Lead the design and implementation of Databricks-based ETL frameworks using Apache NiFi.
• Develop high-throughput data pipelines using Databricks and streaming technologies.
• Write Python and Java-based scripts for data processing and transformation.
• Manage DevOps activities, including infrastructure setup and Spark job deployments.
• Mentor junior team members and provide technical guidance.

Required Skills

• 8–12 years of overall IT experience.
• 5+ years of hands-on experience focused on ETL and Python.
• Expertise in designing data solutions using Databricks and AWS.
• Extensive experience with Apache NiFi and streaming development.
• Architectural experience with Databricks running on AWS deployments.
• Proficiency in Python and Java for data manipulation.
• Ability to troubleshoot and optimize Databricks queries for BI use cases.
• Knowledge of DevOps practices for managing Databricks job deployments.
• Bachelor’s or Master’s degree in Computer Science, IT, or a related field.

Preferred Skills

• Databricks Certified Professional in Advanced Data Engineering.

Key Skills

Education

Any Graduate