Description

You will lead the migration of legacy data pipelines from Apache NiFi and Hive on Cloudera to Python-based architectures on Databricks.

Responsibilities

• Migrate existing Apache NiFi and Hive on Cloudera workloads to Databricks.
• Develop and maintain data processing logic using Python and PySpark.
• Optimize SQL queries and data structures within the Databricks environment.
• Manage or migrate Hive environments to ensure data integrity during transition.

Required Skills

• 10+ years of professional software development experience.
• Strong proficiency in Python development.
• Hands-on experience with Databricks.
• Deep understanding of SQL.
• Experience with PySpark.
• Background in managing or migrating Hive environments.

Preferred Skills

• Degree in any graduate field.

Key Skills

Education

Any Graduate