Description

You will modernize a retail analytics platform within a Hadoop ecosystem using Spark and Python.

Responsibilities

• Modernize existing retail analytics platforms sitting on Hadoop, Spark, and Python ecosystems.
• Develop software solutions, detailed specifications, application coding, and unit/systems testing.
• Design and implement job orchestration, metadata design, audit trails, and error handling.
• Formulate system scope and objectives for assigned projects while coordinating production schedules.
• Build reusable components and implement new design patterns to solve complex system problems.

Required Skills

• 2+ years of hands-on PySpark coding and Azure experience.
• Proven experience building ETL/ELT pipelines using PySpark.
• Experience with Azure Data Factory (ADF) for job orchestration, sequencing, and dynamic parameter passing.
• Proficiency in Databricks, Delta Lake, and notebook development.
• Exposure to Scala, DevOps, and CI/CD workflows.
• Ability to perform low-level design and create reusable components.
• Strong competency in test case creation, including unit, integration, and QA testing.
• 4-8 years of experience with a focus on EDW and ETL.
• Bachelor’s Degree in Computer Science, Computer Engineering, or a closely related field.

Key Skills

Education

Any Graduate