Description

You will build and maintain data pipelines using modern big data technologies and ETL processes.

Responsibilities

• Design and implement ETL processes using the Microsoft stack and Azure Data Factory.
• Develop and optimize data workflows using Apache Spark and Databricks.
• Own tasks from requirement gathering to delivery with minimal oversight.
• Collaborate with non-technical stakeholders to translate abstract ideas into technical requirements.
• Maintain code quality through Git-based PR reviews and CI/CD pipeline integration.

Required Skills

• 5+ years of professional software engineering experience.
• Expertise in Microsoft SQL or other relational database products and SQL language.
• Strong experience with Microsoft ETL stack, including SSIS and Azure Data Factory.
• Hands-on experience with Apache Spark and Databricks.
• Intermediate to advanced proficiency in Python or similar programming languages.
• Practical knowledge of testing within big data environments.
• Proficiency with Git SCM and PR-based code review processes.
• Experience working in Agile/Scrum environments.
• Understanding of CI/CD pipelines and SDLC practices.

Preferred Skills

• Familiarity with Azure DevOps.
• Knowledge of event-driven architecture and unit testing frameworks.

Key Skills

Education

Any Graduate