Description

Responsibilities

• Design and develop large-scale, scalable data pipelines using Databricks and Apache Spark
• Mentor and provide technical guidance to other developers on the engineering team
• Contribute to architecture and system design decisions for scalable and distributed data platforms
• Implement software engineering best practices including CI/CD, automated testing, code reviews, source control, and application security
• Troubleshoot complex technical issues and solve problems across the data platform
• Collaborate with cross-functional teams in an Agile/Scrum environment
• Lead technical discussions and communicate solutions effectively with stakeholders

Qualifications

• Required
  • 8+ years of professional software development or software engineering experience
  • 2+ years of experience as a Technical Lead, Lead Engineer, or Senior Engineer with responsibility for mentoring other developers
  • Strong hands-on experience with Python
  • Strong hands-on experience with PySpark
  • Strong production experience with Databricks
  • Strong experience with Apache Spark
  • Experience designing and developing large-scale, scalable data pipelines
  • Strong experience working in Microsoft Azure environments
  • Strong understanding of distributed data processing and modern data architectures
  • Experience with CI/CD practices
  • Experience with automated testing
  • Experience with code reviews and source control
  • Experience with application security
  • Experience contributing to architecture and system design for scalable and distributed applications or data platforms
  • Strong troubleshooting and problem-solving skills
  • Experience working in Agile/Scrum environments
  • Excellent communication and collaboration skills
  • Ability to mentor engineers and provide technical guidance

Key Skills

Education

Any Graduate