Description

You will design, develop, and maintain scalable data pipelines.

Responsibilities

• Design, develop, and optimize data pipelines using Databricks (Spark/Scala) on Azure.
• Write efficient SQL queries for ETL processes, data validation, and reporting.
• Optimize performance of complex data processing workflows.
• Implement best practices for code quality, testing, deployment, and documentation.
• Collaborate with stakeholders to translate business requirements into technical solutions.

Required Skills

• 6+ years of hands-on experience in software or data engineering roles.
• Strong experience with Databricks platform for big data analytics.
• Proficiency in Scala programming language and solid understanding of Spark (RDD/DataFrame/SQL).
• Hands-on experience with Azure cloud services (e.g., Azure Data Factory).
• Strong background in Database design and development with excellent SQL skills (T-SQL/PLSQL).
• Advanced knowledge of Python for scripting and automation.
• Understanding of CI/CD pipelines for automated cloud deployments.
• Experience working within Agile/Scrum methodologies.
• Familiarity with .Net APIs or React-based UIs.

Key Skills

Education

Bachelor's or Master's degrees