Description

Design, develop, and deploy data products while continuously improving team technologies.

Responsibilities

• Implement application requirements covering functional, security, integration, performance, quality, and operations.
• Collaborate with Product Owners and Architects to develop Azure Data Platforms.
• Assess and transform data from various corporate sources, including structured, semi-structured, and unstructured formats.
• Design effective BI data models in line with BUS-Matrix.
• Provide technical guidance and assistance to the team.

Required Skills

• 3 to 6 years of experience managing, designing, and maintaining large-scale data solutions.
• Proficiency in Python, Spark, and Scala for data engineering.
• Experience with ELT/ETL pipeline design and development, including ingestion, transformation, schematization, and orchestration.
• Hands-on experience with Hadoop ecosystem and cloud storage.
• Expertise in both batch and real-time data ingestion processing.
• Strong SQL skills for writing complex queries.
• Experience with Unix Shell scripting.
• Knowledge of DevOps CI/CD pipeline development and deployment.
• Experience with serverless technologies such as Azure Functions or AWS Lambda.

Preferred Skills

• Experience with Azure Data Factory, Azure SQL DW/Synapse, or Databricks.
• Hands-on Azure Databricks pipeline development and PySpark.
• Knowledge of AWS Cloud and EMR.

Key Skills

Education

Any Graduate