Description

You will design and develop Azure data stores, warehouses, and ADF pipelines to move data into Azure Databases and Data Lake.

This role is on-site.

Responsibilities

• Design and develop ADF data pipelines using self-integration runtime to ingest data from various sources.
• Collaborate with Product Owners and Solution Architects to capture requirements and deliver Azure solutions.
• Estimate, plan, and execute Proof of Concept (PoC) projects using Agile methodology.
• Design and deliver quality solutions independently or lead a team of Data Engineers.

Required Skills

• Minimum 5 years of PySpark Development experience, focusing on Spark SQL and complex transformations.
• Minimum 3 years of Python development experience.
• Minimum 2 years of experience with Azure Databricks, Azure Databases, PySpark, Databricks, Azure ADLS, and Azure Cloud.
• Hands-on experience with Delta Lake, SQL, and complex data handling scenarios.
• Proficiency in PySpark transformations, aggregations, window functions, and schema handling.
• Strong capability in Big Data handling, performance tuning, and complex SQL problem-solving.
• Demonstrated ability to handle tricky programming scenarios and share solutioning skills.

Preferred Skills

• Experience leading a team of Data Engineers.
• Strong experience in designing data warehouse solutions.

Key Skills

Education

Any Graduate