Description

Lead complex data engineering initiatives and large-scale planning for data systems. You will design, develop, and implement integrated system components while optimizing for performance, scalability, and maintainability.

Responsibilities

• Design and implement data integration, quality processes, and architecture including staging areas, data marts, and data lakes.
• Build and maintain ETL processes using Azure Data Factory or Azure Synapse.
• Manage Azure Storage Containers, Key Vault, and Log Analytics for secure and efficient data operations.
• Implement Lakehouse, Delta Lake, and Medallion architecture patterns.
• Collaborate with stakeholders to resolve technical challenges and ensure compliance with policies.

Required Skills

• 5+ years of software engineering or equivalent professional experience.
• Strong proficiency in SQL and relational database concepts.
• Hands-on experience with Oracle, SQL Server, PostgreSQL, or MySQL.
• Expertise in ETL processes using Azure Data Factory or Azure Synapse.
• Proficiency in Python, Perl, or Shell scripting for automation and data manipulation.
• Experience with Azure Synapse components: Integration runtimes, Linked Services, and Serverless SQL Pools.
• Knowledge of Spark Pools, PySpark, and Spark SQL via Synapse notebooks.
• Understanding of data modeling techniques including star schema and snowflake schema.

Preferred Skills

• Experience consulting on complex, multi-faceted engineering challenges.
• Background in optimizing system performance within existing operational environments.

Key Skills

Education

Any Graduate