Description
You will build and maintain scalable data pipelines and ETL frameworks within an Azure environment.
Responsibilities
- Implement ETL processes for Data Warehouse and Business Intelligence solutions.
- Structure Data Lakes using Azure Data Lake Storage to ensure reliability, security, and performance.
- Write modular, dynamic, and parameterized code following established standards.
- Perform root cause analysis on data and processes to identify improvements.
- Collaborate within an agile team using DevOps processes including CI/CD and Infrastructure as Code.
Required Skills
- 5+ years of data engineering experience.
- Expertise in Snowflake, Azure Data Factory, ADLS Gen 2, Logic Apps, and Azure Functions.
- Proficiency with Databricks, Apache Spark, Scala, Synapse, and SQL Server.
- 5+ years of experience writing and optimizing SQL queries against RDBMS.
- Strong scripting skills in Python, Scala, Java, PowerShell, and XML.
- Experience with unit testing, software change management, and release management.
Preferred Skills
- Knowledge of Infrastructure as Code fundamentals.