Description

Lead development of ETL processes and data pipelines within Snowflake and SQL Server environments.

You will own the release pipeline for Program Integrity data marts, designing and maintaining data workflows using Azure Data Factory. Implement DevOps practices to ensure efficient data loading and workload balancing. Monitor, optimize, and troubleshoot workflows to maintain performance. Identify and resolve complex technical issues independently.

Responsibilities

• Drive ETL process development for Program Integrity data marts.
• Design and maintain data pipelines using Azure Data Factory.
• Implement DevOps practices for efficient data loading and workload balancing.
• Monitor, optimize, and troubleshoot workflows to ensure performance.
• Resolve complex technical issues independently to maintain operational efficiency.

Required Skills

• 5+ years in cloud-based data engineering and data warehousing.
• 2+ years focused experience with Snowflake, including automated ingestion via SQL procedures and Python.
• Proficiency in Python and scripting languages for data manipulation and automation.
• Experience with Azure Data Factory (ADF), SQL Server, and Blob Storage.
• Hands-on expertise with Snowflake SQL, SnowSQL, and managing formats like CSV, JSON, and VARIANT.
• Technical proficiency in Snowpark, Apache Spark, pandas, NumPy, and Scikit-learn.
• Ability to develop cloud-native ETL processes on Azure and Snowflake.

Preferred Skills

• Azure or Snowflake certification.
• Experience with data modeling, database design, and data governance.
• Familiarity with AWS or Google Cloud.

Key Skills

Education

Any Gradute