Description

Lead the design, development, and optimization of large-scale data warehousing and data lake solutions on the Snowflake Cloud Data Platform.

Responsibilities

• Design and implement highly performant data ingestion pipelines using SnowPipe, SnowSQL, and DataStage for structured and semi-structured datasets.
• Build and manage data transformation processes, including metadata, dependency, and workload management.
• Establish end-to-end monitoring for data pipelines and manage on-call support on a rotation basis.
• Convert business requirements into technical solutions, including feasibility analysis and effort estimation.
• Provide technical guidance to the engineering team and coordinate with offshore teams on pipeline design and enhancements.

Required Skills

• 10+ years of experience in data engineering roles with at least 4 years of recent experience using Snowflake.
• Advanced proficiency in writing complex SQL statements and manipulating large datasets.
• Hands-on experience with Snowflake features: SnowPipe, SnowSQL, tasks, stored procedures, streams, RBAC controls, virtual warehouse sizing, and time travel.
• Extensive experience with DataStage for ETL design, development, and support.
• Experience with data loading patterns involving Kafka, APIs, and streaming data.
• Familiarity with AWS or Azure cloud platforms.
• Practical understanding of DevOps CI/CD lifecycles and Git workflows using tools like GitHub, GitLab, or Bitbucket.
• Experience working within Agile/Scrum environments and prioritizing tasks with Product Owners.

Preferred Skills

• Experience with Python and BigData cloud platforms.
• Expertise in Unix and Shell Scripting.
• Valid professional certification.

Key Skills

Education

Any Graduate