Description

You will build and maintain data pipelines to ingest data from multiple sources including MSSQL, Oracle, files, and APIs into cloud storage and Snowflake databases.

Responsibilities

• Build and maintain optimal data pipelines to ensure operational efficiency and meet business performance metrics.
• Assemble large, complex data sets that meet functional and non-functional business requirements.
• Automate manual processes, optimize data delivery, and redesign infrastructure for greater scalability.
• Implement data privacy standards and ensure pipelines meet compliance with regulatory and contractual obligations.
• Collaborate with Executive, Product, Data, and Design teams to resolve technical issues and support infrastructure needs.

Required Skills

• 8+ years of experience in data engineering with at least 3 years focused on building data pipelines.
• Strong SQL experience and proficiency in Python scripting.
• ETL experience pulling from SQL Server and Oracle, including Change Data Capture (CDC) implementation.
• Experience with Snowflake or other Big Data platforms like Greenplum, Netezza, or Teradata.
• Cloud platform experience with Azure, AWS, or Google Cloud.
• Proficiency with source code control tools such as Git or SVN.
• Experience with database configuration.
• Bachelor’s Degree in Computer Science, MIS, or Information Management preferred.

Preferred Skills

• Experience with Streamsets and Spark.
• Experience with Java, Scala, or Groovy.
• Experience with Kafka consumers, CI/CD tools, and containerization like Docker or Kubernetes.

Key Skills

Education

Any Graduate