Description

Lead the design and maintenance of scalable data pipelines and warehouse solutions to support increasing data volume and complexity.

Responsibilities

• Design, develop, and maintain scalable data pipelines, data lakes, and data warehouse architectures.
• Build and implement processes for data mining, data modeling, and data production.
• Develop data quality frameworks and monitoring systems to ensure production data accuracy and availability.
• Migrate data from legacy systems to new distributed solutions and monitor system performance.
• Collaborate with architects, product managers, and engineers to improve data models for business intelligence tools.

Required Skills

• 8+ years of data engineering experience in a software delivery environment.
• Proficiency in Python, Java, or Scala.
• Experience building petabyte-scale distributed systems.
• Hands-on expertise with Hadoop, Hive, and Spark.
• Experience with Snowflake, Presto, or Redshift.
• Working knowledge of NoSQL databases including Cassandra and MongoDB.
• Experience with Kafka and streaming technologies.
• Strong understanding of SDLC methodologies, design principles, and agile methods.
• Bachelor's degree in Computer Science or equivalent experience.

Preferred Skills

• Experience with Databricks and segmentation.

Key Skills

Education

Bachelor's degree