Description

You will build and maintain data ingestion and curation pipelines across diverse sources including DBMS, file systems, APIs, and streaming systems.

Responsibilities

• Develop and enhance data ingestion solutions for structured and unstructured data on on-prem and cloud infrastructure.
• Build and test data curation pipelines to integrate various sources for KPI and metric development.
• Host data models via REST APIs, ensuring proper logging, authentication, error capturing, and concurrency management.
• Collaborate with Data Science teams to implement mathematical models and recommend improvements to analytic methods.
• Maintain data system health by triaging maintenance issues and ensuring compliance with enterprise governance and security standards.

Required Skills

• 6+ years of progressive experience in Data or Software Engineering.
• Proficiency in Python, Scala, and Java.
• Strong experience with Spark and Kafka.
• Advanced SQL query development skills.
• Experience managing data ingestion from APIs, streaming systems, and DBMS.
• Ability to implement non-functional requirements like authentication and error handling in model hosting.
• Bachelor's degree in Computer Science or a related engineering field.

Key Skills

Education

Bachelor's degree in Computer Science