Description

You will own the development and maintenance of data ingestion pipelines across on-prem and cloud infrastructure, including DBMS, file systems, APIs, and streaming platforms.

Responsibilities

• Build and test data curation pipelines to integrate diverse sources for KPI and metric development.
• Host mathematical models via REST APIs with logging, authentication, error capturing, and concurrency management.
• Collaborate with Data Science teams to implement algorithms and recommend analytic improvements.
• Maintain data integrity and security by adhering to enterprise governance and industry best practices.
• Monitor and triage maintenance issues for assigned data engineering capabilities.

Required Skills

• 6 years of progressive experience in Data or Software Engineering.
• Expertise in Java, Scala, Python, and SQL queries.
• Hands-on experience with Spark and Kafka.
• Experience with Hadoop ecosystems including Sqoop, Hive, Pig, and MapReduce.
• Proficiency in Spark SQL, Spark Streaming, and MLlib.
• Experience building RESTful APIs and microservices using Spring Boot.
• Experience with source code control systems and CI/CD tools.

Preferred Skills

• Master's Degree in Computer Science or CIS.
• 6-8 years of experience with Hadoop, NoSQL, RDBMS, or Cloud Big Data components.

Key Skills

Education

Any Graduate