Description

Build and manage data processing pipelines in UNIX/Linux environments.

This role is on-site.

Responsibilities

• Design and develop data sourcing routines using standardization, transformation, and rationalization techniques.
• Implement in-memory computing solutions using R, Python, PySpark, and Scala.
• Parse and shred XML and JSON data structures.
• Manage data quality through linking, matching, and metadata standards.
• Deploy code using DevOps and CI/CD patterns including Docker or Jenkins.

Required Skills

• 6+ years of professional experience in data development.
• Proficiency in Python, R, Scala, and PySpark.
• Strong SQL skills with experience in DB2, SQLServer, Sybase, or Oracle.
• Hands-on experience with NoSQL databases including MongoDB and DynamoDB.
• Competency in shell scripting and UNIX/Linux environments.
• Experience with XML and JSON parsing.
• Knowledge of master data and metadata management processes.
• Experience with TDD or CI/CD tools like Docker or Jenkins.

Preferred Skills

• Experience with AWS services including EC2, EMR, ECS, S3, SNS, SQS, CloudFormation, CloudWatch, or Lambda.
• Agile methodology experience using Jira or VersionOne.

Key Skills

Education

Any Graduate