Description

You will design and develop data sourcing routines and ETL processes within Hadoop and AWS environments.

Responsibilities

• Build data sourcing routines using standardization, transformation, rationalization, linking, and matching functions.
• Implement data integration techniques to parse and shred XML and JSON data.
• Develop in-memory computing solutions using R, Python, PySpark, and Scala.
• Manage data quality and adhere to master data and metadata standards.
• Deploy code using DevOps, CI/CD, Docker, or Jenkins patterns.

Required Skills

• 5+ years of professional experience in data development.
• Proficiency in Python, PySpark, Scala, R, and SAS.
• Strong SQL skills with experience in DB2, SQLServer, Sybase, or Oracle.
• Experience with NoSQL databases including MongoDB and DynamoDB.
• Hands-on experience in UNIX/Linux environments and shell scripting.
• Working knowledge of the Hadoop ecosystem and Spark.
• Expertise in XML and JSON parsing.
• Experience with ETL techniques in AWS ecosystems.
• Familiarity with Docker, Jenkins, or Test Driven Development.

Preferred Skills

• Experience with AWS services including EC2, EMR, ECS, S3, SNS, SQS, CloudFormation, CloudWatch, or Lambda.
• Background in Financial services or information management.
• Experience working in Agile environments using Jira or VersionOne.

Key Skills

Education

Any Graduate