Description

Lead data engineering efforts to build scalable systems capable of processing millions of records under tight time constraints.

Responsibilities

• Develop secure API calls and data transfer protocols to protect sensitive information.
• Implement data masking for on-prem PII to ensure compliance and security.
• Build scalable solutions to handle high-volume workloads, such as masking 10 million records within 15 minutes.
• Enable secured channels for data models and data science activities.
• Manage the movement of standard output data to lower environments for pre-production testing.

Required Skills

• 6+ years of software development experience.
• Proficiency in Java, Scala, and Python.
• Hands-on experience with Spark.
• Experience with AWS services including S3, Glue, and Redshift.
• Strong SQL and NoSQL database knowledge.
• Experience with data masking and securing PII.
• Ability to design systems that scale to millions of data points.

Preferred Skills

• Familiarity with agile methodologies and standard development tools.

Key Skills

Education

Any Graduate