Description

You will design and implement scalable data processing solutions and manage complex data architectures within an AWS environment.

Responsibilities

• Design and implement scalable data processing solutions using PySpark and AWS Big Data technologies.
• Oversee the integration of Palantir into the data architecture to enhance analysis capabilities.
• Develop and manage AWS Batch workflows for efficient data processing and job scheduling.
• Monitor and troubleshoot data pipelines to maintain seamless data flow and minimize downtime.
• Mentor junior team members and provide technical documentation for stakeholders.
• Optimize AWS Cloud services for performance, security, and cost-efficiency.

Required Skills

• 12+ years of experience in data engineering roles.
• Expertise in PySpark.
• Hands-on experience with Palantir.
• Deep knowledge of AWS Cloud services.
• Proficiency with AWS Big Data technologies.
• Experience managing AWS Batch workflows.
• Proven track record designing complex data solutions.
• Strong understanding of data governance, security, and compliance.
• Any Graduate degree.

Key Skills

Education

Any Graduate