Description

You will build and optimize data-processing systems, orchestration, and monitoring using open-source software, AWS, and GitLab automation.

Responsibilities

• Develop and enhance data-processing systems and orchestration workflows.
• Collaborate with product and technology teams to design and validate data platform capabilities.
• Implement process improvements by automating manual tasks and scaling processes.
• Provide technical support and guidance to platform service users.
• Create and refine metrics, monitoring, and alerting mechanisms for production services.

Required Skills

• 6-8 years of experience building and optimizing data pipelines in distributed environments.
• 4+ years of advanced working knowledge of SQL, Python, and PySpark.
• Proficiency in Linux environments.
• Experience with AWS and GitLab automation.
• Experience using Git, Bitbucket, Jenkins, CodeBuild, or CodePipeline.
• Experience with platform monitoring and alert tools.
• Experience working with cross-functional teams.
• Knowledge of Palantir.

Key Skills

Education

Not specified