Description

You will build and maintain data pipelines while supporting the migration of existing infrastructure to Kubernetes.

Responsibilities

• Write ETL code using Python and Spark to manage data movement.
• Perform ETL tasks on SQL Server to support data science workflows.
• Migrate legacy infrastructure and processes to Kubernetes environments.
• Verify tables and identify data discrepancies or issues within the pipeline.
• Spot and correct bugs within data processing scripts and workflows.

Required Skills

• 10+ years of professional experience in data engineering.
• Advanced Python programming skills, including passing a live coding assessment.
• Extensive experience with Apache Spark.
• Proficiency with MSSQL or SQL Server.
• Hands-on experience with AWS.
• Degree in any graduate field.

Preferred Skills

• Experience with Azure, Terraform, Scala, Docker, or Kubernetes.

Key Skills

Education

Any Graduate