Description

You will manage and optimize data pipelines to ensure high availability and quality across the data lifecycle.

Responsibilities

• Monitor, support, and triage data pipelines that ingest, move, transform, and integrate data from acquisition to consumption.
• Proactively identify and resolve recurring data quality or data availability issues.
• Develop queries and metrics for data platform ad-hoc reporting and ETL batch triage.
• Maintain knowledge base and FAQ documentation to provide instructions for common pipeline issues.
• Collaborate within an agile environment to solve complex data patterns and troubleshooting tasks.

Required Skills

• 5+ years of software engineering experience including hands-on development and data engineering.
• 3+ years of hands-on coding experience with SQL, Python, and PySpark.
• 3+ years of experience with orchestration tools like Apache Airflow.
• 2+ years of experience with cloud platforms, specifically GCP, Azure, or AWS.
• Experience building CI/CD processes and pipelines.
• Proven ability to analyze data to identify patterns and resolve technical issues.
• Strong communication skills for interacting with both technical and business teams.

Key Skills

Education

Not specified