Description

You build and maintain scalable data pipelines, ELT processes, and workflow orchestration.

This role is on-site.

Responsibilities

• Design and maintain data pipelines, ELT processes, and workflow orchestration using Apache Airflow, Python, and PySpark.
• Build custom data connectors to ingest structured and unstructured data from diverse sources and document formats.
• Implement DataOps best practices and design CI/CD pipelines for automated integration, transformation, testing, and deployment.
• Monitor, troubleshoot, and optimize data pipelines to resolve ingestion and load issues proactively.
• Perform data validation, quality checks, and testing to ensure accuracy, consistency, and compliance.

Required Skills

• 6–8+ years of hands-on experience in data engineering, ELT development, and data modeling.
• Strong expertise in Python, PySpark, and Apache Airflow.
• Proficiency in SQL and database performance optimization.
• Experience with Snowflake, Hive, Trino, and relational databases (e.g., MySQL).
• Proven experience implementing DataOps and data CI/CD pipelines.
• Strong understanding of distributed systems and large-scale data processing.
• Experience working in Agile/Scrum environments with Git version control.

Preferred Skills

• Experience with Apache Superset and dashboard development.
• Knowledge of data streaming and real-time processing (Apache Kafka).
• Familiarity with data governance frameworks and compliance practices.

Key Skills

Education

Bachelor's degree