Description

You will design, build, and maintain large-scale data pipelines and infrastructure for the engineering team.

This role is on-site.

Responsibilities

• Build and manage ETL processes and data pipelines using Airflow.
• Develop data solutions using Spark, Databricks, and Kafka to handle big data workloads.
• Implement and optimize data warehousing solutions using Delta Lake and Parquet.
• Manage infrastructure with Terraform and maintain CI/CD processes via GitHub.
• Work with SQL and NoSQL databases to ensure data integrity and availability.

Required Skills

• 8+ years of experience in data engineering.
• Proficiency in Python, PySpark, and either Java or Scala.
• Strong expertise in SQL and NoSQL databases.
• Hands-on experience with AWS cloud data platforms.
• Experience with big data frameworks including Spark, Kafka, and Hive.
• Practical knowledge of Parquet, Delta Lake, and other OTF file formats.
• Experience building CI/CD processes and using GitHub.
• Familiarity with workflow orchestration and ETL.

Preferred Skills

• Experience with GCP.
• Knowledge of dbt.

Key Skills

Education

Any Gradute