Description

You will design, build, and optimize scalable data pipelines and cloud-based storage solutions.

This role is on-site.

Responsibilities

• Design and maintain ETL/ELT pipelines using PySpark and Python.
• Develop big data processing workflows on AWS and GCP.
• Manage data lake, data warehouse, and cloud storage solutions.
• Implement data transformation, aggregation, and cleansing processes.
• Optimize performance and troubleshoot distributed computing environments.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency in Python.
• Extensive experience with PySpark.
• Hands-on experience with AWS.
• Hands-on experience with GCP.
• Experience working with structured and unstructured data.
• Ability to support analytical and machine learning workloads.
• Degree in any graduate field.

Preferred Skills

• Experience with CI/CD pipelines for data workflows.
• Knowledge of streaming technologies like Kafka, Spark Streaming, or Pub/Sub.
• Familiarity with Terraform or CloudFormation for infrastructure automation.

Key Skills

Education

Any Graduate