Description

You will build and maintain ETL and streaming pipelines using PySpark, Java, and Spark on AWS infrastructure.

This role is remote.

Responsibilities

• Develop and optimize data pipelines using PySpark and Java.
• Manage infrastructure and deployments with AWS services including EMR, Lambda, S3, and RDS.
• Orchestrate workflows using Databricks and Airflow.
• Write and maintain Shell scripts for Unix systems.

Required Skills

• 3+ years of hands-on experience with PySpark, Java, or Spark.
• Proven experience building ETL and streaming pipelines.
• Proficiency in Python or Java programming.
• Experience with Databricks and Airflow.
• Knowledge of AWS services (EMR, Lambda, S3, RDS).

Preferred Skills

• Experience with Terraform for infrastructure management.
• Familiarity with modern LLM techniques and practical use case development.

Key Skills

Education

Any Graduate