Description

Build and maintain scalable data pipelines on AWS for batch and real-time processing.

This role is hybrid.

Responsibilities

• Design and develop data processing solutions using PySpark and streaming tools like Kafka or Kinesis.
• Implement data ingestion, transformation, and orchestration workflows across AWS services including S3, EMR, Glue, Lambda, and Step Functions.
• Write optimized SQL queries for analytics, reporting, and data validation against Redshift, Athena, and DynamoDB.
• Process and analyze high-volume IoT data streams from connected devices and sensors.
• Ensure data quality, security, governance, and performance optimization for end-to-end data solutions.

Required Skills

• 4–10 years of Data Engineering experience.
• Strong hands-on experience with AWS Cloud Services.
• Excellent programming skills in Python and PySpark.
• Advanced SQL proficiency for large datasets.
• Experience with Kafka or AWS Kinesis for streaming data architectures.
• Strong understanding of data modelling and distributed systems.
• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.

Preferred Skills

• Experience with Databricks on AWS and Apache Airflow for orchestration.
• Knowledge of CI/CD, Infrastructure-as-Code, and AI/ML data pipelines.

Key Skills

Education

Bachelor's or Master's degrees