Description

Key Responsibilities

• Design, develop, and maintain scalable data pipelines on AWS
• Build batch and real‑time data processing solutions using PySpark
• Develop streaming pipelines using Kafka / Kinesis for high‑volume event data
• Process and analyze IoT data streams from connected devices and sensors
• Write optimized SQL queries for analytics, reporting, and data validation
• Work extensively with AWS services such as S3, EC2, EMR, Glue, Lambda, Step Functions, Athena, Redshift, DynamoDB
• Implement data ingestion, transformation, and orchestration workflows
• Ensure data quality, security, governance, and performance optimization
• Collaborate with architects, product teams, and stakeholders on end‑to‑end data solutions

Mandatory Skills

• Strong hands‑on experience in AWS Cloud Services
• Excellent programming skills in Python / PySpark
• Advanced SQL for large datasets
• Experience with Kafka or AWS Kinesis for streaming data
• Exposure to real‑time / streaming architectures
• Experience handling IoT data pipelines
• Strong understanding of data modelling and distributed systems

Good to Have

• Databricks on AWS
• Apache Airflow for orchestration
• Experience with Redshift, Snowflake, or other cloud DWs
• Knowledge of CI/CD and Infrastructure‑as‑Code
• Exposure to AI/ML data pipelines

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field
• 4–10 years of relevant Data Engineering experience
• Strong problem‑solving and communication skills
• Experience working in agile, fast‑paced environments

Key Skills

Education

Bachelor's or Master's degrees