Description

Role Overview:

We are seeking an experienced Tech lead- Data Engineer with a strong background in Java, AWS, Python, PySpark, and event-driven architectures.

Key Responsibilities:

• Architect, build, and maintain event-driven data pipelines using AWS services such as Kinesis, MSK/Kafka, Lambda, Step Functions, SQS/SNS, and Glue/EMR.
• Develop ETL/ELT workflows using Python and PySpark, ensuring performance, scalability, and cost efficiency.
• Implement and optimize Spark-based data transformations, partitioning strategies, and data processing frameworks.
• Design and manage data lake and warehouse structures using S3, Glue Catalog, Athena, and/or Redshift.
• Build streaming solutions with checkpointing, stateful transformations, idempotency, and schema evolution.

Required Skills:

• 15+ years of professional experience in Data Engineering.
• Strong expertise in Python and PySpark for large-scale data processing.
• Advanced hands-on experience with AWS (S3, Glue, EMR, Lambda, Step Functions, Kinesis/MSK, DynamoDB, Athena, Redshift).

Qualifications:

• Deep experience building event-driven and streaming data pipelines.
• Strong SQL experience for analytical and ETL workloads.
• Hands-on experience with workflow orchestration tools such as Airflow or Step Functions

Key Skills

Education

Any Graduate