Description

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines for ingesting, transforming, and processing large-scale datasets.
• Build and optimize batch and streaming data solutions using Apache Spark (PySpark/Scala).
• Develop robust ETL/ELT workflows and integrate data from multiple internal and external sources.
• Work with cloud-based Data Lake architectures using AWS S3, Parquet, Iceberg, and related technologies.
• Design and maintain data models and solutions using Amazon Redshift and/or Snowflake.
• Implement and support CDC (Change Data Capture) pipelines and event-driven data integrations using Kafka.
• Optimize SQL queries and improve performance, scalability, and data reliability.
• Build and maintain data quality, validation, monitoring, and governance frameworks.
• Collaborate with cross-functional teams including Data Architects, Analytics, and Business stakeholders.
• Support deployment, troubleshooting, and production monitoring of data workflows.

Required Skills:

• 8+ years of experience in Data Engineering / Big Data development.
• Strong hands-on experience with Apache Spark (PySpark and/or Scala).
• Advanced proficiency in SQL development and performance tuning.
• Experience with Amazon Redshift and/or Snowflake.
• Expertise in building ETL/ELT pipelines and orchestration workflows.
• Experience with Data Lake technologies (AWS S3, Parquet, Iceberg).
• Experience implementing Kafka-based streaming solutions and CDC pipelines.
• Experience with AWS EMR and cloud-based data platforms.
• Strong understanding of data modeling, optimization, and distributed processing concepts.

Preferred Qualifications:

• Experience with CI/CD and data pipeline automation.
• Exposure to modern orchestration tools (Airflow or equivalent).
• Experience working in Agile environments.
• Strong communication and stakeholder management skills

Key Skills

Education

Bachelor's degree