Description

You will design, build, and optimize scalable data pipelines and cloud-based data solutions.

Responsibilities

• Develop and maintain scalable data pipelines using Python, PySpark, and AWS services.
• Implement ETL/ELT processes for ingesting, transforming, and processing large datasets.
• Optimize data workflows and Spark performance tuning for efficiency and cost-effectiveness.
• Ensure data quality, integrity, and governance across the data platform.
• Monitor and troubleshoot data pipelines and cloud infrastructure for reliability and performance.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency with Python and PySpark.
• Hands-on experience with AWS services including S3, Glue, Lambda, EMR, Redshift, and Athena.
• Experience designing and implementing ETL/ELT processes.
• Knowledge of big data processing and distributed computing.
• Experience implementing CI/CD pipelines and automation for data infrastructure.
• Any Graduate degree.

Key Skills

Education

Any Graduate