Description

You will design, develop, and maintain scalable data pipelines and frameworks to process large volumes of data.

Responsibilities

• Build and optimize ETL pipelines and data ingestion processes.
• Design and implement data transformation jobs and ETL execution workflows.
• Write optimized queries using HQL, SQL, and Spark.
• Develop solutions using Big Data technologies including MapReduce, Hive, and Spark.
• Maintain code quality through peer reviews and CI/CD pipeline management.
• Troubleshoot, debug, and upgrade existing applications and ETL job chains.

Required Skills

• 5+ years of experience as a Data Engineer or in a similar role.
• Proficiency in object-oriented programming with Python and PySpark APIs.
• Hands-on experience with Spark, MapReduce, and Hive.
• Strong command of SQL and HQL for data modeling and querying.
• Experience designing and implementing ETL processes.
• Proficiency with Git for version control.
• Experience implementing CI/CD pipelines using Jenkins.
• Familiarity with Unix/Linux environments and shell scripting.
• Bachelor's degree in Computer Science, Engineering, or a related field.

Preferred Skills

• Experience with Google Cloud platforms.
• Relevant certifications in Big Data, Cloud, or Data Engineering.

Key Skills

Education

Any Gradute