Description

You will design, develop, and maintain Python applications for large-scale data processing within the Hadoop ecosystem.

Responsibilities

• Design and maintain Python applications for large-scale data processing.
• Develop and optimize ETL pipelines for efficient data extraction and transformation.
• Troubleshoot, debug, and optimize Python code and Hadoop processes.
• Collaborate with data engineers and scientists to implement data solutions.
• Participate in code reviews and document data processing workflows.

Required Skills

• 5+ years of experience as a Python Developer focused on data processing.
• Hands-on experience with Hadoop ecosystem components (HDFS, MapReduce, Hive, Pig, Spark).
• Proficiency in writing complex, optimized Python code for data processing tasks.
• Strong understanding of data structures, algorithms, and software design principles.
• Experience with version control systems like Git and collaborative development workflows.
• Knowledge of database systems and SQL.
• Experience with AWS, Azure, or Google Cloud.

Preferred Skills

• Experience with big data technologies such as Kafka, HBase, or Flink.
• Knowledge of containerization and orchestration tools like Docker or Kubernetes.

Key Skills

Education

Any Graduate