Description

You will design, develop, and maintain Python-based applications for large-scale data processing using Hadoop technologies.

Responsibilities

• Design and maintain Python applications for large-scale data processing within the Hadoop ecosystem.
• Develop and optimize ETL pipelines for efficient data extraction, transformation, and loading.
• Troubleshoot, debug, and optimize Python code and Hadoop processes to ensure performance.
• Collaborate with data engineers and scientists to implement data solutions meeting business requirements.
• Participate in code reviews and document data processing workflows, models, and application architecture.

Required Skills

• 5+ years of experience as a Python Developer focused on data processing applications.
• Hands-on experience with Hadoop ecosystem components including HDFS, MapReduce, Hive, Pig, and Spark.
• Proficiency in writing complex, optimized Python code for data processing tasks.
• Strong understanding of data structures, algorithms, and software design principles.
• Experience with version control systems like Git and collaborative development workflows.
• Knowledge of database systems and SQL.

Preferred Skills

• Experience with big data technologies such as Kafka, HBase, or Flink.
• Familiarity with cloud platforms including AWS, Azure, or Google Cloud.
• Knowledge of containerization and orchestration tools like Docker or Kubernetes.

Key Skills

Education

Any Graduate