Description

You will design, develop, and maintain Python-based applications for large-scale data processing using Hadoop technologies.

Responsibilities

• Design and maintain Python applications for large-scale data processing.
• Develop and optimize ETL pipelines for efficient data extraction, transformation, and loading.
• Troubleshoot, debug, and optimize Python code and Hadoop processes to ensure performance.
• Collaborate with data engineers and scientists to implement data solutions.
• Participate in code reviews and document data processing workflows and application architecture.

Required Skills

• 5+ years of experience as a Python Developer focused on data processing.
• Hands-on experience with HDFS, MapReduce, Hive, Pig, and Spark.
• Proficiency in writing complex, optimized Python code for data tasks.
• Strong understanding of data structures and algorithms.
• Solid grasp of software design principles.
• Experience with version control systems like Git.
• Knowledge of database systems and SQL.

Preferred Skills

• Experience with Kafka, HBase, or Flink.
• Familiarity with cloud platforms such as AWS, Azure, or Google Cloud.
• Understanding of containerization and orchestration tools like Docker or Kubernetes.

Key Skills

Education

Any Graduate