Description

You will design, develop, and maintain Python-based applications for large-scale data processing using Hadoop technologies.

Responsibilities

• Design and maintain Python applications for large-scale data processing using Hadoop.
• Develop and optimize ETL pipelines for efficient data extraction, transformation, and loading.
• Troubleshoot, debug, and optimize Python code and Hadoop processes to ensure performance.
• Collaborate with data engineers and scientists to implement data solutions.
• Participate in code reviews and document data processing workflows and application architecture.

Required Skills

• 8 years of experience in Python development with a focus on data processing.
• Hands-on experience with Hadoop ecosystem components including HDFS, MapReduce, Hive, Pig, and Spark.
• Proficiency in writing complex, optimized Python code for data tasks.
• Technical proficiency with PySpark, Hive, and Hadoop.
• Strong understanding of data structures, algorithms, and software design principles.
• Experience with SQL and database systems.
• Proficiency with version control systems such as Git.
• Bachelor's degree in Computer Science, Engineering, or a related field.

Preferred Skills

• Experience with Kafka, HBase, or Flink.
• Familiarity with cloud platforms such as AWS, Azure, or Google Cloud.
• Understanding of containerization and orchestration tools like Docker and Kubernetes.

Key Skills

Education

Not specified