Description

You will develop optimized Python code and manage data pipelines to process large datasets.

Responsibilities

• Write optimized Python code focusing on efficient memory and CPU usage.
• Build and maintain ETL processes and data pipelines.
• Develop and execute optimized SQL queries for Big Data analytics.
• Perform data cleaning, transformation, and visualization.
• Apply distributed computing principles and performance tuning to large-scale data tasks.

Required Skills

• 6+ years of experience in Python development.
• Strong proficiency with Pandas and NumPy for data manipulation.
• Hands-on experience with HDFS, Hive, Spark, HBase, and YARN.
• Experience working with large datasets and performance tuning in Pandas.
• Proficiency in SQL and relational databases such as PostgreSQL and MySQL.
• Experience with version control using Git.
• Understanding of Kafka, Flume, or Sqoop for data ingestion.
• Knowledge of the Autosys Scheduler tool.
• Bachelor's degree in any field.

Key Skills

Education

Any Gradute