Description

Design and manage data pipelines and distributed systems for large-scale processing.

Responsibilities

• Build and maintain data pipelines using Python and Shell scripting.
• Manage distributed systems and workloads with Hadoop and Spark.
• Implement ETL processes to move and transform data across environments.
• Apply statistical models and machine learning principles to data workflows.

Required Skills

• 10+ years of experience in Python and Shell scripting.
• Proficiency in Python or Scala.
• Hands-on experience with Spark and Hadoop.
• Experience with cloud computing platforms including Azure and AWS.
• Strong knowledge of relational databases.
• Proficiency with ETL tools.
• Solid understanding of machine learning principles, statistics, algorithms, and mathematics.

Preferred Skills

• Experience in the healthcare industry.

Key Skills

Education

Any Graduate