Description

You will design, develop, and implement large-scale big data products and production machine learning systems.

Responsibilities

• Build and deploy production-level data processing workflows, pipelines, and data-driven applications.
• Lead the implementation of machine learning production systems, including ingestion, feature engineering, modelling, and monitoring.
• Develop data warehousing and data lake solutions using advanced data modelling and integration techniques.
• Automate processes using shell scripts and manage software lifecycles through version control, testing, and deployment.
• Manage complex engineering projects within Agile/Scrum environments, providing technical leadership and issue escalation.

Required Skills

• 10+ years of hands-on experience as a Data Engineer in an individual contributor capacity.
• Expert proficiency in Python, Scala, and Java.
• Advanced experience with Spark, specifically tuning jobs for performance.
• Deep knowledge of Hadoop platforms and tools including Hive, Impala, Airflow, and NiFi.
• Strong SQL skills for data mining and complex analysis.
• Experience implementing machine learning systems and algorithms at scale.
• Proven ability to lead collaborative work in complex engineering projects using Agile methodologies.
• Solid foundation in computer science, data engineering, and statistical analytical techniques.

Preferred Skills

• Experience developing and configuring dashboards using Tableau or Looker.

Key Skills

Education

Any Graduate