Description

You will own the design and implementation of scalable data architectures and pipelines that support machine learning workflows.

This role is on-site.

Responsibilities

• Develop and maintain large-scale ETL processes managing 1TB of data daily.
• Design ML solutions by integrating new data sources and optimizing data usage for model training and real-time inference.
• Implement monitoring and alerting systems to detect anomalies in processing and model performance.
• Ensure data quality, integrity, and security across all pipeline stages, including versioning and lineage.
• Drive Kanban processes for product maintenance and advocate for tools to reduce technical debt.

Required Skills

• 5+ years of experience in data engineering with a focus on machine learning workflows.
• Hands-on experience with Kafka for streaming data.
• Strong proficiency in SQL and NoSQL database management systems.
• Experience with distributed data processing using Hadoop and Spark.
• Experience with cloud platforms such as AWS or GCP.
• Practical knowledge of data wrangling, preprocessing, and feature engineering.
• Understanding of data governance, security protocols, and compliance requirements.
• Ability to optimize data flows to meet latency requirements.

Preferred Skills

• Experience with Snowflake and Databricks.
• Background in video-game development, the advertising industry, or automated online transaction businesses.

Key Skills

Education

Any Graduate