Description

Design and develop Big Data engineering solutions and generative AI applications focused on data management efficiencies and query optimization.

Responsibilities

• Collaborate with data scientists and product partners to identify Big Data and query tuning opportunities using machine learning and generative AI.
• Build backend applications using Python, Docker, and Google Cloud to orchestrate end-to-end workflows.
• Design scalable and maintainable generative AI applications, including implementing prompt engineering to fine-tune LLMs.
• Write clean, high-performance, and maintainable code following software engineering best practices.
• Research and develop new generative AI applications to stay current with field advancements.

Required Skills

• 5+ years of experience as a Big Data and MLOps Engineer.
• Deep experience tuning Dataproc, BigQuery, and Spark applications.
• Proficiency in Python and Google Cloud.
• Hands-on expertise with Spark, Spark SQL, BigQuery, Hive SQL, and SQL.
• Experience with Generative AI application development and prompt engineering.
• Competency with Docker and version control systems like Git.
• Solid understanding of code reviews and testing methodologies.
• Ability to work with in-house ML frameworks.

Preferred Skills

• Experience with in-house ML frameworks.

Key Skills

Education

Any Graduate