Description

You will design and optimize end-to-end data pipelines to support machine learning and AI applications.

Responsibilities

• Design and implement data pipelines for AI and machine learning using Azure and AWS.
• Develop data processing and feature engineering workflows for model training.
• Automate workflows and orchestration using Apache Airflow, Azure Data Factory, or AWS Step Functions.
• Process large datasets using Apache Spark and Databricks.
• Implement data versioning and lineage tracking to ensure model reproducibility.

Required Skills

• 5+ years of experience in data engineering with a focus on AI, machine learning, and AI enrichment.
• Proficiency in Azure, AWS, and Databricks.
• Strong programming skills in Python or Scala.
• Experience with machine learning frameworks and libraries.
• Knowledge of data modeling, database design, and optimization.
• Familiarity with data warehousing concepts and technologies.
• Degree in Computer Science or a related field.

Preferred Skills

• Experience with MLOps, Docker, and Kubernetes.
• Knowledge of NLP, computer vision, and distributed computing.
• Microsoft Certified: Azure AI Engineer Associate or similar industry certifications.

Key Skills

Education

Any Graduate