Description

You will architect and implement large-scale AI/ML solutions leveraging Databricks and AWS.

Responsibilities

• Design and build data pipelines using PySpark and Spark SQL on Databricks.
• Implement and manage ML workflows using MLflow and Databricks Jobs.
• Process and optimize multi-terabyte datasets using Delta Lake.
• Develop production-ready ML models using Python and standard ML libraries.
• Integrate ML solutions into deployment pipelines using CI/CD tools.

Required Skills

• 12+ years of experience in AI/ML architecture or data engineering.
• Deep expertise in Databricks on AWS, including Delta Lake and Unity Catalog.
• Strong proficiency in Python, including pandas, numpy, and scikit-learn.
• Proven experience with PySpark and Spark SQL for large-scale data processing.
• Experience with MLflow for experiment tracking and model registry.
• Familiarity with CI/CD tools such as GitHub Actions or GitLab CI.
• Strong understanding of distributed systems and data optimization techniques.
• Knowledge of core ML algorithms and architectural patterns.

Key Skills

Education

Any Graduate