Description

You will build and optimize scalable data platforms supporting advanced analytics and machine learning initiatives.

This role is remote.

Responsibilities

• Design and maintain scalable data pipelines using Apache Spark on Databricks.
• Build and manage Delta Lake architectures for efficient data storage and retrieval.
• Implement ETL/ELT workflows using Databricks notebooks, SQL, and Python.
• Collaborate with AI/ML teams to operationalize models within the Databricks environment.
• Monitor and troubleshoot production data pipelines and model deployments.

Required Skills

• 5+ years of experience in Data Engineering.
• Strong hands-on experience with Databricks, including Spark and Delta Lake.
• Proficiency in Python and SQL for distributed data processing.
• Experience with cloud platforms: AWS, Azure, or GCP.
• Familiarity with the machine learning lifecycle and model integration into data pipelines.
• Understanding of data lakehouse architecture and real-time streaming (Kafka).
• Experience with version control (Git) and CI/CD practices.
• Knowledge of Unity Catalog and MLflow for data governance and tracking.

Preferred Skills

• Experience with Unity Catalog for data governance.

Key Skills

Education

Any Graduate