Description
Build and maintain data architectures for AI/ML model consumption.
This role is remote.
Responsibilities
- Build AI/ML ready data ponds using Azure Databricks Delta Lake.
- Optimize data ingestion models, ETL jobs, and alerting systems for data integrity.
- Partner with AI/ML product teams to identify data requirements for experimentation.
- Write complex, efficient queries to transform conventional data sources.
- Suggest optimal data paths to support rapid experimentation and MLOps.
Required Skills
- 7+ years of hands-on data engineering experience.
- Proficiency with Databricks and Apache Spark/PySpark.
- Experience with Azure Data services and Azure Data Factory for pipeline orchestration.
- Advanced SQL development skills for complex data transformations.
- Experience with modern DevOps practices, including Git and CI/CD.
- Strong background in Data Modeling and data management fundamentals.
- Proven experience in ETL design, mapping, and configuration within complex environments.
Preferred Skills
- Knowledge of MDM techniques and data storage principles.
- Domain experience in ERP (Sales, Inventory, AP, AR) or industrial manufacturing/chemical lab systems.