Description
You will design, build, and operate scalable data services for analytics, AI, and GenAI use cases.
This role is on-site.
Responsibilities
- Build and maintain scalable data pipelines and ingestion frameworks for batch, streaming, and event-driven data using Spark, SQL, and Python.
- Develop modular data models and semantic layers optimized for BI self-service and AI applications, leveraging Databricks and Delta Lake.
- Implement orchestration workflows (e.g., Databricks Workflows) and ensure data quality, lineage, and observability using tools like Great Expectations.
- Deliver curated datasets and reusable data assets for analytics and GenAI, including support for embeddings and RAG pipelines.
- Collaborate with AI Engineers and Data Scientists to enable self-service data products and adhere to data governance standards.
Required Skills
- 9+ years of experience in data engineering and building production-grade data pipelines.
- Strong hands-on experience with Databricks, Spark, SQL, and Python.
- Experience with storage technologies such as Delta Lake, ADLS, and vector stores.
- Proficiency in data quality tooling, observability practices, and CI/CD workflows.
- Knowledge of data modeling, orchestration frameworks, and data governance (e.g., GDPR).
- Ability to process structured, graph, and unstructured data for AI use cases.
Preferred Skills
- Experience working in a global, multi-team environment.
- Exposure to unstructured data processing and AI/GenAI data pipelines.