← Back to jobs
Dallas, TX, USA
No related jobs found
Design the target-state Databricks architecture, including a scalable Lakehouse across Bronze, Silver, and Gold layers.
Evaluate existing Palantir Foundry pipelines, datasets, applications, workflows, and AI capabilities; build feature-by-feature mappings to Databricks and flag components that need redesign.
Lead the migration of data ingestion, transformation, data quality, validation, large-scale data comparison, and reconciliation capabilities.
Build and tune data pipelines using Databricks Lakeflow, Spark Declarative Pipelines, Apache Spark, and Delta Lake for both batch and near-real-time processing.
Architect AI/GenAI solutions covering structured and unstructured document extraction, large language models (LLMs), retrieval-augmented generation (RAG), embeddings, vector search, model serving, and the migration of Palantir AIP capabilities.
Design target-state solutions for dashboards, drill-down reporting, analytical applications, chatbots, conversational analytics, and business workflows.
Put enterprise data governance, security, access controls, lineage, and auditing in place using Unity Catalog and suitable role-based access control (RBAC) policies.
Tune Databricks workloads, data pipelines, and compute resources for scalability, performance, reliability, and cost efficiency.
Lead data engineering and AI/ML teams, run architecture reviews, produce high-level design (HLD) and low-level design (LLD) documents and migration roadmaps, and work with business stakeholders, Palantir subject matter experts (SMEs), cloud architects, and application teams.
15+ years of IT experience.
Deep experience in Databricks solution architecture, Lakehouse design, Delta Lake, Apache Spark, and enterprise data engineering.
Strong command of Bronze/Silver/Gold architecture, data ingestion, transformation, data quality, change data capture (CDC), and batch and near-real-time processing.
Demonstrated experience with Palantir Foundry and/or Palantir AIP, ideally on platform migrations, capability mapping, and application or workflow redesign.
Solid knowledge of AI/GenAI solutions, including LLMs, RAG, document intelligence, information extraction, embeddings, vector search, and model serving.
Hands-on experience with Unity Catalog, enterprise data governance, security, access controls, and cloud platforms such as AWS, Azure, or GCP.
Experience architecting analytics, dashboards, chatbots, application integrations, and business workflows in enterprise environments.
Proven technical leadership, architecture documentation, performance optimization, and stakeholder management, with the ability to lead complex migration programs.
Bachelor's degree in Computer Science, Engineering, or a related field
Bachelor's degree
No related jobs found
← Back to jobs