Description

Key Responsibilities:

• Design the data architecture for platform and organizational use cases, focusing on scalability, performance, and support for advanced analytics and data science workloads.
• Ensure the platform supports multi-cloud operations, data sharing, and integration with various data sources, including on-premises and cloud environments.
• Ensure the platform integrates with various data delivery mechanisms including streaming, batch, and leveraging APIs for encryption and decryption.
• Collaborate with Cloud Architects and Engineers to implement data governance, financial governance, and compliance features.
• Identify and architect automation approaches to simplify and accelerate CI/CD across multiple environments, onboarding of new users to the platform, creation of development workspaces, data quality enforcement, and integrated testing.
• Optimize the platform for handling diverse workloads, relational operations, and multi-model data processing.
• Develop and implement data ingestion and data processing frameworks, ensuring efficient and scalable data pipelines for common patterns.
• Architect and maintain data lakehouse storage solutions, emphasizing best practices for data modeling and schema design.
• Provide guidance on best practices for data preparation, exploration, and visualization within the platform.
• Lead client-facing workshops and influence change and acceptance to modern data platform practices.
• Provide client-facing support, ensuring clear communication and understanding of technical solutions.
• Collaborate with stakeholders to document enterprise rollout strategies, data security blueprints, and architectural standards.
• Represent the platform team at Product Increment (PI) Planning events, driving alignment on deliverables, dependencies, and cross-team commitments.
• Lead through others, guiding Data Engineers across multiple parallel projects without becoming overly focused on any single deliverable.

Qualifications:

• Degree in Computer Science, a technical field, or a related business field. Master’s degree preferred.
• 3+ years of experience in designing and leading Databricks implementations including:
  • Lakeflow Connect
  • Lakeflow Jobs and Lakeflow Spark Declarative Pipelines
  • Databricks Declarative Automation Bundles (DABs), formerly Databricks Asset Bundles
  • Databricks Apps
  • Unity Catalog architecture, implementation, and governance
  • Unity AI Gateway (optional)
  • MLflow (optional)
• 8+ years of experience in relevant technology roles, with a focus on data architecture.
• 5+ years in an advanced role, demonstrating leadership in guiding teams directly or indirectly.
• 5+ years of experience in architecting and maintaining big data platform architecture utilizing AWS Data & Analytics Services.
• Strong data modeling experience.
• Strong expertise in designing and supporting complex application architectures, including integration of disparate data sources.
• Experience in documenting enterprise rollout strategies and developing data security blueprints.
• Excellent communication skills

Key Skills

Education

Bachelor's or Master's degrees