Description

You will define data models across the lifecycle to support business domains.

Responsibilities

• Define conceptual, logical, and physical data models in Databricks, leveraging financial industry expertise.
• Collaborate with product owners, architects, and engineers to optimize models for query performance, compute, and storage costs.
• Define best practices for implementing the Bronze/Silver/Gold data layers of the lakehouse.
• Generate and maintain data model documentation, data dictionaries, and data definitions.

Required Skills

• 10+ years of experience in AI, Data Science, or Software Engineering with a focus on data ecosystems.
• Expertise in designing and implementing data models optimized for storage and analytics within Databricks on AWS (conceptual, logical, physical).
• In-depth, hands-on experience with the AWS Databricks platform (SQL, Runtime, clusters, notebooks).
• Proficiency in developing ETL pipelines using Databricks tools and Spark for data ingestion and transformation.
• Experience integrating data from heterogeneous sources (APIs, databases, files) ensuring data quality and lineage.
• Ability to optimize data processing workflows and SQL queries in Databricks using partitioning, clustering, and Spark techniques.
• Proficiency in writing complex SQL and Spark code (Scala/Python) for data manipulation.
• Understanding of cloud computing principles and AWS architecture for scalable data solutions.
• Familiarity with government cloud compliance policies (e.g., FedRAMP, FISMA).

Key Skills

Education

Any Graduate