Description

Design and manage data architectures and pipelines within an AWS environment.

Responsibilities

• Map cross-functional business processes to identify data locations and define data models based on reporting requirements.
• Develop and maintain data pipelines using Databricks DeltaLake format to consolidate, transform, and ensure data quality.
• Establish medallion-style architecture in the data lake to define authoritative data sources and gold-standard data products.
• Implement unit tests for new pipelines and provide guidance on CI/CD processes for data development.
• Collaborate with Business Analysts and act as a consultant for ad-hoc analytic queries executed via Looker.

Required Skills

• 5+ years of experience in data architecture with a focus on data governance.
• Expertise in developing data pipelines using Databricks DeltaLake.
• Proficiency with Spark and Airflow for data orchestration.
• Hands-on experience with Amazon Redshift, Snowflake, Amazon Athena, or Presto for end-user querying.
• Experience with Python, Node.js, and AWS services.
• Strong knowledge of unit testing and CI/CD best practices for data pipelines.
• Ability to map business processes and define formal data models.
• Familiarity with Looker for analytics.

Preferred Skills

• AWS Data Analytics Specialty Certification.
• Experience with Agile development methodologies.

Key Skills

Education

Bachelor's degree in Computer Science