Description

You will lead the development of data solutions and manage scalable data pipelines to support organizational objectives.

Responsibilities

• Design, build, and maintain scalable data pipelines for ingestion, transformation, and integration using Kafka and Databricks.
• Develop feature engineering pipelines for machine learning workflows using Vertex AI and BigQuery ML.
• Implement automated unit, integration, and performance testing frameworks to ensure data quality and compliance.
• Optimize data workflows for performance, cost efficiency, and scalability across large datasets.
• Provide technical leadership and mentor team members on data principles, patterns, and processes.
• Draft architectural diagrams and technical documentation to communicate design requirements.
• Conduct cost/benefit analyses to guide architectural decisions for data solutions.

Required Skills

• 10+ years of experience in data engineering.
• Expertise in GCP and Vertex AI.
• Proficiency with Python and custom Python libraries.
• Hands-on experience with Kafka and Databricks.
• Experience with BigQuery ML and feature engineering techniques.
• Ability to manage SQL-based, NoSQL-based, and real-time streaming data platforms.
• Any Graduate degree.

Key Skills

Education

Any Graduate