Description

Lead the design, development, and maintenance of scalable data pipelines and platforms to support organizational data needs.

Responsibilities

• Build and maintain scalable data pipelines for ingestion, transformation, and integration using Kafka and Databricks.
• Develop and manage feature engineering pipelines for machine learning workflows using Vertex AI and BigQuery ML.
• Modernize core data assets across SQL, NoSQL, cloud, and real-time streaming platforms.
• Implement automated unit, integration, and performance testing frameworks to ensure data quality and reliability.
• Optimize data workflows for performance, cost efficiency, and scalability across large datasets.

Required Skills

• 4+ years of professional data development experience.
• 5+ years of experience developing with Java.
• 4+ years of experience with SQL and NoSQL technologies.
• 3+ years of experience building and maintaining data pipelines and workflows.
• 3+ years of experience developing Kafka solutions.
• 2+ years of experience developing with Python.
• 2+ years of experience in feature engineering for machine learning pipelines.
• Hands-on experience with GCP services including BigQuery, Vertex AI Platform, Cloud Storage, AutoMLOps, and Dataflow.
• Experience with CI/CD pipelines, Git version control, and automated testing frameworks.

Preferred Skills

• Knowledge of Structured Streaming using Spark, Kafka, or EventHub.
• Experience with PySpark, Spark development, and Databricks concepts.
• Experience with GitHub SaaS and GitHub Actions.

Key Skills

Education

Any Graduate