← Back to jobs

CareerNet Technologies Pvt Ltd Logo
Data Platform Architect

CareerNet Technologies Pvt Ltd

 

Thane, Maharashtra, India

Posted On: Just posted
Experience: 7+ years
Availability: Onsite
Openings: 1
Category: Data Platform Architect
Tenure: No Preference/Any
Related Jobs

No related jobs found

Description

Design, implement, and own the Databricks-based data platform architecture across Dev, UAT, and Prod environments.

This role is on-site.

Responsibilities

  • Build end-to-end batch and streaming data pipelines using Spark, Delta Lake, and structured streaming.
  • Implement medallion architecture (bronze/silver/gold) and scalable ingestion from RDBMS, cloud storage, and streaming sources.
  • Optimize Spark job performance via partitioning, caching, joins, and shuffle tuning to ensure reliability and fault tolerance.
  • Lead cluster auto-scaling, job orchestration, and FinOps governance including cost allocation and usage monitoring.
  • Define platform best practices for code structure, version control, CI/CD, and monitoring, including Unity Catalog adoption.

Required Skills

  • 7+ years of experience in data engineering and large-scale data processing.
  • Strong expertise in Databricks and Spark-based lakehouse architectures, including Delta Lake.
  • Hands-on experience building batch and streaming pipelines using Kafka and Spark.
  • Proficiency in Scala and PySpark for distributed data processing.
  • Solid understanding of RDBMS/SQL and data engineering principles.
  • Experience implementing cloud platforms (AWS, Azure, or GCP) for data infrastructure.
  • Knowledge of data governance tools like Unity Catalog and security controls (RBAC, lineage).

Preferred Skills

  • Familiarity with cloud-native services such as Kinesis, SQS, and SNS.
  • Experience enabling enterprise AI/ML and GenAI/LLM integrations, including RAG patterns.

Education

Any Graduate

Related Jobs

No related jobs found

← Back to jobs