← Back to jobs

CareerNet Technologies Pvt Ltd Logo
Data Platform Architect

CareerNet Technologies Pvt Ltd

 

Thane, Maharashtra, India

Posted On: 5 days ago
Experience: 10+ years
Availability: Onsite
Openings: 1
Category: Data Platform Architect
Tenure: No Preference/Any
Related Jobs

No related jobs found

Description

Design and own the end-to-end architecture for a Databricks-based data lakehouse platform across Dev, UAT, and Prod environments.

This role is on-site.

Responsibilities

  • Build batch and streaming data pipelines using Spark, Delta Lake, and Delta Live Tables, implementing medallion architecture patterns.
  • Define and enforce platform best practices including CI/CD, version control, reusable libraries, and robust monitoring.
  • Optimize Spark job performance through partitioning, caching, join strategies, and shuffle tuning to ensure reliability.
  • Lead FinOps governance, managing cluster auto-scaling, cost allocation, budgets, and usage monitoring.
  • Implement data governance, security controls (RBAC, row/column-level security), and lineage auditing via Unity Catalog.

Required Skills

  • 10+ years of experience in Data Engineering and platform architecture.
  • Deep expertise in Databricks, Spark, and Delta Lake for large-scale data processing.
  • Hands-on experience with Kafka and structured streaming for real-time ingestion.
  • Proficiency in cloud platforms (AWS, Azure, or GCP) and their data services.
  • Strong SQL and RDBMS skills with a focus on data modeling.
  • Proven track record in cost optimization and FinOps practices for cloud data platforms.
  • Experience with Scala and PySpark for developing scalable data applications.

Preferred Skills

  • Familiarity with AWS native services like Kinesis, SQS, and SNS.
  • Knowledge of enabling AI/ML integrations, including RAG patterns and GenAI use cases.

Education: B.E/ B.Tech/ MCA

Education

Any Graduate

Related Jobs

No related jobs found

← Back to jobs