Description

You will lead the design and architecture of large-scale big data systems as a subject matter expert.

Responsibilities

• Design and develop big data pipelines using Spark for both batch and streaming workloads.
• Act as a consultant and SME to provide technical solutioning for big data systems.
• Implement ETL processes, data governance frameworks, and data quality standards.
• Manage data operations and ensure system integrity across cloud environments.

Required Skills

• 15-20 years of professional experience in data engineering and architecture.
• Expertise in Spark (DataFrame, Spark SQL) and Databricks.
• Proficiency in Scala/Spark or PySpark.
• Hands-on experience with Kafka or similar streaming technologies.
• Deep knowledge of AWS Glue or similar ETL tools.
• Proven experience in solutioning and consulting as a technical SME.
• Experience with AWS or Azure cloud platforms.
• Strong understanding of technical architecture, data governance, and data quality.
• Ability to design and manage complex big data pipelines.

Key Skills

Education

Any Graduate