Description

You will lead the design and architecture of large-scale big data systems as a subject matter expert.

This role is remote.

Responsibilities

• Design and develop big data pipelines using Spark for batch and streaming workloads.
• Act as a technical consultant and SME to provide solutioning for complex big data systems.
• Implement ETL processes, data governance frameworks, and data quality standards.
• Manage data operations and ensure system integrity across cloud environments.

Required Skills

• 15-20 years of professional experience in data engineering and architecture.
• Expertise in Spark (DataFrame, Spark SQL) and Databricks.
• Proficiency in Scala/Spark or PySpark.
• Hands-on experience with Kafka or similar streaming technologies.
• Deep knowledge of AWS Glue or similar ETL tools.
• Experience with AWS or Azure cloud platforms.
• Strong understanding of technical architecture, data governance, and data quality.

Key Skills

Education

Any Graduate