Description

You will architect and implement large-scale data solutions within the Big Data ecosystem, owning the design of automation frameworks and data platform architectures.

This role is hybrid.

Responsibilities

• Design and implement large-scale data automation frameworks.
• Define and enforce data quality engineering principles, including profiling, validation, cleansing, and anomaly detection.
• Architect data platforms with a focus on data quality, governance, and distributed storage.
• Lead and mentor technical teams to drive technical excellence and best practices.

Required Skills

• 15+ years of progressive software engineering experience.
• 5+ years in a Technical Architect or Principal Data Engineer role.
• Expert-level proficiency with Apache Spark (PySpark, Scala, or Java), including Spark SQL and advanced performance tuning.
• Deep architectural understanding of distributed storage using Hadoop, HDFS, and S3.
• Expert-level hands-on experience with Python for data engineering and automation.
• Proven experience designing solutions on cloud platforms (AWS, Azure, GCP).
• Expert-level hands-on experience with Advanced SQL for complex data validation and optimization.
• Strong understanding of SDLC, DevOps practices, and CI/CD pipeline integration.

Preferred Skills

• Experience with Delta Lake and Dataproc.

Key Skills

Education

Any Graduate