← Back to jobs

Quantum Technologies LLC Logo
Big Data Technical Architect

Quantum Technologies LLC

 

New York City, NY, USA

Posted On: Just posted
Experience: 15+ years
Availability: Onsite
Openings: 1
Category: Big Data Architect
Tenure: No Preference/Any
Related Jobs

No related jobs found

Description

You will define and implement large-scale data platform architectures, focusing on data quality, governance, and automation frameworks. You lead technical teams, mentor engineers, and integrate quality gates into CI/CD pipelines.

This role is on-site.

Responsibilities

  • Architect and build robust data platforms using Apache Spark, Hadoop, and cloud-native services.
  • Design and implement large-scale data automation frameworks for quality, validation, and reconciliation.
  • Lead and mentor technical teams, fostering technical excellence and continuous improvement.
  • Integrate data quality gates into DevOps practices and SDLC workflows.

Required Skills

  • 15+ years of software engineering experience, with 5+ years in Technical Architect or Principal Data Engineer roles.
  • Expert-level proficiency in Apache Spark (PySpark, Scala, or Java), including SQL, DataFrames, streaming, and performance tuning.
  • Deep knowledge of distributed storage: Hadoop, HDFS, S3, Delta Lake, and Apache Iceberg.
  • Hands-on experience with streaming technologies like Apache Kafka or AWS Kinesis.
  • Extensive experience designing solutions on AWS (EMR, Glue, Redshift, Lambda), Azure (Databricks, Synapse), or GCP (Dataproc, BigQuery).
  • Expert-level SQL for complex analysis and optimization.
  • Strong Python skills for data engineering, automation, and data quality solutions.
  • Bachelor's or master's degree in Computer Science, Engineering, or a related quantitative field.

Preferred Skills

  • Proven track record of defining data quality engineering methodologies and tools.
  • Experience articulating complex technical concepts to senior leadership and non-technical stakeholders.

Education

Any Graduate

Related Jobs

No related jobs found

← Back to jobs