← Back to jobs

Q1 Technologies  Inc Logo
Big Data Engineer

Q1 Technologies Inc

 

Phoenix, AZ 85044, USA

Posted On: 4 days ago
Experience: 5+ years
Availability: Onsite
Openings: 1
Category: BigData Engineer
Tenure: No Preference/Any
Related Jobs

No related jobs found

Description

Design system solutions and develop custom applications to transition legacy batch-based platforms to event-driven, microservices-based architectures.

This role is on-site.

Responsibilities

  • Translate user needs into system architecture and develop software applications.
  • Build and execute data pipelines for batch and real-time data processing.
  • Collaborate with SRE, testing, and production support teams to ensure smooth development life-cycle transitions.
  • Perform coding, debugging, documentation, and performance validation of component systems.
  • Test the engineering resilience of software and automation tools.

Required Skills

  • 5+ years of software development experience, including leading engineering and scrum teams.
  • 3+ years of hands-on experience with Map-Reduce, Hive, and Spark (core, SQL, and PySpark).
  • Expertise in writing and optimizing complex SQL, Hive, and PySpark dataframes for large-scale data processing.
  • Proficiency in UNIX shell scripting.
  • Strong understanding of Hadoop and Spark architecture principles.
  • Ability to design and develop optimized data pipelines.
  • Experience in the analysis, design, testing, and implementation of system applications.
  • Hands-on experience with NoSQL databases including HBase, Couchbase, or MongoDB.

Preferred Skills

  • Knowledge of cloud platforms such as GCP or AWS.
  • 2+ years of experience designing solutions using Kafka streams or queues.
  • Experience with GitHub and CI/CD pipelines.

Education

Any Graduate

Related Jobs

No related jobs found

← Back to jobs