← Back to jobs

Prophecy Technologies Logo
Senior AWS Spark Performance Engineer

Prophecy Technologies

 

Tampa, FL, USA

Posted On: Just posted
Experience: 6+ years
Availability: Onsite
Openings: 1
Category: Performance Engineer
Tenure: No Preference/Any
Related Jobs

No related jobs found

Description

Optimize AWS EMR clusters and Spark applications to enhance performance, reduce costs, and improve resource utilization.

This role is on-site.

Responsibilities

  • Analyze EMR cluster metrics, Spark application telemetry, and YARN resource utilization to identify inefficiencies and suboptimal configurations.
  • Implement cluster-level optimizations including instance right-sizing, node count adjustments, EBS configurations, and spot/on-demand fleet composition.
  • Tune Spark runtime configurations such as executor memory/core ratios, YARN container sizing, and shuffle service settings to optimize memory utilization.
  • Validate optimization impact through custom operations, end-to-end deployment, and metric validation using internal tooling.
  • Develop operational runbooks, SOPs, and programmatic frameworks to scale optimization workflows across hundreds of EMR clusters.

Required Skills

  • 6+ years of experience in cloud computing and data engineering.
  • Advanced Java concepts and Java Performance Tools (Jprobe, Jensor, OptimizeIT).
  • Deep expertise in AWS EMR cluster operations and Spark & YARN tuning.
  • Experience with Memory Optimization and Capacity Analysis.
  • Proficiency in EC2 Right-Sizing and Cost Optimization strategies.
  • Strong skills in Monitoring & Troubleshooting using CloudWatch and Logs.
  • Experience with Microsoft SQL Server 2019.

Education

Any Graduate

Related Jobs

No related jobs found

← Back to jobs