Description

You will lead the design, implementation, and optimization of data processing solutions for attribution and forecasting engines.

Responsibilities

• Design and maintain scalable, fault-tolerant data pipelines and DAG workflows across on-premises and cloud environments.
• Optimize Spark job performance through tuning resource usage, shuffling, partitioning, and caching.
• Own the full software lifecycle, from defining business requirements and specifications to documentation and deployment.
• Mentor junior staff and collaborate with cross-functional stakeholders to deliver solutions under tight deadlines.
• Perform data integration, transformation, and troubleshooting within large-scale distributed systems.

Required Skills

• 12+ years of experience in Scala programming.
• Extensive experience with Apache Spark for big data processing on AWS and GCP.
• In-depth knowledge of the Hadoop ecosystem, including HDFS, YARN, and MapReduce.
• Hands-on experience with Python for developing infrastructure modules.
• Proficiency in writing efficient SQL for RDBMS and data warehouses handling terabytes of data.
• Experience with Git and unit/integration testing frameworks.
• Strong understanding of Agile/Scrum methodologies.
• Ability to communicate technical concepts clearly in English across multiple time zones.
• Any Graduate degree.

Preferred Skills

• Experience with Databricks, Delta Lake, and the ELK stack.
• Familiarity with Docker, Kubernetes, and data visualization tools like Tableau, Power BI, or Grafana.

Key Skills

Education

Any Graduate