Description

You will develop and optimize data processing pipelines using Apache Spark and PySpark for large-scale datasets.

Responsibilities

• Build and maintain high-performance data processing pipelines using Apache Spark and PySpark.
• Manage and query large-scale datasets within distributed Big Data environments.
• Integrate and maintain data flows from MySQL and Oracle Database systems.
• Ensure data integrity and optimize performance across distributed computing frameworks.

Required Skills

• 5+ years of professional experience in big data development.
• Deep expertise with Apache Spark architecture and optimization.
• Proficiency in PySpark for data manipulation and analysis.
• Strong practical experience with core Big Data technologies and ecosystems.
• Hands-on experience with MySQL database management and querying.
• Experience working with Oracle Database systems.
• Bachelor's degree or equivalent graduate-level education.

Preferred Skills

• Experience with additional distributed computing frameworks beyond Spark.
• Background in data warehousing or ETL pipeline architecture.

Key Skills

Education

ANY GRADUATE