Description

You will lead the design and implementation of scalable, distributed data architectures for large-scale data processing.

Responsibilities

• Design, develop, and maintain distributed data architectures capable of processing large data volumes.
• Implement and optimize data storage solutions using Hadoop, Spark, and PySpark.
• Develop efficient ETL processes using PySpark to extract, transform, and load large datasets.
• Optimize PySpark applications for performance, scalability, and resource management.

Required Skills

• 6+ years of experience as a Big Data Engineer.
• Strong proficiency in PySpark for developing ETL processes and data transformations.
• Expertise in Python and Big Data technologies.
• Deep understanding of Big Data processing frameworks.
• Experience with distributed computing and parallel processing.
• Proven track record in managing ETL processes.
• Any Graduate degree.

Key Skills

Education

Any Graduate