Description

You will design, develop, and optimize big data applications using Python and PySpark.

Responsibilities

• Develop and maintain scalable data pipelines using Python and PySpark.
• Implement ETL processes to extract, transform, and load data.
• Optimize and troubleshoot existing PySpark applications to improve performance.
• Collaborate with data engineers and data scientists to implement data processing needs.
• Participate in design and code reviews while ensuring data integrity and quality.

Required Skills

• 5+ years of experience as a Python Developer with expertise in PySpark.
• Proficiency in Python and Apache Spark.
• Strong understanding of Big Data technologies and frameworks.
• Experience with distributed computing and parallel processing.
• Proficiency in SQL and experience with database systems.
• Solid understanding of Data Engineering concepts and Data Warehousing.
• Experience working with Azure or AWS.
• Bachelor's Degree in Computer Science, Information Technology, or a related field.

Key Skills

Education

Bachelor's Degree