Description

You will build and maintain data-intensive applications using Python and PySpark.

Responsibilities

• Develop and optimize data processing logic using PySpark, Spark SQL, and Spark APIs.
• Build and manage data pipelines using Airflow, Kafka, and Jenkins.
• Write and maintain unit tests using Pytest or PyUnit to ensure application stability.
• Deploy and manage workloads within AWS environments, specifically utilizing S3, Databricks, and Data Lake Storage.

Required Skills

• 8+ years of professional software development experience.
• Expert proficiency in Python programming.
• Advanced experience with PySpark, including Spark SQL and various Spark APIs.
• Hands-on experience with Python testing frameworks such as Pytest or PyUnit.
• Deep understanding of web frameworks like Django or Flask.
• Experience with AWS cloud services, including S3, Databricks, and Data Lake Storage.
• Proficiency with data orchestration and streaming tools like Airflow and Kafka.
• Experience managing CI/CD processes with Jenkins.
• Any Graduate degree.

Key Skills

Education

Any Graduate