Description

You will build and maintain scalable data applications using Python and PySpark.

Responsibilities

• Design, implement, and maintain scalable AWS solutions and architecture patterns.
• Develop data pipelines using Airflow, Kafka, and Jenkins.
• Build and test applications using Pytest and PyUnit frameworks.
• Manage continuous integration and continuous deployment (CI/CD) pipelines.
• Collaborate with cross-functional teams to solve complex analytical problems.

Required Skills

• 10+ years of experience in software development.
• Expert proficiency in Python programming.
• Expert proficiency in PySpark, including Spark SQL and Spark APIs.
• Hands-on experience with AWS services including S3, Databricks, and Data Lake Storage.
• Experience with Python frameworks such as Django or Flask.
• Proficiency with testing tools like Pytest and PyUnit.
• Experience with data pipeline tools including Airflow and Kafka.
• Experience with Jenkins and CI/CD workflows.
• Bachelor's degree in Computer Science, Information Technology, or a related field.

Preferred Skills

• Master's degree in a relevant technical field.

Key Skills

Education

Bachelor's or Master's degrees