Description

You will build and maintain data pipelines and full-stack applications using Python, PySpark, and SQL.

Responsibilities

• Develop and optimize PySpark jobs for data processing and analytics on cloud platforms.
• Write complex SQL queries and structure data for business requirements.
• Build MongoDB APIs, including aggregations and data retrieval.
• Maintain CI/CD pipelines using Gitlab and oversee application deployments.
• Collaborate with product and design teams to translate requirements into technical specifications.

Required Skills

• 5+ years of experience in data engineering or full-stack development.
• Strong proficiency in Python and PySpark for data transformation.
• Deep understanding of Hadoop ecosystem technologies (Hive, HBase).
• Experience with stream-processing applications (Spark Streaming, Kafka, or Flink).
• Proficient in SQL, data structuring, and storage practices.
• Hands-on experience with Gitlab for CI/CD pipeline management.
• Ability to build and maintain both front-end and back-end systems.
• Experience assembling large, complex structured and unstructured datasets.

Preferred Skills

• Experience with Java or Scala for development within the Spark ecosystem.
• Knowledge of DevOps practices and cloud data platform services.

Key Skills

Education

Bachelor’s degree in Computer Science, Engineering