Description

You will build and maintain end-to-end data pipelines and full-stack systems using Python and PySpark.

Responsibilities

• Write, test, and maintain Python and PySpark code for data processing and analytics.
• Develop complex SQL queries and MongoDB aggregations to support business requirements.
• Build and maintain stream-processing applications using Spark streaming, Apache-Flink, or Kafka.
• Oversee application deployment, monitor performance, and troubleshoot issues in production.
• Collaborate with product managers and designers to translate requirements into technical specifications.
• Maintain CI/CD pipelines based on GitLab and ensure code quality through unit testing and reviews.

Required Skills

• 5+ years of experience as a Data Engineer or similar role within a product company.
• Proficiency in Python and PySpark for data transformation and processing.
• Strong SQL skills for structuring data and performing complex analysis.
• Hands-on experience with MongoDB, including writing queries and building APIs.
• Experience with the Hadoop ecosystem, including Spark, HBase, and Hive.
• Proven ability to build and maintain both front-end and back-end systems.
• Experience implementing end-to-end data pipelines on cloud-based data platforms.
• Experience managing CI/CD pipelines using GitLab.
• Bachelor’s degree in Computer Science, Engineering, or equivalent experience.

Preferred Skills

• Experience with Java or Scala development.
• Knowledge of DevOps practices.

Key Skills

Education

Bachelor’s degree in Computer Science, Engineering