Description

You will develop and maintain data processing pipelines using Python and PySpark for real-time and batch workloads.

This role is on-site.

Responsibilities

• Build and maintain data processing pipelines using Big Data frameworks.
• Process real-time data streams using Kafka and Kinesis.
• Interact with NoSQL data stores including HBase and DynamoDB.
• Participate in business, architectural, and technical decision-making.
• Write efficient SQL queries and analytical functions.

Required Skills

• 3+ years of industry experience, including 2+ years in big data.
• Strong programming proficiency in Python and PySpark.
• Experience with Big Data frameworks: Spark, Hive, NiFi, Spark-Streaming, Flink.
• Experience with streaming services: Kafka and Kinesis.
• Familiarity with NoSQL stores: HBase and DynamoDB.
• Proficiency in SQL and SQL Analytical functions.

Preferred Skills

• Scala experience.

Key Skills

Education

Any Gradute