Description

You will design and build scalable data pipelines and distributed event processing systems.

Responsibilities

• Develop high-volume, multi-threaded event processing systems.
• Ingest data from files, streams, and databases using Spark and Python.
• Build and maintain scalable data pipelines to handle increasing volumes.
• Write PySpark and Python programs for data cleaning and processing.
• Ensure platform stability and high availability.

Required Skills

• 7 to 12 years of total IT experience.
• 4+ years of experience with Spark, Python, Scala, or Java.
• Extensive experience in Big Data, Analytics, and ETL technologies.
• Expertise in Python and SQL databases.
• Hands-on experience with non-relational databases.
• Strong background in application development and statistical libraries.
• Experience designing and coding complex data pipelines.

Key Skills

Education

Any gradudate