Description

You will build and maintain distributed systems for high-volume, high-velocity data processing.

Responsibilities

• Ingest data from files, streams, and databases.
• Process data using Python, PySpark, and Scala.
• Develop Spark, Scala, and Hive programs for data cleaning and processing.
• Design multi-threaded event processing systems.
• Implement scalable solutions to handle increasing data volumes on cloud platforms.

Required Skills

• 5+ years of experience in Big Data technologies.
• 4+ years with Python, Spark, PySpark, Scala, and Hive.
• Hands-on experience with Databricks and Azure Data Factory.
• Proficiency with Hadoop and its ecosystem.
• Experience developing applications on AWS, Azure, or GCP.
• Ability to develop APIs and work with analytics libraries.
• Background in open-source NLP and statistical computing libraries.
• Proven track record in data ingestion, cleansing, ETL, and data mart creation.

Key Skills

Education

Any Graduate