Description

You will own the end-to-end development and architecture of data platforms and streaming systems.

Responsibilities

• Define end-to-end architecture for data platforms, streaming systems, and web applications, ensuring alignment with enterprise standards.
• Design and implement data ingestion, transformation, and processing pipelines using Hadoop and PySpark.
• Optimize ETL workflows for large-scale datasets and integrate Apache Kafka for event-driven architectures.
• Build and maintain backend services using Python in a microservices architecture.
• Collaborate with product owners and DevOps teams while mentoring junior engineers.

Required Skills

• 9+ years in software development, with 4+ years focused on architecture and Big Data technologies.
• Strong experience with Hadoop ecosystem components (HDFS, Hive, Spark).
• Proficiency in PySpark for distributed data processing.
• Advanced programming skills in Python.
• Hands-on experience with Apache Kafka for real-time streaming.
• Familiarity with microservices, API design, and scalable system architecture.
• Experience with frontend development using Angular, TypeScript, HTML, and CSS.
• Exposure to cloud platforms including AWS, Azure, or GCP.

Key Skills

Education

Any Graduate