Description

Job Description

We are seeking an experienced Data Engineer with strong expertise in large-scale data processing and cloud-based application development. The role requires deep hands-on technical skills along with a strong understanding of end-to-end data solutions and the ability to collaborate across teams to deliver scalable and high-performance systems.

Experience Requirements

• 8+ years of experience in software development, preferably in distributed data systems or large-scale applications
• Minimum 5+ years of hands-on experience in designing and developing Big Data solutions
• At least one end-to-end Big Data implementation experience

Key Responsibilities

• Design, develop, and maintain scalable Big Data pipelines and data processing systems
• Build distributed data processing applications using modern frameworks
• Develop backend services and APIs for data platforms
• Optimize performance and troubleshoot issues in large-scale data systems
• Work closely with cross-functional teams to deliver end-to-end solutions
• Ensure data quality, scalability, and reliability of data pipelines

Technical Skills

Processing & Compute

• Strong hands-on experience with Apache Spark (mandatory)
• Experience with Hadoop ecosystem including HDFS, Hive, Impala, HBase

Programming

• Proficiency in Java / Scala / Python

Data & Query

• Strong expertise in SQL (query building, debugging, optimization)

Data Formats

• Experience working with XML, JSON, Parquet, Avro, Protobuf

Application Development

• Experience in Spring Boot / Microservices architecture
• Strong experience in backend development and API design

Messaging & Streaming

• Hands-on experience with Apache Kafka for real-time data pipelines

AI & Productivity Tools

• Ability to leverage tools such as GitHub Copilot, ChatGPT, or similar to enhance development efficiency, code quality, and debugging

Nice-to-Have Skills

• Experience with Databricks Lakehouse Platform, including:
  • Delta Lake
  • Databricks Workflows
  • Unity Catalog
  • MLflow

Soft Skills

• Strong analytical and troubleshooting capabilities
• Ability to work under pressure in a fast-paced environment
• Strong multitasking and prioritization skills
• High learning agility and proactive mindset
• Adaptable, flexible, and energetic approach to problem-solving

Key Skills

Education

Any Graduate