Description

Job Description:

We are seeking a skilled Big Data Engineer with handson experience in Apache Spark Java and Apache Iceberg to design and develop scalable data processing solutions The ideal candidate will build high performance ETL pipelines optimize data lake architectures and ensure reliability quality and efficiency across largescale datasets

Key Responsibilities

• Develop and optimize distributed data processing pipelines using Apache Spark Java
• Design build and maintain Iceberg based data lake tables with strong focus on ACID transactions schema evolution and timet ravel capabilities
• Implement efficient data ingestion transformation and storage strategies using Parquet ORC
• Collaborate with data architects product teams and DevOps to deliver scalable production grade data solutions
• Troubleshoot performance issues and ensure high availability of data systems

Required Skills

• Strong programming experience in Java with deep understanding of Spark APIs
• Handson experience working with Apache Iceberg in real world data lake implementations
• Solid understanding of distributed computing file formats Parquet Avro ORC and data lake architectures
• Experience with cloud platforms AWS/Azure/GCP is a plus
• Familiarity with SQL versioned datasets and metadata management

Key Skills

Education

Any Graduate