Description

The role will focus on building scalable data pipelines and integrating Amazon S3, AWS Glue, Apache Iceberg, and Apache Polaris with Snowflake and other analytics platforms.

Responsibilities

• Design and develop data ingestion, transformation, and processing pipelines using AWS Glue, Apache Spark, and Amazon S3.
• Build and manage Apache Iceberg tables supporting open, interoperable access across multiple compute engines.
• Configure and integrate Apache Polaris to Iceberg REST Catalog with AWS and Snowflake.
• Support platform security, access controls, networking, performance optimization, and operational monitoring.
• Participate in workload benchmarking and evaluate the cost and performance of AWS Glue versus Snowflake.

Qualifications

• Required
  • Strong hands-on experience with Amazon S3
  • Strong hands-on experience with AWS Glue
  • Strong hands-on experience with Apache Spark
  • Strong hands-on experience with Apache Iceberg
  • Strong hands-on experience with IAM
  • Strong hands-on experience with AWS networking
  • Experience designing and implementing cloud-based data pipelines and large-scale data processing solutions
  • Understanding of Apache Iceberg architecture, table maintenance, schema evolution, partitioning, and performance optimization

Key Skills

Education

Any Graduate