Description

You will build and maintain data integration pipelines within an enterprise data platform.

Responsibilities

• Design, develop, test, and deploy data integration solutions to connect enterprise systems.
• Build and enhance Apache Spark-based integration capabilities.
• Author technical design documents, testing plans, and scripts.
• Facilitate requirements gathering, process mapping workshops, and technical review sessions.
• Maintain and improve existing data integration pipelines and standard operating procedures.

Required Skills

• 5+ years of experience in data integration and pipeline development.
• 2+ years of experience with AWS Cloud data integration.
• Extensive Python development experience, specifically using PySpark.
• Proficiency with Apache Spark, EMR, Glue, Kafka, Kinesis, and Lambda.
• Hands-on experience with S3, Redshift, RDS, and MongoDB/DynamoDB ecosystems.
• Strong database skills including complex query writing, optimization, debugging, UDFs, views, and indexes.
• Experience with Git, Bitbucket, and Jenkins for continuous integration.
• Ability to work with common Python libraries.

Preferred Skills

• Experience with Databricks or Apache Spark.

Key Skills

Education

Any Graduate