Description

You will manage the migration, integration, and maintenance of data within Amazon S3 and Amazon Redshift environments.

Responsibilities

• Design and implement ETL and ELT data pipelines to load data into Amazon S3 using Azure Data Factory.
• Set up automated workflows for continuous data migration and configure Amazon S3 buckets for storage.
• Establish data schemas and tables in Amazon Redshift and migrate preprocessed data into these environments.
• Monitor data pipelines for performance, reliability, and security to ensure data integrity.
• Maintain and improve data integration processes for better efficiency and scalability.

Required Skills

• 5+ years of experience creating, documenting, implementing, and maintaining data pipelines.
• Proficiency with Azure Data Factory.
• Strong knowledge of Amazon AWS S3, AWS Redshift, and AWS Lambda.
• Advanced SQL skills and experience with ETL tools.
• Experience working with various cloud platforms.
• Expertise in implementing data security and integrity measures.
• Ability to document technical integration processes and procedures.

Preferred Skills

• Experience using AWS Glue.
• Proficiency in Python, C#, and SparkSQL.

Key Skills

Education

Any Graduate