Description

You will manage and improve ETL pipelines to ensure accurate data integration and processing.

Responsibilities

• Design, develop, and maintain ETL processes to extract, transform, and load data from various source systems into target databases.
• Create and optimize data pipelines for real-time, batch, and near-real-time data processing.
• Write complex SQL queries, scripts, and stored procedures for data manipulation.
• Integrate data from multiple sources including APIs, flat files, databases, and cloud platforms.
• Implement data quality checks and validation routines to resolve discrepancies and ensure data reliability.
• Monitor ETL job performance and optimize processes to reduce resource usage and processing time.

Required Skills

• 5 to 8 years of experience in data analysis or ETL development.
• Expertise in Python and SQL for data manipulation and transformation.
• Strong experience with relational databases such as PostgreSQL and SQL Server.
• Hands-on experience with AWS services, specifically Athena and Kubernetes.
• Ability to integrate data via APIs and flat files.
• Any Graduate degree.

Preferred Skills

• Experience designing and managing CI/CD pipelines to automate software build and deployment.
• Proficiency in container orchestration using Docker and Kubernetes.
• Experience managing cloud infrastructure using AWS EC2 and S3.

Key Skills

Education

Any Graduate