Description

You will own the ETL pipeline architecture and data integration workflows, ensuring reliable data flow from source systems to target databases.

Responsibilities

• Design, develop, and maintain ETL processes for real-time, batch, and near-real-time data processing.
• Write complex SQL queries, scripts, and stored procedures to manipulate and transform data.
• Integrate data from diverse sources including APIs, flat files, databases, and cloud platforms.
• Implement data quality checks and validation routines to resolve discrepancies and ensure reliability.
• Monitor ETL job performance and optimize processes to reduce resource usage and processing time.

Required Skills

• 5 to 8 years of experience in data analysis or ETL development.
• Expertise in Python and SQL for data manipulation and transformation.
• Strong experience with relational databases such as PostgreSQL and SQL Server.
• Hands-on experience with AWS services, specifically Athena and Kubernetes.
• Ability to integrate data via APIs and flat files.

Preferred Skills

• Experience designing and managing CI/CD pipelines to automate software build and deployment.
• Proficiency in container orchestration using Docker and Kubernetes.
• Experience managing cloud infrastructure using AWS EC2 and S3.

Key Skills

Education

Any Graduate