Description

You will own the design, configuration, and maintenance of data ingestion pipelines, focusing on Fivetran and HVR integrations. You will ensure reliable data flow from enterprise and fintech sources into our analytics infrastructure.

Responsibilities

• Build and support data ingestion pipelines using Fivetran, managing connectors, connectivity, and monitoring.
• Implement Change Data Capture (CDC) and replication-based ingestion using Fivetran HVR.
• Integrate ingestion workflows with Astronomer (Airflow) for scheduling and orchestration.
• Develop and maintain Python-based ingestion components, including custom SDKs and API-based connectors for non-standard sources.
• Perform pipeline monitoring, troubleshooting, and basic performance optimization.

Required Skills

• 6-10 years of experience in Data Engineering or ETL/ELT development.
• Hands-on experience with Fivetran (HVR), including deep understanding of HVR Channel management.
• Proficiency in Python for data ingestion and integration scripts.
• Experience working with relational databases: Oracle, SQL Server, and DB2.
• Working knowledge of Astronomer or similar orchestration tools.
• Bachelor's degree in a relevant field or equivalent experience.

Preferred Skills

• Experience supporting ingestion from Mainframe data sources.
• Ability to document ingestion processes, configurations, and operational procedures.

Key Skills

Education

Bachelor's degree