Description

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines for enterprise data sources. • Build ingestion frameworks for batch, incremental, file, database, and API-based data feeds. • Develop data transformations using SQL, Python and/or PySpark. • Implement source-to-target mappings and support complex data transformation requirements. • Develop reusable and configurable pipeline patterns to accelerate onboarding of new data sources. • Implement data quality, reconciliation, validation, and exception-handling controls. • Optimize pipelines for performance, scalability, reliability, and maintainability. • Implement appropriate logging, monitoring, alerting, and restart/recovery mechanisms. • Support automated deployment using Git and CI/CD practices. • Troubleshoot production data/pipeline issues and perform root-cause analysis. • Work closely with Technical Leads, architects, source-system teams, and downstream consumers. • Participate in code reviews, technical design reviews, testing, deployment, and production support. • Maintain appropriate technical documentation, data mappings, and operational procedures.

Must-Have Skills

• 5–8+ years of hands-on Data Engineering experience. • Strong SQL skills, including complex querying and performance optimization. • Strong programming experience in Python and/or PySpark. • Strong experience with ETL/ELT and data pipeline development. • Experience working with cloud-based enterprise data platforms. • Strong understanding of: o Data ingestion and transformation o Data modeling o Batch and incremental processing o APIs and system integration o Data quality and reconciliation o Pipeline orchestration o Error handling and recovery o Logging and monitoring • Hands-on experience with Git and CI/CD. • Experience handling large-volume enterprise datasets. • Ability to independently troubleshoot complex data issues.

Preferred Skills

• Experience with Snowflake and/or modern cloud data platforms. • Experience with AWS or Azure data services. • Knowledge of CDC and event/streaming-based data integration. • Experience developing reusable data ingestion frameworks and accelerators. • Understanding of data lineage, governance, security, and access controls. • Insurance/Claims domain experience is highly desirable. Required skills: Strong hands-on Data Engineering experience Advanced SQL Python and/or PySpark ETL/ELT and data pipeline development Cloud-based data platforms Data ingestion and transformation Data modeling and source-to-target mapping Preferred skills: Snowflake experience AWS/Azure data engineering services CDC/incremental processing experience Enterprise-scale data integration Data governance, lineage, and security exposure Insurance/Claims domain experience

Key Skills

Education

Bachelor's degree