Description

Key Responsibilities: 
ELT Pipeline Development: Design, build, and maintain robust ELT (Extract, Load, Transform) pipelines to migrate data from various third-party API sources into our enterprise data lake. 
Databricks Ecosystem: Create, manage, and optimize managed Delta tables within a Databricks environment to ensure high data reliability and query performance. 
API Integration: Securely connect to, authenticate with, and extract data from a wide variety of third-party RESTful APIs. 
Data Modeling & Querying: Write clean, scalable, and efficient Python and SQL code to transform raw data into analytics-ready datasets. 
Knowledge Transfer (KT): Actively participate in onboarding and KT sessions with the current offshore team to seamlessly transition ownership of existing pipelines and architecture. 
Documentation: Maintain clear technical documentation for data pipelines, API integrations, and Delta table schemas to support future onshore hires. 

Required Skills & Qualifications: 
Experience: 6+ years of experience in data engineering, data warehousing. 
Core Languages: Strong, hands-on proficiency in Python and SQL. 
Databricks & Data Lakes: Practical experience working with Databricks, specifically building and maintaining Delta tables and working within a modern data lake architecture. 
API Expertise: Proven experience extracting and handling data from third-party APIs (handling pagination, rate limiting, JSON/XML parsing, etc.). 
Methodology: Solid understanding of ELT methodologies and cloud-based data architecture. 
Communication: Excellent verbal and written communication skills, with the ability to effectively absorb complex technical information during cross-border knowledge transfers

Key Skills

Education

Bachelor's degree