Description

Design and maintain scalable data pipelines for batch and real-time processing. Own the implementation of operational and analytical data stores using Delta Lake.

Responsibilities

• Design and develop ETL/ELT workflows for reporting and downstream applications.
• Build and maintain data pipelines leveraging open-source frameworks and cloud technologies.
• Implement operational and analytical data stores utilizing Delta Lake and modern database concepts.
• Optimize data structures for performance and scalability across large datasets.
• Collaborate with architects to ensure alignment with target-state architecture.

Required Skills

• 5+ years of Software Engineering experience or equivalent demonstrated through work, training, military experience, or education.
• Strong proficiency in designing and developing data pipelines and workflows.
• Experience building and maintaining data pipelines using open-source frameworks.
• Proficiency in cloud technologies for data engineering.
• Knowledge of Delta Lake and modern database concepts.
• Strong troubleshooting skills to resolve pipeline issues and ensure high availability.

Preferred Skills

• Experience optimizing data structures for large-scale datasets.

Key Skills

Education

Any Graduate