Description

You will design and develop data management and persistence solutions using relational and non-relational databases. You own the build, test, and enhancement of data curation pipelines integrating data from DBMS, file systems, APIs, and streaming systems. You design data solutions for application use cases to improve processing capabilities. You maintain the health and monitoring of data engineering capabilities to ensure high availability, triage maintenance issues, and monitor workload demands. You collaborate with Infrastructure Engineering teams to maintain the data platform.

Responsibilities

• Build, test, and enhance data curation pipelines for DBMS, file systems, APIs, and streaming sources.
• Design data solutions for application use cases to optimize data processing capabilities.
• Maintain health and monitoring of data engineering capabilities to ensure high availability.
• Triage maintenance issues and monitor workload demands for assigned analytic functions.
• Collaborate with Infrastructure Engineering teams to maintain the data platform.

Required Skills

• 6+ years of overall work experience.
• 3+ years of active development experience as a data developer using Python-spark and Spark Streaming.
• Proficiency with Azure SQL Server, Cosmos DB, Mongo DB, Azure Event Hubs, Azure Data Lake Storage, and Azure Search.
• Experience with AWS and Azure cloud computing environments.
• 3+ years of experience with source code control systems and CI/CD tools.
• Knowledge of Micro Services and streaming technologies.

Preferred Skills

• Bachelor's degree in Information Science, Computer Science, Mathematics, Statistics, or a quantitative discipline.

Key Skills

Education

Any Graduate