Description

You will design and maintain data pipeline architectures for data-intensive applications across Azure SQL, Cosmos DB, Databricks, and legacy databases.

Responsibilities

• Build infrastructure for extracting, transforming, and loading data from Azure SQL, Cosmos DB, Databricks, and legacy databases.
• Design and implement process improvements, including automation and infrastructure redesign for scalability.
• Develop analytics dashboards and visualizations to provide insights into business performance metrics.
• Create data tools to support analytics and data science teams in optimizing product performance.
• Manage data security and separation across multiple Azure regions and national boundaries.

Required Skills

• 8+ years of experience in software development or data engineering.
• Expert-level Python programming skills for processing Big Data.
• Extensive experience with Databricks on Azure, including Delta Lake and Lake House Architecture.
• Advanced SQL knowledge for relational database management and query authoring.
• Hands-on experience with Azure SQL and Cosmos DB.
• Proficiency in data visualization tools such as Tableau, Power BI, or Looker.
• Experience with message queuing, stream processing, and scalable big data stores.
• Familiarity with Docker and building CI/CD pipelines.
• Ability to perform root cause analysis on internal and external data processes.

Preferred Skills

• Experience managing metadata, dependencies, and workloads within data transformation processes.

Key Skills

Education

Bachelor's degree