Description

You design and support cloud data solutions for the CyberDW catalogue, focusing on data ingestion, quality, tuning, and performance of upstream cybersecurity data sources.

This role is remote.

Responsibilities

• Lead data feed development and design initiatives for the CyberDW.
• Capture and document metadata for Key Data Elements (KDEs) to support Data Quality rules.
• Partner with data owners to define technical requirements and DQ rules for data elements.
• Establish CyberDW views to encapsulate data for downstream consumption.
• Utilize IBM TWS to schedule and monitor production jobs and resolve DQ issues.

Required Skills

• 10+ years of experience in IT development, data governance, or data architecture.
• 5+ years experience with Azure Databricks, Azure Data Factory, Azure Functions, and Azure Data Lake storage.
• Proficiency in SQL Server, Oracle, PL/SQL, T-SQL, and SQL stored procedures.
• Experience with Azure Event Grid, Azure Log Analytics, Azure Monitor, and Unity catalog configuration.
• Proficiency in Python, Java, or similar high-level server-side languages.
• Strong knowledge of CI/CD and DevOps tooling, specifically Git, Jenkins, and Azure DevOps.
• Strong understanding of ETL/ELT processes and REST API web services.
• Experience with microservice architecture and enterprise Information Security data.

Key Skills

Education

Bachelor's degree