Description

You design, develop, test, and support cloud data solutions for the CyberDW catalogue, focusing on data ingestion, quality, tuning, and performance of upstream cybersecurity data sources.

Responsibilities

• Lead data feed development and design initiatives for the CyberDW.
• Capture and document metadata for Key Data Elements (KDEs) to support Data Quality rules.
• Partner with data owners to define technical requirements and DQ rules for data elements.
• Establish CyberDW views to encapsulate data for downstream consumption.
• Utilize IBM TWS to schedule and monitor production jobs and resolve DQ issues.

Required Skills

• 10+ years of experience in IT development, data governance, or data architecture, preferably in a regulated environment.
• 5+ years experience with Azure Databricks, Azure Data Factory, Azure Functions, Azure Data Lake storage, Azure Event Grid, Azure Log Analytics, Azure Monitor, and Unity catalog configuration.
• Proficiency in SQL Server, Oracle / PL-SQL, T-SQL, and SQL stored procedures.
• Proficiency in Python, Java, or similar high-level server-side languages.
• Strong knowledge of CI/CD and DevOps tooling, specifically Git, Jenkins, and Azure DevOps.
• Strong understanding of ETL/ELT processes.
• Experience with REST API web services and microservice architecture.
• Knowledge of enterprise Information Security data (e.g., Cloud Security, Incident Response).

Key Skills

Education

Bachelor's degree