Description

Role description

Role purpose. Convert legacy artifacts (Alteryx, SSIS, SQL Server Agent) into executable designs on the Medallion/Unity Catalog architecture, with defensible and auditable decisions. This is the role that unblocks development: every job awaiting redesign is development on hold.

Candidate profile summary

Professional with a track record in cloud data platform design who has led the modernization of legacy ETL workloads to lakehouse architectures. Able to make design decisions with incomplete information, document them in an auditable way, and provide technical leadership to a team of up to 3 engineers/developers working in parallel.

Required experience

• Data engineering/architecture.
• Designing cloud data architectures (not just implementing designs created by others).
• Knowledge of Apache Spark in production environments.
• Experience providing technical leadership to other engineers/developers (not necessarily with formal direct reports).

Key Responsibilities

• Critical Reverse Engineering: Analyze the actual production behavior of Alteryx workflows, SSIS packages, and SQL Server jobs, identifying undocumented dependencies, historical patches, and implicit logic.
• Analysis Agent Validation: Audit the reports produced by the automated analysis agents against the original source, resolving discrepancies and blind spots.
• Design and Governance: Define and formalize patterns for ingestion, partitioning, data typing, naming conventions, scalability strategies, and security policies in Unity Catalog.
• Design Plan Approval: Formally approve and take ownership of the plan.md before authorizing automated code generation.
• Blocker Resolution (NEEDS-HUMAN): Act as the highest technical authority for exceptions, business ambiguities, or limitations of the multi-agent system.
• Integration Approval: Act as the mandatory technical approver on Pull Requests to protected branches (preprod / main).

Skills

Technical skills

Delta Lake · Apache Spark · PySpark · Medallion architecture · lakehouse · data warehouse · SQL Server · T-SQL · SSIS · Azure · Azure DevOps · CI/CD · data modeling · data governance · data lineage · Databricks (nice to have)

Nice to have (not required): Streaming (Solace, Kafka, Lakeflow), DevOps, experience with SAST controls (Fluid Attacks or similar for vulnerability management).
 

Key Skills

Education

Any Graduate