Description

You build production-grade data pipelines in Databricks, ingesting data via APIs, sFTP, and MCP, and optimizing it for BI consumption.

This role is remote.

Responsibilities

• Develop and execute data ingestions from core IT and business platforms using APIs, MCP, and sFTP.
• Design and maintain Databricks ODS architecture, schemas, and tables using Delta Lake and Medallion models.
• Map source infrastructure to establish technical data lineage from raw logs to the analytics layer.
• Automate orchestration schedules using Databricks Workflows, Apache Airflow, or cron.
• Optimize data layers for performance reporting in Power BI and Tableau.

Required Skills

• 3+ years building production data pipelines in Databricks.
• Proficiency in PySpark, Scala, or Advanced SQL.
• Experience developing custom REST API clients and handling sFTP transfers.
• Strong foundation in data warehouse modeling and schema evolution.
• Experience with operational data stores (ODS).
• Familiarity with Power BI or Tableau for dashboard consumption.

Preferred Skills

• Familiarity with IT Service Management (ITSM) telemetry data.
• Experience with Application Performance Monitoring (APM) tools or platform log schemas.

Key Skills

Education

Bachelor's degree