Description

You will build and maintain scalable data ingestion and transformation pipelines using Azure PaaS, Databricks, and Delta Lake. You own the end-to-end reliability of data models supporting semantic analytics, reporting, and executive dashboards.

Responsibilities

• Engineer integrations for ServiceNow operational datasets and ApptioOne financial data.
• Implement data quality checks, validation rules, and monitoring for pipeline reliability.
• Apply Unity Catalog governance controls for data access, lineage, and schema enforcement.
• Optimize pipeline performance, storage layouts, and query efficiency within the Databricks Lakehouse.
• Support CI/CD pipelines and DevOps automation using Azure DevOps and GitHub Actions.

Required Skills

• 5+ years of experience in Data Engineering or Analytics Engineering.
• Hands-on experience with Databricks, Delta Lake, and Spark-based data pipelines.
• Strong understanding of Medallion Architecture, particularly Gold/Platinum layer implementation.
• Proficiency in Python, SQL, and Spark (PySpark or SQL).
• Experience integrating enterprise systems such as ServiceNow (SLA, incident, CMDB data).
• Experience working with financial or cost management data (e.g., ApptioOne or equivalent ITFM tools).
• Experience with data modeling methodologies and tools.
• Familiarity with Unity Catalog concepts for data governance and access control.
• Experience with Power BI or similar BI tools consuming curated Lakehouse datasets.

Key Skills

Education

Any Gradute