Description

You will own the automation strategy for the data platform, building scalable infrastructure-as-code modules and self-service workflows.

This role is on-site.

Responsibilities

• Develop and maintain Terraform modules for Azure and GCP infrastructure to support data platform scalability.
• Build and maintain CI/CD pipelines using GitHub Actions for deploying infrastructure, scripts, and configuration changes.
• Automate ServiceNow request workflows for data platform access, onboarding, approvals, and operational fulfillment.
• Create automated backup and restore scripts for Databricks Unity Catalog, Dremio metadata, and ADLS objects.
• Optimize Databricks pipelines for performance and cost while mentoring team members on engineering best practices.

Required Skills

• 5+ years of experience with Python for production-ready automation and scripting.
• Strong expertise in Databricks workspaces, Unity Catalog, jobs, notebooks, and cluster management.
• Proficiency with Terraform for infrastructure provisioning on Azure or GCP.
• Experience integrating with ServiceNow APIs for ticket creation and workflow automation.
• Knowledge of Azure services including ADLS, REST APIs, and CI/CD concepts.
• Scripting capabilities in Bash and PowerShell for system administration and automation.
• Ability to debug post-production issues and design robust, reusable engineering patterns.

Preferred Skills

• Experience with Azure DevOps, Azure Functions, Logic Apps, and Event Grid.
• Knowledge of Kubernetes (AKS/GKE), Helm, and YAML-based deployments.
• Familiarity with Dremio REST APIs, Delta Lake, and cost/performance optimization.

Key Skills

Education

Any Graduate