Description

DataOps builds and manages the automated systems, pipelines, and tools that move and test data inside a company.
Operate, automate, monitor, and continuously improve core data platforms to reduce waste, improve data flow, and ensure maximum uptime for structured datasets and analytics.
Design and optimize robust data pipeline automation for ETL/ELT workloads, including orchestration, scheduling, and CI/CD to streamline data extraction and processing.
Implement deep observability, manage logs, automate health checks, and rigorously track SLAs/SLOs to increase data reliability.
Lead rapid incident response, stakeholder communications, and Root Cause Analysis (RCA) to continuously identify process gaps and correct them.
Manage the end-to-end release lifecycle, executing automated testing (unit, performance, and end-to-end tests), smooth deployments, rollbacks, and performance tuning.
Partner with managed service providers to test and adopt new solutions that adhere to DataOps best practices and ensure contract SLAs are met.
Translate technical metrics into clear executive reports and facilitate collaboration with data and BI teams to enhance the quality of data products.
Strong hands-on experience with AWS, Medallion architecture, and modern DataOps practices. Deep understanding of complex XML handling.
Deep expertise in AWS Infrastructure (EC2, EKS, S3, Glue, Lambda), IAM, and ensuring stringent security standards are applied across all data pipelines.
Proficient in Infrastructure as Code using Terraform and AWS CloudFormation.
Design data engineering assets, develop reusable frameworks and patterns, enforce standards, and leverage AI-assisted coding tools (Codex).
Strong FinOps awareness to monitor, control, and optimize cloud costs across large data environments.
Highly proficient in Python, PySpark, and SQL

Key Skills

Education

Bachelor's degree