Description

•  DBA Development Develop ETL pipelines using Python and PySpark to extract, transform and load data from various sources.
•  Build configuration-driven ETL jobs using YAML files for easy maintenance and deployment.
•  Write Python scripts for data transformations and implement business logic.
•  Create reusable components and maintain code libraries.
•  Database Work Extract data from SQL Server databases using SQL queries.
•  Load and manage data in MongoDB collections.
•  Write complex SQL queries for data extraction and validation.
•  Optimize database queries and improve performance.
•  Design data models for both relational and NoSQL databases Cloud & Deployment Deploy ETL applications on Azure Kubernetes Service.
•  Create and maintain Docker containers for ETL jobs.
•  Write Kubernetes YAML files for deployments and configurations.
•  Set up CI/CD pipelines for automated deployments.
•  Monitor pipeline performance and troubleshoot production issues.
•  General Work with business analysts and architects to understand requirements.
•  Document ETL processes and data flows.
•  Participate in code reviews Provide production support when needed

Key Skills

Education

Any Gradute