Description

You will design, develop, and maintain scalable data pipelines using Python and Big Data frameworks to support governance initiatives. You will architect and implement AWS cloud-based solutions for secure data storage, processing, and analytics.

Responsibilities

• Automate infrastructure deployment and management using Terraform for efficient cloud operations.
• Develop and maintain CI/CD pipelines using Jenkins for automated testing and deployment processes.
• Collaborate with data stewards and compliance officers to deliver compliant data solutions.
• Troubleshoot and optimize data workflows and cloud infrastructure for performance and reliability.
• Ensure best practices in code quality, data security, privacy, and regulatory compliance standards.

Required Skills

• 5+ years of experience in data engineering, cloud automation, or DevOps.
• Strong proficiency in Python for data pipeline development.
• Hands-on experience with AWS cloud services for storage and analytics.
• Practical knowledge of Spark and Hadoop for Big Data processing.
• Experience with Terraform for infrastructure as code.
• Proficiency in Jenkins for CI/CD pipeline management.
• Understanding of data governance, data quality, or data compliance teams.
• Formal training or certification on software engineering concepts.

Preferred Skills

• Experience or interest in agent-based AI systems or autonomous agents.

Key Skills

Education

Any Graduate