Description

You will build and manage data pipelines and automated workflows within an AWS environment. You participate in all SDLC phases, from requirements gathering to development. You configure Jenkins to automate AWS pipeline infrastructure and deploy AWS Lambda functions for workflow automation. You build PySpark applications for data extraction and transformation. You perform testing and debugging to ensure application quality.

Responsibilities

• Build and manage data pipelines and automated workflows in AWS.
• Participate in all SDLC phases, including modeling, analysis, design, and development.
• Configure and deploy Jenkins functions to automate AWS pipeline infrastructure.
• Develop and deploy AWS Lambda functions to automate workflows.
• Perform testing and debugging using test cases to ensure quality.

Required Skills

• 5+ years of experience in data engineering roles.
• Proficiency with PySpark for data processing.
• Experience developing and deploying AWS Lambda functions.
• Hands-on experience with AWS pipeline infrastructure.
• Experience using Jenkins for automation.
• Familiarity with JIRA and Confluence.

Key Skills

Education

Any Graduate