Description

You will design, develop, and maintain scalable data ingestion pipelines and ETL workflows.

Responsibilities

• Build and migrate data ingestion processes from legacy systems to cloud-based frameworks like IICS.
• Develop and optimize Snowflake SQL stored procedures and Python modules to improve pipeline performance.
• Design orchestration workflows using Control-M and AutoSys to automate ETL processes and shell scripts.
• Implement API-driven data ingestion pipelines and manage unstructured data loading.
• Troubleshoot production issues and manage deployment activities including ServiceNow change tickets and Jenkins pipelines.

Required Skills

• 5+ years of experience in data engineering roles.
• Proficiency in Python and PySpark for ETL development.
• Hands-on experience with Snowflake, including stored procedure development.
• Strong knowledge of AWS services, specifically EC2, S3, and Step Functions.
• Experience with Hadoop environments and managing log files.
• Competency in ETL tools such as Talend and Informatica Intelligent Cloud Services (IICS).
• Familiarity with Splunk for data dashboarding and monitoring.
• Experience using EMR and EKS within cloud environments.
• Knowledge of object-oriented design, analysis, and testing methodologies.

Preferred Skills

• Experience with Jenkins for automated code deployment.
• Background working in Agile environments with direct client interaction.

Key Skills

Education

Any Gradute