Description

You will design, develop, and maintain custom ETL infrastructure and data pipelines for large data domains.

Responsibilities

• Design, develop, and implement ETL pipelines, data mining, and batch processing workflows using Python and PySpark.
• Create technical detail design documents, including mapping specifications, HLD, and LLD.
• Collaborate with developers and business analysts to translate user requirements into technical solutions.
• Perform quality analysis, defect tracking, and classification throughout the software development lifecycle.
• Manage risk and resolve technical barriers to ensure project delivery and production support.

Required Skills

• 5+ years of experience in Data Engineering.
• Hands-on development experience using Python and PySpark for ETL processes.
• Proficiency with AWS services including Glue, Lambda, MSK (Kafka), S3, Step Functions, RDS, and EKS.
• Strong SQL skills including performance tuning, stored procedures, views, functions, and triggers.
• Experience with relational databases such as Postgres, SQL Server, Oracle, and Sybase.
• Proficiency in developing UNIX scripts and Oracle SQL/PL-SQL.
• Experience working with REST APIs and workload automation tools like Control-M or Autosys.
• Practical knowledge of CI/CD DevOps tools including Bitbucket, GitHub, and Jenkins.
• Experience working within Agile/SCRUM methodologies.

Preferred Skills

• Experience with ETL tools such as DataStage, Informatica, or Pentaho.
• Knowledge of MDM, Data Warehousing, and Data Analytics.

Key Skills

Education

Any Graduate