Description

You will design, develop, and maintain large-scale ETL pipelines and data infrastructure to support complex business domains.

Responsibilities

• Design and develop data/batch processing, data mining, and ETL pipelines for large data domains.
• Create technical detail design documents, including mapping specifications, HLD, and LLD.
• Configure, deploy, and maintain custom ETL infrastructure.
• Collaborate with business analysts to define requirements and present solution alternatives.
• Execute the full software development lifecycle, including coding, unit testing, and defect tracking.

Required Skills

• 6+ years of experience in Data Engineering development.
• Expertise in Python and PySpark for ETL processes.
• Hands-on experience with AWS services including Glue, RDS, S3, Step Functions, Event Bridge, Lambda, MSK (Kafka), and EKS.
• Proficiency in SQL database programming, performance tuning, and relational model analysis.
• Experience with databases such as Postgres, SQL Server, Oracle, and Sybase.
• Strong foundation in data modeling, data warehousing, data mining, and data profiling.
• Experience with CI/CD tools including Bitbucket, GitHub, and Jenkins.
• Working knowledge of Agile/SCRUM methodologies.
• Any Graduate degree.

Preferred Skills

• Experience with Unix scripts, ERWIN, and REST APIs.
• Knowledge of reporting tools like Tableau or Power BI.
• Experience with workload automation tools like Control-M or Autosys.
• Experience with ETL tools such as DataStage, Informatica, or Pentaho.
• Familiarity with Data Science concepts.

Key Skills

Education

Any Graduate