Description

Design, build, and maintain data ingestion and processing pipelines on AWS. You will own the migration of legacy Talend workflows to modern frameworks and ensure reliable data delivery for analytics.

Responsibilities

• Design, develop, and maintain ETL frameworks for new data sources.
• Migrate existing Talend ETL workflows to AWS Glue, EMR, PySpark, or Python-based pipelines.
• Execute ad-hoc data ingestion to support business analytics and reporting.
• Evaluate tools and services to meet business requirements and write technical specifications.

Required Skills

• 5+ years of experience with cloud platforms (AWS, Azure, or GCP).
• 5+ years with AWS services: VPC, S3, EC2, Redshift, RDS, EMR, Athena, IAM, Glue, DMS, Data Pipeline, and API.
• 5+ years in ETL and data engineering using Python, AWS Glue, AWS EMR, PySpark, and Talend.
• 5+ years with SQL, Python, and source control systems.
• Proficiency in PostgreSQL, SQLServer, MySQL, and Oracle databases.
• Experience with MPP systems such as AWS Redshift and EMR.
• Experience in distributed programming using Python, Unix Scripting, MPP, and RDBMS.
• Experience building distributed systems using Spark, PySpark, and AWS Glue for loading and streaming data.
• Experience working within Agile methodology.

Preferred Skills

• Experience with analytic tools such as Tableau or Qlik.
• Experience with big data processing using Sqoop, Spark, or Hive.

Key Skills

Education

Bachelor's degree in Computer Science