Description

You will design and implement data modeling, ingestion, and processing pipelines for various datasets.

This role is on-site.

Responsibilities

• Design, develop, and maintain ETL frameworks for new data sources.
• Build data ingestion pipelines using AWS Glue, EMR, PySpark, Python, and Databricks.
• Execute ad-hoc data ingestion to support business analytics and reporting.
• Evaluate and integrate new tools and services to meet evolving business requirements.

Required Skills

• 5+ years of experience in ETL and data engineering using Python, AWS Glue, AWS EMR, PySpark, and Airflow.
• 5+ years of experience with AWS services including VPC, S3, EC2, Redshift, RDS, Athena, IAM, Glue, DMS, and Lambda.
• 5+ years of experience in SQL, Python, source control (Bitbucket), and CI/CD practices.
• 2+ years of experience with Databricks, including Unity Catalog and job workflow orchestration.
• Experience with MPP systems such as AWS Redshift, AWS EMR, and Databricks SQL warehouse.
• Proficiency with PostgreSQL, SQL Server, MySQL, and Oracle databases.
• Experience in distributed programming with Python, Unix scripting, and Spark/PySpark.

Preferred Skills

• Experience with big data processing using Sqoop, Spark, or Hive.
• Knowledge of data visualization tools such as PowerBI or Tableau.

Key Skills

Education

Any Graduate