Description

Key Skills: Redshift, SQL, Pyspark, AWS, Airflow, Python, Data Catalog, Terraform, ETL, Data quality, Data Warehouse, Data Governance, Apache Spark

Roles and Responsibilities:

• Design, develop and maintain data ingestion and transformation pipelines using AWS, Python, PySpark and SQL.
• Build data integration solutions for database sources, delimited files and other enterprise data feeds.
• Develop and maintain analytical data structures and loading processes for Amazon Redshift.
• Use Apache Airflow to schedule, orchestrate and manage production data workflows.
• Assess source data for reliability, completeness and usability while supporting standardized enterprise data deliverables.

Skills Required:

• 2-4 years of relevant experience in data engineering, cloud data platforms or enterprise data integration.
• Hands-on experience with AWS cloud services and cloud-based data engineering solutions.
• Strong programming experience in Python and PySpark, with advanced SQL capability.
• Experience designing data pipelines and transformation workflows for analytical platforms, including Amazon Redshift.
• Practical experience with Apache Airflow for workflow orchestration and production pipeline management.

Good to Have:

• Experience with Amazon S3, AWS Glue, AWS Glue Data Catalog, ETL, data pipelines and data warehousing.
• Exposure to change data capture, event streaming, Apache Kafka, Apache Parquet or Apache Iceberg.
• Knowledge of data governance, data quality, Terraform and Git.

Education: B.E. or B.Tech degree in Computer Science, Information Technology, Engineering or a related discipline

Key Skills

Education

Any Graduate