Description

Key Responsibilities

Design, develop, and maintain scalable data pipelines using AWS services.

Build and optimize ETL/ELT workflows to process structured and unstructured data.

Develop data ingestion, transformation, and integration solutions from multiple sources.

Work with AWS services such as Amazon S3, AWS Glue, Amazon Redshift, AWS Lambda, and Amazon CloudWatch.

Develop efficient data processing solutions using Python, SQL, and other relevant technologies.

Implement data quality checks, validation, monitoring, and error-handling mechanisms.

Optimize data pipelines and database queries for performance, scalability, and cost efficiency.

Work with data lakes, data warehouses, and cloud-based storage solutions.

Collaborate with data analysts, data scientists, architects, and business stakeholders to understand data requirements.

Implement data security, access controls, encryption, and governance best practices.

Troubleshoot production issues and ensure the reliability and availability of data pipelines.

Maintain technical documentation and follow established development and deployment standards.

Required Skills and Qualifications

Strong hands-on experience in AWS data engineering and cloud-based data solutions.

Proficiency in Python and SQL.

Experience building ETL/ELT pipelines and data integration workflows.

Hands-on experience with AWS services such as:

Amazon S3

AWS Glue

Amazon Redshift

AWS Lambda

Amazon CloudWatch

Experience with data warehousing, data lakes, and large-scale data processing.

Strong understanding of data modeling, database design, and query optimization.

Experience with data quality, data validation, and pipeline monitoring.

Knowledge of cloud security, IAM roles, permissions, and data protection practices.

Strong analytical, troubleshooting, and problem-solving skills.

Ability to work effectively in a collaborative team environment.

Preferred Qualifications

Experience with Apache Spark or PySpark.

Knowledge of AWS Step Functions, Amazon Kinesis, or Amazon EMR.

Experience with orchestration tools such as Apache Airflow.

Familiarity with CI/CD pipelines, Git, and infrastructure automation.

Experience working in Agile development environments.

Knowledge of financial services data and enterprise data governance practices

Key Skills

Education

Bachelor's degree