Description

Key Responsibilities:

Design, develop, and maintain ETL processes for real-time and batch data processing.

Build and support database systems, data pipelines, and analytics tools.

Integrate data from multiple sources to support business reporting and decision-making.

Troubleshoot data issues related to consistency, quality, integrity, and performance.

Translate business requirements into technical design, code, reports, and data solutions.

Develop Spark-based data processing applications using Python and PySpark.

Write complex SQL queries, ad hoc reports, and reusable data scripts.

Build and maintain reporting solutions using Tableau.

Work with AWS cloud services such as ECS, Glue, S3, Glue Crawler, Redshift, Step Functions, IAM, CloudWatch, CloudFormation, Lambda, Secrets Manager, and SageMaker.

Participate in all phases of the SDLC, including design, development, testing, deployment, and support.

Follow test automation and test-driven development practices.

Use CI/CD tools such as GitHub for version control and deployment support.

Document code, data flows, process designs, and technical solutions clearly.

Review deliverables with business users to ensure accuracy and usability.

Provide data analysis support and technical guidance to business users and team members.

Required Skills:

Strong experience as a Data Engineer.

Hands-on experience with ETL design and development.

Strong experience with Python and PySpark.

Expertise in Spark data processing applications.

Strong SQL development experience.

Experience with AWS cloud data services, including S3, Glue, Redshift, Lambda, IAM, CloudWatch, CloudFormation, Step Functions, Secrets Manager, and related services.

Experience with Tableau reporting.

Good understanding of data quality, data integrity, and data troubleshooting.

Experience with CI/CD tools such as GitHub.

Experience with test automation or test-driven development.

Strong documentation and communication skills.

Preferred Skills:

Experience with SageMaker or cloud-based analytics platforms.

Experience working with large-scale enterprise data environments.

Experience supporting both real-time and batch analytics solutions.

Ability to explain technical concepts clearly to business stakeholders.

Preferred Qualifications:

Bachelor’s degree in Computer Science, Information Technology, Data Engineering, or a related field.

Experience working in Agile development environments.

Ability to work independently and collaborate with cross-functional teams

Key Skills

Education

Bachelor's degree