Description

Design and maintain scalable data architecture and cloud strategy on AWS, leading data engineering initiatives and infrastructure automation.

This role is remote.

Responsibilities

• Architect and manage cloud-based data infrastructure using AWS services (S3, Glue, EMR, Redshift, Lambda, Kinesis, MWAA) to ensure scalability and cost-efficiency.
• Lead data engineering teams in developing, maintaining, and optimizing ETL processes for data ingestion, transformation, and loading.
• Design and implement real-time data processing solutions using Apache Kafka and AWS Kinesis, alongside batch processing pipelines.
• Configure workflow orchestration using Apache Airflow to automate ETL processes, monitoring job scheduling, error handling, and performance.
• Implement data security protocols, access controls, and encryption to safeguard sensitive data and ensure compliance with privacy regulations.

Required Skills

• 6+ years of proven experience in data engineering, cloud architecture, and AWS services.
• Strong expertise in SQL, Python, and PySpark for data processing and transformation.
• Extensive knowledge of data lakehouse technologies, including Hudi, DBT, Airbyte, Redshift, Glue, Kinesis, and Apache Airflow.
• Experience with data integration solutions for API ingestion and diverse data sources.
• Proficiency in infrastructure automation using tools like Terraform, CloudFormation, and Jenkins.
• Ability to guide cross-functional teams and collaborate with data scientists and business stakeholders.

Preferred Skills

• Bachelor's or higher degree in a relevant field.
• Passion for Sports/Gaming/Entertainment.

Key Skills

Education

Bachelor's degree