Description

Own the design, migration, and optimization of scalable data platforms and pipelines on AWS.

This role is hybrid.

Responsibilities

• Design and maintain scalable data pipelines using AWS services (EMR, S3, Redshift, Glue, EC2, VPC, RDS) and Big Data technologies.
• Lead data platform migration initiatives from on-premises environments to AWS cloud infrastructure.
• Develop, manage, and optimize ETL/ELT processes using SSIS, Pentaho, or similar tools, ensuring data integrity and performance.
• Build and orchestrate data workflows using Airflow and optimize SQL queries for scalability across SQL Server, PostgreSQL, Teradata, and Netezza.
• Implement real-time streaming solutions with Kafka and Spark Streaming, monitor system performance, and troubleshoot production issues.

Required Skills

• 10+ years of experience in Data Engineering, Big Data, or Platform Engineering.
• Strong hands-on experience with AWS Cloud services, including EMR, EC2, S3, Redshift, Glue, IAM, CloudWatch, and CloudFormation.
• Proficiency in SQL-based systems (SQL Server, PostgreSQL) and data warehousing concepts.
• Experience with ETL tools such as SSIS and Pentaho.
• Expertise in query optimization, performance tuning, and data migration projects (e.g., Hadoop to AWS).
• Strong understanding of Unix/Linux environments and shell scripting.
• Bachelor’s degree in Computer Science, Engineering, or a related field.

Preferred Skills

• Experience with real-time streaming technologies like Kafka and Spark Streaming.
• Knowledge of DevOps tools, CI/CD pipelines, and platform engineering frameworks.
• Familiarity with data lake architecture, governance, and security frameworks.

Key Skills

Education

Bachelor's degree