Description

You will design, build, and optimize data pipelines and applications for large-scale, real-time, and batch data processing across the enterprise.

Responsibilities

• Design, build, and maintain scalable data pipelines, applications, and services using Python, SQL, or TypeScript.
• Develop and support real-time data streaming solutions using Kafka.
• Utilize AWS cloud services (EMR, Redshift, Lambda, S3, ECS/EKS) and distributed tools like Spark, Hadoop, Hive, MapReduce, and EMR.
• Implement and optimize NoSQL data stores for high-velocity workloads.
• Collaborate in an Agile environment, participating in planning, reviews, and standups.

Required Skills

• 10+ years of experience as a Senior Data Engineer.
• Hands-on experience developing data pipelines using Python or TypeScript and SQL.
• Practical experience with AWS services (E.g., EMR, Redshift, Lambda, S3, ECS/EKS).
• Experience with distributed computing tools including Spark, Hadoop, Hive, MapReduce, and EMR.
• Experience with real-time data streaming platforms, specifically Kafka.
• Experience with data warehousing solutions, including Snowflake.
• Expertise with NoSQL technologies.
• Strong foundation in UNIX/Linux environments with shell scripting.
• Experience working within Agile/Scrum engineering environments.

Key Skills

Education

Any Graduate