Description

Lead the design and implementation of large-scale data pipelines and distributed systems within AWS environments.

Responsibilities

• Build and deploy data applications and services on AWS infrastructure.
• Discover patterns in large datasets using data mining and ETL tools.
• Create and manage databases across various storage technologies.
• Document processes and develop database reporting for data discovery and decision-making.
• Collaborate within Agile frameworks like Scrum or Kanban to deliver features.

Required Skills

• 4+ years of experience with Big Data Hadoop clusters (HDFS, Yarn, Hive, MapReduce).
• 4+ years of experience building and deploying applications in AWS (S3, Glue, AWS Batch, DynamoDB, Redshift, EMR, Cloudwatch, RDS, Lambda, SNS, SQS).
• 4+ years of proficiency in Python, SQL, SparkSQL, and PySpark.
• Hands-on experience with Spark and AWS EMR.
• Experience managing MySQL, Hadoop, or MongoDB.
• Proficiency in data mining or ETL software such as Oracle Data Mining or Informatica.
• Ability to perform reporting using Crystal Reports, Excel, or SSRS.

Preferred Skills

• Bachelor's degree or equivalent experience.

Key Skills

Education

Bachelor degree