Description

You will build and document automated data pipelines designed for scale and high volume.

Responsibilities

• Build automated data pipelines from diverse sources with an emphasis on scale.
• Handle large streaming datasets and perform initial data pattern identification using AWS Kinesis.
• Develop highly available applications and REST APIs using Swagger to support near-real-time integrations.
• Scale prototypes into production using AWS Event Bridge, SQS, SNS, and Confluent Kafka.
• Contribute to architecture and design patterns for storing and processing unstructured and semi-structured data.
• Author software design specifications based on product requirements and user stories within an agile cadence.

Required Skills

• 3+ years of experience in a data engineering role.
• 2+ years of experience with AWS services including EC2, S3, SNS, Lambda, and IAM.
• Proficiency in Python and ANSI SQL.
• Experience with RDBMS platforms such as MySQL, Postgres, or MS SQL Server.
• Experience with NoSQL databases.
• Experience with Snowflake and AWS S3, DynamoDB, or RDS.
• Knowledge of API development and integration.
• Bachelor's degree in Computer Science, Informatics, or a related field.

Preferred Skills

• Hands-on experience with ETL tools and techniques.

Key Skills

Education

Bachelors; degree in Computer Science, Informatics