Description

You will design and optimize large-scale data architectures and pipelines to extract value from disconnected datasets. You will own the full lifecycle of data transformation, metadata management, and workload dependency.

Responsibilities

• Build and maintain robust data pipelines, architectures, and datasets.
• Implement message queuing, stream processing, and scalable storage solutions.
• Perform root cause analysis on internal and external data processes to identify improvements.
• Collaborate with cross-functional teams to solve complex data problems.
• Process and manipulate unstructured datasets effectively.

Required Skills

• 5+ years of experience in a Data Engineering role.
• Advanced SQL knowledge and experience with relational databases and query authoring.
• Experience with Hadoop, Apache Spark, and Kafka.
• Proficiency in Python, Java, or Scala scripting.
• Hands-on experience with Snowflake and NoSQL databases including Postgres and Cassandra.
• Experience with AWS services including S3, EC2, EMR, RDS, and Redshift.
• Experience with stream-processing systems such as Storm or Spark-Streaming.
• Graduate degree in Computer Science, Statistics, Informatics, Information Systems, or a quantitative field.

Key Skills

Education

Any Graduate