Description

You will design and optimize large-scale data architectures and pipelines to extract value from disconnected datasets.

Responsibilities

• Build and maintain processes for data transformation, metadata management, and workload dependency.
• Develop and optimize big data pipelines, architectures, and datasets.
• Perform root cause analysis on internal and external data processes to identify improvements.
• Implement message queuing, stream processing, and scalable big data storage solutions.
• Collaborate with cross-functional teams to solve complex data problems.

Required Skills

• 5+ years of experience in a Data Engineering role.
• Graduate degree in Computer Science, Statistics, Informatics, Information Systems, or a quantitative field.
• Advanced SQL knowledge and experience with relational databases and query authoring.
• Experience with Hadoop, Apache Spark, and Kafka.
• Proficiency in Python, Java, Scala, or Spark scripting.
• Hands-on experience with Snowflake and NoSQL databases including Postgres and Cassandra.
• Experience with AWS services including S3, EC2, EMR, RDS, and Redshift.
• Experience with stream-processing systems such as Storm or Spark-Streaming.
• Ability to manipulate and process unstructured datasets.

Key Skills

Education

Any Graduate