Description

You will scale data processing flows to handle rapid data growth and evolve data models and schemas to meet engineering needs.

Responsibilities

• Scale data processing flows to accommodate rapid data growth.
• Evolve data models and schemas based on business and engineering requirements.
• Implement systems to track data quality and consistency.
• Develop tools to support self-service ETL pipeline management.
• Tune SQL and MapReduce jobs to improve processing performance.

Required Skills

• 5+ years of professional data engineering experience.
• Experience with the Hadoop ecosystem, including MapReduce, Yarn, HDFS, Hive, Spark, Presto, Pig, HBase, or Parquet.
• Proficiency in at least one SQL language: MySQL, PostgreSQL, SQL Server, or Oracle.
• Deep understanding of SQL engines and advanced performance tuning.
• Strong scripting skills in Python, Ruby, or Bash.
• 1+ years of experience with workflow management tools such as Airflow, Oozie, Azkaban, or UC4.

Key Skills

Education

Not specified