Description

Design and develop big data applications, manage large-scale data products, and lead data engineering initiatives for assigned business domains.

This role is on-site.

Responsibilities

• Design and develop logical and physical data models for big data platforms.
• Create and maintain data pipelines using Apache Hive, Apache Spark, Scala, and Apache Kafka.
• Automate workflows and orchestrate data processing using Apache Airflow.
• Mentor junior engineers and serve as the primary technical point of contact for specific business domains.
• Maintain existing systems and participate in rotational on-call support.

Required Skills

• 12+ years of total professional experience in data engineering.
• 8+ years of hands-on experience with Spark, Scala, Hive, SQL, and ETL processes/data pipelines.
• 5+ years of experience with Google Cloud Platform (GCP), specifically GCS, Dataproc, and BigQuery.
• 4+ years of experience developing distributed data processing platforms using Hadoop, Hive, Scala, or Airflow.
• 4+ years of experience developing data warehouse solutions and data products.
• 2+ years of experience in data modeling (Erwin) and designing schemas for data lakes or RDBMS platforms.
• Proficiency in Python, Java, and Scala.
• Experience with scripting languages such as Perl and Shell.
• Proven track record managing multi-TB/PB scale datasets.

Preferred Skills

• Experience with GitFlow and Atlassian products (BitBucket, JIRA, Confluence).
• Familiarity with CI tools such as Bamboo, Jenkins, or TFS.
• Exposure to test-driven development and automated testing frameworks.

Key Skills

Education

ANY GRADUATE