Data Engineer

Responsibilities

• Will drive balanced, scalable, and automated solutions for our Performance, Power, and Stability data analysis capability.
• Optimize data processes & transformations to improve efficacy and efficiency of both staff and compute resources, defining and publishing critical metrics where appropriate.
• Assist multi-functional technical teams with targeted ad hoc analyses methodologies.
• Collaborate with other specialists on common program test & debug activities and process improvements.
• Engage with multi-functional leadership & teams to integrate your results with other project work streams.

Required Skills

• Strong Python engineering skills, programming skills, Restful API’s.
• Ability to create clean, easy to maintain, modify and support code.
• Expertise in creating and automated unit test cases.
• Understanding the pre-processing analytical methods.
• Knowledge of at least any one of the leading cloud offerings (GCP / AWS).
• Strong expertise in Unix shell scripting.
• Expertise with ETL tools, specifically DataStage.
• Proficient in Python best practices, logging, and strong debugging skills.
• Data querying and storage including SQL, Parquet, XML, JSON.
• Understanding of Python environments, package management.
• Strong SQL knowledge, able to read and understand SQL.
• Familiar with git source control, CI/CD, agile development practices.
• Able to work independently and ask for guidance when needed.
• Strong communication skills and delivery focused.
• Must be a self-starter and self-motivated.

Required Experience

• 8+ years of overall experience in data warehousing and data lakes.
• Experience with automated testing systems.
• Experience in distributed computing frameworks such as Spark.
• Experience with SQL and NoSQL systems.
• Experience working in a fast-paced matrixed technical organization and multi-site environment.
• Experience working with CI/CD, containerization, and virtualization tools such as Gitlab, Jenkins, Kubernetes, Docker.

Education Requirements

• Bachelor’s Degree in Computer Science, Computer Engineering or a closely related field.