Description

What you’ll do

Apply the knowledge of data characteristics and data supply pattern, develop rules and tracking process to support data quality model. 

Prepare data for analytical use by building data pipelines to gather data from multiple sources and systems.

Integrate, consolidate, cleanse and structure data for use by our clients in our solutions.

Perform design, creation, and interpretation of large and highly complex datasets.

Stay up-to-date with the latest trends and advancements in GCP and related technologies, actively proposing and evaluating new solutions.

Understand best practices for data management, maintenance, reporting and security and use that knowledge to implement improvements in our solutions.
Implement security best practices in pipelines and infrastructure.

Develop and implement data quality checks and troubleshoot data anomalies.

Provide guidance and mentorship to junior data engineers.

Review dataset implementations performed by junior data engineers.

Perform general application development activities, including unit testing, code deployment to development environment and technical documentation.

Work on one or more projects, making contributions to unfamiliar code written by team members.

What experience you need  

BS degree in a STEM major or equivalent discipline; Master’s Degree strongly preferred

3+ years of experience as a data engineer or related role

3+ years of Data engineering experience

Advanced skills using programming languages such as SQL , Spotfire, Looker, Python or Java

SQL and intermediate level experience with scripting languages

3+ years experience with Cloud technology: GCP, AWS, or Azure

3+ years experience designing and developing cloud-native solutions

Experience building and maintaining moderately-complex data pipelines, troubleshooting issues, transforming and entering data into a data pipeline in order for the content to be digested and usable for future projects

Demonstrates advanced Git usage and CI/CD integration skills

What could set you apart

3+ years of hands-on experience in data engineering or a related field.

Expert-level proficiency in SQL, Spotifre, Tableau, Looker or relevant programming languages

Demonstrated experience designing and implementing distributed data processing solutions at scale.

In-depth, hands-on experience with BigData tools such as  Apache Spark or Apache Beam.

Proven experience with Google Cloud Platform (GCP) services for data engineering, including Dataflow, Dataproc, BigQuery, and Pub/Sub .

Experience with workflow orchestration tools like Apache Airflow (or GCP's Cloud Composer).

Solid understanding of data modeling, data warehousing, and ETL/ELT best practices.

Background of Large Data Analysis and building AI ML Models would be a huge plus.

Excellent problem-solving skills and the ability to troubleshoot complex distributed systems.

Effective communication and collaboration skills, with the ability to articulate technical concepts to both technical and non-technical stakeholders

Key Skills

Education

Any Graduate