Description

Job Description

We’re seeking an enthusiastic, experienced, Cloud Data Engineer.  The ideal candidate must have a DevOps background, strong Python and SQL programming, and have a solid grasp of data warehousing fundamentals.  Beyond technical prowess, the Cloud Data Engineer will need soft skills for clearly communicating to both technical and non-technical colleagues.  The Cloud Data Engineer must be comfortable developing reliable documentation and visualizations. We’re looking for someone who appreciates the importance of a data-driven approach to create optimal outcomes and is comfortable working in a high-speed environment.

Responsibilities

• Builds the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS cloud native technologies.
• Performs ETL/ELT development.
• Creates and maintains optimal data pipeline architecture integrating large, complex data sets that meet functional and non-functional business requirements.
• Works with project stakeholders to create reports and develop extracts for dashboards.
• Identifies, designs, and implements internal process improvements such as automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability.
• Participates in rotating 24x7 production support.
• Works with technical and non-technical business owners to streamline and update business logic and processes as needed.

Required For All Jobs

• Participates in special projects and performs other duties as assigned.

Additional Job Description

Education

• BS or MS degree in Computer Science or a related technical field

Work Experience

• 4+ years of Python required.
• 4+ years of Shell scripting required.
• 4+ years working on relational databases and data warehouses.
• Experience with AWS cloud services such as EC2, RDS, S3, Athena, Redshift, and Dynamo required.
• Experience with Gitlab/Github CI/CD required.
• Experience with data pipeline and workflow management tools such as AWS Glue and Apache Airflow required.
• Candidates with demonstrable Snowflake experience strongly preferred.
• Candidates with demonstrable Apache Iceberg experience strongly preferred.
• Candidates with demonstrable experience with stream-processing systems including PySpark, Storm, and Spark-Streaming strongly preferred.
• Candidates with previous 24x7 production support experience preferred.
• Strong proficiency to write complex SQL to perform common types of data extraction and manipulation required

Key Skills

Education

Any Graduate