Description

You will design and implement data pipelines and warehousing solutions as a consultant.

Responsibilities

• Develop and implement ETL jobs for data warehouses using SQL and Python.
• Build and manage data lakes, star schemas, and data warehouse architectures.
• Provision data APIs and manage cloud-based storage and functions.
• Implement CI/CD pipelines and manage infrastructure via code.
• Articulate complex technical concepts to non-technical stakeholders.

Required Skills

• 5+ years of experience in data engineering.
• Expertise in SQL and Python for ETL development.
• Hands-on experience with Spark.
• Proficiency with AWS tools including Glue, Athena, Redshift, and Kinesis.
• Experience with Azure Big Data tools and Azure Analytics.
• Experience with Databricks.
• Knowledge of cloud storage and functions (S3, Blob, Lambda, Azure Functions).
• Experience with Infrastructure as Code using AWS CloudFormation or Terraform.
• Competency with Git, Bitbucket, and Jira.
• Understanding of CI/CD concepts and AWS CodePipeline, CodeBuild, and CodeDeploy.
• Knowledge of information security concepts and terminology.

Preferred Skills

• Familiarity with the Gartner ODM framework.

Key Skills

Education

Any Graduate