Description

You will build and maintain data pipelines to support AI lab initiatives.

This role is remote.

Responsibilities

• Build data pipelines sourcing data from internal and third-party sources.
• Design effective data solutions that meet specific business needs.
• Collaborate with cross-functional teams to deliver data for downstream consumption.
• Manage timelines, milestones, and delivery to ensure initiative value.

Required Skills

• 5+ years of experience in enterprise IT applications within cloud and big data environments.
• Expert proficiency in Python and SQL.
• Experience with Azure Databricks, Google Cloud Platform, or AWS.
• Building data pipelines using batch processing with Apache Spark (Spark SQL, Dataset/Dataframe API) or Hive Query Language (HQL).
• Experience with Hive and Hadoop file formats (Avro, Parquet, ORC).
• Knowledge of Big Data ETL processing tools.
• Familiarity with relational (SQL Server, Oracle, DB2, Netezza) and NoSQL/document databases.
• Ability to debug, fine-tune, and optimize large-scale data processing jobs.

Preferred Skills

• Basic understanding of CI/CD tools like Jenkins, Jira, Bitbucket, or Azure DevOps.

Key Skills

Education

Any Graduate