Description

You will lead data development efforts focusing on large-scale pipelines and AI-augmented engineering workflows.

Responsibilities

• Build and maintain complex data pipelines using Python and PySpark.
• Optimize data processing performance across relational and NoSQL databases.
• Integrate GenAI and LLMs into all phases of the SDLC to augment development.
• Troubleshoot microservices and API interactions using logging and monitoring tools like Kibana.
• Manage data orchestration and streaming workflows.

Required Skills

• 5+ years of experience in data engineering or development.
• Advanced SQL proficiency.
• Expertise in Python and PySpark.
• Experience with Relational databases such as Oracle or Postgres.
• Experience with NoSQL databases including MongoDB or Cassandra.
• Hands-on with at least two or three of these tools: Spark, Databricks, Hadoop, Kafka, Airflow, or Apache NiFi.
• Experience with at least one cloud platform: AWS, Azure, or GCP.
• Proficiency in AI-augmented development using Copilot and other LLMs.
• Practical knowledge of CI/CD, Git, and data pipeline architecture.

Preferred Skills

• Scala programming experience.
• Reporting experience with PowerBI or Tableau.

Key Skills

Education

Any Gradute