Description

Key Skills: Pyspark, Databricks, Python, Natural Language Processing, Ms Sql, AWS

Roles and Responsibilities:

• Design, develop, and implement scalable data engineering solutions using Python, PySpark, and Databricks.
• Architect and build end-to-end data pipelines across cloud platforms such as AWS, Azure, and GCP.
• Develop and integrate NLP and LLM-based solutions, including prompt engineering and AI workflows.
• Work with Snowflake and SQL/NoSQL databases for data storage and processing.
• Build and maintain CI/CD pipelines using tools such as Jenkins, Git, and Airflow.
• Develop internal tools, user-facing applications, and data infrastructure components.
• Monitor data quality and proactively identify and resolve issues before business impact.
• Drive automation, optimization, and continuous improvement of engineering workflows.
• Collaborate with cross-functional teams to gather requirements and deliver solutions.
• Stay updated with emerging technologies in AI, data engineering, and cloud ecosystems.

Skills Required:

• Strong experience in Python, PySpark, and Databricks is required.
• Hands-on expertise in Snowflake and cloud platforms such as AWS, Azure, or GCP is essential.
• Experience in NLP, LLMs, prompt engineering, and tools such as OpenAI and LangChain is required.
• Knowledge of data pipeline orchestration tools like Airflow is expected.
• Experience with CI/CD tools such as Jenkins and version control systems like Git is important.
• Familiarity with NoSQL databases and distributed data systems is beneficial.
• Understanding of DevOps practices and automation is required.
• Strong analytical, problem-solving, and collaboration skills are expected.
• Exposure to healthcare data domains is an added advantage.

Education: Bachelor's or Master's degree in Computer Science, Engineering, or a related field is preferred.

Key Skills

Education

Any Graduate