Description

Job Description :

- Design and develop AI-driven data ingestion frameworks and real-time processing solutions that enhance data analysis and machine learning capabilities across the full technology stack.

- Deploy, maintain, and support application codes and machine learning models in production environments, ensuring seamless integration with front-end and back-end systems.

- Create and enhance AI solutions that facilitate seamless integration and flow of data across the data ecosystem, enabling advanced analytics and insights for end users.

- Conduct business analysis to gather requirements and develop ETL processes, scripts, and machine learning pipelines that meet technical specifications and business needs, utilizing both server-side and client-side technologies.

- Develop real-time data ingestion and stream-analytic solutions utilizing technologies such as Kafka, Apache Spark (SQL, Scala, Java), Python, and cloud platforms to support AI applications. Utilize multiple programming languages and tools, including Python, Spark, Hive, Presto, Java, and JavaScript frameworks (e.g., React, Angular) to build prototypes for AI models and evaluate their effectiveness and feasibility.

- Develop application systems that adhere to standard software development methodologies, ensuring robust design, programming, backup, and recovery processes to deliver high-performance AI solutions across the full stack.

- Provide system support as part of a team rotation, collaborating with other engineers to resolve issues and enhance system performance, including both front-end and back-end components.

- Operationalize open-source AI and data-analytic tools for enterprise-scale applications, ensuring they align with organizational needs and user interfaces.

- Ensure compliance with data governance policies by implementing and validating data lineage, quality checks, and data classification in AI projects.

- Understand and follow the companys software development lifecycle to effectively develop, deploy, and deliver AI solutions.

- Design and develop AI frameworks leveraging open-source tools and advanced data processing frameworks, integrating them with user-facing applications. Lead the design and execution of complex AI projects, ensuring alignment with ethical guidelines and principles under the guidance of senior team members.

Mandatory technical & functional skills :

- Technical Skills : Strong proficiency in Databricks/Fabric/Spark Notebooks, SQL/NoSQL databases, Redis, and other data engineering tools, as well as familiarity with machine learning frameworks (e.g., TensorFlow, PyTorch).

- Strong programming skills on Python, Java, C++ and hands on experience in building backend services with frameworks like FastAPI, LangChain/LlamaIndex /LlamaPrase/LlamaCloud /Semantic Kernel etc.

- Full-Stack Development : Proficiency in front-end and back-end technologies, including JavaScript frameworks (e.g., React, Angular), to build and integrate user interfaces with AI models and data solutions.

- Data Management : Design, implement, and manage AI-driven data solutions on the Microsoft Azure cloud platform, ensuring scalability and performance.

- Data Integration : Develop and maintain data pipelines for AI applications, ensuring efficient data extraction, transformation, and loading (ETL) processes using Azure Data Factory.

- Big Data Processing : Utilize big data technologies such as Azure Databricks and Apache Spark to handle, analyze, and process large datasets for machine learning and AI applications.

Preferred technical & functional skills :

- Develop real-time data ingestion and stream-analytic solutions leveraging technologies such as Kafka, Apache Spark (SQL, Scala, Java), Python and Hadoop Platform and any Cloud Data Platform.

- Certifications : Relevant certifications such as Microsoft Certified : Azure Data Engineer Associate, Azure AI Engineer or any other cloud certification are a plus.

Key behavioral attributes/requirements :

- Collaborative Learning : Open to learning and working with others.

- Project Responsibility : Able to manage project components beyond individual tasks

Key Skills

Education

Any Graduate