Description

5+ years of experience with SQL, data engineering & data modeling techniques 

Strong experience building out data warehouse and/or data lake. 

1+ year of experience with Snowflake 

Experience leading full lifecycle, large, complex reporting, or data engineering efforts 

Strong experience in working with heterogeneous datasets in building and optimizing data pipelines, pipeline architectures and integrated datasets using various data integration technologies (ETL/ELT, data replication/CDC, message-oriented data movement, API design, etc.) 

Experience with DevOps, CI/CD pipelines and automated testing 

Experience with any scripting languages, preferably Python 

Must be self-motivated and able to work with minimal supervision. 

Must possess strong analytical, problem solving and root cause analysis skills. 

Good communicator and coordinator 

Working knowledge of data security practices, including encryption, anonymization, and masking 

Demonstrated familiarity with modern data architectures, including data lake house, data warehouse, and data lake. 

Proficient in data pipeline orchestration techniques for efficient data flow management 

Experience in metadata management and employing metadata-driven design principles to enhance data usability and governance. 

Strong understanding of version control systems for maintaining code integrity and collaboration 

Knowledge of infrastructure considerations for scaling data pipelines within distributed computing environments 

 

Competencies-Skills (Preferred):

Experience working with Snowflake, Databricks (or Spark) & Qlik technologies (Replicate and Compose) 

Experience working using agile methodologies. 

Experience with .NET required, Nice to have > Python, React or Node.js. Java

AI Services in Cloud (AI Foundry project would be a value, if the developer has prior experience working designing or developing custom AI Apps in Azure, rare to find)

Designing Agent Apps using Microsoft Agent Framework or Lang graph or Lang Chain or other similar open-source frameworks. We will prefer to use MS Agent Framework (.NET and Python supported language for development)

Experience in Manufacturing or Agriculture industry 

Knowledge of the data vault method, model, and architecture 

Ability to think innovatively to implement creative solutions to problems. 

Experience with machine learning operations  

Experience building data integration solutions using tools like such as Talend, Informatica, Mulesoft, Qlik, etc

Education

Bachelor's degree