Description

You will design, develop, and maintain an enterprise Big Data and Data Lake system architecture to support District-wide data operations, including IoT/Smart City projects and data science applications.

Responsibilities

• Design and implement enterprise Big Data/Data Lake architectures using Databricks and Microsoft Azure.
• Develop and maintain data pipelines and ETL processes using Streamsets, Apache NiFi, and Azure Data Factory.
• Coordinate engineering, maintenance, QA, and risk management for IT projects.
• Enforce documented standards for system design, installation, and modification.
• Troubleshoot system problems, monitor real-time databases, and perform application upgrades.

Required Skills

• 5+ years of experience in big data architecture or enterprise data operations.
• Expertise with Microsoft Azure platform tools, including Azure Data Lake and Synapse.
• Hands-on experience with Databricks.
• Proficiency with Apache platform tools: Hadoop, Hive, Impala, Spark, Sedona, and Airflow.
• Experience with ETL/data pipeline tools: Streamsets, Apache NiFi, and Azure Data Factory.
• Knowledge of integrating data with Oracle databases.
• Experience with Esri, Tableau, MicroStrategy, and API Gateways.
• Ability to manage the full project life-cycle in compliance with organizational standards.
• Bachelor’s degree in Information Technology, a related field, or equivalent experience.

Key Skills

Education

Any Graduated