Description

Design and support Big Data and Data Lake system architectures to enable enterprise data operations, IoT/Smart City projects, and data science applications.

Responsibilities

• Develop Big Data and Data Lake architectures supporting enterprise data warehouses and open data portals.
• Implement data pipelines and ETL processes to ingest data between various systems.
• Integrate data platforms with enterprise tools including Esri, Tableau, MicroStrategy, and Oracle databases.
• Manage the deployment and configuration of data tools and storage on Microsoft Azure.
• Lead project activities to ensure compliance with established technical standards and procedures.

Required Skills

• 5+ years of experience implementing Big Data storage and analytics platforms like Databricks and Data Lakes.
• 5+ years of experience with Big Data and Data Architecture implementation best practices.
• 5+ years of experience with Microsoft Azure architecture, including networking, security, and storage.
• 5+ years of experience querying structured and unstructured data using SQL and NoSQL.
• 5+ years of experience with Apache products including Spark, Sedona, Airflow, Atlas, NiFi, Hive, and Impala.
• 5+ years of experience modeling and ingesting data via data pipelines using Streamsets, Apache NiFi, or Azure Data Factory.
• 3+ years of experience with Web Services (REST/SOAP).
• 3+ years of experience with complex event processing and real-time streaming data.
• 3+ years of experience with Python, Java, or R for ETL and data processing.

Preferred Skills

• 3+ years of experience with Cloudera Data Platform.
• 10+ years of experience with data visualization tools such as Tableau, MicroStrategy, ArcGIS, or Kibana.

Key Skills

Education

ANY GRADUATE