Description

Key Skills:

  • Cloud Platforms: AWS ( Kubernetes), Google Cloud, Microsoft Azure.
  • Data Storage: Data lakes, data warehouses, cloud storage service (S3)
  • Data Pipelines: Developing and maintaining data pipelines for ETL processes leveraging GitHub Actions for promotions of code assets.
  • Programming Languages: Python, Java, Scala
  • Data Management Systems: SQL, NoSQL, Hadoop, Postgress
  • Data Security and Governance: Understanding of data security best practices and compliance regulations.
  • Problem-Solving and Analytical Skills: Ability to troubleshoot issues and analyze data to identify patterns and trends.

Responsibilities

  • Designing and implementing scalable and secure data storage solutions in the cloud, ensuring optimal performance and accessibility.
  • Developing and maintaining robust data pipelines for the ingestion, transformation, and distribution of large datasets.
  • Automating data processes and integrating third-party services
  • Utilizing cloud services and tools to automate data workflows and streamline the data engineering process.
  • Ensuring compliance with data governance and security policies, including data encryption and access controls.
  • Monitoring cloud data systems' performance (Cloud Watch), identifying bottlenecks, and implementing improvements to enhance efficiency.
  • Conducting data quality checks and implementing measures to ensure data accuracy and integrity.
  • Optimizing data retrieval and developing APIs for data consumption by various enterprise consumers.
  • Providing technical expertise and support for data-related issues, including troubleshooting and resolving data pipeline failures.
  • Collaborating with IT and security teams to plan and execute disaster recovery strategies for cloud-based data systems.
  • Documenting data engineering processes, creating data flow diagrams, and maintaining metadata for data lineage and cataloging.
  • Collaborating with architects, analysts, and other engineers to support data modeling, analysis, and reporting needs.
  • Staying current with emerging cloud technologies and data engineering practices to recommend and adopt innovations that improve data systems

Key Skills
Education

Any Graduate

Back To Jobs