Description

Key Skills:

• Cloud Platforms: AWS ( Kubernetes), Google Cloud, Microsoft Azure.
• Data Storage: Data lakes, data warehouses, cloud storage service (S3)
• Data Pipelines: Developing and maintaining data pipelines for ETL processes leveraging GitHub Actions for promotions of code assets.
• Programming Languages: Python, Java, Scala
• Data Management Systems: SQL, NoSQL, Hadoop, Postgress
• Data Security and Governance: Understanding of data security best practices and compliance regulations.
• Problem-Solving and Analytical Skills: Ability to troubleshoot issues and analyze data to identify patterns and trends.

Responsibilities

• Designing and implementing scalable and secure data storage solutions in the cloud, ensuring optimal performance and accessibility.
• Developing and maintaining robust data pipelines for the ingestion, transformation, and distribution of large datasets.
• Automating data processes and integrating third-party services
• Utilizing cloud services and tools to automate data workflows and streamline the data engineering process.
• Ensuring compliance with data governance and security policies, including data encryption and access controls.
• Monitoring cloud data systems' performance (Cloud Watch), identifying bottlenecks, and implementing improvements to enhance efficiency.
• Conducting data quality checks and implementing measures to ensure data accuracy and integrity.
• Optimizing data retrieval and developing APIs for data consumption by various enterprise consumers.
• Providing technical expertise and support for data-related issues, including troubleshooting and resolving data pipeline failures.
• Collaborating with IT and security teams to plan and execute disaster recovery strategies for cloud-based data systems.
• Documenting data engineering processes, creating data flow diagrams, and maintaining metadata for data lineage and cataloging.
• Collaborating with architects, analysts, and other engineers to support data modeling, analysis, and reporting needs.
• Staying current with emerging cloud technologies and data engineering practices to recommend and adopt innovations that improve data systems

Key Skills

Education

Any Graduate