Description

Requirements

• 7+ years of experience in software development, with a focus on data engineering and cloud architectures.
• Strong experience with Snowflake, DBT, and Data Pipeline design.
• Expertise in Kafka, Argo, Kubernetes, and ML Flow.
• Deep understanding of AWS services, particularly Lambdas, Step Functions, Serverless, and SageMaker.
• Solid Python development skills for data engineering and backend services.
• Experience with Terraform for infrastructure automation.
• Strong architectural experience in designing scalable, fault-tolerant, and distributed systems.

Nice-to-Have (RoR Focused) Skills:

• Experience with Ruby on Rails (RoR) for API development and backend services.
• Familiarity with Sidekiq, MySQL, Redis, and performance tuning in a RoR environment.
• Understanding of CI/CD pipelines, DevOps best practices, and microservices architecture.

Responsibilities :

• Design and implement scalable, high-performance data architectures in cloud environments (AWS preferred).
• Drive best practices for data pipeline design, integration, and orchestration using technologies like DBT, Kafka, Argo, and Kubernetes.
• Define and implement data governance, security, and compliance best practices.
• Provide technical guidance on integrating MLFlow, SageMaker, and other AI/ML solutions into data pipelines.
• Build and maintain data pipelines using Snowflake, DBT, Kafka, and AWS serverless services (Lambdas, Step Functions).
• Ensure optimal performance, scalability, and cost-effectiveness of data workflows.
• Utilize Terraform for infrastructure as code to manage cloud environments efficiently.
• Work with Kubernetes for deploying, scaling, and managing applications.
• Develop and optimize data-driven applications leveraging Python.
• Collaborate with software teams to integrate data pipelines into RoR applications.
• Implement and optimize background job processing using Sidekiq, Redis, and MySQL.
• Ensure scalability, reliability, and maintainability of services

Key Skills

Education

Any Gradute