← Back to jobs

Prophecy Technologies Logo
Kafka AWS MSK Migration Engineer

Prophecy Technologies

 

Alpharetta, GA, USA

Posted On: Just posted
Experience: 5+ years
Availability: Onsite
Openings: 1
Category: Kafka Engineer
Tenure: No Preference/Any
Related Jobs

No related jobs found

Description

You will lead the migration of on-premises Apache Kafka clusters to AWS Managed Streaming for Apache Kafka (MSK), owning the full lifecycle from discovery and capacity planning through to execution, validation, and decommissioning.

This role is on-site.

Responsibilities

  • Assess on-prem Kafka clusters to map topics, partitions, consumer groups, and dependencies, determining migration readiness and workload criticality.
  • Plan and provision AWS MSK infrastructure, including broker sizing, storage, networking, and security configurations to support production execution.
  • Execute parallel testing using MirrorMaker 2 or similar replication tools, comparing message delivery outcomes and troubleshooting discrepancies.
  • Perform performance tuning and optimization of MSK clusters to ensure migrated workloads are stable, scalable, and efficient.
  • Design automation frameworks and AI-assisted solutions to enable repeatable, large-scale Kafka-to-MSK migrations and document operational best practices.

Required Skills

  • 5+ years of experience with distributed messaging systems, specifically deep knowledge of Kafka architecture (brokers, topics, partitions, replication, consumer groups).
  • Practical experience with Amazon Managed Streaming for Apache Kafka (MSK), including cluster provisioning, configuration, IAM, mTLS, and SASL/SCRAM security.
  • Professional proficiency in Python, Java, or Shell scripting to build utilities for topic creation, ACL migration, and consumer group management.
  • Strong Unix/Linux fundamentals with the ability to troubleshoot application, deployment, environment, and runtime issues.
  • Hands-on experience with AWS services including ECS, EKS, Lambda, and CloudWatch for compute and migration patterns.
  • Experience with cross-cluster replication tools such as MirrorMaker 2 or Confluent Replicator for data migration and offset synchronization.
  • Familiarity with SDLC practices, CI/CD delivery models, GitLab, Jenkins, Maven, and Kubernetes-based deployments.
  • Understanding of VPC peering, PrivateLink, TLS encryption, and network connectivity patterns for hybrid architectures.

Preferred Skills

  • Experience with Schema Registry, Kafka Connect, and Kafka Streams ecosystem components.
  • Working knowledge of RESTful APIs, API design, and API Gateway patterns.

Education

Any Graduate

Related Jobs

No related jobs found

← Back to jobs