Description

You will build and maintain data processing pipelines and continuous learning streams in Austin, TX.

Responsibilities

• Develop programming and processing logic using object-oriented languages and algorithms.
• Build automated machine learning pipelines for continuous data processing.
• Collect data from diverse sources including APIs, web scraping, and public repositories.
• Process structured, semi-structured, and unstructured data formats.
• Implement real-time data processing and data warehousing solutions.

Required Skills

• 5+ years of experience in data development.
• Proficiency in Scala, Java, or Python.
• Hands-on experience with Apache Spark and Spark Structured Streaming.
• Experience with real-time processing using Apache Beam and Kafka.
• Strong SQL skills including joins, aggregations, and subqueries.
• Experience with relational databases like Postgres and NoSQL systems like Apache Hadoop.
• Practical knowledge of AWS cloud platforms.
• Proficiency with Git and GitHub for version control.
• Familiarity with Kubernetes for abstraction and orchestration.

Preferred Skills

• Experience with time series databases like InfluxDB and graph databases like Neo4j.
• Knowledge of Apache Hive for data warehousing.

Key Skills

Education

Any Gradute