Confluent

Confluent Stream Processing Using KSQL & Apache Kafka Streams

21 hours
1995 €
Classroom or Live Virtual Class
Classroom or Live Virtual Class

Description

The Confluent Kafka Platform is a data transmission environment that allows organize and manage large amounts of data, as well as the second, the entry points of organizations in social networks. With Confluent, this growing flow of data is organized in Publication / Subscription, often unstructured, but incredibly valuable, Kafka Confluent becomes a unified and easily accessible data platform that is always available for many uses throughout the organization. These uses can easily be covered in Big Data analysis in batches with Hadoop and the feeding of real-time monitoring systems, up to the more traditional large-volume data integration tasks that require a high-performance backbone, extraction, transformation and load (ETL). Confluent Kafka offer customers different training classes, in particular, for administrators (implementation), for developers and the most modern communication of data with KSQL.

During this hands-on course, you will learn how to use Confluent KSQL to transform, enrich, filter and add data flows in real time using a language similar to SQL. You will also learn how to use the Apache Kafka Streams lib to create streaming applications. In addition, you will learn how to test, monitor, protect and scale those applications. The course deals with how these applications integrate with the Confluent platform with Apache Kafka technology, Kafka Connect, Confluent Schema Registry, Confluent REST Proxy and Confluent Control Center. The architecture concepts and the components of KSQL and Kafka Streams will be analyzed.

PUE is official Training Partner of Confluent, is authorized by this multinational company to deliver official training in Confluent technologies.

PUE is accredited and recognized to deliver consulting services and mentoring on implementing Confluent solutions in business environment with the added value in the practical business-centered focus of knowledge transferred from the official courses.

Audience and prerequisites

This training is designed for application developers, ETL (extract, transform, and load) developers, DevOps engineers and data scientists who need to interact with Kafka clusters as a source of real-time data streams and transform, enrich and join those streams to discover anomalies, analyze behavior or monitor complex systems.

The students should be familiar with developing in Java (preferred), .NET or Python. Attendees should also be familiar with the Apache Kafka architecture and Kafka client application development, either through prior experience or by taking Confluent Fundamentals of Apache Kafka and Confluent Developer Skills for Building Apache Kafka.

Objectives

At the end of the training, the student will get skills related to:

  • Identify common patterns and use cases for real-time stream processing
  • Understand the high level architecture of Kafka Streams
  • Write real-time applications with the Kafka Streams API to filter, transform, enrich, aggregate, and join data streams
  • Describe how KSQL combines the elastic, fault-tolerant, high-performance stream processing capabilities of Kafka Streams with the simplicity of a SQL-like syntax
  • Author KSQL queries that showcase its balance of power and simplicity
  • Test, secure, deploy, and monitor Kafka Streams applications and KSQL queries

Topics

Motivation and Concepts for Streams

  • Motivation and Use Cases for Real-Time Streaming
  • High Level Comparison of Kafka Streams and KSQL
  • Stream Processing Concepts

Kafka Streams Architecture

  • Kafka Streams' Place in the Kafka Ecosystem
  • High Level Architecture Design
  • Kafka Streams Data Types

Writing Kafka Streams Applications

  • Anatomy of a Kafka Streams Application
  • Kafka Streams DSL - Stateless Operations
  • Kafka Streams DSL - Aggregations
  • Kafka Streams DSL - Windowed aggregations
  • Kafka Streams DSL - Joins
  • Kafka Streams DSL - Summary
  • Processor API
  • Optimizations

Testing Kafka Streams Applications

  • Unit Tests
  • Integration Tests
  • Stress Tests
  • End-to-end Tests

Introduction to KSQL

  • Sample Use Cases
  • End-to-end Examples
  • Interacting with KSQL

Using KSQL

  • Data Manipulation
  • Aggregations
  • Testing

Deployment

  • Parallelism
  • Elasticity
  • Fault tolerance
  • Capacity planning
  • Troubleshooting
  • KSQL-specific considerations

Security

  • Security Overview
  • Access Control
  • Examples
  • KSQL-specific considerations

Monitoring

  • JMX
  • Confluent Control Center
  • KSQL-specific Considerations

Open calls