Skip to main content
HomeData EngineeringStreaming Concepts

Streaming Concepts

Learn about the difference between batching and streaming, scaling streaming systems, and real-world applications.

Start Course for Free
2 hours16 videos47 exercises3,634 learnersTrophyStatement of Accomplishment

Create Your Free Account

GoogleLinkedInFacebook

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.
Group

Training 2 or more people?

Try DataCamp for Business

Loved by learners at thousands of companies


Course Description

Streaming is a huge aspect of the data world right now and is being used by nearly every industry from manufacturing to healthcare. Would you like to learn more about the general concepts behind data pipelines and how the processes work?

This course provides a general introduction to streaming concepts including batching, queuing, and stream processing along with where they fit into data processing frameworks. It covers real-world examples of how streaming is implemented in production. It is designed as a general introduction to these concepts and does not require an extensive background in data processing.
For Business

GroupTraining 2 or more people?

Get your team access to the full DataCamp library, with centralized reporting, assignments, projects and more
Try DataCamp for BusinessFor a bespoke solution book a demo.

In the following Tracks

Professional Data Engineer in Python

Go To Track
  1. 1

    Methods for Processing Data

    Free

    In this chapter, you’ll be given an introduction to basic concepts in batch data processing, revolving around implementation, scaling, and potential challenges during use.

    Play Chapter Now
    Intro to batch processing
    50 xp
    Batch characteristics
    100 xp
    Batch ordering
    100 xp
    Scaling batch processing
    50 xp
    On the scale
    100 xp
    Horizontally opposed
    50 xp
    Batch issues
    50 xp
    Batch problems
    50 xp
    Batch scenarios
    100 xp
For Business

GroupTraining 2 or more people?

Get your team access to the full DataCamp library, with centralized reporting, assignments, projects and more

In the following Tracks

Professional Data Engineer in Python

Go To Track

collaborators

Collaborator's avatar
Hadrien Lacroix
Collaborator's avatar
Kelsey McNeillie

prerequisites

Understanding Data Engineering
Mike Metzger HeadshotMike Metzger

Data Engineer Consultant @ Flexible Creations

Mike is a consultant focusing on data engineering and analysis using SQL, Python, and Apache Spark among other technologies. He has a 20+ year history of working with various technologies in the data, networking, and security space.
See More

What do other learners have to say?

Join over 15 million learners and start Streaming Concepts today!

Create Your Free Account

GoogleLinkedInFacebook

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.