Direkt zum Inhalt
StartseiteShell

Data Processing in Shell

Learn powerful command-line skills to download, process, and transform data, including machine learning pipeline.

Kurs Kostenlos Starten
4 Stunden13 Videos46 Übungen19.467 LernendeTrophyLeistungsnachweis

Kostenloses Konto erstellen

GoogleLinkedInFacebook

oder

Durch Klick auf die Schaltfläche akzeptierst du unsere Nutzungsbedingungen, unsere Datenschutzrichtlinie und die Speicherung deiner Daten in den USA.
Group

Trainierst du 2 oder mehr?

Versuchen DataCamp for Business

Beliebt bei Lernenden in Tausenden Unternehmen

Kursbeschreibung

We live in a busy world with tight deadlines. As a result, we fall back on what is familiar and easy, favoring GUI interfaces like Visual Studio and RStudio. However, taking the time to learn data analysis on the command line is a great long-term investment because it makes us stronger and more productive data people.

In this course, we will take a practical approach to learn simple, powerful, and data-specific command-line skills. Using publicly available Spotify datasets, we will learn how to download, process, clean, and transform data, all via the command line. We will also learn advanced techniques such as command-line based SQL database operations. Finally, we will combine the powers of command line and Python to build a data pipeline for automating a predictive model.
Für Unternehmen

Trainierst du 2 oder mehr?

Verschaffen Sie Ihrem Team Zugriff auf die vollständige DataCamp-Plattform, einschließlich aller Funktionen.
DataCamp Für UnternehmenFür eine maßgeschneiderte Lösung buchen Sie eine Demo.
  1. 1

    Downloading Data on the Command Line

    Kostenlos

    In this chapter, we learn how to download data files from web servers via the command line. In the process, we also learn about documentation manuals, option flags, and multi-file processing.

    Kapitel Jetzt Abspielen
    Downloading data using curl
    50 xp
    Using curl documentation
    50 xp
    Downloading single file using curl
    100 xp
    Downloading multiple files using curl
    100 xp
    Downloading data using Wget
    50 xp
    Installing Wget
    50 xp
    Downloading single file using wget
    100 xp
    Advanced downloading using Wget
    50 xp
    Setting constraints for multiple file downloads
    50 xp
    Creating wait time using Wget
    100 xp
    Data downloading with Wget and curl
    100 xp
  2. 2

    Data Cleaning and Munging on the Command Line

    We continue our data journey from data downloading to data processing. In this chapter, we utilize the command line library csvkit to convert, preview, filter and manipulate files to prepare our data for further analyses.

    Kapitel Jetzt Abspielen
    Getting started with csvkit
    50 xp
    Installation and documentation for csvkit
    100 xp
    Converting and previewing data with csvkit
    100 xp
    File conversion and summary statistics with csvkit
    100 xp
    Filtering data using csvkit
    50 xp
    Printing column headers with csvkit
    100 xp
    Filtering data by column with csvkit
    100 xp
    Filtering data by row with csvkit
    100 xp
    Stacking data and chaining commands with csvkit
    50 xp
    Stacking files with csvkit
    100 xp
    Chaining commands using operators
    100 xp
    Data processing with csvkit
    100 xp
  3. 3

    Database Operations on the Command Line

    In this chapter, we dig deeper into all that csvkit library has to offer. In particular, we focus on database operations we can do on the command line, including table creation, data pull, and various ETL transformation.

    Kapitel Jetzt Abspielen
    Pulling data from database
    50 xp
    Using sql2csv documentation
    50 xp
    Understand sql2csv connectors
    50 xp
    Practice pulling data from database
    100 xp
    Manipulating data using SQL syntax
    50 xp
    Applying SQL to a local CSV file
    100 xp
    Cleaner scripting via shell variables
    100 xp
    Joining local CSV files using SQL
    100 xp
    Pushing data back to database
    50 xp
    Practice pushing data back to database
    100 xp
    Database and SQL with csvkit
    100 xp
  4. 4

    Data Pipeline on the Command Line

    In the last chapter, we bridge the connection between command line and other data science languages and learn how they can work together. Using Python as a case study, we learn to execute Python on the command line, to install dependencies using the package manager pip, and to build an entire model pipeline using the command line.

    Kapitel Jetzt Abspielen
    Python on the command line
    50 xp
    Finding Python version on the command line
    50 xp
    Executing Python script on the command line
    100 xp
    Python package installation with pip
    50 xp
    Understanding pip's capabilities
    50 xp
    Installing Python dependencies
    100 xp
    Running a Python model
    100 xp
    Data job automation with cron
    50 xp
    Understanding cron scheduling syntax
    50 xp
    Scheduling a job with crontab
    100 xp
    Model production on the command line
    100 xp
    Course recap
    50 xp
Für Unternehmen

Trainierst du 2 oder mehr?

Verschaffen Sie Ihrem Team Zugriff auf die vollständige DataCamp-Plattform, einschließlich aller Funktionen.

Datensätze

Spotify Songs Popularity RankingSpotify Song Attributes

Mitwirkende

Collaborator's avatar
Hillary Green-Lerman
Collaborator's avatar
Adrián Soto

Voraussetzungen

Introduction to ShellIntermediate PythonIntermediate SQL
Susan Sun HeadshotSusan Sun

Data Person

Mehr Anzeigen

Was sagen andere Lernende?

    Melden Sie sich an 15 Millionen Lernende und starten Sie Data Processing in Shell Heute!

    Kostenloses Konto erstellen

    GoogleLinkedInFacebook

    oder

    Durch Klick auf die Schaltfläche akzeptierst du unsere Nutzungsbedingungen, unsere Datenschutzrichtlinie und die Speicherung deiner Daten in den USA.