Saltar al contenido principal
InicioSpark

Feature Engineering with PySpark

Learn the gritty details that data scientists are spending 70-80% of their time on; data wrangling and feature engineering.

Comienza El Curso Gratis
4 horas16 vídeos60 ejercicios14.806 aprendicesTrophyDeclaración de cumplimiento

Crea Tu Cuenta Gratuita

GoogleLinkedInFacebook

o

Al continuar, acepta nuestros Términos de uso, nuestra Política de privacidad y que sus datos se almacenan en los EE. UU.
Group

¿Entrenar a 2 o más personas?

Probar DataCamp for Business

Preferido por estudiantes en miles de empresas

Descripción del curso

The real world is messy and your job is to make sense of it. Toy datasets like MTCars and Iris are the result of careful curation and cleaning, even so the data needs to be transformed for it to be useful for powerful machine learning algorithms to extract meaning, forecast, classify or cluster. This course will cover the gritty details that data scientists are spending 70-80% of their time on; data wrangling and feature engineering. With size of datasets now becoming ever larger, let's use PySpark to cut this Big Data problem down to size!
Empresas

¿Entrenar a 2 o más personas?

Obtén a tu equipo acceso a la plataforma DataCamp completa, incluidas todas las funciones.
DataCamp Para EmpresasPara obtener una solución a medida, reserve una demostración.

En las siguientes pistas

Big Data con PySpark

Ir a la pista
  1. 1

    Exploratory Data Analysis

    Gratuito

    Get to know a bit about your problem before you dive in! Then learn how to statistically and visually inspect your dataset!

    Reproducir Capítulo Ahora
    Where to Begin
    50 xp
    Where to begin?
    50 xp
    Check Version
    100 xp
    Load in the data
    100 xp
    Defining A Problem
    50 xp
    What are we predicting?
    100 xp
    Verifying Data Load
    100 xp
    Verifying DataTypes
    100 xp
    Visually Inspecting Data / EDA
    50 xp
    Using Corr()
    100 xp
    Using Visualizations: distplot
    100 xp
    Using Visualizations: lmplot
    100 xp
  2. 2

    Wrangling with Spark Functions

    Real data is rarely clean and ready for analysis. In this chapter learn to remove unneeded information, handle missing values and add additional data to your analysis.

    Reproducir Capítulo Ahora
    Dropping data
    50 xp
    Dropping a list of columns
    100 xp
    Using text filters to remove records
    100 xp
    Filtering numeric fields conditionally
    100 xp
    Adjusting Data
    50 xp
    Custom Percentage Scaling
    100 xp
    Scaling your scalers
    100 xp
    Correcting Right Skew Data
    100 xp
    Working with Missing Data
    50 xp
    Visualizing Missing Data
    100 xp
    Imputing Missing Data
    100 xp
    Calculate Missing Percents
    100 xp
    Getting More Data
    50 xp
    A Dangerous Join
    100 xp
    Spark SQL Join
    100 xp
    Checking for Bad Joins
    100 xp
  3. 3

    Feature Engineering

    In this chapter learn how to create new features for your machine learning model to learn from. We'll look at generating them by combining fields, extracting values from messy columns or encoding them for better results.

    Reproducir Capítulo Ahora
    Feature Generation
    50 xp
    Differences
    100 xp
    Ratios
    100 xp
    Deeper Features
    100 xp
    Time Features
    50 xp
    Time Components
    100 xp
    Joining On Time Components
    100 xp
    Date Math
    100 xp
    Extracting Features
    50 xp
    Extracting Text to New Features
    100 xp
    Splitting & Exploding
    100 xp
    Pivot & Join
    100 xp
    Binarizing, Bucketing & Encoding
    50 xp
    Binarizing Day of Week
    100 xp
    Bucketing
    100 xp
    One Hot Encoding
    100 xp
  4. 4

    Building a Model

    In this chapter we'll learn how to choose which type of model we want. Then we will learn how to apply our data to the model and evaluate it. Lastly, we'll learn how to interpret the results and save the model for later!

    Reproducir Capítulo Ahora
    Choosing the Algorithm
    50 xp
    Which MLlib Module?
    50 xp
    Creating Time Splits
    100 xp
    Adjusting Time Features
    100 xp
    Feature Engineering Assumptions for RFR
    50 xp
    Feature Engineering For Random Forests
    50 xp
    Dropping Columns with Low Observations
    100 xp
    Naively Handling Missing and Categorical Values
    100 xp
    Building a Model
    50 xp
    Building a Regression Model
    100 xp
    Evaluating & Comparing Algorithms
    100 xp
    Understanding Metrics
    50 xp
    Interpreting, Saving & Loading
    50 xp
    Interpreting Results
    100 xp
    Saving & Loading Models
    100 xp
    Final Thoughts
    50 xp
Empresas

¿Entrenar a 2 o más personas?

Obtén a tu equipo acceso a la plataforma DataCamp completa, incluidas todas las funciones.

En las siguientes pistas

Big Data con PySpark

Ir a la pista

conjuntos de datos

2017 St Paul MN Real Estate Dataset

colaboradores

Collaborator's avatar
Adrián Soto
Collaborator's avatar
Nick Solomon

requisitos previos

Introduction to PySparkSupervised Learning with scikit-learn
John Hogue HeadshotJohn Hogue

Lead Data Scientist, General Mills

Ver Más

¿Qué tienen que decir otros alumnos?

    ¡Únete a 15 millones de estudiantes y empieza Feature Engineering with PySpark hoy mismo!

    Crea Tu Cuenta Gratuita

    GoogleLinkedInFacebook

    o

    Al continuar, acepta nuestros Términos de uso, nuestra Política de privacidad y que sus datos se almacenan en los EE. UU.