Pular para o conteúdo principal
InícioSparkFeature Engineering with PySpark

Feature Engineering with PySpark

Learn the gritty details that data scientists are spending 70-80% of their time on; data wrangling and feature engineering.

Comece O Curso Gratuitamente
4 horas16 vídeos60 exercícios
14.421 aprendizesTrophyDeclaração de Realização

Crie sua conta gratuita

GoogleLinkedInFacebook

ou

Ao continuar, você aceita nossos Termos de Uso, nossa Política de Privacidade e que seus dados são armazenados nos EUA.
GroupTreinar 2 ou mais pessoas?Experimente o DataCamp For Business

Amado por alunos de milhares de empresas

Descrição do Curso

The real world is messy and your job is to make sense of it. Toy datasets like MTCars and Iris are the result of careful curation and cleaning, even so the data needs to be transformed for it to be useful for powerful machine learning algorithms to extract meaning, forecast, classify or cluster. This course will cover the gritty details that data scientists are spending 70-80% of their time on; data wrangling and feature engineering. With size of datasets now becoming ever larger, let's use PySpark to cut this Big Data problem down to size!
Para Empresas

GroupTreinar 2 ou mais pessoas?

Obtenha acesso à biblioteca completa do DataCamp, com relatórios, atribuições, projetos e muito mais centralizados
Experimente O DataCamp for BusinessPara uma solução sob medida , agende uma demonstração.

Nas seguintes faixas

Big Data com PySpark

Ir para a trilha
  1. 1

    Exploratory Data Analysis

    Gratuito

    Get to know a bit about your problem before you dive in! Then learn how to statistically and visually inspect your dataset!

    Reproduzir Capítulo Agora
    Where to Begin
    50 xp
    Where to begin?
    50 xp
    Check Version
    100 xp
    Load in the data
    100 xp
    Defining A Problem
    50 xp
    What are we predicting?
    100 xp
    Verifying Data Load
    100 xp
    Verifying DataTypes
    100 xp
    Visually Inspecting Data / EDA
    50 xp
    Using Corr()
    100 xp
    Using Visualizations: distplot
    100 xp
    Using Visualizations: lmplot
    100 xp
  2. 2

    Wrangling with Spark Functions

    Real data is rarely clean and ready for analysis. In this chapter learn to remove unneeded information, handle missing values and add additional data to your analysis.

    Reproduzir Capítulo Agora
    Dropping data
    50 xp
    Dropping a list of columns
    100 xp
    Using text filters to remove records
    100 xp
    Filtering numeric fields conditionally
    100 xp
    Adjusting Data
    50 xp
    Custom Percentage Scaling
    100 xp
    Scaling your scalers
    100 xp
    Correcting Right Skew Data
    100 xp
    Working with Missing Data
    50 xp
    Visualizing Missing Data
    100 xp
    Imputing Missing Data
    100 xp
    Calculate Missing Percents
    100 xp
    Getting More Data
    50 xp
    A Dangerous Join
    100 xp
    Spark SQL Join
    100 xp
    Checking for Bad Joins
    100 xp
  3. 3

    Feature Engineering

    In this chapter learn how to create new features for your machine learning model to learn from. We'll look at generating them by combining fields, extracting values from messy columns or encoding them for better results.

    Reproduzir Capítulo Agora
    Feature Generation
    50 xp
    Differences
    100 xp
    Ratios
    100 xp
    Deeper Features
    100 xp
    Time Features
    50 xp
    Time Components
    100 xp
    Joining On Time Components
    100 xp
    Date Math
    100 xp
    Extracting Features
    50 xp
    Extracting Text to New Features
    100 xp
    Splitting & Exploding
    100 xp
    Pivot & Join
    100 xp
    Binarizing, Bucketing & Encoding
    50 xp
    Binarizing Day of Week
    100 xp
    Bucketing
    100 xp
    One Hot Encoding
    100 xp
  4. 4

    Building a Model

    In this chapter we'll learn how to choose which type of model we want. Then we will learn how to apply our data to the model and evaluate it. Lastly, we'll learn how to interpret the results and save the model for later!

    Reproduzir Capítulo Agora
    Choosing the Algorithm
    50 xp
    Which MLlib Module?
    50 xp
    Creating Time Splits
    100 xp
    Adjusting Time Features
    100 xp
    Feature Engineering Assumptions for RFR
    50 xp
    Feature Engineering For Random Forests
    50 xp
    Dropping Columns with Low Observations
    100 xp
    Naively Handling Missing and Categorical Values
    100 xp
    Building a Model
    50 xp
    Building a Regression Model
    100 xp
    Evaluating & Comparing Algorithms
    100 xp
    Understanding Metrics
    50 xp
    Interpreting, Saving & Loading
    50 xp
    Interpreting Results
    100 xp
    Saving & Loading Models
    100 xp
    Final Thoughts
    50 xp
Para Empresas

GroupTreinar 2 ou mais pessoas?

Obtenha acesso à biblioteca completa do DataCamp, com relatórios, atribuições, projetos e muito mais centralizados

Nas seguintes faixas

Big Data com PySpark

Ir para a trilha

conjuntos de dados

2017 St Paul MN Real Estate Dataset

colaboradores

Collaborator's avatar
Adrián Soto
Collaborator's avatar
Nick Solomon

pré-requisitos

Introduction to PySparkSupervised Learning with scikit-learn
John Hogue HeadshotJohn Hogue

Lead Data Scientist, General Mills

Ver Mais

O que os outros alunos têm a dizer?

    Junte-se a mais de 14 milhões de alunos e comece Feature Engineering with PySpark hoje mesmo!

    Crie sua conta gratuita

    GoogleLinkedInFacebook

    ou

    Ao continuar, você aceita nossos Termos de Uso, nossa Política de Privacidade e que seus dados são armazenados nos EUA.