Writing Efficient Code with pandas
Learn efficient techniques in pandas to optimize your Python code.
Start Course for Free4 hours14 videos45 exercises20,162 learnersStatement of Accomplishment
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.Training 2 or more people?
Try DataCamp for BusinessLoved by learners at thousands of companies
Course Description
The ability to efficiently work with big datasets and extract valuable information is an indispensable tool for every aspiring data scientist. When working with a small amount of data, we often don’t realize how slow code execution can be. This course will build on your knowledge of Python and the pandas library and introduce you to efficient built-in pandas functions to perform tasks faster. Pandas’ built-in functions allow you to tackle the simplest tasks, like targeting specific entries and features from the data, to the most complex tasks, like applying functions on groups of entries, much faster than Python's usual methods. By the end of this course, you will be able to apply a function to data based on a feature value, iterate through big datasets rapidly, and manipulate data belonging to different groups efficiently. You will apply these methods on a variety of real-world datasets, such as poker hands or restaurant tips.
Training 2 or more people?
Get your team access to the full DataCamp platform, including all the features.- 1
Selecting columns and rows efficiently
FreeThis chapter will give you an overview of why efficient code matters and selecting specific and random rows and columns efficiently.
The need for efficient coding I50 xpWhat does time.time() measure?50 xpMeasuring time I100 xpMeasuring time II100 xpLocate rows: .iloc[] and .loc[]50 xpRow selection: loc[] vs iloc[]100 xpColumn selection: .iloc[] vs by name100 xpSelect random rows50 xpRandom row selection100 xpRandom column selection100 xp - 2
Replacing values in a DataFrame
This chapter shows the usage of the replace() function for replacing one or multiple values using lists and dictionaries.
Replace scalar values using .replace()50 xpReplacing scalar values I100 xpReplace scalar values II100 xpReplace values using lists50 xpReplace multiple values I100 xpReplace multiple values II100 xpReplace values using dictionaries50 xpReplace single values I100 xpReplace single values II100 xpReplace multiple values III100 xpMost efficient method for scalar replacement50 xp - 3
Efficient iterating
This chapter presents different ways of iterating through a Pandas DataFrame and why vectorization is the most efficient way to achieve it.
Looping using the .iterrows() function50 xpCreate a generator for a pandas DataFrame100 xpThe iterrows() function for looping100 xpLooping using the .apply() function50 xp.apply() function in every cell100 xp.apply() for rows iteration100 xpVectorization over pandas series50 xpWhy vectorization in pandas is so fast?50 xppandas vectorization in action100 xpVectorization with NumPy arrays using .values()50 xpBest method of vectorization50 xpVectorization methods for looping a DataFrame100 xp - 4
Data manipulation using .groupby()
This chapter describes the groupby() function and how we can use it to transform values in place, replace missing values and apply complex functions group-wise.
Data transformation using .groupby().transform50 xpThe min-max normalization using .transform()100 xpTransforming values to probabilities100 xpValidation of normalization100 xpWhen to use transform()?50 xpMissing value imputation using transform()50 xpIdentifying missing values100 xpMissing value imputation100 xpData filtration using the filter() function50 xpWhen to use filtration?50 xpData filtration100 xpCongratulations!50 xp
Training 2 or more people?
Get your team access to the full DataCamp platform, including all the features.Leonidas Souliotis
See MorePhD @ University of Warwick
Leonidas Souliotis is a PhD student at the University of Warwick, UK. His research interests lie in the field of bioinformatics, machine learning, and deep learning. Before that, he completed his MSc in Statistics degree from Imperial College London, UK, and his BSc in Statistics and Insurance Science from the University of Piraeus.
He has worked in different areas of applied statistics and machine learning, both inside and outside academia. This includes stock trading, epidemiology and biology.
Join over 15 million learners and start Writing Efficient Code with pandas today!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.