Contents:

Discover the world of data science on a free course. Learn three key areas in data science and Determine your future specialization Earn useful gifts while developing your data science skills
Learn MorePython is used in the field of data analysis and machine learning, and it integrates with a variety of libraries such as Pandas, Matplotlib, NumPy, TensorFlow, and many others. Each of these libraries is designed to perform specific tasks.
In today's discussion, we will focus on the Pandas library: we will explore its role, how it can be imported into the Python environment, and we will analyze our first dataset to determine which countries have the fastest and slowest internet speeds.
The Purpose of the Pandas Library in Python
Pandas is a key library in Python designed for data processing. It is widely used by specialists in the field of data analysis and data science. Created in 2008 by AQR Capital, the library became an open source project in 2009 and received the support of a large community of developers.
Here are the purposes of this library.
Data analysis covers various areas, such as product and marketing analytics, and many other areas. Working with any dataset requires careful analysis and preparation: you need to eliminate or fill gaps, filter, sort, or otherwise modify the information. The Pandas library in Python provides the ability to effectively perform all of these tasks and, in most cases, automate their process.
Data science research and processing of large arrays of information. The Pandas library serves as a tool for pre-processing and analysis of data, which allows for subsequent application in machine and deep learning tasks.
Statistical methods. The library offers a set of key statistical tools necessary for data analysis. Such methods include, for example, calculating average values, their distribution by quantiles, and many other approaches.
Working with Pandas
In the field of data analysis and machine learning, specialized tools such as Google Colab and Jupyter Notebook are often used. These integrated development environments (IDEs) allow you to work with data incrementally and iteratively, without having to build a full-fledged application.
In this article, we'll look at Google Colab, a cloud-based data analysis platform accessible through a browser on a variety of devices, from desktops and laptops to tablets and smartphones.

Each line of code displayed on the screen corresponds to a specific action, the result of which is instantly displayed to the user in Google Colab and Jupyter Notebook. This is especially convenient for working with tasks related to analytics and data science.
When using Jupyter Notebook or Google Colab, there is no need to install the Pandas library, as it is already included in the standard package and will be available immediately after opening the environment. You only need to import it into your code.
pd is a widely used abbreviation for the Pandas library in programming. It appears in literature, articles, and educational materials. It's recommended that you use it in your own projects to avoid having to type the full pandas name.
Series and DataFrame
In the Pandas library, information can be organized in two main formats: Series and DataFrame. Let's take a closer look at each of these types.
A Series is a data structure resembling a one-dimensional array that can store elements of different types. It can be most clearly represented as a table column, where the values are arranged in sequential order, and each of them corresponds to an index that indicates the row number.
Let's create an elementary Series object:
Let's display this on the screen.

The series is presented in a table format, where the first column contains the element indices, and the second contains the corresponding values.
A DataFrame is a key structural element in the Pandas library, on the basis of which all work with data is carried out. It can be thought of as a standard table consisting of many columns and rows. The cells of this "table" can contain data of various types, including numeric values, logical variables, text, and much more.
A DataFrame includes both row indices and column indices. This structure makes the process of sorting and filtering data more convenient and also facilitates quick search for the necessary cells.
Let's create a basic DataFrame using a dictionary, and then consider what it will look like.
Let's evaluate the result.

The table shows rows indexed from 0 to 3, while the column names serve as their indices. It is clear that this dataframe includes three Series: City, Founding Year, and Population. Both types of indexes provide the ability to navigate through the information.
Importing Data
Pandas provides a variety of methods for importing data. For example, you can load information from dictionaries, lists, or tuples. However, the most common option is working with .csv files, which are widely used in data analytics. The pd.read_csv() function is used to load them.
The read_csv function provides users with a number of parameters that allow you to control the data import process.
- The sep parameter allows you to explicitly set the character that serves as a separator in the imported document. By default, it is set to a comma, which corresponds to the standard separator in .csv files. This parameter is especially useful when the source file uses non-standard delimiters, such as tabs or semicolons.
- dtype allows you to specify the data type for columns after importing a .csv file. This feature is especially useful when automatic data format detection fails. For example, dates are often imported as string variables, despite the existence of a specific data type for them.
The documentation provides detailed parameters to help you properly configure the CSV file import process.
We'll import a dataset containing information about mobile and landline internet speeds in various countries. We'll download this ready-made dataset from the Kaggle platform; it's presented in .csv format. We won't specify any parameters for the read_csv function, since the data is already prepared for analysis.
Let's take a closer look at the generated dataframe:
Important!
When using Google Colab or Jupyter Notebook, you don't need to use the print command to display a DataFrame or Series. The Pandas library can output data without it. If you use print(df), the table formatting will be corrupted. Try displaying the data in two different ways and evaluate the results.
The following table appeared on the display:

At the beginning of the data frame are The column headers are country, broadband, and mobile. To the left of these are indexes, which range from 0 to 176, for a total of 177 rows. This information is also presented at the bottom of the table, as is common in Pandas.
There's no need to display the entire table. To get a feel for the data, simply look at the first five or last five rows. This can be accomplished using the df.head() and df.tail() functions. You can specify the number of lines to display in brackets, with 5 being the default.
Result:

This makes the task much easier. We can quickly see the column headers and the data types they contain. Additionally, some cells contain NaN values, which we'll return to later.
Analyzing and Interpreting the Collected Data
Now we need to analyze the imported data. We'll do this step by step.
The first step is to check the data type contained in the table. This is necessary to determine how exactly the information in the dataset is represented and to identify any anomalies. For example, dates may be stored as strings, which can create difficulties during further analysis. To perform this check, you can use the standard method:
A table will be displayed indicating the type of data represented in each column of the data frame.

What we see:
- The country column has the object data type. This type is designed to store string values and mixed data.
- The broadband and mobile columns have the float data type, which means they represent floating-point numbers.
Step 2. At this stage, it is necessary to quickly analyze the information and formulate preliminary conclusions. This can be done easily using the describe() method, which is provided in the Pandas library. This method outputs mean values with standard deviations, as well as the maxima and minima of variables, and their distribution by quantiles.
Let's see how this method works in practice:
Result:

Let's look at each row in detail.
- The word "count" refers to the number of completed rows in each column. Looking at the column containing information about mobile internet speed, you can see that there are gaps.
- The average internet speed, both wired and mobile, indicates that in most countries, mobile internet is slower than its cable counterpart.
- Standard deviation, denoted as std, is a key statistic that reflects the degree of variability of values in a data set.
- min and max refer to the minimum and maximum values, respectively.
- The internet speed values at the 25%, 50%, and 75% levels represent percentile scores. Simply put, a percentile is a metric that illustrates how values are distributed in a particular sample. For example, in a mobile dataset, the 25th percentile indicates that 25% of all recorded internet speeds are below 24.4.
Keep in mind that this method only applies to numeric data. The column containing country names is blank.
What can we conclude? In most countries, wired internet speeds are higher than mobile internet speeds. However, in 75% of cases, wired internet speeds do not exceed 110 Mbps, while mobile internet speeds reach 69 Mbps.
Step 3. Organizing and filtering data. In our dataframe, the information is already sorted by descending wired internet speed. Now we will try to determine the country with the highest quality mobile internet. To do this, we will use the standard sort_values method, which requires two arguments:
- The column by which sorting is performed must be enclosed in single or double quotation marks.
- The ascending= parameter determines the sorting method. If our goal is to sort the values in descending order, we set this parameter to False. In case you need to sort the data in ascending order, you should set the value to True.
Let's move on to the code:
Result:

The country rankings have now been updated, and the top five have changed as we've revised the sorting order of the data. As a result, we've determined that the United Arab Emirates has the fastest mobile internet.
However, there's one important caveat. If you look back at the original table, where the data is organized by wired internet speed, you'll see that Monaco, which ranks first, has a NaN value in the second column.
In Python, a NaN value indicates missing data. This means that the dataset provided is missing information about mobile internet speeds in Monaco, making it difficult to form a clear picture of the world's mobile leaders.
Let's filter the data by excluding those countries from the dataframe for which mobile internet speeds were not reported. This will allow us to focus on those countries that perform the worst. Leaving NaN values in will clutter the table and make it difficult to identify the real data about the slowest mobile internet.
The Pandas library provides several methods for filtering and removing NaN values. One of these is the dropna() method, which excludes all rows with gaps. It should be noted that this method eliminates the entire row with NaN, and not just the cells containing missing data in a specific column.
Result:

After removing empty values, the number of rows in the dataframe was reduced to 136. If you return to the second stage of the analysis, you will notice that this number matches the number of non-empty records in the mobile column in the original dataframe.
Let's create a new dataframe, which we will call df_without_nan, to save the obtained result. In this case, we will try to leave the original DataFrame unchanged, since it may be useful later.
Now let's sort the obtained data by the mobile speed column, starting with the smallest values and moving to the largest, to determine the country with the slowest mobile Internet speed.
Result:

The worst mobile internet quality is observed in Afghanistan, followed by Palestine and Venezuela.
Ways to change and edit a dataframe
In addition to working with an existing dataframe, we have the ability to modify it to suit our needs: we can add new records, delete old ones, aggregate information, and perform other operations.
Let's restore the original state of our DataFrame. To do this, let's re-upload the CSV file with the dataset:
Let's pay attention to the dataframe:
Let's check if all the data is present:

There are 177 countries in the list, including those for which there is no internet speed information.
We will add a new country to our dataframe. Since there are already 177 countries in the list, let's add the Galactic Republic from the Star Wars universe.
To add data to the dataframe, we use the concat method.
Let's look at the code line by line:
- First, we create a dictionary that will collect data on countries, including information on average internet speed and mobile internet speed.
- Using the pd.DataFrame constructor, we convert the dictionary into a dataframe.
- Using the concat method, we concatenate the original dataframe with the new one, which is located in new_list. It is important to set the ignore_index=True parameter so that the new line appears at the beginning.
Let's check the result:

Everything worked. The Galactic Republic is now on our list.
Pandas uses the drop method to delete rows. Let's use it to remove a country that doesn't actually exist and that we previously added to our dataframe:
The method takes two arguments:
- The indices of the rows to be deleted, in this example, are the row with index 0. If you want to delete multiple rows at once, you must pass their indices as a list, for example, [0, 1, 2].
- The inplace=True parameter resets the indices, so the first row after deletion gets index 0.
Let's run the code and see the results as a dataframe:

The Galactic Republic has ceased to exist. The dataframe has been restored to its original state.
In some cases, it is necessary to find a specific row in a dataframe. There are two approaches to accomplishing this: using the index value or accessing the index. Let's look at both of these methods.
Filtering is performed based on the index values that are presented in the first column of the dataframe. In this context, index values range from 0 to 177.
Let's display the countries with index values 10 and 11.
Note the result:

Filtering data by indexes. In a data frame, indexing starts at zero. Let's look at how to extract data on countries with indexes from 5 to 8. The iloc method is used for this operation.
We evaluate the result:

We extracted part of the list, starting from the 6th and ending with the 8th element. It should be noted that the indexes and row values \u200b\u200bdo not match.
Let's display only those countries in which the mobile Internet speed exceeds 100 Mbps.
As a result, a table was compiled that includes 18 countries:

Important!
Using this method, you can preserve the index values of the data frame being studied.
Data aggregation is the process of combining several individual values to obtain generalized information.
To calculate the average internet speed for all countries, we use the agg function, passing the mean parameter.
Result:

The average Internet speed in various countries is 72.67 Mbps.
Once you have finished editing the dataframe, you can save it in CSV format or select another suitable format.
The saved document will be available at the specified address.
Important!
When using this approach, the index values of the data frame under study remain unchanged.
What's next?
Pandas is one of the most effective libraries for data analysis in Python. In this article, we covered the basic operations that can be performed with it. More detailed information about the library's functionality can be found in the official documentation. For a more in-depth understanding of Pandas, we recommend reading the following:
- Learning Pandas by Michael Haidt and Artem Gruzdev is a fascinating and useful guide to working with the pandas library.
- Thoughts on Pandas: How to Properly Use the Python Data Analysis Library by Hannah Stepanek.
- Practical Data Analysis with Pandas: Effectively Collecting, Processing, Analyzing, and Visualizing Data Using Python by Stephanie Molin.
Read also:
- Libraries play an important role in the programming world, providing developers with ready-made solutions for various tasks. Their main function is to facilitate the process of creating software by allowing you to use already written code instead of developing everything from scratch.
There are several types of libraries, each designed for specific needs. For example, a programming language's library of standard functions contains the basic tools needed to perform simple operations. Others may be more specialized, offering functionality for working with graphics, networking, or data processing.
Libraries may also be open source, allowing developers to make modifications and improvements, or closed source, restricting access to the source code. Some libraries are widely used across various projects, while others may be more specialized and designed for specific applications.
Ultimately, libraries significantly speed up the development process by allowing programmers to focus on more complex aspects of a project without wasting time on routine tasks.
- Test: Determine which of the presented programming languages are esoteric and which are not.
- Getting Started with Python Programming: A Quick Guide.
