Data collection and analysis is an old practice going back to the beginning of civilization. Records from ancient Egyptian papyrus suggest that pharaohs collected census information from villages, possibly to determine the number of soldiers that could be enlisted for the war. However, it was after the arrival of modern computers that the art of data analytics became a significant phenomenon that is changing people's lives every day.

This book, as its name suggests, teaches you how to use Structured Query Language (SQL) for data analytics. SQL is the tool that you will be focusing on in the rest of the book. But before diving into SQL, this chapter will provide an overview of data analytics. You will be introduced to fundamental concepts such as the definition and type of statistics and different methods of statistics, which will lay the foundation for the concepts that future chapters will be based on, define the purpose of the SQL operations that you will learn...

Raw data is a group of values that you can extract from a source. It becomes useful when it is processed to find different patterns in the data that was extracted. These patterns, also referred to as information, help you to interpret the data, make predictions, and identify unexpected changes in the future. This information is then processed into knowledge.

Knowledge is a large, organized collection of persistent and extensive information and experience that can be used to describe and predict phenomena in the real world. Data analysis is the process by which you convert data into information and, thereafter, knowledge. Data analytics is when data analysis is combined with making predictions.

There are several data analysis techniques available to make sense of data. One of them is statistics, which uses mathematical techniques on datasets.

Statistics is the science of collecting and analyzing a large amount of data to identify the characteristics...

Statistics can be further divided into two subcategories: descriptive statistics and inferential statistics.

Descriptive statistics are used to describe a collection of data. For example, the average age of people in a country is a descriptive statistics indicator that describes an aspect of the country's residents. Descriptive statistics on a single variable in a dataset are called univariate analysis, while descriptive statistics that look at two or more variables at the same time are called multivariate analysis. In particular, statistics that look at two variables are called bivariate analysis. The average age of a country is an example of univariate analysis, while an analysis examining the interaction between GDP per capita, healthcare spending per capita, and age is multivariate analysis.

In contrast, inferential statistics allows datasets to be collected as a sample or a small portion of measurements from a larger group, called a population....

In all the examples so far, you have been dealing with datasets that are clean and easy to decipher. However, datasets in real world are more complicated than these. One of the many problems you may have to deal with when working with datasets is missing values.

You will further learn the specifics of preparing data in Chapter 3, SQL for Data Preparation. However, in this section, you will learn several strategies that you can use to handle missing data. Some of your strategies include the following:

• Deleting rows: If a very small number of rows (that is, less than 5% of your dataset) is missing data, then the simplest solution may be to just delete the data points from your set. This would not impact your results too much.
• Mean/median/mode imputation: If 5% to 25% of your data for a variable is missing, another option is to take the mean, median, or mode of that column and fill in the blanks with that value. It may provide a...

Often, an analyst is interested in comparing the statistical properties of two groups, or perhaps just one group before and after a change. Of course, the difference between these two groups may just be due to chance.

An example of where this comes up is in marketing A/B tests. Companies often test two different types of landing pages for a product and measure how many clicks it will receive on each of the landing pages. For example, if you make the image of a product two times larger, will this make people more likely to click it? You may find that 10% of the visitors for variation A of the landing page clicked on the product, and 11% for variation B. So, does that mean variation B is 10% better than A or is this just a result of day-to-day variance? You need a method based on statistics to determine just that.

Statistical significance testing is the method of determining whether the data that you have supports a certain hypothesis. To build...

Throughout this chapter, you have learned about different techniques used in data analytics. All these analytics techniques inevitably lead to the storage and processing of massive data. While there are many tools in today's market that can help you with these tasks, a relational database is the most important one.

A relational database is a convenient and easy-to-understand way to store datasets. Modern relational database management systems, such as PostgreSQL databases, also provide a powerful tool for processing and analyzing data, which is SQL. Using SQL, you can clean data, transform data into more useful formats, and analyze data with statistics to find interesting patterns. The rest of this book will be dedicated to understanding how you can use SQL for these purposes productively and efficiently.

Data analytics is a powerful method through which you analyze raw data to find patterns and gather predictions that help you to understand the world. The goal of analytics is to turn data into information and knowledge. To accomplish this goal, statistics, or descriptive statistics and statistical significance testing, are used to understand data.

Univariate analysis, a branch of descriptive statistics, can be utilized to understand a single variable of data. It can also be used to find outliers and the distribution of data by utilizing frequency distributions and quantiles. It is useful in finding the central tendency of a variable by calculating the mean, median, and mode of data and the dispersion of data using the range, standard deviation, and IQR.

Bivariate analysis is also used to understand the relationship between datasets. You can determine trends, changes in trends, periodic behavior, and anomalous points regarding two variables by using scatterplots. You can...