There are different approaches to creating and training ML models. In this section, we show what these approaches are and how they differ. Apart from the approach we use to create an ML model, there are also parameters that manage how this model behaves in the training and evaluation processes. Model parameters can be divided into two distinct groups, which should be configured in different ways. The first group of parameters is the model weights in ML algorithms that are used to adjust the model’s predictions. They are assigned numerical values during the training process, and these values determine how the model makes decisions or predictions based on new data. The second group is the model hyperparameters that control the behavior of an ML model during training. They are not learned from the data like other parameters in the model but, rather, are set by the user or algorithm before training begins. The last crucial part of the ML process...

The concepts of linear algebra are essential for understanding the theory behind ML because they help us understand how ML algorithms work under the hood. Also, most ML algorithm definitions use linear algebra terms.

Linear algebra is not only a handy mathematical instrument but also the concepts of linear algebra can be very efficiently implemented with modern computer architectures. The rise of ML, and especially deep learning, began after significant performance improvement of the modern graphics processing unit (GPU). GPUs were initially designed to work with linear algebra concepts and massively parallel computations used in computer games. After that, special libraries were created to work with general linear algebra concepts. Examples of libraries that implement basic linear algebra routines are Cuda and OpenCL, and one example of a specialized linear algebra library is cuBLAS. Moreover, it became more common to use general-purpose graphics processing...

Consider an example of the real-world supervised ML algorithm called linear regression. In general, linear regression is an approach for modeling a target value (dependent value) based on an explanatory value (independent value). This method is used for forecasting and finding relationships between values. We can classify regression methods by the number of inputs (independent variables) and the type of relationship between the inputs and outputs (dependent variables).

Simple linear regression is the case where the number of independent variables is 1, and there is a linear relationship between the independent (x) and dependent (y) variables.

Linear regression is widely used in different areas such as scientific research, where it can describe relationships between variables, as well as in applications within industry, such as revenue prediction. For example, it can estimate a trend line that represents the long-term movement in the stock price...

In this chapter, we learned what ML is, how it differs from other computer algorithms, and how it became so popular. We also became familiar with the necessary mathematical background required to begin working with ML algorithms. We looked at software libraries that provide APIs for linear algebra and also implemented our first ML algorithm—linear regression.

There are other linear algebra libraries for C++. Moreover, the popular deep learning frameworks use their own implementations of linear algebra libraries. For example, the MXNet framework is based on the mshadow library, and the PyTorch framework is based on the ATen library. Some of these libraries can use GPU or special CPU instructions to speed up calculations. Such features do not usually change the API but require some additional library initialization settings or explicit object conversion to different backends such as CPUs or GPUs.

Real ML projects can be challenging and complex. Common pitfalls include...

• Basic Linear Algebra for Deep Learning: https://towardsdatascience.com/linear-algebra-for-deep-learning-f21d7e7d7f23
• Deep Learning, an MIT Press book: https://www.deeplearningbook.org/contents/linear_algebra.html
• What is Machine Learning?: https://www.mathworks.com/discovery/machine-learning.html
• The Eigen library documentation: https://gitlab.com/libeigen/eigen
• The xtensor library documentation: https://xtensor.readthedocs.io/en/latest/
• The Dlib library documentation: http://dlib.net/
• The blaze library documentation: https://bitbucket.org/blaze-lib/blaze/wiki/Home
• The ArrayFire library documentation: https://arrayfire.org/docs/index.htm