Gradient Descent Method

We always try our best to create best results but how we can do so? Mathematical optimization is a good answer for above question if our problems can be modeled by mathematical model. The common model is analytic function and it is very easy for us to know that optimization becomes finding out extreme points of such function. The issue focuses on global optimization which means that how to find out the global peak over the whole function. It is very interesting problem because there are two realistic cases as follows: 1. We want to get the best solution and there is no one better than this solution. 2. Given a good solution, we want to get another better solution. However, global optimization is also complicated because it is relevant to other mathematical subject such as solution existence and approximation. The issue also mentions these subjects. Your attention please, the issue focuses on algorithms and applied methods to solve problem of global optimization. Thus, theoretical aspects relevant to functional analysis are mentioned very little.

Global optimization is necessary in some cases when we want to achieve the best solution or we require a new solution which is better the old one. However global optimization is a hazard problem. Gradient descent method is a well-known technique to find out local optimizer whereas approximation solution approach aims to simplify how to solve the global optimization problem. In order to find out the global optimizer in the most practical way, I propose a so-called descending region (DR) algorithm which is combination of gradient descent method and approximation solution approach. The ideology of DR algorithm is that given a known local minimizer, the better minimizer is searched only in a so-called descending region under such local minimizer. Descending region is begun by a so-called descending point which is the main subject of DR algorithm. Descending point, in turn, is solution of intersection equation (A). Finally, I prove and provide a simpler linear equation system (B) which is derived from (A). So (B) is the most important result of this research because (A) is solved by solving (B) many enough times. In other words, DR algorithm is refined many times so as to produce such (B) for searching for the global optimizer. I propose a so-called simulated Newton – Raphson (SNR) algorithm which is a simulation of Newton – Raphson method to solve (B). The starting point is very important for SNR algorithm to converge. Therefore, I also propose a so-called RTP algorithm, which is refined and probabilistic process, in order to partition solution space and generate random testing points, which aims to estimate the starting point of SNR algorithm. In general, I combine three algorithms such as DR, SNR, and RTP to solve the hazard problem of global optimization. Although the approach is division and conquest methodology in which global optimization is split into local optimization, solving equation, and partitioning, the solution is synthesis in which DR is backbone to connect itself with SNR and RTP.

Increasing use of accelerometer and protractor sensors in recent years has created a field of study for the definition of human activities. This issue is tried to be solved by using machine learning methods. For this, it is solved by extracting different properties from the obtained signals, obtaining the characteristics specific to the activity and classifying these properties. In this study, the time and frequency domain properties of 4 different human activities were extracted, then a pre-treatment step was applied in accordance with the obtained feature set, and then the size was reduced with PCA and Fisher 'LDA methods. The k-NN classifier and perceptron classifiers were designed for the obtained feature set and the classification process was performed. In this study, the classification success of these methods using different parameters has been examined and the results are shown.