CN110134874A - A Collaborative Filtering Method for Optimizing User Similarity - Google Patents

Abstract

The invention proposes a kind of collaborative filtering methods for optimizing user's similarity.While not increasing server delay, the precision of proposed algorithm is improved.The characteristics of this method, is: by being standardized pretreatment to user's score data, calculate Pearson similarity, the evaluation weight of user vector distance and asymmetrical similarity weight, and then Pearson similarity is optimized, so that traditional collaborative filtering recommends precision to be improved.This method is suitable for user --- the data set of project scoring.

Description

A Collaborative Filtering Method for Optimizing User Similarity

technical field

Aiming at the recommendation system based on collaborative filtering, the present invention proposes a collaborative filtering method for optimizing user similarity.

Background technique

The rapid development and popularization of the Internet has provided great convenience for users to obtain, share and disseminate information. At the same time, the substantial increase in the amount of information reduces the utilization rate of information, making it difficult for users to obtain information that is truly useful to them from the network in a timely manner, resulting in information overload. A method that can effectively deal with the problem of information overload is to design a recommender system, which recommends content and products that users are interested in based on information such as users' needs and interests. Compared with search engines, recommender systems conduct personalized calculations by studying the interests and preferences of users, so as to discover the points of interest of users, and then guide users to discover their own information needs and obtain useful information for them. A good recommendation system can not only provide users with personalized services, but also establish a close relationship between different users, allowing users to rely on recommendations.

Recommendation systems mainly include content filtering and collaborative filtering. The recommendation system based on content filtering obtains the characteristics of the user's attention items according to the user's previous browsing or purchase records, and recommends new items that best match the user's interest characteristics to the user. The recommendation system based on collaborative filtering obtains the similarity between users by calculating the similarity of historical records between users, searches for other users with similar preferences to the target user, and recommends items of interest to such users to the target user.

The recommendation system based on content filtering only considers the target user itself, while the recommendation system based on collaborative filtering makes full use of collective intelligence, that is, collects answers from the behavior and data of a large number of people, and the recommendation is more personalized, so collaborative filtering recommends Algorithms are the most widely used and effective recommendation algorithms in personalized recommendation services.

The recommendation system based on collaborative filtering is further divided into model-based collaborative filtering recommendation system and memory-based collaborative filtering recommendation system. The former mainly uses methods such as machine learning, data mining and statistics to train users' historical data, then constructs a corresponding user model, and uses the model to provide users with predictions and recommendations, involving matrix decomposition, latent semantic analysis and other technologies . The latter is divided into user-based collaborative filtering recommender systems and item-based collaborative filtering recommender systems.

Although the traditional user-based collaborative filtering recommendation system uses the Pearson formula to measure the similarity, it does not preprocess the data set, does not consider the distance between user rating vectors, and does not consider the inequality of the similarity relationship between users. The recommendation quality of the recommender system decreases. Therefore, the present invention optimizes the above three points for the user-based collaborative filtering recommendation algorithm to improve the recommendation quality.

SUMMARY OF THE INVENTION

The present invention is dedicated to reducing the average absolute error value of the traditional user-based collaborative filtering recommendation algorithm, effectively improving the recommendation quality of the recommending system, and providing a collaborative filtering method for optimizing user similarity.

To achieve the above object, the present invention proposes the following technical solutions:

A collaborative filtering method for optimizing user similarity. By standardizing the user's score vector, combining the evaluation weight of the user vector distance and the asymmetric similarity weight to optimize the Pearson similarity, and finally predicting the user's score, the specific steps are as follows:

1) Prepare the experimental database: collect the scores of a certain number of users for different items, and establish an experimental database;

2) Standardization preprocessing: use the Z-score method to standardize the rating vector of each user, and generate a user-item rating matrix based on the standardized user rating vector;

3) Calculate the similarity matrix of the user: According to the user-item scoring matrix generated in step 2), calculate the Pearson similarity, the evaluation weight of the user vector distance, and the asymmetric similarity weight; Symmetrical similarity weights optimize the Pearson similarity, obtain the optimized similarity formula, calculate the similarity between each user and other users according to the optimized similarity formula, and finally generate a similarity matrix;

4) Prediction score: According to the similarity between the target user and other users, the neighbor user set of the target user is calculated, and the unscored items of the target user are predicted by the scoring formula.

Compared with the prior art, the present invention has the following advantages:

The method of the invention optimizes the user similarity for the recommendation algorithm module in the collaborative filtering technology, so that the recommendation system can effectively improve the recommendation quality without increasing the server delay.

Description of drawings

Figure 1 is a flow chart of the method of the present invention.

Detailed ways

The specific embodiments of the present invention will be further described below with reference to the accompanying drawings.

In this embodiment, instance analysis is performed on the MovieLens-100k dataset (available for download from the website https://movielens.org/), which covers a total of 100,000 rating records of 1,682 movies by 943 users, and the rating value is 1 An integer between 5 and 1, where 1 is the lowest rating and 5 is the highest rating. Each user has rated at least 20 movies. 80% of the data in the dataset is the training set and 20% of the data is the test set.

As shown in Figure 1, a collaborative filtering method for optimizing user similarity, by standardizing the user's rating vector, combining the evaluation weight of the user vector distance and the asymmetric similarity weight to optimize the Pearson similarity, and finally the user rating is calculated. The specific steps are as follows:

1) Prepare the experimental database: collect the scores of a certain number of users for different items, and establish an experimental database.

2) Standardization preprocessing: Use the Z-score method to standardize the rating vector of each user, and generate a user-item rating matrix based on the standardized user rating vector. The specific steps are as follows:

Let the rating vector of the uth user in the training set be R _u =(r _(u,1) ,r _(u,2) ,...,r _(u,m) ), where r _(u,m) represents the pair of user u The score of item m; as shown in formula (1), the Z-score method is used to standardize R _u , where z _{(u, m)} is the score of user u on item m after normalization, is the mean of each component of R _u , and σ _u is the standard deviation of each component of R _u :

The standardized rating vector of user u is denoted as Z _u =(z _(u,1) ,z _(u,2) ,...,z _(u,m) ), where the mean value of Z _u is 0 and the standard deviation is 1. A user-item rating matrix of size 943 × 1682 is generated, where 943 is the number of users and 1682 is the number of items. Zu is recorded in the _uth row of the user-item rating matrix, and the rating value of the item that user u has not rated is recorded as 0.

3) Calculate the similarity matrix of the user: According to the user-item scoring matrix generated in step 2), calculate the Pearson similarity, the evaluation weight of the user vector distance, and the asymmetric similarity weight; Symmetrical similarity weights are used to optimize the Pearson similarity, and an optimized similarity formula is obtained. According to the optimized similarity formula, the similarity between each user and other users is calculated, and a similarity matrix is finally generated. Taking any two users u and user v in the training set of MovieLens-100k as an example, to calculate the similarity between user u and user v, the specific steps are as follows:

3.1) Calculate Pearson similarity: As shown in formula (2), the Pearson similarity sim _{(u, v)} of user u and user v is measured by the Pearson similarity formula, where set S is the common score of user u and user v. A collection of items:

3.2) Calculate the evaluation weight of the user vector distance: As shown in formula (3), calculate the evaluation weight D _{(u, v)} of the user vector distance between Z _u and Z _v , where S is the common rating item of user u and user v Set, N(S) is the number of elements in set S, α represents the threshold of the score difference of a single item, if α is larger, D _{(u, v) is} closer to 1, if α is smaller, D _{(u, v ) is} closer to 0:

3.3) Calculate the asymmetric similarity weight: as shown in formula (4), calculate the asymmetric similarity weight w _{(u, v)} of user u to user v, where S is the common score of user u and user v Item set, I _u is the set of rated items of user u, N(S) is the number of elements in the set S, N(I _u ) is the number of elements in the set I _u :

3.4) User similarity formula: as shown in formula (5), by fusing formula (2), formula (3) and formula (4), the similarity between user u and user v after optimization is obtained as sim′ _{(u, v )} :

sim′ _(u,v) = D _(u,v) *w _(u,v) *sim _(u,v) (5)

3.5) Calculate the user similarity matrix: Calculate the similarity between different users according to formula (5), and finally obtain a user similarity matrix of 943×943.

4) Prediction score: Calculate the neighbor user set of the target user according to the similarity between the target user and other users, and predict the unscored items of the target user through the scoring formula. In this example, an unscored item of any user u in the training set is used. Taking a as an example, to calculate the predicted score of user u for item a, the specific steps are as follows:

4.1) Calculate the set of neighbor users: In the training set, find the set of users who have evaluated item a, denoted as U _a ={u _(1,a) ,u _(2,a) ,...,u _(q,a) }, Among them, u _{(q, a)} represents the qth user who has evaluated item a; according to the similarity between these q users and user u, sort them in descending order of similarity, and denote it as U′ _a = { u′ _(1,a) ,u′ _(2,a) ,…,u′ _(q,a) }; then select the first k users from the sorted user set U′ _a as the neighbor users of user u Set, denoted as U={u′ _(1,a) ,u′ _(2,a) ,…,u′ _(k,a) }.

4.2) Predict the score of user u on item a: Calculate the predicted score p _(u,a) of user u on item a according to formula (6), where set U is the set of neighbor users of user u, is the average value of each component of R _u , σ _u is the standard deviation of each component of R _u , z _(v,a) is the standardized score of user v to item a, sim′ _(u,v) is the score of user u to user v Similarity:

As shown in formula (7), the mean absolute error (MAE) is used to describe the recommendation accuracy. The smaller the MAE, the smaller the error and the higher the accuracy, where pi is the user's predicted score for item _i , and _ri is the test set. User's actual rating for item i, n is the number of ratings in the test set:

In this embodiment, the size of the neighbor set is 10, and the MAE value of the present invention is 0.74086, which is 3.06% lower than the original Pearson similarity. The time required for calculating the user similarity matrix by the method of the present invention is 61.3 seconds, and the time required for scoring prediction is 7.04 seconds. In practical applications, the calculation of the user similarity matrix can be completed by offline calculation, and when the user uses the recommendation algorithm of the present invention to predict the item score online, the real-time calculation time used is almost the same as the original method, and the user's online waiting time is not increased.

Claims (4)

1. A collaborative filtering method for optimizing user similarity. By standardizing the user's rating vector, combining the evaluation weight of the user vector distance and the asymmetric similarity weight to optimize the Pearson similarity, and finally predicting the user's rating, its characteristics Yes, the specific steps are as follows: 1) Prepare the experimental database: collect the scores of a certain number of users for different items, and establish an experimental database; 2) Standardization preprocessing: use the Z-score method to standardize the rating vector of each user, and generate a user-item rating matrix based on the standardized user rating vector; 3) Calculate the similarity matrix of the user: According to the user-item scoring matrix generated in step 2), calculate the Pearson similarity, the evaluation weight of the user vector distance, and the asymmetric similarity weight; Symmetrical similarity weights optimize the Pearson similarity, obtain the optimized similarity formula, calculate the similarity between each user and other users according to the optimized similarity formula, and finally generate a similarity matrix; 4) Prediction score: According to the similarity between the target user and other users, the neighbor user set of the target user is calculated, and the unscored items of the target user are predicted by the scoring formula. 2. the collaborative filtering method of optimizing user similarity according to claim 1, is characterized in that, the concrete steps of described step 2) are as follows: let the rating vector of the uth user in the training set be R _u =(r _{(u ,1)} ,r _(u,2) ,…,r _(u,m) ), where z _(u,m) is the user u’s rating on item m after normalization, and r _(u,m) is the user u’s rating on item m The score of m; as shown in formula (1), use the Z-score method to standardize R _u , where z _{(u, m)} is the score of user u on item m after normalization, is the mean of each component of R _u , and σ _u is the standard deviation of each component of R _u : The standardized rating vector of user u is denoted as Z _u = (z _(u,1) ,z _(u,2) ,...,z _(u,m) ), the mean value of Z _u is 0, and the standard deviation is 1; then , generate a user-item rating matrix; Zu is recorded in the _uth row of the user-item rating matrix, and the rating value of the item that user u has not scored is recorded as 0. 3. the collaborative filtering method of optimizing user similarity according to claim 1, is characterized in that, in described step 3), take any two users u and user v in training set as an example, calculate the similarity of user u to user v degree, the specific steps are as follows: 3.1) Calculate Pearson similarity: As shown in formula (2), the Pearson similarity sim _{(u, v)} of user u and user v is measured by the Pearson similarity formula, where set S is the common score of user u and user v. A collection of items: 3.2) Calculate the evaluation weight of the user vector distance: As shown in formula (3), calculate the evaluation weight D _{(u, v)} of the user vector distance between Z _u and Z _v , where S is the common rating item of user u and user v Set, N(S) is the number of elements in set S, α represents the threshold of the score difference of a single item, if α is larger, D _{(u, v) is} closer to 1, if α is smaller, D _{(u, v ) is} closer to 0: 3.3) Calculate the asymmetric similarity weight: as shown in formula (4), calculate the asymmetric similarity weight w _{(u, v)} of user u to user v, where S is the common score of user u and user v Item set, I _u is the set of rated items of user u, N(S) is the number of elements in the set S, N(I _u ) is the number of elements in the set I _u : 3.4) User similarity formula: as shown in formula (5), by fusing formula (2), formula (3) and formula (4), the similarity between user u and user v after optimization is obtained as sim′ _{(u, v )} : sim′ _(u,v) = D _(u,v) *w _(u,v) *sim _(u,v) (5) 3.5) Calculate the user similarity matrix: Calculate the similarity between different users according to formula (5), and finally obtain the user similarity matrix. 4. the collaborative filtering method of optimizing user similarity according to claim 1, it is characterized in that, in described step 4), take an unscored item a of any user u in training set as an example, calculate user u to item a. To predict the score, the specific steps are as follows: 4.1) Calculate the set of neighbor users: In the training set, find the set of users who have evaluated item a, denoted as U _a ={u _(1,a) ,u _(2,a) ,...,u _(q,a) }, Among them, u _{(q, a)} represents the qth user who has evaluated item a; according to the similarity between these q users and user u, sort them in descending order of similarity, and denote it as U′ _a = { u′ _(1,a) ,u′ _(2,a) ,…,u′ _(q,a) }; then select the first k users from the sorted user set U′ _a as the neighbor users of user u Set, denoted as U={u′ _(1,a) ,u′ _(2,a) ,…,u′ _(k,a) }; 4.2) Predict the score of user u on item a: Calculate the predicted score p _(u,a) of user u on item a according to formula (6), where set U is the set of neighbor users of user u, is the average value of each component of R _u , σ _u is the standard deviation of each component of R _u , z _(v,a) is the standardized score of user v to item a, sim′ _(u,v) is the score of user u to user v Similarity: As shown in formula (7), the average absolute error MAE is used to describe the recommendation accuracy. The smaller the MAE, the smaller the error and the higher the accuracy, where pi is the user's predicted score for item _i , and ri is the user's score for item _i in the test set. Actual ratings, n is the number of ratings in the test set: