Multi-language implementation of Collaborative filtering algorithm R/mapreduce/spark mllib

Download the user movie rating dataset

Http://grouplens.org/datasets/movielens/

1) Item-Based, impersonal, everyone sees the same thing

2) User-Based, personalized, everyone sees differently

After analyzing the user's behavior, we can calculate the similar users and items according to the users' preferences, and then make recommendations based on the similar users or items. These are the two branches of collaborative filtering, user-based and item-based collaborative filtering.

When calculating the similarity between users, it takes a user's preference for all items as a vector, while when calculating the similarity between items, it takes all users' preference for an item as a vector. After finding the similarity, we can find the similar neighbors.

3) Model-based (ModelCF)

According to the model, it can be divided into:

1) nearest neighbor model: distance-based collaborative filtering algorithm

2) Latent Factor Mode (SVD): a model based on matrix decomposition

3) Graph: Graph model, social network graph model

Applicable scenario

For an online website, the number of users often exceeds the number of items, and the item data is relatively stable, so calculating the similarity of items is not only

The amount of calculation is small, and there is no need to update frequently. But this situation only applies to e-commerce sites, such as news, blogs and so on.

System recommendation, the situation is often the opposite, the number of items is massive, and frequently updated.

Implementation of object-based Collaborative filtering algorithm in r language

# reference plyr package library (plyr) # read dataset train