How to use Tf-idf in Python

This article mainly introduces how to use Tf-idf in Python, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand it.

Description

1. TF-IDF is that if the probability of words or phrases appearing in articles is high, but rarely in other articles, then it is considered to have a good ability to distinguish categories and is suitable for classification.

2. Extract text features to evaluate the importance of words to files in a file set or a corpus.

Example

Def tfidf_demo (): "" using tfidf method for text feature extraction: return: "" # 1. Segmenting the Chinese text into words data = "Today is cruel, tomorrow is crueler, the day after tomorrow is beautiful, but absolutely most of them die tomorrow night, so everyone should not give up today." The light we see from distant galaxies was emitted millions of years ago so that when we look at the universe we are looking at its past. " If you know something in only one way, you won't really understand it. The secret of knowing what things really mean depends on how you relate them to what we know. "] Data_new = [] for sent in data: data_new.append (cut_word (sent)) # print (data_new) # 2. Instantiate a converter class transfer = TfidfVectorizer (stop_words= ["one", 'because']) # 3. Call fit_transform data_final = transfer.fit_transform (data_new) print ("data_new:\ n", data_final.toarray ()) print ("feature name:\ n", transfer.get_feature_names ()) return None to thank you for reading this article carefully. I hope the article "how to use Tf-idf in Python" shared by the editor will be helpful to you, and I also hope you will support it. Pay attention to the industry information channel, more related knowledge is waiting for you to learn!