How to use python to climb the ranking data of World Universities
Today, I would like to share with you how to use python to climb the world university ranking data related knowledge points, the content is detailed, the logic is clear, I believe most people still know too much about this knowledge, so share this article for your reference, I hope you can get something after reading this article, let's take a look at it.
Data acquisition
What we choose here is the ARWU website of Shanghai Jiaotong University.
The website contains college scores and rankings over the years.
Through the analysis of the page, we can find that it is most convenient to obtain relevant information through the read_html function of pandas.
Table = pd.read_html (url)
College = table [0]
At the same time, we also found that the national data corresponding to the university are pictures, so it needs special processing.
Def get_country_name (html):
Soup = BeautifulSoup (html,'lxml')
Countries = soup.select ('td > a > img')
Lst = []
For i in countries:
Src = I ['src']
Pattern = re.compile ('flag.*/ (. *?). Png')
Country = re.findall (pattern,src) [0]
Lst.append (country)
Return lst
Finally, we process the obtained data, remove the unneeded fields, and then add the year field and so on.
For i in range (2005, 2020):
Print ('year', I)
Url = 'http://www.shanghairanking.com/ARWU%s.html'% I
Html = requests.get (url) .content
Table = pd.read_html (url)
College = table [0]
College.columns = ['world rank','university', 2, 3,' score', 5]
College.drop ([2jue 3jue 5], axis = 1je inplace = True)
College ['year'] = I
College ['index_rank'] = college.index
College ['index_rank'] = college [' index_rank'] .astype (int) + 1
College ['country'] = get_country (html)
College.to_csv (ringing College.csvents, mode='a', encoding='utf_8_sig', header=True, index=0)
So, we get the College.csv file.
These are all the contents of the article "how to use python to climb World University ranking data". Thank you for reading! I believe you will gain a lot after reading this article. The editor will update different knowledge for you every day. If you want to learn more knowledge, please pay attention to the industry information channel.
