Shulou(Shulou.com)05/31 Report--

Today, I would like to share with you how to use python to climb the world university ranking data related knowledge points, the content is detailed, the logic is clear, I believe most people still know too much about this knowledge, so share this article for your reference, I hope you can get something after reading this article, let's take a look at it.

Data acquisition

What we choose here is the ARWU website of Shanghai Jiaotong University.

The website contains college scores and rankings over the years.

Through the analysis of the page, we can find that it is most convenient to obtain relevant information through the read_html function of pandas.

Table = pd.read_html (url)

College = table [0]

At the same time, we also found that the national data corresponding to the university are pictures, so it needs special processing.

Def get_country_name (html):

Soup = BeautifulSoup (html,'lxml')

Countries = soup.select ('td > a > img')

Lst = []

For i in countries:

Src = I ['src']

Pattern = re.compile ('flag.*/ (. *?). Png')

Country = re.findall (pattern,src) [0]

Lst.append (country)

Return lst

Finally, we process the obtained data, remove the unneeded fields, and then add the year field and so on.

For i in range (2005, 2020):

Print ('year', I)

Url = 'http://www.shanghairanking.com/ARWU%s.html'% I

Html = requests.get (url) .content

Table = pd.read_html (url)

College = table [0]

College.columns = ['world rank','university', 2, 3,' score', 5]

College.drop ([2jue 3jue 5], axis = 1je inplace = True)

College ['year'] = I

College ['index_rank'] = college.index

College ['index_rank'] = college [' index_rank'] .astype (int) + 1

College ['country'] = get_country (html)

College.to_csv (ringing College.csvents, mode='a', encoding='utf_8_sig', header=True, index=0)

So, we get the College.csv file.

These are all the contents of the article "how to use python to climb World University ranking data". Thank you for reading! I believe you will gain a lot after reading this article. The editor will update different knowledge for you every day. If you want to learn more knowledge, please pay attention to the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.