How to crawl stock trading data and display them visually by Python
This article introduces the relevant knowledge of "how Python crawls stock trading data and visually displays". In the operation of actual cases, many people will encounter such a dilemma. Next, let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!
Development environment
Interpreter version: python 3.8
Code Editor: pycharm 2021.2
Third-party module
Requests: pip install requests
Csv
Steps of a crawler case
1. Determine the url address (link address)
two。 Send a network request
3. Data parsing (filtering data)
4. Data preservation (database (mysql\ mongodb\ redis), local files)
Crawler full code analysis web page
Open the developer tool, search for keywords, and find the correct url
Import module import requests # sends network request import csv request data url = f 'https://xueqiu.com/service/v5/stock/screener/quote/list?page=1&size=30&order=desc&order_by=amount&exchange=CN&market=CN&type=sha&_=1637908787379'# camouflage headers = {# browser camouflage' User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64 X64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36'} response = requests.get (url Headers=headers) json_data = response.json () parsing data data_list = json_data ['data'] [' list'] for data in data_list: data1 = data ['symbol'] data2 = data [' name'] data3 = data ['current'] data4 = data [' chg'] data5 = data ['percent'] data6 = data [' current_year_percent'] data7 = data ['volume'] data8 = data [' Amount'] data9 = data ['turnover_rate'] data10 = data [' pe_ttm'] data11 = data ['dividend_yield'] data12 = data [' market_capital'] print (data1 Data2, data3, data4, data5, data6, data7, data8, data9, data10, data11, data12) data_dict = {'stock symbol': data1, 'stock name': data2, 'current price': data3,'up and down': data4,'up and down': data5, 'year to date': data6, 'volume': data7, 'turnover': data8 Turnover: data9, TTM: data10, dividend yield: data11, Market capitalization: data12,} csv_write.writerow (data_dict)
Compare the url of 1, 2 and 3 pages of data to find the rules.
For page in range (1,56): url = f 'https://xueqiu.com/service/v5/stock/screener/quote/list?page={page}&size=30&order=desc&order_by=amount&exchange=CN&market=CN&type=sha&_=1637908787379' save data file = open (' data2.csv', mode='a', encoding='utf-8', newline='') csv_write = csv.DictWriter (file, fieldnames= ['stock symbol', 'stock name', 'current price','up and down','up and down' "year-to-date", "volume", "turnover", "turnover", "TTM", "dividend yield", "market capitalization"]) csv_write.writeheader () file.close () achieve results
Data visualization full code import data import pandas as pdfrom pyecharts import options as optsfrom pyecharts.charts import Bar read data data_df = pd.read_csv ('data2.csv') df = data_df.dropna () df1 = df [[' stock name'' Df2 = df1.iloc [: 20] print (df2 ['stock name'] .values) print (df2 ['stock name'] .values) visual chart c = (Bar () .add _ xaxis (list (stock ['stock name'])) .add _ yaxis ("stock trading volume") List (df2 ['Trading Volume']) .set _ global_opts (title_opts=opts.TitleOpts (title= "Trading Volume Chart-Volume chart"), datazoom_opts=opts.DataZoomOpts (),) .render ("data.html") print ('data visualization result is complete, please find the open data.html file in the current directory!') Effect display
This is the end of the content of "how Python crawls stock trading data and visualizes it". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!
