Shulou(Shulou.com)06/01 Report--

Python how to implement statistical osd-pg and export to csv, for this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible method.

Analyze the pg distribution of each OSD in the current cluster

Import jsonfrom collections import OrderedDictimport csvimport subprocessdef get_pg_dump (): cmd = "ceph pg dump-- format=json" out = subprocess.Popen (cmd, shell=True Stdout=subprocess.PIPE) result = out.stdout.readlines () [1] .split ('\ n') [0] return resultcsv_file ='. / demo.csv' # generated csv file path pg_data = json.loads (get_pg_dump ()) pg_dict = {} osd_pg_dict = {} for i in pg_data: for j in pg_data ['pg_stats']: pg_dict [j [' pgid'] ] = j ['up'] pool_pg_total = {} osd_pg_num_dict = {} tmp_pool_list = {} tmp_osd_list = {} start_osd = 0start_pool_id = 0all_result = {} for i in pg_dict: pool_id = int (i.split ('.') [0]) for osd_id in pg_dict [I]: tmp_pool_ list [pool _ id] = pool_id Tmp_osd_ list [OSD _ id] = osd_idnum_pool = max (tmp_pool_list.iterkeys () Key=lambda k: tmp_pool_ list [k]) + 1num_osd = max (tmp_osd_list.iterkeys () Key=lambda k: tmp_osd_ list [k]) + 1pool_totol_pg = [0] * num_poolmax_osd_list = [0] * num_poolmin_osd_list = [999] * num_poolmax_osdid_list = ['No_OSD'] * num_poolmin_osdid_list = [' No_OSD'] * num_poolave_osd_list = [0] * num_poolave_osd_list_result = [0] * num_poolmin_osd_per_ List = [0] * num_poolmax_osd_per_list = [0] * num_poolfor i in range (0 Num_osd): keyname = str (I) all_ result [keyname] = [0] * num_poolfor I in pg_dict: pool_id = int (i.split ('.') [0]) for osd_id in pg_dict [I]: tmp_keyname = str (osd_id) tmp_list = all_ result [t mp_keyname] tmp_ list [pool _ id] = tmp_ list [pool _ id] ] + 1for I in all_result: tmp_num = 0 for j in all_result [I]: if j

Max_osd_ list [tmp _ num]: max_osd_ list [tmp _ num] = j max_osdid_ list [tmp _ num] = i if j! = 0: ave_osd_ list [tmp _ num] = ave_osd_ list [tmp _ num] + 1 pool_totol_ PG [tmp _ num] = pool_totol_ PG [tmp _ num] + j tmp_num = tmp_num + 1for i in range (0 Len (ave_osd_list): if ave_osd_ list [I]! = 0: ave_osd_list_ result [I] = pool_totol_ PG [I] / ave_osd_ list [I] all_result = OrderedDict (sorted (all_result.items (), key=lambda t: int (t [0])) for i in range (0) Num_pool): if max_osd_ list [I]! = 0: max_osd_per_ list [I] = round (100* (max_osd_ list [I]-ave_osd_list_ result [I]) / float (ave_osd_list_ result [I]), 2) for i in range (0 Num_pool): if min_osd_ list [I]! = 999: min_osd_per_ list [I] = round (100 * (min_osd_ list [I]-ave_osd_list_ result [I]) / float (ave_osd_list_ result [I]), 2) with open (csv_file,'w') as csvfile: pool_list = [] for i in range (0 Num_pool): pool_list.append ('pool-'+str (I)) fieldnames= [' osd_name'] + pool_list + ['total'] writer = csv.DictWriter (csvfile, fieldnames=fieldnames) writer.writeheader () for i in all_result: tmp_dict = {' osd_name': I 'total':sum (all_ result [I])} tmp_num = 0 for j in all_result [I]: keyname =' pool-' + str (tmp_num) tmp_num = tmp_num + 1 tmp_ [keyname] = j writer.writerow (tmp_dict) fieldnames = ['SUM'] + pool_totol_pg + [sum (pool_totol_pg)] writer = csv.DictWriter (csvfile Fieldnames=fieldnames) writer.writeheader () fieldnames= ['AVE'] + ave_osd_list_result + [sum (ave_osd_list_result)] writer = csv.DictWriter (csvfile, fieldnames=fieldnames) writer.writeheader () fieldnames= [' MAX'] + max_osd_list + [sum (max_osd_list)] writer = csv.DictWriter (csvfile, fieldnames=fieldnames) writer.writeheader () fieldnames= ['MAX-OSD-ID'] + max_osdid_list writer = csv.DictWriter (csvfile) Fieldnames=fieldnames) writer.writeheader () fieldnames= ['MAX-PER'] + max_osd_per_list + [sum (max_osd_per_list)] writer = csv.DictWriter (csvfile, fieldnames=fieldnames) writer.writeheader () fieldnames= [' MIN'] + min_osd_list + [sum (min_osd_list)] writer = csv.DictWriter (csvfile) Fieldnames=fieldnames) writer.writeheader () fieldnames= ['MIN-OSD-ID'] + min_osdid_list writer = csv.DictWriter (csvfile, fieldnames=fieldnames) writer.writeheader () fieldnames= [' MIN-PER'] + min_osd_per_list + [sum (min_osd_per_list)] writer = csv.DictWriter (csvfile) Fieldnames=fieldnames) writer.writeheader () print'- 'for i in all_result: print iMagazine all _ result [I] Sum (all_ result [I]) print'- 'print' SUM',pool_totol_pgprint 'MAX',max_osd_listprint' MIN-OSD-ID',max_osdid_listprint 'MIN' Min_osd_listprint 'MIN-OSD-ID',min_osdid_listprint' AVE',ave_osd_list_resultprint 'MIN-PER',min_osd_per_listprint' MAX-PER',max_osd_per_list 's answers to questions about how python implements statistical osd-pg and exports to csv are shared here. I hope the above content can help you to a certain extent, if you still have a lot of doubts to be solved, you can follow the industry information channel to learn more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.