How to debug radosgw of ceph by gdb

Editor to share with you how gdb debugs ceph's radosgw. I hope you will get something after reading this article. Let's discuss it together.

# gdb debugging rgw

Modify gdb configuration, set to save history commands, p variables to beautify output, etc.

[root@ceph04 src] # cat ~ / .gdbinitset history save onset print pretty onset pagination offset confirm off

Switch to / home/yuliyang/ceph/src [root@ceph04 src] # pwd / home/yuliyang/ceph/src of ceph04 (10.254.3.81) to start a cluster using vstart mode

. / vstart.sh-d-n-X-r-- short

Create an admin user

. / radosgw-admin user create-uid=admin-display-name=admin-access-key=admin-secret=admin

Rgw checks the process number on port 8000 ps.

[root@ceph04 src] # ps aux | grep cephroot 10486 1.5 0.1 404376 201076 pts/2 Sl 11:40 1:22. / ceph-mon-I a-c / home/yuliyang/ceph/src/ceph.confroot 10501 0.9 0.1 413592 202224 pts/2 Sl 11:40 0:54. / ceph-mon-I b-c / home/yuliyang/ceph/src/ceph.confroot 10520 0.9 0.1 413592 201968 pts/2 Sl 11 : 40 0:51. / ceph-mon-I c-c / home/yuliyang/ceph/src/ceph.confroot 10854 0.6 0.1 899744 226972? Ssl 11:40 0:35. / ceph-osd-I 0-c / home/yuliyang/ceph/src/ceph.confroot 11067 0.6 0.1 899748 226280? Ssl 11:40 0:34. / ceph-osd-I 1-c / home/yuliyang/ceph/src/ceph.confroot 11304 0.5 0.1 898720 222624? Ssl 11:40 0:31. / ceph-osd-I 2-c / home/yuliyang/ceph/src/ceph.confroot 11634 0.00.1 351560 157888? Ssl 11:40 0:01. / ceph-mds-I a-c / home/yuliyang/ceph/src/ceph.confroot 11723 0.00.1 383336 164808? Ssl 11:40 0:03. / ceph-mds-I b-c / home/yuliyang/ceph/src/ceph.confroot 11826 0.00.1 351556 158000? Ssl 11:40 0:01. / ceph-mds-I c-c / home/yuliyang/ceph/src/ceph.confroot 12282 0.1 12470764 209416? Ssl 11:40 0:05 / home/yuliyangceph/src/.libs/lt-radosgw-c / home/yuliyang/ceph/src/ceph.conf-- log-file=/home/yuliyang/ceph/src/out/rgw.log-- debug-rgw=20-- debug-ms=1root 22717 0.0 112648 952 pts/2 S + 13:11 0:00 grep-- color=auto ceph

Attach process

Gdb attach 12282

Break point

(gdb) b process_requestBreakpoint 1 at 0x7f465253ea20: file rgw/rgw_process.cc, line 37. (gdb) b rgw/rgw_process.cc:38Breakpoint 2 at 0x7f465253ea2f: file rgw/rgw_process.cc, line 38. (gdb) b rgw/rgw_process.cc:39Breakpoint 3 at 0x7f465253ea39: file rgw/rgw_process.cc, line 39.

Running

(gdb) cContinuing. [New Thread 0x7f462c7f8700 (LWP 23422)] [Thread 0x7f463c1ae700 (LWP 21983) exited] [Thread 0x7f463c5b2700 (LWP 21980) exited] [New Thread 0x7f465c472700 (LWP 23425)]

Stuck here. Perform the creation of 100 buckets at intervals of 1000s

For i in `seq 1 100`; do s3cmd-c yuliyang.s3cfg mb s3://test-$i; sleep 1000; done

Stop at the breakpoint

(gdb) cContinuing. [New Thread 0x7f462c7f8700 (LWP 23422)] [Thread 0x7f463c1ae700 (LWP 21983) exited] [Thread 0x7f463c5b2700 (LWP 21980) exited] [New Thread 0x7f465c472700 (LWP 23425)] [Switching to Thread 0x7f453affd700 (LWP 12363)] Breakpoint 1, process_request (store=0x7f465dd42840, rest=0x7ffd760c17e0, req=0x7f453affa800, client_io=0x7f453affa940, olog=0x0) at rgw/rgw_process.cc:3737 {(gdb)

Ctr x an opened tui and found it was parked at the breakpoint.

N next step, after continuing with a few n

View userinfo

(gdb) p userinfo$2 = {auid = 0, user_id = {tenant = ", id ="}, display_name = ", user_email =", access_keys = std::map with 0 elements, swift_keys = std::map with 0 elements, subusers = std::map with 0 elements, suspended = 0'\ 000mm, max_buckets = 1000, op_mask = 7, caps = {caps = std::map with 0 elements} System = 0'\ 000mm, default_placement = "", placement_tags = empty std::list, bucket_quota = {max_size_kb =-1, max_objects =-1, enabled = false, max_size_soft_threshold =-1, max_objs_soft_threshold =-1}, temp_url_keys = std::map with 0 elements, user_quota = {max_size_kb =-1, max_objects =-1 Enabled = false, max_size_soft_threshold =-1, max_objs_soft_threshold =-1}}

List looks at the code around the current breakpoint

(gdb) list56 s-> obj_ctx = & rados_ctx;5758 s-> req_id = store- > unique_id (req- > id); 59 s-> trans_id = store- > unique_trans_id (req- > id); 60 s-> host_id = store- > host_id;6162 req- > log_format (s, "initializing for trans_id =% s", s-> trans_id.c_str ()); 6364 RGWOp* op = NULL 65 int init_error = 0

Continue the n execution and view the * s variable

(gdb) p * sroom3 = {cct = 0x7f465dcc5e10, cio = 0x0, req = 0x6, op = OP_UNKNOWN, op_type = RGW_OP_UNKNOWN, content_started = false, format = 0, formatter = 0x0, decoded_uri = ", relative_uri =", length = 0x0, content_length = 0, generic_attrs = std::map with 0 elements, err = {http_ret = 200, ret = 0, s3_code =" Message = ""}, expect_cont = false, header_ended = false, obj_size = 0, enable_ops_log = false, enable_usage_log = false, defer_to_bucket_acls = 0'\ 000mm, perm_mask = 0, header_time = {tv = {tv_sec = 0, tv_nsec = 0}}, account_name = ", bucket_tenant =", bucket_name =" Bucket = {tenant = ", name =", data_pool = ", data_extra_pool =", index_pool = ", marker =", bucket_id =", oid = ""}, object = {name = "", instance = ""}, src_tenant_name = ", src_bucket_name =", src_object = {name =" Instance = ""}, bucket_owner = {id = {tenant = ", id ="}, display_name ="}, owner = {id = {tenant =", id ="}, display_name ="}, zonegroup_name =", zonegroup_endpoint = ", bucket_instance_id =", bucket_instance_shard_id =-1 Redirect = "", bucket_info = {bucket = {tenant = "", name = ", data_pool =", data_extra_pool = "", index_pool = "", marker = "", bucket_id = "", oid = ""}, owner = {tenant = "", id = ""}, flags = 0, zonegroup = "" Creation_time = {_ d = {_ r = 0}}, placement_rule = "", has_instance_obj = false, objv_tracker = {read_version = {ver = 0, tag = ""}, write_version = {ver = 0, tag = ""}} Ep_objv = {ver = 0, tag = ""}, quota = {max_size_kb =-1, max_objects =-1, enabled = false, max_size_soft_threshold =-1, max_objs_soft_threshold =-1}, num_shards = 0, bucket_index_shard_hash_type = 0'\ 000mm, static NUM_SHARDS_BLIND_BUCKET = 4294967295 Requester_pays = false, has_website = false, website_conf = {redirect_all = {protocol = ", hostname =", http_redirect_code = 0}, index_doc_suffix = "", error_doc = "", routing_rules = {rules = empty std::list}}, index_type = (unknown: 805334416) Swift_versioning = false, swift_ver_location = ""}, bucket_attrs = std::map with 0 elements, bucket_exists = false, has_bad_meta = false, user = 0x7f453affa060, bucket_acl = 0x0, object_acl = 0x0, system_request = false, aws4_auth_needs_complete = false, aws4_auth = 0x0, canned_acl = "", has_acl_header = false, http_auth = 0x0, local_source = false, prot_flags = 0 Os_auth_token = 0x0, swift_user = "", swift_groups = "", content_disp = {override = "", fallback = ""}, host_id = "", info = {env = 0x7f453affa950, args = {str = "", empty_str = ", val_map = std::map with 0 elements, sys_val_map = std::map with 0 elements Sub_resources = std::map with 0 elements, has_resp_modifier = false, admin_subresource_added = false}, x_meta_map = std::map with 0 elements, host = "127.0.0.1", method = 0x7f4530017e38 "PUT", script_uri = "/ test-1/", the bucket name created by the  request request_uri = "/ test-1/", request_uri_aws4 = "" Effective_uri = "", request_params = "", domain = ""}, init_state = {url_bucket = "", src_bucket = ""}, time = {tv = {tv_sec = 1470979550, tv_nsec = 592751627}}, obj_ctx = 0x0, dialect = ", req_id =", trans_id = ""}

View other information about structure * s

(gdb) p (* s). Req_id$4 = "8e78e271-aa38-4121-bfc5-c9a74b816fc5.4115.9" (gdb) p (* s). Trans_id$5 = "tx",'0', "9-0057ad5e60-1013-default" (gdb) p (* s). Host_id $6 = ""

Continue n until the get_handler function, press s to enter the function

Because it's a put type.

Directly break exit view op type is create_bucket

View current thread

(gdb) thread

[Current thread is 106 (Thread 0x7f4538ff9700 (LWP 12367))]

Q exit

After reading this article, I believe you have a certain understanding of "how gdb debugs ceph's radosgw". If you want to know more about it, you are welcome to follow the industry information channel. Thank you for reading!