Interpretation of PostgreSQL Source Code (130)-MVCC#14 (vacuum procedure-lazy_scan_heap function)

This section briefly introduces the processing flow of manual vacuum execution by PostgreSQL, and mainly analyzes the implementation logic of the ExecVacuum- > vacuum- > vacuum_rel- > heap_vacuum_rel- > lazy_scan_heap function, which scans open heap relation and cleans up every page in the heap.

I. data structure

Macro definition

Vacuum and Analyze command options

/ *-- * Vacuum and Analyze Statements * Vacuum and Analyze command options * * Even though these are nominally two statements, it's convenient to use * just one node type for both. Note that at least one of VACOPT_VACUUM * and VACOPT_ANALYZE must be set in options. Although there are two different statements here, you only need to use a uniform Node type. * Note that at least VACOPT_VACUUM/VACOPT_ANALYZE is set in the options. *-- * / typedef enum VacuumOption {VACOPT_VACUUM = 1 rel_pages = nblocks; vacrelstats- > scanned_pages = 0; vacrelstats- > tupcount_pages = 0; vacrelstats- > nonempty_pages = 0; vacrelstats- > latestRemovedXid = InvalidTransactionId; / / each block is recorded separately lazy_space_alloc (vacrelstats, nblocks) / / allocate memory space for frozen array frozen = palloc (sizeof (xl_heap_freeze_tuple) * MaxHeapTuplesPerPage); / * Report that we're scanning the heap, advertising total # of blocks * / / reports scanning heap and broadcasts the total number of blocks / / PROGRESS_VACUUM_PHASE_SCAN_HEAP status initprog_val [0] = PROGRESS_VACUUM_PHASE_SCAN_HEAP; initprog_val [1] = nblocks / / Total number of blocks initprog_val [2] = vacrelstats- > max_dead_tuples;// maximum number of abandoned tuples pgstat_progress_update_multi_param (3, initprog_index, initprog_val); / * * Except when aggressive is set, we want to skip pages that are * all-visible according to the visibility map, but only when we can skip * at least SKIP_PAGES_THRESHOLD consecutive pages. Since we're reading * sequentially, the OS should be doing readahead for us, so there's no * gain in skipping a page now and then; that's likely to disable * readahead and so be counterproductive. Also, skipping even a single * page means that we can't update relfrozenxid, so we only want to do it * if we can skip a goodly number of pages. * unless aggressive is set, we want to skip all visible pages determined by vm, * but only if we can skip at least SKIP_PAGES_THRESHOLD consecutive pages. * because we read in order, the operating system should read it for us in advance, * so it's no good to skip a page from time to time; this may disable readahead, which is counterproductive. * and even skipping one page means we can't update relfrozenxid, so we just want to skip a considerable number of pages. * * When aggressive is set, we can't skip pages just because they are * all-visible, but we can still skip pages that are all-frozen, since * such pages do not need freezing and do not affect the value that we can * safely set for relfrozenxid or relminmxid. * when aggressive (T) is set, we can't skip pages just because they are visible, * but we can still skip all frozen pages because they don't need to be frozen, * and it doesn't affect that we can safely set new values for relfrozenxid or relminmxid. * * Before entering the main loop, establish the invariant that * next_unskippable_block is the next block number > = blkno that we can't * skip based on the visibility map, either all-visible for a regular scan * or all-frozen for an aggressive scan. We set it to nblocks if there's * no such block. We also set up the skipping_blocks flag correctly at * this stage. * create an invariant, that is, next_unskippable_block: the next block number > = blkno, before entering the main loop, * then we cannot skip it based on vm, which is fully visible for regular scans and completely frozen for active scans. * if such a block does not exist, then let's set it to nblocks. * at the same time, we have set the skipping_blocks flag correctly at this stage. * * Note: The value returned by visibilitymap_get_status could be slightly * out-of-date, since we make this test before reading the corresponding * heap page or locking the buffer. This is OK. If we mistakenly think * that the page is all-visible or all-frozen when in fact the flag's just * been cleared, we might fail to vacuum the page. It's easy to see that * skipping a page when aggressive is not set is not a very big deal; we * might leave some dead tuples lying around, but the next vacuum will * find them. But even when aggressive * is* set, it's still OK if we miss * a page whose all-frozen marking has just been cleared. Any new XIDs * just added to that page are necessarily newer than the GlobalXmin we * computed, so they'll have no effect on the value to which we can safely * set relfrozenxid. A similar argument applies for MXIDs and relminmxid. * Note: the value returned by visibilitymap_get_status may be a bit out of date, * because we tested it before reading the corresponding heap page or locking the buffer. There's nothing wrong with that. * if we mistakenly think that the page is all visible or frozen, * when we have just cleared the flag, then we may not be able to execute vacuum. * obviously, skipping a page without aggressive is a big problem; * We may leave some DEAD tuples, but the next vacuum will find them. * however, even if aggressive is set, it doesn't matter if we miss the page that has just cleared all the frozen tags. * any new xid just added to the page must be newer than the GlobalXmin we calculated, * so they do not affect our safe setting of the relfrozenxid value. * A similar view applies to mxid and relminmxid. * * We will scan the table's last page, at least to the extent of * determining whether it has tuples or not, even if it should be skipped * according to the above rules; except when we've already determined that * it's not worth trying to truncate the table. This avoids having * lazy_truncate_heap () take access-exclusive lock on the table to attempt * a truncation that just fails immediately because there are tuples in * the last page. This is worth avoiding mainly because such a lock must * be replayed on any hot standby, where it can be disruptive. * even if pages should be skipped according to the above rules, we will scan the last page of the table, * at least to the extent that can determine whether the table has tuples to determine if there are tuples, * unless we have determined that it is not worth trying to truncate the table, then we do not need to perform such a scan. * this prevents the lazy_truncate_heap () function from accessing the table exclusively and trying to truncate immediately because there are tuples in the last page. * this is worth it, mainly because such a lock must be replay on all hot standby, because it can cause damage. * / / the next unskipped block next_unskippable_block = 0; if ((options & VACOPT_DISABLE_PAGE_SKIPPING) = = 0) {/ / option does not disable skipping PAGE while (next_unskippable_block)

< nblocks)//循环k { uint8 vmstatus;//vm状态 vmstatus = visibilitymap_get_status(onerel, next_unskippable_block, &vmbuffer); if (aggressive) { if ((vmstatus & VISIBILITYMAP_ALL_FROZEN) == 0) break;//遇到全冻结的block,跳出循环 } else { if ((vmstatus & VISIBILITYMAP_ALL_VISIBLE) == 0) break;//如非强制扫描,遇到全可见block,跳出循环 } vacuum_delay_point(); next_unskippable_block++; } } if (next_unskippable_block >

= SKIP_PAGES_THRESHOLD) skipping_blocks = true;// is greater than the threshold, then set to T else skipping_blocks = false;// otherwise F for (blkno = 0; blkno)

< nblocks; blkno++) { //循环处理每个block Buffer buf;//缓冲区编号 Page page;//page OffsetNumber offnum,//偏移 maxoff; bool tupgone, hastup; int prev_dead_count;//上次已废弃元组统计 int nfrozen;//冻结统计 Size freespace;//空闲空间 bool all_visible_according_to_vm = false;//通过vm判断可见性的标记 bool all_visible;//全可见? bool all_frozen = true; /* provided all_visible is also true */ bool has_dead_tuples;//是否存在dead元组? TransactionId visibility_cutoff_xid = InvalidTransactionId;//事务ID /* see note above about forcing scanning of last page */ //请查看上述关于最后一个page的强制扫描注释 //全部扫描&尝试截断#define FORCE_CHECK_PAGE() \ (blkno == nblocks - 1 && should_attempt_truncation(vacrelstats)) //更新统计信息 pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_SCANNED, blkno); if (blkno == next_unskippable_block) { //到达了next_unskippable_block标记的地方 /* Time to advance next_unskippable_block */ //是时候增加next_unskippable_block计数了 next_unskippable_block++; //寻找下一个需跳过的block if ((options & VACOPT_DISABLE_PAGE_SKIPPING) == 0) { while (next_unskippable_block < nblocks) { uint8 vmskipflags; vmskipflags = visibilitymap_get_status(onerel, next_unskippable_block, &vmbuffer); if (aggressive) { if ((vmskipflags & VISIBILITYMAP_ALL_FROZEN) == 0) break; } else { if ((vmskipflags & VISIBILITYMAP_ALL_VISIBLE) == 0) break; } vacuum_delay_point(); next_unskippable_block++; } } /* * We know we can't skip the current block. But set up * skipping_blocks to do the right thing at the following blocks. * 不能跳过当前block. * 但设置skipping_blocks标记处理接下来的blocks */ if (next_unskippable_block - blkno >

SKIP_PAGES_THRESHOLD) skipping_blocks = true; else skipping_blocks = false; / * * Normally, the fact that we can't skip this block must mean that * it's not all-visible. But in an aggressive vacuum we know only * that it's not all-frozen, so it might still be all-visible. * usually, the fact that we can't skip this block must mean that it is not completely visible. * but in an aggressive vacuum, all we know is that it is not completely frozen, so it may still be completely visible. * / if (aggressive & & VM_ALL_VISIBLE (onerel, blkno, & vmbuffer)) all_visible_according_to_vm = true;} else {/ / has not yet reached the next_unskippable_block mark / * * The current block is potentially skippable If we've seen a * long enough run of skippable blocks to justify skipping it, and * we're not forced to check it, then go ahead and skip. * Otherwise, the page must be at least all-visible if not * all-frozen, so we can set all_visible_according_to_vm = true. * the current block may be skippable; if we have seen enough skippable block run time, we can skip it, * and if we do not need to check, then continue to skip it. * otherwise, the page must be at least visible (if not all frozen), * so we can set all_visible_according_to_vm = true. * / if (skipping_blocks & &! FORCE_CHECK_PAGE ()) {/ * * Tricky, tricky. If this is in aggressive vacuum, the page * must have been all-frozen at the time we checked whether it * was skippable, but it might not be any more. We must be * careful to count it as a skipped all-frozen page in that * case, or else we'll think we can't update relfrozenxid and * relminmxid. If it's not an aggressive vacuum, we don't * know whether it was all-frozen, so we have to recheck; but * in this case an approximate answer is OK. * difficult, difficult. If this is in aggressive vacuum, * then by the time we check to see if the page can be skipped, the page must have been completely frozen, but it probably won't now. * in this case, we must be careful to think of it as all frozen pages skipped, * otherwise we will consider it impossible to update relfrozenxid and relminmxid. * if it's not an aggressive vacuum, we don't know if it's completely frozen, * so we have to re-check; but in this case, the approximate answer is yes. * / if (aggressive | | VM_ALL_FROZEN (onerel, blkno, & vmbuffer) vacrelstats- > page count completely frozen by frozenskipped_pages++;// + 1 continue;// jumps to the next block} all_visible_according_to_vm = true;} vacuum_delay_point () / * * If we are close to overrunning the available space for dead-tuple * TIDs, pause and do a cycle of vacuuming before we tackle this page. * if you are about to run out of free space in the DEAD tuple tid, pause and execute a vacuuming loop before processing this page. * / if ((vacrelstats- > max_dead_tuples-vacrelstats- > num_dead_tuples)

< MaxHeapTuplesPerPage && vacrelstats->

Num_dead_tuples > 0) {/ / there are obsolete tuples, and / / MaxHeapTuplesPerPage + vacrelstats- > num_dead_tuples > vacrelstats- > max_dead_tuples const int hvp_index [] = {PROGRESS_VACUUM_PHASE, PROGRESS_VACUUM_NUM_INDEX_VACUUMS}; int64 hvp_val [2] / * Before beginning index vacuuming, we release any pin we may * hold on the visibility map page. This isn't necessary for * correctness, but we do it anyway to avoid holding the pin * across a lengthy, unrelated operation. * release all pin held on vm page before starting index vacuuming. * this is not necessary for correctness, but we do so to avoid holding pin in a lengthy, irrelevant operation. * / if (BufferIsValid (vmbuffer)) {ReleaseBuffer (vmbuffer); vmbuffer = InvalidBuffer;} / * Log cleanup info before we touch indexes * / clear the log information vacuum_log_cleanup_info (onerel, vacrelstats) before starting processing indexes / * Report that we are now vacuuming indexes * / / cleaning vacuum indexes pgstat_progress_update_param (PROGRESS_VACUUM_PHASE, PROGRESS_VACUUM_PHASE_VACUUM_INDEX) / * Remove index entries * / / traverses index relation, executes vacuum / / deletes index entries pointing to vacrelstats- > dead_tuples tuple, and updates runtime statistics for (I = 0; I

< nindexes; i++) lazy_vacuum_index(Irel[i], &indstats[i], vacrelstats); /* * Report that we are now vacuuming the heap. We also increase * the number of index scans here; note that by using * pgstat_progress_update_multi_param we can update both * parameters atomically. * 报告正在vacumming heap. * 这里会增加索引扫描,注意通过设置pgstat_progress_update_multi_param参数可以同时自动更新参数. */ hvp_val[0] = PROGRESS_VACUUM_PHASE_VACUUM_HEAP; hvp_val[1] = vacrelstats->

< nindexes; i++) lazy_vacuum_index(Irel[i], &indstats[i], vacrelstats); /* Report that we are now vacuuming the heap */ //报告我们正在vacuuming heap hvp_val[0] = PROGRESS_VACUUM_PHASE_VACUUM_HEAP; hvp_val[1] = vacrelstats->

Num_index_scans+ 1; pgstat_progress_update_multi_param (2, hvp_index, hvp_val); / * Remove tuples from heap * / / Clean tuple pgstat_progress_update_param (PROGRESS_VACUUM_PHASE, PROGRESS_VACUUM_PHASE_VACUUM_HEAP); lazy_vacuum_heap (onerel, vacrelstats); vacrelstats- > num_index_scans++ } / * Vacuum the remainder of the Free Space Map. We must do this whether or * not there were indexes. * vacuum FSM. * this must be done regardless of whether an index exists or not. * / if (blkno > next_fsm_block_to_vacuum) FreeSpaceMapVacuumRange (onerel, next_fsm_block_to_vacuum, blkno); / * report all blocks vacuumed; and that we're cleaning up * / / report all blocks vacuumed completed. Pgstat_progress_update_param (PROGRESS_VACUUM_HEAP_BLKS_VACUUMED, blkno); pgstat_progress_update_param (PROGRESS_VACUUM_PHASE, PROGRESS_VACUUM_PHASE_INDEX_CLEANUP); / * Do post-vacuum cleanup and statistics update for each index * / / perform the vacuum wrap-up, updating statistics for (I = 0; I) for each index

< nindexes; i++) lazy_cleanup_index(Irel[i], indstats[i], vacrelstats); /* If no indexes, make log report that lazy_vacuum_heap would've made */ //如无索引,写日志 if (vacuumed_pages) ereport(elevel, (errmsg("\"%s\": removed %.0f row versions in %u pages", RelationGetRelationName(onerel), tups_vacuumed, vacuumed_pages))); /* * This is pretty messy, but we split it up so that we can skip emitting * individual parts of the message when not applicable. * 一起写日志会非常混乱,但我们把它拆分了,因此我们可以跳过发送消息的各个部分. */ initStringInfo(&buf); appendStringInfo(&buf, _("%.0f dead row versions cannot be removed yet, oldest xmin: %u\n"), nkeep, OldestXmin); appendStringInfo(&buf, _("There were %.0f unused item pointers.\n"), nunused); appendStringInfo(&buf, ngettext("Skipped %u page due to buffer pins, ", "Skipped %u pages due to buffer pins, ", vacrelstats->

Pinskipped_pages), vacrelstats- > pinskipped_pages); appendStringInfo (& buf, ngettext ("% u frozen page.\ n", "% u frozen pages.\ n", vacrelstats- > frozenskipped_pages), vacrelstats- > frozenskipped_pages) AppendStringInfo (& buf, ngettext ("% u page is entirely empty.\ n", "% u pages are entirely empty.\ n", empty_pages), empty_pages); appendStringInfo (& buf, _ ("% s."), pg_rusage_show (& ru0)) Ereport (elevel, (errmsg ("\"% s\ ": found% .0f removable,% .0f nonremovable row versions in% u out of% u pages", RelationGetRelationName (onerel), tups_vacuumed, num_tuples, vacrelstats- > scanned_pages, nblocks), errdetail_internal ("% s", buf.data)); pfree (buf.data) Third, follow-up and analysis

Test script, execute vacuum while performing stress test

-- session 1pgbench-c 2-C-f. / update.sql-j 1-n-T 600-U xdb testdb-- session 217 xdb@ 52 xdb@ (local]: 5432) testdb=# vacuum verbose T1

Start gdb and set breakpoint

(gdb) b lazy_scan_heapBreakpoint 1 at 0x6bc38a: file vacuumlazy.c, line 470. (gdb) cContinuing.Breakpoint 1, lazy_scan_heap (onerel=0x7f224a197788, options=5, vacrelstats=0x296d7b8, Irel=0x296d8b0, nindexes=1, aggressive=false) at vacuumlazy.c:470470 TransactionId relfrozenxid = onerel- > rd_rel- > relfrozenxid; (gdb)

Input parameters

1-relation

(gdb) p * onerel$1 = {rd_node = {spcNode = 1663, dbNode = 16402, relNode = 50820}, rd_smgr = 0x2930270, rd_refcnt = 1, rd_backend =-1, rd_islocaltemp = false, rd_isnailed = false, rd_isvalid = true, rd_indexvalid = 1'\ 001, rd_statvalid = false, rd_createSubid = 0, rd_newRelfilenodeSubid = 0, rd_rel = 0x7f224a197bb8, rd_att = 0x7f224a0d8050, rd_id = 50820, rd_lockInfo = {relId = 50820, dbId = 16402}} Rd_rules = 0x0, rd_rulescxt = 0x0, trigdesc = 0x0, rd_rsdesc = 0x0, rd_fkeylist = 0x0, rd_fkeyvalid = false, rd_partkeycxt = 0x0, rd_partkey = 0x0, rd_pdcxt = 0x0, rd_partdesc = 0x0, rd_partcheck = 0x0, rd_indexlist = 0x7f224a198fe8, rd_oidindex = 0, rd_pkindex = 0, rd_replidindex = 0, rd_statlist = 0x0, rd_indexattr = 0x0, rd_projindexattr = 0x0, rd_keyattr = 0x0, rd_pkattr = rd_pkattr, 0x0 = 0x0 Rd_projidx = 0x0, rd_pubactions = 0x0, rd_options = 0x0, rd_index = 0x0, rd_indextuple = 0x0, rd_amhandler = 0, rd_indexcxt = 0x0, rd_amroutine = 0x0, rd_opfamily = 0x0, rd_opcintype = 0x0, rd_support = 0x0, rd_supportinfo = 0x0, rd_indoption = 0x0, rd_indexprs = 0x0, rd_indpred = 0x0, rd_exclops = 0x0, rd_exclprocs = 0x0, rd_exclstrats = 0x0, rd_amcache = rd_amcache, 0x0 = 0x0, 0x0 = rd_indcollation Rd_toastoid = 0, pgstat_info = 0x2923e50} (gdb)

2-options=5, i.e. VACOPT_VACUUM | VACOPT_VERBOSE

3-vacrelstats

(gdb) p * vacrelstats$2 = {hasindex = true, old_rel_pages = 75, rel_pages = 0, scanned_pages = 0, pinskipped_pages = 0, frozenskipped_pages = 0, tupcount_pages = 0, old_live_tuples = 10000, new_rel_tuples = 0, new_live_tuples = 0, new_dead_tuples = 0, pages_removed = 0, tuples_deleted = 0, nonempty_pages = 0, num_dead_tuples = 0, max_dead_tuples = 0, dead_tuples = 0x0 Num_index_scans = 0, latestRemovedXid = 0, lock_waiter_detected = false} (gdb)

4-Irel

(gdb) p * Irel$3 = (Relation) 0x7f224a198688 (gdb) p * * Irel$4 = {rd_node = {spcNode = 1663, dbNode = 16402, relNode = 50823}, rd_smgr = 0x29302e0, rd_refcnt = 1, rd_backend =-1, rd_islocaltemp = false, rd_isnailed = false, rd_isvalid = true, rd_indexvalid = 0000, rd_statvalid = false, rd_createSubid = 0, rd_newRelfilenodeSubid = 0, rd_rel = 0x7f224a1988a0, rd_att = 0x7f224a1989b8, rd_id = 50823 Rd_lockInfo = {lockRelId = {relId = 50823, dbId = 16402}}, rd_rules = 0x0, rd_rulescxt = 0x0, trigdesc = 0x0, rd_rsdesc = 0x0, rd_fkeylist = 0x0, rd_fkeyvalid = false, rd_partkeycxt = 0x0, rd_partkey = 0x0, rd_pdcxt = 0x0, rd_partdesc = 0x0, rd_partcheck = 0x0, rd_indexlist = 0x0, rd_oidindex = 0, rd_pkindex = 0, rd_replidindex = 0, rd_statlist = 0x0, rd_indexattr = 0x0, rd_projindexattr = rd_projindexattr Rd_keyattr = 0x0, rd_pkattr = 0x0, rd_idattr = 0x0, rd_projidx = 0x0, rd_pubactions = 0x0, rd_options = 0x0, rd_index = 0x7f224a198d58, rd_indextuple = 0x7f224a198d20, rd_amhandler = 330,330,rd_indexcxt = 0x28cb340, rd_amroutine = 0x28cb480, rd_opfamily = 0x28cb598, rd_opcintype = 0x28cb5b8, rd_support = 0x28cb5d8, rd_supportinfo = 0x28cb600, rd_indoption = 0x28cb738, rd_indexprs = 0x0, rd_indpred = 0x0, rd_exclops = rd_exclops, 0x0 = 0x0, 0x0 = rd_exclprocs Rd_amcache = 0x0, rd_indcollation = 0x28cb718, rd_fdwroutine = 0x0, rd_toastoid = 0, pgstat_info = 0x2923ec8} (gdb)

5-nindexes=1, there is an index

6-aggressive=false, no need to perform a full table scan

Let's initialize the relevant variables

(gdb) n471 TransactionId relminmxid = onerel- > rd_rel- > relminmxid; (gdb) 483 Buffer vmbuffer = InvalidBuffer; (gdb) 488 const int initprog_index [] = {(gdb) 495 pg_rusage_init (& ru0); (gdb) 497 relname = RelationGetRelationName (onerel); (gdb) 498 if (aggressive) (gdb) 504 ereport (elevel, (gdb) 509 empty_pages = vacuumed_pages = 0 (gdb) 510 next_fsm_block_to_vacuum = (BlockNumber) 0; (gdb) 511 num_tuples = live_tuples = tups_vacuumed = nkeep = nunused = 0; (gdb) 514 palloc0 (nindexes * sizeof (IndexBulkDeleteResult *)); (gdb) 513 indstats = (IndexBulkDeleteResult *) (gdb) 516 nblocks = RelationGetNumberOfBlocks (onerel) (gdb) p relminmxid$5 = 1 (gdb) p ru0 $6 = {tv = {tv_sec = 1548669429, tv_usec = 578779}, ru = {ru_utime = {tv_sec = 0, tv_usec = 29531}, ru_stime = {tv_sec = 0, tv_usec = 51407}, {ru_maxrss = 7488, _ _ ru_maxrss_word = 7488}, {ru_ixrss = 0, _ ru_ixrss_word = 0}, {ru_idrss = 0 _ _ ru_idrss_word = 0}, {ru_isrss = 0, _ _ ru_isrss_word = 0}, {ru_minflt = 1819, _ _ ru_minflt_word = 1819}, {ru_majflt = 0, _ _ ru_majflt_word = 0}, {ru_nswap = 0, _ _ ru_nswap_word = 0}, {ru_inblock = 2664, _ _ ru_inblock_word = 2664}, {ru_oublock = 328 _ _ ru_oublock_word = 328}, {ru_msgsnd = 0, _ _ ru_msgsnd_word = 0}, {ru_msgrcv = 0, _ _ ru_msgrcv_word = 0}, {ru_nsignals = 0, _ _ ru_nsignals_word = 0}, {ru_nvcsw = 70, _ _ ru_nvcsw_word = 70}, {ru_nivcsw = 3, _ _ ru_nivcsw_word = 3}} (gdb) p relname$7 = 0x7f224a197bb8 "T1" (gdb)

Get the total number of blocks

(gdb) n517 vacrelstats- > rel_pages = nblocks; (gdb) p nblocks$8 = 75 (gdb)

Initialize statistics and related arrays

(gdb) n518 vacrelstats- > scanned_pages = 0; (gdb) 519 vacrelstats- > tupcount_pages = 0; (gdb) 520 vacrelstats- > nonempty_pages = 0; (gdb) 521 vacrelstats- > latestRemovedXid = InvalidTransactionId; (gdb) 523 lazy_space_alloc (vacrelstats, nblocks); (gdb) 524 frozen = palloc (sizeof (xl_heap_freeze_tuple) * MaxHeapTuplesPerPage); (gdb) 527 initprog_val [0] = PROGRESS_VACUUM_PHASE_SCAN_HEAP (gdb) 528 initprog_val [1] = nblocks; (gdb) 529 initprog_val [2] = vacrelstats- > max_dead_tuples; (gdb) 530 pgstat_progress_update_multi_param (3, initprog_index, initprog_val) (gdb) p * vacrelstats$9 = {hasindex = true, old_rel_pages = 75, rel_pages = 75, scanned_pages = 0, pinskipped_pages = 0, frozenskipped_pages = 0, tupcount_pages = 0, old_live_tuples = 10000, new_rel_tuples = 0, new_live_tuples = 0, new_dead_tuples = 0, pages_removed = 0, tuples_deleted = 0, nonempty_pages = 0, num_dead_tuples = 0, max_dead_tuples = 21825, dead_tuples = 0x297e820 Num_index_scans = 0, latestRemovedXid = 0, lock_waiter_detected = false} (gdb)

Calculate the next block that cannot be skipped

The 0th block cannot be skipped (0

< 32),设置标记skipping_blocks为F (gdb) n576 next_unskippable_block = 0;(gdb) 577 if ((options & VACOPT_DISABLE_PAGE_SKIPPING) == 0)(gdb) 579 while (next_unskippable_block < nblocks)(gdb) 583 vmstatus = visibilitymap_get_status(onerel, next_unskippable_block,(gdb) 585 if (aggressive)(gdb) p vmstatus$10 = 0 '\000'(gdb) n592 if ((vmstatus & VISIBILITYMAP_ALL_VISIBLE) == 0)(gdb) 593 break;(gdb) 600 if (next_unskippable_block >

= SKIP_PAGES_THRESHOLD) (gdb) p next_unskippable_block$11 = 0 (gdb) p SKIP_PAGES_THRESHOLD$12 = 32 (gdb) n603 skipping_blocks = false; (gdb)

Start traversing each block

Initialize related variables

(gdb) 605 for (blkno = 0; blkno)

< nblocks; blkno++)(gdb) 616 bool all_visible_according_to_vm = false;(gdb) 618 bool all_frozen = true; /* provided all_visible is also true */(gdb) 620 TransactionId visibility_cutoff_xid = InvalidTransactionId;(gdb) 626 pgstat_progress_update_param(PROGRESS_VACUUM_HEAP_BLKS_SCANNED, blkno);(gdb) 628 if (blkno == next_unskippable_block)(gdb) blkno == next_unskippable_block,获取下一个不可跳过的block (gdb) p blkno$13 = 0(gdb) p next_unskippable_block$14 = 0(gdb) n631 next_unskippable_block++;(gdb) 632 if ((options & VACOPT_DISABLE_PAGE_SKIPPING) == 0)(gdb) 634 while (next_unskippable_block < nblocks)(gdb) 638 vmskipflags = visibilitymap_get_status(onerel,(gdb) 641 if (aggressive)(gdb) p vmskipflags$15 = 0 '\000'(gdb) n648 if ((vmskipflags & VISIBILITYMAP_ALL_VISIBLE) == 0)(gdb) 649 break;(gdb) 660 if (next_unskippable_block - blkno >

SKIP_PAGES_THRESHOLD) (gdb) p next_unskippable_block$16 = 1 (gdb) n 1047 if (onerel- > rd_rel- > relhasoids & & (gdb) 1132 if (tupgone) (gdb)

Tupgone is F to determine whether it needs to be frozen (F)

Get offset, traverse tuple

(gdb) p tupgone$17 = false (gdb) n1144 num_tuples + = 1; (gdb) 1145 hastup = true (gdb) 1151 if (heap_prepare_freeze_tuple (tuple.t_data, (gdb) 1154 & frozen [nfrozen], (gdb) p nfrozen$18 = 0 (gdb) n1151 if (heap_prepare_freeze_tuple (tuple.t_data) (gdb) 1158 if (! tuple_totally_frozen) (gdb) 1159 all_frozen = false (gdb) 958 offnum = OffsetNumberNext (offnum)) (gdb) 956 for (offnum = FirstOffsetNumber; (gdb)

The tuple is normal

(gdb) p offnum$19 = 3 (gdb) n962 itemid = PageGetItemId (page, offnum); (gdb) 965 if (! ItemIdIsUsed (itemid)) (gdb) 972 if (ItemIdIsRedirected (itemid)) (gdb) 978 ItemPointerSet (& (tuple.t_self), blkno, offnum); (gdb) 986 if (ItemIdIsDead (itemid)) (gdb) 993 Assert (ItemIdIsNormal (itemid)) (gdb) 995 tuple.t_data = (HeapTupleHeader) PageGetItem (page, itemid); (gdb) 996 tuple.t_len = ItemIdGetLength (itemid); (gdb) 997 tuple.t_tableOid = RelationGetRelid (onerel); (gdb) 999 tupgone = false; (gdb)

Call HeapTupleSatisfiesVacuum to determine the tuple state, the main purpose is whether a tuple can be visible to all running transactions

The tuple is Live tuple

1012 switch (& tuple, OldestXmin, buf) (gdb) (gdb) n1047 if (onerel- > rd_rel- > relhasoids & & (gdb) n1056 live_tuples + = 1; (gdb) 1067 if (all_visible) (gdb) p all_visible$20 = false

Jump out of the cycle

(gdb) b vacuumlazy.c:1168Breakpoint 2 at 0x6bd4e7: file vacuumlazy.c, line 1168. (gdb) cContinuing.Breakpoint 2, lazy_scan_heap (onerel=0x7f224a197788, options=5, vacrelstats=0x296d7b8, Irel=0x296d8b0, nindexes=1, aggressive=false) at vacuumlazy.c:11681168 if (nfrozen > 0) (gdb)

Update statistics

(gdb) n1203 if (nindexes = = 0 & & (gdb) p nfrozen$23 = 0 (gdb) n1232 freespace = PageGetHeapFreeSpace (page) (gdb) 1235 if (all_visible & &! all_visible_according_to_vm) (gdb) 1268 else if (all_visible_according_to_vm & & PageIsAllVisible (page) (gdb) 1290 else if (PageIsAllVisible (page) & & has_dead_tuples) (gdb) 1305 else if (all_visible_according_to_vm & all_visible & & all_frozen & & (gdb) 1318 UnlockReleaseBuffer (buf)) (gdb) 1321 if (hastup) (gdb) 1322 vacrelstats- > nonempty_pages = blkno + 1; (gdb) p hastup$24 = true (gdb) n1331 if (vacrelstats- > num_dead_tuples = = prev_dead_count) (gdb) 1332 RecordPageWithFreeSpace (onerel, blkno, freespace)

Move on to the next block

(gdb) 605 for (blkno = 0; blkno)

< nblocks; blkno++)(gdb) p blkno$25 = 0(gdb) n616 bool all_visible_according_to_vm = false;(gdb) p blkno$26 = 1(gdb) 判断(vacrelstats->

Max_dead_tuples-vacrelstats- > num_dead_tuples)

< MaxHeapTuplesPerPage && vacrelstats->

Num_dead_tuples > 0. If not satisfied, continue execution.

(gdb) 701 vacuum_delay_point (); (gdb) 707 if ((vacrelstats- > max_dead_tuples-vacrelstats- > num_dead_tuples)

< MaxHeapTuplesPerPage &&(gdb) p vacrelstats->

Max_dead_tuples$27 = 21825 (gdb) p vacrelstats- > num_dead_tuples$28 = 0 (gdb) p MaxHeapTuplesPerPageNo symbol "_ builtin_offsetof" in current context. (gdb)

Read buffer in an extended way

(gdb) n783 visibilitymap_pin (onerel, blkno, & vmbuffer); (gdb) 785 buf = ReadBufferExtended (onerel, MAIN_FORKNUM, blkno, (gdb) 789 if (! ConditionalLockBufferForCleanup (buf)) (gdb)

Get buffer cleanup lock, success!

Call heap_page_prune to clean up all HOT-update chains in the page

(gdb) n847 vacrelstats- > scanned_pages++; (gdb) 848 vacrelstats- > tupcount_pages++; (gdb) 850 page = BufferGetPage (buf); (gdb) 852 if (PageIsNew (page)) (gdb) 894 if (PageIsEmpty (page)) (gdb) 938 tups_vacuumed + = heap_page_prune (onerel, buf, OldestXmin, false, (gdb) 945 all_visible = true; (gdb)

Traversing row pointers in page

956 for (offnum = FirstOffsetNumber; (gdb) p maxoff$29 = 291 (gdb) $30 (gdb) n962 itemid = PageGetItemId (page, offnum); (gdb) n965 if (! ItemIdIsUsed (itemid)) (gdb) 972 if (ItemIdIsRedirected (itemid)) (gdb) 978 ItemPointerSet (& (tuple.t_self), blkno, offnum) (gdb) 986 if (ItemIdIsDead (itemid)) (gdb) 993 Assert (ItemIdIsNormal (itemid)); (gdb) 995 tuple.t_data = (HeapTupleHeader) PageGetItem (page, itemid); (gdb) 996 tuple.t_len = ItemIdGetLength (itemid); (gdb) 997 tuple.t_tableOid = RelationGetRelid (onerel); (gdb) 999 tupgone = false (gdb) 1012 switch (& tuple, OldestXmin, buf) (gdb) 1099 nkeep + = 1; (gdb) 1100 all_visible = false; (gdb) 1101 break; (gdb) 1132 if (tupgone) (gdb) 1144 num_tuples + = 1

Jump out of the cycle

(gdb) cContinuing.Breakpoint 2, lazy_scan_heap (onerel=0x7f224a197788, options=5, vacrelstats=0x296d7b8, Irel=0x296d8b0, nindexes=1, aggressive=false) at vacuumlazy.c:11681168 if (nfrozen > 0) (gdb)

DONE!

IV. Reference materials

PG Source Code