tech-kern archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
High kernel time, page scan rate & reclaims?
I recently bumped my generally idle home server from 4GiB to 16GiB RAM,
and have noticed a strange behaviour occasionally. While running:
find /b -name \*gz -printx | nice xargs gzip -vt
from a ~30MiB/s USB drive to verify some archives, I see a sudden huge
burst of page scanning, page reclaims, and high kernel time (x86_64,
8 core machine). During this time, performance obviously suffers.
vmstat 1:
procs memory page disks faults cpu
r b avm fre flt re pi po fr sr w0 w1 in sy cs us sy id
0 1 11656768 41772 2569 0 0 0 9277 9515 3 3 1665 2025 1655 1 11 88
0 1 11688900 12656 4017 0 0 0 753 821 213 213 1811 3166 2670 1 4 95
1 0 11675024 28172 2551 0 0 0 8972 9227 0 0 1562 2627 1504 2 11 87
0 1 11680508 42028 2649 0 0 0 8773 8909 0 0 1225 2217 1559 3 11 86
0 2 11716988 11816 4744 0 0 0 421 472 13 13 2506 2500 2637 1 2 97
0 1 11702644 32312 2034 14325 0 0 9179 548630 42 67 960 2259 1542 0 12 88
0 2 11696984 43672 2880 0 0 0 8602 8705 0 0 1814 2059 1647 0 4 96
0 2 11728608 12028 3970 0 0 0 0 0 2 4 1615 4365 2328 1 0 99
0 1 11742108 24492 2705 0 0 0 8521 8607 57 53 1739 1944 1994 7 5 88
0 1 11750532 38572 2769 0 0 0 9053 9241 0 0 1077 2182 1708 4 7 89
0 1 12677464 26356 3768 4476 0 0 4466 228811 14 14 2119 2836 2271 8 5 87
2 0 11763924 26672 2312 9130 0 0 4683 320835 181 181 991 2818 1771 0 6 94
0 1 11785516 39212 2761 2 0 0 8667 8782 1 1 1639 2767 1659 0 4 96
1 1 12029352 39784 3961 333 0 0 8079 61311 0 0 1520 2900 2162 1 2 97
0 1 11830136 23448 2096 3914 0 0 105 487391 0 0 1385 2128 1413 0 9 91
1 1 11846328 11936 2025 2844 0 0 1151 541475 0 0 830 2104 1350 1 6 93
0 3 11945764 10796 138 2693 0 0 7 1080642 0 0 803 1710 590 0 12 88
0 2 12377108 10388 54 2178 0 0 4 1212719 0 3 72 3561 700 0 13 87
4 0 13558624 10324 41 2428 0 0 21 1375681 12 10 794 2644 619 0 40 60
2 1 14009652 9788 185 1549 0 0 5 1192572 0 0 117 3187 712 0 31 69
3 0 14009796 9772 7 1882 0 0 0 1080436 22 19 969 2361 1084 0 35 65
5 0 14009952 9744 20 5581 0 0 0 1079241 45 45 344 2113 929 0 39 60
4 0 14010280 9568 24 5330 0 0 0 1060952 1 2 682 2600 555 0 59 41
0 3 14010312 9580 9 1299 0 0 6 1080506 1 1 273 3402 816 11 50 38
3 1 14010284 9472 19 1745 0 0 8 1077594 25 25 698 1659 626 0 23 77
1 2 14010420 9464 13 1609 0 0 0 1080089 0 0 42 2898 560 0 19 81
0 2 14010436 9316 23 1838 0 0 0 1061402 0 0 728 3458 851 0 13 87
0 2 14010532 9292 14 2299 0 0 2 1080078 0 1 113 1884 576 0 15 85
3 2 14010556 9288 7 1454 0 0 8 1080671 12 16 973 2995 1074 0 38 62
5 1 14010592 9284 10 1808 0 0 8 1080241 7 10 231 1794 556 0 61 39
0 3 14010656 9076 23 1713 0 0 4 1080539 36 37 874 2521 1087 0 35 65
2 1 14011328 8164 176 1971 0 0 8 1080458 28 24 257 2938 902 0 38 62
0 4 14011444 8160 7 3074 0 0 1 1080940 0 0 839 1681 471 0 57 43
5 0 14011876 7660 164 2756 0 0 23 1080608 2 4 709 1621 656 0 25 75
0 3 14012504 6844 205 2898 0 0 0 1080425 1 5 237 1623 582 0 49 51
5 1 13794208 6892 15 4542 0 0 20 1081070 2 0 772 5039 1196 0 46 54
13 3 11868880 6784 22 5656 0 0 28 1625980 1 0 255 1343 413 0 77 23
5 1 11941952 6760 15 3600 0 0 11 1076077 7 8 886 2977 586 0 68 32
2 3 12062924 3120 959 2393 0 0 8 1133045 3 3 768 6350 1584 0 38 61
4 4 12187780 2756 59 3661 0 0 9 1112234 17 23 1357 7822 2072 1 62 37
6 1 11996460 2556 59 10342 0 0 4 1034585 94 55 615 3225 1266 0 68 31
23 1 11929016 3164 36 7641 0 0 26 1054593 43 61 1201 2863 839 0 100 0
6 3 11910632 3444 8 7435 0 0 16 1052934 16 17 594 4200 1171 0 78 21
7 0 12163800 4088 29 2777 0 0 17 1080661 0 2 370 1463 396 0 69 31
10 4 13160972 4588 17 2104 0 0 11 1081658 0 2 472 2303 484 1 67 32
8 1 13740000 5232 12 1600 0 0 52 1081632 0 2 885 1059 327 0 84 16
11 3 14013912 5684 8 1371 0 0 0 1042740 0 1 410 1557 416 4 73 23
11 0 11929820 6020 16 1432 0 0 2 1042952 0 1 530 1431 361 0 82 18
6 0 12630784 6204 75 2257 0 0 3 1042260 3 27 285 1834 612 0 70 30
6 2 12712484 6124 35 5872 0 0 26 1060289 1 1 114 2246 362 0 91 9
4 2 13605572 6212 562 3721 0 0 5 1079855 60 48 745 2298 1100 0 62 38
1 7 14013352 6224 31 2298 0 0 25 1078799 29 27 496 1795 798 0 68 32
5 3 14013716 5660 89 7506 0 0 12 1080311 16 26 937 2892 685 0 57 43
1 5 14014188 5016 160 2367 0 0 17 1079323 8 7 705 2063 691 0 18 82
0 6 13716768 4560 108 2282 0 0 17 1080616 3 7 674 2123 740 0 14 85
0 5 12962172 4228 164 2119 0 0 38 1080527 3 3 186 1944 632 0 14 86
3 2 12320244 4240 14 2461 0 0 12 1058865 0 2 394 2152 518 0 39 61
9 0 12563024 4180 13 1552 0 0 0 1079018 0 1 357 3519 926 0 76 23
0 5 12339224 1784 741 2864 0 0 25 1079423 3 8 713 4092 1077 0 15 85
3 4 11924212 1236 423 2381 0 0 61 1099422 19 22 446 3158 1004 0 15 85
2 2 12027760 4060 58 2486 0 0 4 1063631 4 0 585 2315 455 0 62 38
1 2 12374168 3436 354 2274 0 0 62 1080451 17 16 306 2367 912 0 46 54
2 1 12949216 3492 24 2507 0 0 58 1080756 29 24 369 2315 799 0 50 49
5 0 9376604 43776 820 2125 0 0 10288 1091040 72 54 729 2378 1252 0 46 54
0 1 9408724 12592 4348 0 0 0 0 0 82 106 2300 3196 2863 1 3 96
1 1 9395364 27804 3254 0 0 0 10268 10710 2 1 1770 6378 2710 5 11 84
1 1 9382756 39492 3208 0 0 0 9345 9447 0 0 2687 9029 3538 6 8 86
1 0 9414960 22432 4025 0 0 0 3770 3821 0 0 2265 4224 2674 1 1 98
vmstat -s:
4096 bytes per page
8 page colors
4056270 pages managed
10321 pages free
2767312 pages active
747242 pages inactive
0 pages paging
3688 pages wired
7960 zero pages
1 reserve pagedaemon pages
40 reserve kernel pages
354538 anonymous pages
2971292 cached file pages
192524 cached executable pages
2048 minimum free pages
2730 target free pages
1352090 maximum wired pages
1 swap devices
2621439 swap pages
0 swap pages in use
0 swap allocations
949887396 total faults taken
774924866 traps
288358214 device interrupts
922225669 CPU context switches
546785719 software interrupts
2867987678 system calls
0 pagein requests
0 pageout requests
0 pages swapped in
0 pages swapped out
1604751 forks total
1252330 forks blocked parent
1252330 forks shared address space with parent
203775339 pagealloc zero wanted and avail
306860999 pagealloc zero wanted and not avail
58583 aborts of idle page zeroing
728854095 pagealloc desired color avail
78479879 pagealloc desired color not avail
358153947 pagealloc local cpu avail
449180027 pagealloc local cpu not avail
43 faults with no memory
0 faults with no anons
0 faults had to wait on pages
0 faults found released page
354190 faults relock (352928 ok)
202394428 anon page faults
0 anon retry faults
42172180 amap copy faults
23031397 neighbour anon page faults
663678460 neighbour object page faults
204244025 locked pager get faults
354188 unlocked pager get faults
191281904 anon faults
11118481 anon copy on write faults
163305028 object faults
40964361 promote copy faults
475034438 promote zero fill faults
25320 times daemon wokeup
25363 revolutions of the clock hand
215576033 pages freed by daemon
894484695 pages scanned by daemon
0 anonymous pages scanned by daemon
215576033 object pages scanned by daemon
22678911 pages reactivated
2004 pages found busy by daemon
0 total pending pageouts
909312286 pages deactivated
2183398810 total name lookups
1927927827 good hits
106350213 negative hits
2457145 bad hits
6724600 false hits
132111755 miss
7827270 too long
63323906 pass2 hits
64765455 2passes
cache hits (88% pos + 4% neg) system 2% per-process
deletions 0%, falsehits 0%, toolong 0%
The only tunings I have are from way back:
vm.anonmin=30 # default 10
vm.filemax=30 # default 50
Kernel is:
NetBSD slave 8.0_RC1 NetBSD 8.0_RC1 (SLAVE) #0: Mon Apr 30 14:24:07 AEST 2018 stix@slave:/home/netbsd/netbsd-8/obj.amd64/home/netbsd/netbsd-8/src/sys/arch/amd64/compile/SLAVE amd64
--
Paul Ripke
"Great minds discuss ideas, average minds discuss events, small minds
discuss people."
-- Disputed: Often attributed to Eleanor Roosevelt. 1948.
Home |
Main Index |
Thread Index |
Old Index