ceph large omap objects

问题

出现下面告警信息

 ceph -s
  cluster:
    id:     xxxxxxxxxxxxxxxxxxxxxxx
    health: HEALTH_WARN
            7 large omap objects

获取问题 pool

# ceph health detail
HEALTH_WARN 7 large omap objects; 4 clients failing to respond to cache pressure
[WRN] LARGE_OMAP_OBJECTS: 7 large omap objects
    7 large objects found in pool 'k8s-metadata'

获取有问题 pg 信息

结果返回时间跟你当前集群拥有 PG 数量成正比
PG 越少，返回速度越快

for i in `ceph pg ls-by-pool k8s-metadata | tail -n +2 | head -n -2 | awk '{print $1}'`; do echo -n "$i: "; ceph pg $i query | grep num_large_omap_objects | head -1 | awk '{print $2}'; done | grep ": 1"
11.18a: 1,
11.1fc: 1,
11.215: 1,
11.44f: 1,
11.5d7: 1,
11.808: 1,
11.c23: 1,

获取每个 PG 信息

通过下面命令得知对应 OSD 信息

# for id in 11.18a 11.1fc 11.215 11.44f 11.5d7 11.808 11.c23
 do
   ceph pg map $id
 done

osdmap e64761 pg 11.18a (11.18a) -> up [44,215,107] acting [44,215,107]
osdmap e64761 pg 11.1fc (11.1fc) -> up [240,50,196] acting [240,50,196]
osdmap e64761 pg 11.215 (11.215) -> up [20,56,198] acting [20,56,198]
osdmap e64761 pg 11.44f (11.44f) -> up [64,102,89] acting [64,102,89]
osdmap e64761 pg 11.5d7 (11.5d7) -> up [71,28,228] acting [71,28,228]
osdmap e64761 pg 11.808 (11.808) -> up [11,8,219] acting [11,8,219]
osdmap e64761 pg 11.c23 (11.c23) -> up [28,43,108] acting [28,43,108]

针对 osd 执行 deep-scrub 操作

# for id in 44 215 107 240 50 196 20 56 198 64 102 89 71 28 228 11 8 219 28 43 108
do
    ceph osd deep-scrub osd.$id
done

instructed osd(s) 44 to deep-scrub
instructed osd(s) 215 to deep-scrub
instructed osd(s) 107 to deep-scrub
instructed osd(s) 240 to deep-scrub
instructed osd(s) 50 to deep-scrub
instructed osd(s) 196 to deep-scrub
instructed osd(s) 20 to deep-scrub
instructed osd(s) 56 to deep-scrub
instructed osd(s) 198 to deep-scrub
instructed osd(s) 64 to deep-scrub
instructed osd(s) 102 to deep-scrub
instructed osd(s) 89 to deep-scrub
instructed osd(s) 71 to deep-scrub
instructed osd(s) 28 to deep-scrub
instructed osd(s) 228 to deep-scrub
instructed osd(s) 11 to deep-scrub
instructed osd(s) 8 to deep-scrub
instructed osd(s) 219 to deep-scrub
instructed osd(s) 28 to deep-scrub
instructed osd(s) 43 to deep-scrub
instructed osd(s) 108 to deep-scrub

或者调大下面参数

ceph tell 'osd.*' injectargs --osd_deep_scrub_large_omap_object_key_threshold 2000000
ceph tell 'osd.*' injectargs --osd_deep_scrub_large_omap_object_value_sum_threshold 2147483648

原文地址：https://blog.csdn.net/signmem/article/details/137885923

免责声明：本站文章内容转载自网络资源，如本站内容侵犯了原著者的合法权益，可联系本站删除。更多内容请关注自学内容网（zxcms.com）！