forum.opennet.ru

Составление сообщения

Исходное сообщение

"Solaris ZFS жутко медленный под NFS"
Отправлено Urgen, 31-Окт-08 16:11

>Так вроде всё нормально. Надеюсь нагрузка шла во время съёма lockstat-а? :)
Как г-рят наши прибалтийские братья - исфини-и-и-ите стор-мо-зи-и-ил :)
вот локстат под нагрузкой и тут же появились adaptive mutex block:
Adaptive mutex spin: 236 events in 5.043 seconds (47 events/sec)
-------------------------------------------------------------------------------
Count indv cuml rcnt     nsec Lock                   Hottest Caller
   31  25%  25% 0.00    18859 0xffffff014a55b2d8     cv_wait+0x69
      nsec ------ Time Distribution ------ count     Stack
       256 |                               1         taskq_thread+0x11e
       512 |@@@@                           5         thread_start+0x8
      1024 |@@@@@@                         7
      2048 |@                              2
      4096 |@@@@@                          6
      8192 |@@@@@                          6
     16384 |@                              2
     32768 |                               0
     65536 |                               0
    131072 |                               0
    262144 |                               1
    524288 |                               1
-------------------------------------------------------------------------------
Count indv cuml rcnt     nsec Lock                   Hottest Caller
   46  19%  43% 0.00     9561 0xffffff01480c8e40     lookuppnat+0x67
      nsec ------ Time Distribution ------ count     Stack
       512 |@@@                            6         lookupnameat+0x82
      1024 |@@@@                           7         cstatat_getvp+0x160
      2048 |@@@                            6         cstatat32+0x7d
      4096 |@@@@@@@@                       13        lstat32+0x31
      8192 |@@@@@                          9         lxstat32+0x24
     16384 |@                              3         _sys_sysenter_post_swapgs+0x14b
     32768 |                               0
     65536 |                               0
    131072 |                               0
    262144 |@                              2
-------------------------------------------------------------------------------
Count indv cuml rcnt     nsec Lock                   Hottest Caller
    2  15%  58% 0.00   177580 0xffffff0147fbe2d0     cv_wait+0x69
      nsec ------ Time Distribution ------ count     Stack
    262144 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 2         txg_thread_wait+0x60
                                                     txg_timelimit_thread+0xaf
                                                     thread_start+0x8
-------------------------------------------------------------------------------
Count indv cuml rcnt     nsec Lock                   Hottest Caller
    2  13%  71% 0.00   158750 0xffffff014a7d7cd0     cv_wait+0x69
      nsec ------ Time Distribution ------ count     Stack
    262144 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 2         txg_thread_wait+0x60
                                                     txg_timelimit_thread+0xaf
                                                     thread_start+0x8
-------------------------------------------------------------------------------
Count indv cuml rcnt     nsec Lock                   Hottest Caller
   37   6%  78% 0.00     4167 0xffffff014a80b118     sd_start_cmds+0x304
      nsec ------ Time Distribution ------ count     Stack
       256 |                               1         sd_core_iostart+0x17b
       512 |                               0         sd_mapblockaddr_iostart+0x185
      1024 |@                              2         sd_xbuf_strategy+0x50
      2048 |@@@                            4         xbuf_iostart+0x103
      4096 |@@@@@@@@@@@@@@                 18        ddi_xbuf_qstrategy+0x60
      8192 |@@@@@                          7         sdstrategy+0xec
     16384 |@@@@                           5         bdev_strategy+0x77
                                                     ldi_strategy+0x54
                                                     vdev_disk_io_start+0x154
-------------------------------------------------------------------------------
Count indv cuml rcnt     nsec Lock                   Hottest Caller
    8   6%  83% 0.00    16504 0xffffff0147fbf988     vdev_queue_io+0x85
      nsec ------ Time Distribution ------ count     Stack
      2048 |@@@                            1         vdev_disk_io_start+0x54
      4096 |@@@@@@@                        2         zio_vdev_io_start+0x148
      8192 |@@@@@@@@@@@@@@@                4         zio_execute+0x7f
     16384 |                               0         zio_nowait+0x11
     32768 |                               0         vdev_mirror_io_start+0x18f
     65536 |                               0         zio_vdev_io_start+0x156
    131072 |@@@                            1         zio_execute+0x7f
                                                     taskq_thread+0x1a7
                                                     thread_start+0x8
-------------------------------------------------------------------------------
Count indv cuml rcnt     nsec Lock                   Hottest Caller
   13   5%  88% 0.00     8355 0xffffff01471a10c0     ehci_intr+0x2a
      nsec ------ Time Distribution ------ count     Stack
      1024 |@@@@                           2         av_dispatch_autovect+0x78
      2048 |                               0         dispatch_hardint+0x2f
      4096 |@@@@                           2         switch_sp_and_call+0x13
      8192 |@@@@@@                         3
     16384 |@@@@@@@@@@@                    5
     32768 |@@                             1
-------------------------------------------------------------------------------
Count indv cuml rcnt     nsec Lock                   Hottest Caller
   32   3%  91% 0.00     2387 0xffffff014b6221f0     aac_tran_start+0xc6
      nsec ------ Time Distribution ------ count     Stack
       256 |                               1         scsi_transport+0x7a
       512 |@@@@                           5         sd_start_cmds+0x2f4
      1024 |@@@@                           5         sd_core_iostart+0x17b
      2048 |@@@@@@@@@@                     11        sd_mapblockaddr_iostart+0x185
      4096 |@@@                            4         sd_xbuf_strategy+0x50
      8192 |@@@                            4         xbuf_iostart+0x103
     16384 |@                              2         ddi_xbuf_qstrategy+0x60
                                                     sdstrategy+0xec
                                                     bdev_strategy+0x77
-------------------------------------------------------------------------------
Count indv cuml rcnt     nsec Lock                   Hottest Caller
   17   2%  93% 0.00     3139 0xffffff014a643658     sd_start_cmds+0x304
      nsec ------ Time Distribution ------ count     Stack
      2048 |@@@                            2         sd_core_iostart+0x17b
      4096 |@@@@@@@@@@@@@@@@@@@@@@@@       14        sd_mapblockaddr_iostart+0x185
      8192 |@                              1         sd_xbuf_strategy+0x50
                                                     xbuf_iostart+0x103
                                                     ddi_xbuf_qstrategy+0x60
                                                     sdstrategy+0xec
                                                     bdev_strategy+0x77
                                                     ldi_strategy+0x54
                                                     vdev_disk_io_start+0x154
-------------------------------------------------------------------------------
Count indv cuml rcnt     nsec Lock                   Hottest Caller
    4   1%  95% 0.00     6779 0xffffff0147ced528     spa_config_enter+0x34
      nsec ------ Time Distribution ------ count     Stack
      1024 |@@@@@@@                        1         bp_get_dasize+0x3d
      2048 |                               0         dsl_dataset_block_born+0x32
      4096 |                               0         dbuf_write_ready+0x20f
      8192 |@@@@@@@@@@@@@@@                2         arc_write_ready+0x49
     16384 |@@@@@@@                        1         zio_ready+0x2c
                                                     zio_execute+0x7f
                                                     taskq_thread+0x1a7
                                                     thread_start+0x8
-------------------------------------------------------------------------------
Adaptive mutex block: 16 events in 5.043 seconds (3 events/sec)
-------------------------------------------------------------------------------
Count indv cuml rcnt     nsec Lock                   Hottest Caller
    7  66%  66% 0.00    57913 0xffffff014a55b2d8     taskq_dispatch+0x2c9
      nsec ------ Time Distribution ------ count     Stack
      4096 |@@@@                           1         zio_interrupt+0x30
      8192 |@@@@@@@@                       2         vdev_disk_io_intr+0x58
     16384 |                               0         biodone+0x8a
     32768 |                               0         sd_buf_iodone+0x62
     65536 |@@@@                           1         sd_mapblockaddr_iodone+0x48
    131072 |@@@@@@@@                       2         sd_return_command+0x158
    262144 |@@@@                           1         sdintr+0x50a
                                                     aac_drain_comp_q+0x10b
                                                     aac_intr_new+0x40
-------------------------------------------------------------------------------
Count indv cuml rcnt     nsec Lock                   Hottest Caller
    3  18%  84% 0.00    37741 0xffffff014b6221f0     aac_tran_start+0xc6
      nsec ------ Time Distribution ------ count     Stack
      8192 |@@@@@@@@@@                     1         scsi_transport+0x7a
     16384 |                               0         sd_start_cmds+0x2f4
     32768 |                               0         sd_core_iostart+0x17b
     65536 |@@@@@@@@@@@@@@@@@@@@           2         sd_mapblockaddr_iostart+0x185
                                                     sd_xbuf_strategy+0x50
                                                     xbuf_iostart+0x103
                                                     ddi_xbuf_qstrategy+0x60
                                                     sdstrategy+0xec
                                                     bdev_strategy+0x77
-------------------------------------------------------------------------------
Count indv cuml rcnt     nsec Lock                   Hottest Caller
    1   8%  92% 0.00    48984 0xffffff014a3acec8     vdev_queue_io_done+0x3b
      nsec ------ Time Distribution ------ count     Stack
     65536 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1         vdev_disk_io_done+0x18
                                                     zio_vdev_io_done+0x22
                                                     zio_execute+0x7f
                                                     taskq_thread+0x1a7
                                                     thread_start+0x8
-------------------------------------------------------------------------------
Count indv cuml rcnt     nsec Lock                   Hottest Caller
    4   6%  98% 0.00     9927 0xffffff014a80b118     sdintr+0x33
      nsec ------ Time Distribution ------ count     Stack
      8192 |@@@@@@@                        1         aac_drain_comp_q+0x10b
     16384 |@@@@@@@@@@@@@@@@@@@@@@         3         aac_intr_new+0x40
                                                     av_dispatch_autovect+0x78
                                                     dispatch_hardint+0x2f
                                                     switch_sp_and_call+0x13
-------------------------------------------------------------------------------
Count indv cuml rcnt     nsec Lock                   Hottest Caller
    1   2% 100% 0.00    10245 0xffffff014b90cd68     txg_list_add+0x2d
      nsec ------ Time Distribution ------ count     Stack
     16384 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1         dsl_dir_dirty+0x2d
                                                     dsl_dir_diduse_space+0x40
                                                     dsl_dataset_block_born+0x137
                                                     dbuf_write_ready+0x20f
                                                     arc_write_ready+0x49
                                                     zio_ready+0x2c
                                                     zio_execute+0x7f
                                                     taskq_thread+0x1a7
                                                     thread_start+0x8
-------------------------------------------------------------------------------
Spin lock spin: 11 events in 5.043 seconds (2 events/sec)
-------------------------------------------------------------------------------
Count indv cuml rcnt     nsec Lock                   Hottest Caller
    9 100% 100% 0.00    19177 cpu0_disp[56]          disp_lock_enter_high+0x11
      nsec ------ Time Distribution ------ count     Stack
       128 |@@@                            1         setfrontdq+0x190
       256 |                               0         ts_wakeup+0x149
       512 |@@@@@@@@@@@@@                  4         sleepq_wakeone_chan+0x83
      1024 |@@@                            1         cv_signal+0x89
      2048 |                               0         lwp_unpark+0x6e
      4096 |                               0         lwp_park+0x47
      8192 |@@@@@@                         2         syslwp_park+0x2d
     16384 |                               0         _sys_sysenter_post_swapgs+0x14b
     32768 |                               0
     65536 |                               0
    131072 |                               0
    262144 |@@@                            1
-------------------------------------------------------------------------------
Count indv cuml rcnt     nsec Lock                   Hottest Caller
    1   0% 100% 0.00      467 turnstile_table[4096]  disp_lock_enter+0x31
      nsec ------ Time Distribution ------ count     Stack
       512 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1         turnstile_lookup+0x61
                                                     mutex_vector_enter+0x1b4
                                                     taskq_dispatch+0x2c9
                                                     zio_interrupt+0x30
                                                     vdev_disk_io_intr+0x58
                                                     biodone+0x8a
                                                     sd_buf_iodone+0x62
                                                     sd_mapblockaddr_iodone+0x48
                                                     sd_return_command+0x158
-------------------------------------------------------------------------------
Count indv cuml rcnt     nsec Lock                   Hottest Caller
    1   0% 100% 0.00      150 0xffffff0149fba968     disp_lock_enter_high+0x11
      nsec ------ Time Distribution ------ count     Stack
       256 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1         setbackdq+0x226
                                                     sleepq_wakeall_chan+0x78
                                                     cv_broadcast+0x5b
                                                     txg_quiesce_thread+0x115
                                                     thread_start+0x8
-------------------------------------------------------------------------------

Исходное сообщение
"Solaris ZFS жутко медленный под NFS" Отправлено Urgen, 31-Окт-08 16:11
>Так вроде всё нормально. Надеюсь нагрузка шла во время съёма lockstat-а? :) Как г-рят наши прибалтийские братья - исфини-и-и-ите стор-мо-зи-и-ил :) вот локстат под нагрузкой и тут же появились adaptive mutex block: Adaptive mutex spin: 236 events in 5.043 seconds (47 events/sec) ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Hottest Caller 31 25% 25% 0.00 18859 0xffffff014a55b2d8 cv_wait+0x69 nsec ------ Time Distribution ------ count Stack 256 \| 1 taskq_thread+0x11e 512 \|@@@@ 5 thread_start+0x8 1024 \|@@@@@@ 7 2048 \|@ 2 4096 \|@@@@@ 6 8192 \|@@@@@ 6 16384 \|@ 2 32768 \| 0 65536 \| 0 131072 \| 0 262144 \| 1 524288 \| 1 ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Hottest Caller 46 19% 43% 0.00 9561 0xffffff01480c8e40 lookuppnat+0x67 nsec ------ Time Distribution ------ count Stack 512 \|@@@ 6 lookupnameat+0x82 1024 \|@@@@ 7 cstatat_getvp+0x160 2048 \|@@@ 6 cstatat32+0x7d 4096 \|@@@@@@@@ 13 lstat32+0x31 8192 \|@@@@@ 9 lxstat32+0x24 16384 \|@ 3 _sys_sysenter_post_swapgs+0x14b 32768 \| 0 65536 \| 0 131072 \| 0 262144 \|@ 2 ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Hottest Caller 2 15% 58% 0.00 177580 0xffffff0147fbe2d0 cv_wait+0x69 nsec ------ Time Distribution ------ count Stack 262144 \|@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 2 txg_thread_wait+0x60 txg_timelimit_thread+0xaf thread_start+0x8 ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Hottest Caller 2 13% 71% 0.00 158750 0xffffff014a7d7cd0 cv_wait+0x69 nsec ------ Time Distribution ------ count Stack 262144 \|@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 2 txg_thread_wait+0x60 txg_timelimit_thread+0xaf thread_start+0x8 ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Hottest Caller 37 6% 78% 0.00 4167 0xffffff014a80b118 sd_start_cmds+0x304 nsec ------ Time Distribution ------ count Stack 256 \| 1 sd_core_iostart+0x17b 512 \| 0 sd_mapblockaddr_iostart+0x185 1024 \|@ 2 sd_xbuf_strategy+0x50 2048 \|@@@ 4 xbuf_iostart+0x103 4096 \|@@@@@@@@@@@@@@ 18 ddi_xbuf_qstrategy+0x60 8192 \|@@@@@ 7 sdstrategy+0xec 16384 \|@@@@ 5 bdev_strategy+0x77 ldi_strategy+0x54 vdev_disk_io_start+0x154 ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Hottest Caller 8 6% 83% 0.00 16504 0xffffff0147fbf988 vdev_queue_io+0x85 nsec ------ Time Distribution ------ count Stack 2048 \|@@@ 1 vdev_disk_io_start+0x54 4096 \|@@@@@@@ 2 zio_vdev_io_start+0x148 8192 \|@@@@@@@@@@@@@@@ 4 zio_execute+0x7f 16384 \| 0 zio_nowait+0x11 32768 \| 0 vdev_mirror_io_start+0x18f 65536 \| 0 zio_vdev_io_start+0x156 131072 \|@@@ 1 zio_execute+0x7f taskq_thread+0x1a7 thread_start+0x8 ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Hottest Caller 13 5% 88% 0.00 8355 0xffffff01471a10c0 ehci_intr+0x2a nsec ------ Time Distribution ------ count Stack 1024 \|@@@@ 2 av_dispatch_autovect+0x78 2048 \| 0 dispatch_hardint+0x2f 4096 \|@@@@ 2 switch_sp_and_call+0x13 8192 \|@@@@@@ 3 16384 \|@@@@@@@@@@@ 5 32768 \|@@ 1 ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Hottest Caller 32 3% 91% 0.00 2387 0xffffff014b6221f0 aac_tran_start+0xc6 nsec ------ Time Distribution ------ count Stack 256 \| 1 scsi_transport+0x7a 512 \|@@@@ 5 sd_start_cmds+0x2f4 1024 \|@@@@ 5 sd_core_iostart+0x17b 2048 \|@@@@@@@@@@ 11 sd_mapblockaddr_iostart+0x185 4096 \|@@@ 4 sd_xbuf_strategy+0x50 8192 \|@@@ 4 xbuf_iostart+0x103 16384 \|@ 2 ddi_xbuf_qstrategy+0x60 sdstrategy+0xec bdev_strategy+0x77 ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Hottest Caller 17 2% 93% 0.00 3139 0xffffff014a643658 sd_start_cmds+0x304 nsec ------ Time Distribution ------ count Stack 2048 \|@@@ 2 sd_core_iostart+0x17b 4096 \|@@@@@@@@@@@@@@@@@@@@@@@@ 14 sd_mapblockaddr_iostart+0x185 8192 \|@ 1 sd_xbuf_strategy+0x50 xbuf_iostart+0x103 ddi_xbuf_qstrategy+0x60 sdstrategy+0xec bdev_strategy+0x77 ldi_strategy+0x54 vdev_disk_io_start+0x154 ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Hottest Caller 4 1% 95% 0.00 6779 0xffffff0147ced528 spa_config_enter+0x34 nsec ------ Time Distribution ------ count Stack 1024 \|@@@@@@@ 1 bp_get_dasize+0x3d 2048 \| 0 dsl_dataset_block_born+0x32 4096 \| 0 dbuf_write_ready+0x20f 8192 \|@@@@@@@@@@@@@@@ 2 arc_write_ready+0x49 16384 \|@@@@@@@ 1 zio_ready+0x2c zio_execute+0x7f taskq_thread+0x1a7 thread_start+0x8 ------------------------------------------------------------------------------- Adaptive mutex block: 16 events in 5.043 seconds (3 events/sec) ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Hottest Caller 7 66% 66% 0.00 57913 0xffffff014a55b2d8 taskq_dispatch+0x2c9 nsec ------ Time Distribution ------ count Stack 4096 \|@@@@ 1 zio_interrupt+0x30 8192 \|@@@@@@@@ 2 vdev_disk_io_intr+0x58 16384 \| 0 biodone+0x8a 32768 \| 0 sd_buf_iodone+0x62 65536 \|@@@@ 1 sd_mapblockaddr_iodone+0x48 131072 \|@@@@@@@@ 2 sd_return_command+0x158 262144 \|@@@@ 1 sdintr+0x50a aac_drain_comp_q+0x10b aac_intr_new+0x40 ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Hottest Caller 3 18% 84% 0.00 37741 0xffffff014b6221f0 aac_tran_start+0xc6 nsec ------ Time Distribution ------ count Stack 8192 \|@@@@@@@@@@ 1 scsi_transport+0x7a 16384 \| 0 sd_start_cmds+0x2f4 32768 \| 0 sd_core_iostart+0x17b 65536 \|@@@@@@@@@@@@@@@@@@@@ 2 sd_mapblockaddr_iostart+0x185 sd_xbuf_strategy+0x50 xbuf_iostart+0x103 ddi_xbuf_qstrategy+0x60 sdstrategy+0xec bdev_strategy+0x77 ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Hottest Caller 1 8% 92% 0.00 48984 0xffffff014a3acec8 vdev_queue_io_done+0x3b nsec ------ Time Distribution ------ count Stack 65536 \|@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 vdev_disk_io_done+0x18 zio_vdev_io_done+0x22 zio_execute+0x7f taskq_thread+0x1a7 thread_start+0x8 ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Hottest Caller 4 6% 98% 0.00 9927 0xffffff014a80b118 sdintr+0x33 nsec ------ Time Distribution ------ count Stack 8192 \|@@@@@@@ 1 aac_drain_comp_q+0x10b 16384 \|@@@@@@@@@@@@@@@@@@@@@@ 3 aac_intr_new+0x40 av_dispatch_autovect+0x78 dispatch_hardint+0x2f switch_sp_and_call+0x13 ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Hottest Caller 1 2% 100% 0.00 10245 0xffffff014b90cd68 txg_list_add+0x2d nsec ------ Time Distribution ------ count Stack 16384 \|@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 dsl_dir_dirty+0x2d dsl_dir_diduse_space+0x40 dsl_dataset_block_born+0x137 dbuf_write_ready+0x20f arc_write_ready+0x49 zio_ready+0x2c zio_execute+0x7f taskq_thread+0x1a7 thread_start+0x8 ------------------------------------------------------------------------------- Spin lock spin: 11 events in 5.043 seconds (2 events/sec) ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Hottest Caller 9 100% 100% 0.00 19177 cpu0_disp[56] disp_lock_enter_high+0x11 nsec ------ Time Distribution ------ count Stack 128 \|@@@ 1 setfrontdq+0x190 256 \| 0 ts_wakeup+0x149 512 \|@@@@@@@@@@@@@ 4 sleepq_wakeone_chan+0x83 1024 \|@@@ 1 cv_signal+0x89 2048 \| 0 lwp_unpark+0x6e 4096 \| 0 lwp_park+0x47 8192 \|@@@@@@ 2 syslwp_park+0x2d 16384 \| 0 _sys_sysenter_post_swapgs+0x14b 32768 \| 0 65536 \| 0 131072 \| 0 262144 \|@@@ 1 ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Hottest Caller 1 0% 100% 0.00 467 turnstile_table[4096] disp_lock_enter+0x31 nsec ------ Time Distribution ------ count Stack 512 \|@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 turnstile_lookup+0x61 mutex_vector_enter+0x1b4 taskq_dispatch+0x2c9 zio_interrupt+0x30 vdev_disk_io_intr+0x58 biodone+0x8a sd_buf_iodone+0x62 sd_mapblockaddr_iodone+0x48 sd_return_command+0x158 ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Hottest Caller 1 0% 100% 0.00 150 0xffffff0149fba968 disp_lock_enter_high+0x11 nsec ------ Time Distribution ------ count Stack 256 \|@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 setbackdq+0x226 sleepq_wakeall_chan+0x78 cv_broadcast+0x5b txg_quiesce_thread+0x115 thread_start+0x8 -------------------------------------------------------------------------------

Ваше сообщение
Имя*:
EMail:	Для отправки новых сообщений в текущей нити на email укажите знак ! перед адресом, например, !user@host.ru (!! - не показывать email). Более тонкая настройка отправки ответов производится в профиле зарегистрированного участника форума.
Заголовок*:
Сообщение*:	>>Так вроде всё нормально. Надеюсь нагрузка шла во время съёма lockstat-а? :) > Как г-рят наши прибалтийские братья - исфини-и-и-ите стор-мо-зи-и-ил :) > вот локстат под нагрузкой и тут же появились adaptive mutex block: > Adaptive mutex spin: 236 events in 5.043 seconds (47 events/sec) > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock > > Hottest Caller > 31 25% 25% 0.00 > 18859 0xffffff014a55b2d8 cv_wait+0x69 > nsec ------ Time Distribution ------ count > Stack > 256 \| > > > 1 > taskq_thread+0x11e > 512 \|@@@@ > > > 5 thread_start+0x8 > 1024 \|@@@@@@ > > 7 > 2048 \|@ > > > 2 > 4096 \|@@@@@ > > 6 > 8192 \|@@@@@ > > 6 > 16384 \|@ > > > 2 > 32768 \| > > > 0 > 65536 \| > > > 0 > 131072 \| > > > 0 > 262144 \| > > > 1 > 524288 \| > > > 1 > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock > > Hottest Caller > 46 19% 43% 0.00 > 9561 0xffffff01480c8e40 lookuppnat+0x67 > nsec ------ Time Distribution ------ count > Stack > 512 \|@@@ > > > 6 > lookupnameat+0x82 > 1024 \|@@@@ > > > 7 cstatat_getvp+0x160 > 2048 \|@@@ > > > 6 cstatat32+0x7d > 4096 \|@@@@@@@@ > > 13 > lstat32+0x31 > 8192 \|@@@@@ > > 9 > lxstat32+0x24 > 16384 \|@ > > > 3 > _sys_sysenter_post_swapgs+0x14b > 32768 \| > > > 0 > 65536 \| > > > 0 > 131072 \| > > > 0 > 262144 \|@ > > > 2 > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock > > Hottest Caller > 2 15% 58% 0.00 > 177580 0xffffff0147fbe2d0 cv_wait+0x69 > nsec ------ Time Distribution ------ count > Stack > 262144 \|@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 2 > txg_thread_wait+0x60 > > > > > txg_timelimit_thread+0xaf > > > > > thread_start+0x8 > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock > > Hottest Caller > 2 13% 71% 0.00 > 158750 0xffffff014a7d7cd0 cv_wait+0x69 > nsec ------ Time Distribution ------ count > Stack > 262144 \|@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 2 > txg_thread_wait+0x60 > > > > > txg_timelimit_thread+0xaf > > > > > thread_start+0x8 > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock > > Hottest Caller > 37 6% 78% 0.00 > 4167 0xffffff014a80b118 sd_start_cmds+0x304 > nsec ------ Time Distribution ------ count > Stack > 256 \| > > > 1 > sd_core_iostart+0x17b > 512 \| > > > 0 > sd_mapblockaddr_iostart+0x185 > 1024 \|@ > > > 2 > sd_xbuf_strategy+0x50 > 2048 \|@@@ > > > 4 xbuf_iostart+0x103 > 4096 \|@@@@@@@@@@@@@@ > > 18 ddi_xbuf_qstrategy+0x60 > 8192 \|@@@@@ > > 7 > sdstrategy+0xec > 16384 \|@@@@ > > 5 > bdev_strategy+0x77 > > > > > ldi_strategy+0x54 > > > > > vdev_disk_io_start+0x154 > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock > > Hottest Caller > 8 6% 83% 0.00 > 16504 0xffffff0147fbf988 vdev_queue_io+0x85 > nsec ------ Time Distribution ------ count > Stack > 2048 \|@@@ > > > 1 vdev_disk_io_start+0x54 > 4096 \|@@@@@@@ > > 2 > zio_vdev_io_start+0x148 > 8192 \|@@@@@@@@@@@@@@@ > > 4 zio_execute+0x7f > 16384 \| > > > 0 > zio_nowait+0x11 > 32768 \| > > > 0 > vdev_mirror_io_start+0x18f > 65536 \| > > > 0 > zio_vdev_io_start+0x156 > 131072 \|@@@ > > 1 > zio_execute+0x7f > > > > > taskq_thread+0x1a7 > > > > > thread_start+0x8 > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock > > Hottest Caller > 13 5% 88% 0.00 > 8355 0xffffff01471a10c0 ehci_intr+0x2a > nsec ------ Time Distribution ------ count > Stack > 1024 \|@@@@ > > > 2 av_dispatch_autovect+0x78 > 2048 \| > > > 0 > dispatch_hardint+0x2f > 4096 \|@@@@ > > > 2 switch_sp_and_call+0x13 > 8192 \|@@@@@@ > > 3 > 16384 \|@@@@@@@@@@@ > > 5 > 32768 \|@@ > > > 1 > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock > > Hottest Caller > 32 3% 91% 0.00 > 2387 0xffffff014b6221f0 aac_tran_start+0xc6 > nsec ------ Time Distribution ------ count > Stack > 256 \| > > > 1 > scsi_transport+0x7a > 512 \|@@@@ > > > 5 sd_start_cmds+0x2f4 > 1024 \|@@@@ > > > 5 sd_core_iostart+0x17b > 2048 \|@@@@@@@@@@ > > 11 > sd_mapblockaddr_iostart+0x185 > 4096 \|@@@ > > > 4 sd_xbuf_strategy+0x50 > 8192 \|@@@ > > > 4 xbuf_iostart+0x103 > 16384 \|@ > > > 2 > ddi_xbuf_qstrategy+0x60 > > > > > sdstrategy+0xec > > > > > bdev_strategy+0x77 > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock > > Hottest Caller > 17 2% 93% 0.00 > 3139 0xffffff014a643658 sd_start_cmds+0x304 > nsec ------ Time Distribution ------ count > Stack > 2048 \|@@@ > > > 2 sd_core_iostart+0x17b > 4096 \|@@@@@@@@@@@@@@@@@@@@@@@@ > 14 sd_mapblockaddr_iostart+0x185 > 8192 \|@ > > > 1 > sd_xbuf_strategy+0x50 > > > > > xbuf_iostart+0x103 > > > > > ddi_xbuf_qstrategy+0x60 > > > > > sdstrategy+0xec > > > > > bdev_strategy+0x77 > > > > > ldi_strategy+0x54 > > > > > vdev_disk_io_start+0x154 > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock > > Hottest Caller > 4 1% 95% 0.00 > 6779 0xffffff0147ced528 spa_config_enter+0x34 > nsec ------ Time Distribution ------ count > Stack > 1024 \|@@@@@@@ > > 1 > bp_get_dasize+0x3d > 2048 \| > > > 0 > dsl_dataset_block_born+0x32 > 4096 \| > > > 0 > dbuf_write_ready+0x20f > 8192 \|@@@@@@@@@@@@@@@ > > 2 arc_write_ready+0x49 > 16384 \|@@@@@@@ > > 1 > zio_ready+0x2c > > > > > zio_execute+0x7f > > > > > taskq_thread+0x1a7 > > > > > thread_start+0x8 > ------------------------------------------------------------------------------- > Adaptive mutex block: 16 events in 5.043 seconds (3 events/sec) > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock > > Hottest Caller > 7 66% 66% 0.00 > 57913 0xffffff014a55b2d8 taskq_dispatch+0x2c9 > nsec ------ Time Distribution ------ count > Stack > 4096 \|@@@@ > > > 1 zio_interrupt+0x30 > 8192 \|@@@@@@@@ > > 2 > vdev_disk_io_intr+0x58 > 16384 \| > > > 0 > biodone+0x8a > 32768 \| > > > 0 > sd_buf_iodone+0x62 > 65536 \|@@@@ > > 1 > sd_mapblockaddr_iodone+0x48 > 131072 \|@@@@@@@@ > > 2 > sd_return_command+0x158 > 262144 \|@@@@ > > 1 > sdintr+0x50a > > > > > aac_drain_comp_q+0x10b > > > > > aac_intr_new+0x40 > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock > > Hottest Caller > 3 18% 84% 0.00 > 37741 0xffffff014b6221f0 aac_tran_start+0xc6 > nsec ------ Time Distribution ------ count > Stack > 8192 \|@@@@@@@@@@ > > 1 > scsi_transport+0x7a > 16384 \| > > > 0 > sd_start_cmds+0x2f4 > 32768 \| > > > 0 > sd_core_iostart+0x17b > 65536 \|@@@@@@@@@@@@@@@@@@@@ > 2 > sd_mapblockaddr_iostart+0x185 > > > > > sd_xbuf_strategy+0x50 > > > > > xbuf_iostart+0x103 > > > > > ddi_xbuf_qstrategy+0x60 > > > > > sdstrategy+0xec > > > > > bdev_strategy+0x77 > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock > > Hottest Caller > 1 8% 92% 0.00 > 48984 0xffffff014a3acec8 vdev_queue_io_done+0x3b > nsec ------ Time Distribution ------ count > Stack > 65536 \|@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 > vdev_disk_io_done+0x18 > > > > > zio_vdev_io_done+0x22 > > > > > zio_execute+0x7f > > > > > taskq_thread+0x1a7 > > > > > thread_start+0x8 > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock > > Hottest Caller > 4 6% 98% 0.00 > 9927 0xffffff014a80b118 sdintr+0x33 > nsec ------ Time Distribution ------ count > Stack > 8192 \|@@@@@@@ > > 1 > aac_drain_comp_q+0x10b > 16384 \|@@@@@@@@@@@@@@@@@@@@@@ > 3 > aac_intr_new+0x40 > > > > > av_dispatch_autovect+0x78 > > > > > dispatch_hardint+0x2f > > > > > switch_sp_and_call+0x13 > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock > > Hottest Caller > 1 2% 100% 0.00 > 10245 0xffffff014b90cd68 txg_list_add+0x2d > nsec ------ Time Distribution ------ count > Stack > 16384 \|@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 > dsl_dir_dirty+0x2d > > > > > dsl_dir_diduse_space+0x40 > > > > > dsl_dataset_block_born+0x137 > > > > > dbuf_write_ready+0x20f > > > > > arc_write_ready+0x49 > > > > > zio_ready+0x2c > > > > > zio_execute+0x7f > > > > > taskq_thread+0x1a7 > > > > > thread_start+0x8 > ------------------------------------------------------------------------------- > Spin lock spin: 11 events in 5.043 seconds (2 events/sec) > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock > > Hottest Caller > 9 100% 100% 0.00 19177 > cpu0_disp[56] disp_lock_enter_high+0x11 > nsec ------ Time Distribution ------ count > Stack > 128 \|@@@ > > > 1 > setfrontdq+0x190 > 256 \| > > > 0 > ts_wakeup+0x149 > 512 \|@@@@@@@@@@@@@ > > 4 > sleepq_wakeone_chan+0x83 > 1024 \|@@@ > > > 1 cv_signal+0x89 > 2048 \| > > > 0 > lwp_unpark+0x6e > 4096 \| > > > 0 > lwp_park+0x47 > 8192 \|@@@@@@ > > 2 > syslwp_park+0x2d > 16384 \| > > > 0 > _sys_sysenter_post_swapgs+0x14b > 32768 \| > > > 0 > 65536 \| > > > 0 > 131072 \| > > > 0 > 262144 \|@@@ > > 1 > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock > > Hottest Caller > 1 0% 100% 0.00 > 467 turnstile_table[4096] disp_lock_enter+0x31 > nsec ------ Time Distribution ------ count > Stack > 512 \|@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 > turnstile_lookup+0x61 > > > > > mutex_vector_enter+0x1b4 > > > > > taskq_dispatch+0x2c9 > > > > > zio_interrupt+0x30 > > > > > vdev_disk_io_intr+0x58 > > > > > biodone+0x8a > > > > > sd_buf_iodone+0x62 > > > > > sd_mapblockaddr_iodone+0x48 > > > > > sd_return_command+0x158 > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock > > Hottest Caller > 1 0% 100% 0.00 > 150 0xffffff0149fba968 disp_lock_enter_high+0x11 > nsec ------ Time Distribution ------ count > Stack > 256 \|@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 > setbackdq+0x226 > > > > > sleepq_wakeall_chan+0x78 > > > > > cv_broadcast+0x5b > > > > > txg_quiesce_thread+0x115 > > > > > thread_start+0x8 > -------------------------------------------------------------------------------
	Введите код, изображенный на картинке:

При общении не допускается: неуважительное отношение к собеседнику, хамство, унизительное обращение, ненормативная лексика, переход на личности, агрессивное поведение, обесценивание собеседника, провоцирование флейма голословными и заведомо ложными заявлениями. Не отвечайте на сообщения, явно нарушающие правила - удаляются не только сами нарушения, но и все ответы на них. Лог модерирования.

Партнёры:

Хостинг:

Закладки на сайте
Проследить за страницей

Created 1996-2024 by Maxim Chirkov
Добавить, Поддержать, Вебмастеру