Subject: sysrq-b fails miserably to reboot PowerNV box

From: Anton Blanchard <[email-masked]>
Date: Sat, 8 Oct 2016 22:16:56 +1100

Hi,

Unfortunately sysrq-b seems to tie us up in knots, instead of rebooting
the box. This is mainline from today.

Anton
--

Trying to free IRQ 17 from IRQ context!
------------[ cut here ]------------
WARNING: CPU: 32 PID: 0 at kernel/irq/manage.c:1460 __free_irq+0x298/0x380
Modules linked in: kvm_hv kvm binfmt_misc vmx_crypto crc32c_vpmsum leds_powernv powernv_rng led_class powernv_op_panel rng_core ib_iser rdma_cm iw_cm ib_cm ib_core libiscsi scsi_transport_iscsi autofs4
CPU: 32 PID: 0 Comm: swapper/32 Tainted: G      D         4.8.0-11288-gb66484c #785
task: c0000017f91fce00 task.stack: c0000017f927c000
NIP: c00000000012b4e8 LR: c00000000012b4e4 CTR: c000000000668950
REGS: c000001fffe8b5c0 TRAP: 0700   Tainted: G      D          (4.8.0-11288-gb66484c)
MSR: 9000000000029033 <SF,HV,EE,ME,IR,DR,RI,LE>  CR: 28004224  XER: 20000000
CFAR: c00000000000c2e0 SOFTE: 1 
GPR00: c00000000012b4e4 c000001fffe8b840 c000000000dbc100 0000000000000027 
GPR04: c0000007fb80ada0 c0000007fb81f598 0000000000000000 c000000000dee9a8 
GPR08: 0000000000000007 0000000000000000 00000007fab60000 9000000000001003 
GPR12: 0000000000004400 c00000000fe08000 c0000017f927ff90 0000000000000000 
GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
GPR20: 0000000000000001 c000000000d35d70 c000001fffe8bb00 0000000000000000 
GPR24: 0000000000000000 c0000007e3330400 c000000000d31de0 000000000000000f 
GPR28: 0000000000000011 c000000ff7ed1400 0000000000000000 0000000000000011 
NIP [c00000000012b4e8] __free_irq+0x298/0x380
LR [c00000000012b4e4] __free_irq+0x294/0x380
Call Trace:
[c000001fffe8b840] [c00000000012b4e4] __free_irq+0x294/0x380 (unreliable)
[c000001fffe8b8d0] [c00000000012b6ec] free_irq+0x7c/0xb0
[c000001fffe8b900] [c000000000072748] opal_event_shutdown+0x88/0xd0
[c000001fffe8b940] [c00000000006db8c] opal_shutdown+0x1c/0x90
[c000001fffe8b970] [c000000000069d00] pnv_shutdown+0x20/0x40
[c000001fffe8b990] [c000000000026a18] machine_restart+0x38/0x90
[c000001fffe8ba00] [c0000000000e7538] emergency_restart+0x28/0x40
[c000001fffe8ba20] [c000000000568c44] sysrq_handle_reboot+0x24/0x40
[c000001fffe8ba40] [c000000000569490] __handle_sysrq+0xe0/0x250
[c000001fffe8bae0] [c00000000057f1a0] hvc_poll+0x230/0x380
[c000001fffe8bb80] [c0000000005803cc] hvc_handle_interrupt+0x1c/0x50
[c000001fffe8bba0] [c00000000012a2b0] __handle_irq_event_percpu+0x90/0x2f0
[c000001fffe8bc60] [c00000000012a540] handle_irq_event_percpu+0x30/0x90
[c000001fffe8bca0] [c00000000012a5fc] handle_irq_event+0x5c/0xc0
[c000001fffe8bcd0] [c00000000012f5b4] handle_level_irq+0xd4/0x1e0
[c000001fffe8bd00] [c000000000128e44] generic_handle_irq+0x44/0x70
[c000001fffe8bd20] [c000000000072600] opal_handle_events+0xd0/0x120
[c000001fffe8bd70] [c0000000000726a8] opal_interrupt+0x38/0x50
[c000001fffe8bda0] [c00000000012a2b0] __handle_irq_event_percpu+0x90/0x2f0
[c000001fffe8be60] [c00000000012a540] handle_irq_event_percpu+0x30/0x90
[c000001fffe8bea0] [c00000000012a5fc] handle_irq_event+0x5c/0xc0
[c000001fffe8bed0] [c00000000012f9f0] handle_fasteoi_irq+0xe0/0x290
[c000001fffe8bf00] [c000000000128e44] generic_handle_irq+0x44/0x70
[c000001fffe8bf20] [c00000000001641c] __do_irq+0x8c/0x210
[c000001fffe8bf90] [c000000000027ed4] call_do_irq+0x14/0x24
[c0000017f927fac0] [c000000000016634] do_IRQ+0x94/0x140
[c0000017f927fb10] [c000000000008e8c] hardware_interrupt_common+0x10c/0x180
--- interrupt: 501 at arch_local_irq_restore+0x5c/0x90
    LR = arch_local_irq_restore+0x40/0x90
[c0000017f927fe00] [c000000000155e44] tick_broadcast_oneshot_control+0x54/0x70 (unreliable)
[c0000017f927fe20] [c0000000007b6da4] cpuidle_enter_state+0x1b4/0x440
[c0000017f927fe80] [c000000000114da4] call_cpuidle+0x44/0x90
[c0000017f927fea0] [c000000000115138] cpu_startup_entry+0x1f8/0x320
[c0000017f927ff20] [c000000000043540] start_secondary+0x320/0x340
[c0000017f927ff90] [c00000000000b56c] start_secondary_prolog+0x10/0x14
Instruction dump:
7d2903a6 7fc3f378 f8410018 7d2c4b78 4e800421 e8410018 4bfffe68 3c62ffd6 
7f84e378 38639270 48811f35 60000000 <0fe00000> 4bfffdb4 7d2903a6 7fc3f378 
---[ end trace dab28cb0b01424b6 ]---
EEH: Frozen PHB#3-PE#1 detected
EEH: PE location: U78C9.001.WZS003V-P1-C10     , PHB location: N/A
CPU: 25 PID: 0 Comm: swapper/25 Tainted: G      D W       4.8.0-11288-gb66484c #785
Call Trace:
[c0000007f6483690] [c00000000093e6fc] dump_stack+0xb0/0xf0 (unreliable)
[c0000007f64836d0] [c000000000039c30] eeh_dev_check_failure+0x3f0/0x530
[c0000007f6483780] [c000000000039df4] eeh_check_failure+0x84/0xd0
[c0000007f64837c0] [c000000000677fb0] tg3_read32+0x80/0x90
[c0000007f64837f0] [c000000000698acc] tg3_timer+0x5ec/0x1170
[c0000007f6483850] [c000000000142334] call_timer_fn+0x54/0x1d0
[c0000007f64838e0] [c000000000142600] expire_timers+0x120/0x1d0
[c0000007f6483950] [c0000000001428a8] run_timer_softirq+0x1f8/0x260
[c0000007f64839e0] [c0000000000c0478] __do_softirq+0x158/0x3e0
[c0000007f6483ac0] [c0000000000c0958] irq_exit+0xe8/0x120
[c0000007f6483ae0] [c0000000000400f0] doorbell_exception+0xa0/0xd0
[c0000007f6483b10] [c00000000000a20c] h_doorbell_common+0x10c/0x180
--- interrupt: e81 at arch_local_irq_restore+0x5c/0x90
    LR = arch_local_irq_restore+0x40/0x90
[c0000007f6483e00] [c000000000155e44] tick_broadcast_oneshot_control+0x54/0x70 (unreliable)
[c0000007f6483e20] [c0000000007b6da4] cpuidle_enter_state+0x1b4/0x440
[c0000007f6483e80] [c000000000114da4] call_cpuidle+0x44/0x90
[c0000007f6483ea0] [c000000000115138] cpu_startup_entry+0x1f8/0x320
[c0000007f6483f20] [c000000000043540] start_secondary+0x320/0x340
[c0000007f6483f90] [c00000000000b56c] start_secondary_prolog+0x10/0x14
EEH: Detected PCI bus error on PHB#3-PE#1
EEH: This PCI device has failed 1 times in the last hour
EEH: Notify device drivers to shutdown
tg3 0003:09:00.0 eth0: PCI I/O error detected
tg3 0003:09:00.1 eth1: PCI I/O error detected
tg3 0003:09:00.2 eth2: PCI I/O error detected
tg3 0003:09:00.3 eth3: PCI I/O error detected
EEH: Collect temporary log
PHB3 PHB#3 Diag-data (Version: 1)
brdgCtl:     00000002
RootSts:     0002002f 00400000 b0830008 00100147 00002800
RootErrSts:  00000000 00008000 00000000
PhbSts:      0000001c00000000 0000001c00000000
Lem:         4000000050000000 42498e327f502eae 0000000000000000
OutErr:      2000000000000000 2000000000000000 0204006000003b50 1000080000000000
InAErr:      0000008800000000 0000000800000000 001d090000000800 0800000ff3b47380
PE[  0] A/B: 8000002100000000 8000000000000000
PE[  1] A/B: 8000000000000000 8000000000000000
PE[  2] A/B: 8000000000000000 8000000000000000
PE[  3] A/B: 8000000000000000 8000000000000000
PE[  4] A/B: 8000000000000000 8000000000000000
PE[  5] A/B: 8000000000000000 8000000000000000
PE[  6] A/B: 8000000000000000 8000000000000000
PE[  7] A/B: 8000000000000000 8000000000000000
PE[  8] A/B: 8000000000000000 8000000000000000
PE[  9] A/B: 8000000000000000 8000000000000000
PE[ 10] A/B: 8000000000000000 8000000000000000
PE[ 11] A/B: 8000000000000000 8000000000000000
PE[ 12] A/B: 8000000000000000 8000000000000000
PE[ 13] A/B: 8000000000000000 8000000000000000
PE[ 14] A/B: 8000000000000000 8000000000000000
PE[ 15] A/B: 8000000000000000 8000000000000000
PE[ 16] A/B: 8000000000000000 8000000000000000
PE[ 17] A/B: 8000000000000000 8000000000000000
PE[ 18] A/B: 8000000000000000 8000000000000000
PE[ 19] A/B: 8000000000000000 8000000000000000
PE[ 20] A/B: 8000000000000000 8000000000000000
PE[ 21] A/B: 8000000000000000 8000000000000000
PE[ 22] A/B: 8000000000000000 8000000000000000
PE[ 23] A/B: 8000000000000000 8000000000000000
PE[ 24] A/B: 8000000000000000 8000000000000000
PE[ 25] A/B: 8000000000000000 8000000000000000
PE[ 26] A/B: 8000000000000000 8000000000000000
PE[ 27] A/B: 8000000000000000 8000000000000000
PE[ 28] A/B: 8000000000000000 8000000000000000
PE[ 29] A/B: 8000000000000000 8000000000000000
PE[ 30] A/B: 8000000000000000 8000000000000000
PE[ 31] A/B: 8000000000000000 8000000000000000
PE[ 32] A/B: 8000000000000000 8000000000000000
PE[ 33] A/B: 8000000000000000 8000000000000000
PE[ 34] A/B: 8000000000000000 8000000000000000
PE[ 35] A/B: 8000000000000000 8000000000000000
PE[ 36] A/B: 8000000000000000 8000000000000000
PE[ 37] A/B: 8000000000000000 8000000000000000
PE[ 38] A/B: 8000000000000000 8000000000000000
PE[ 39] A/B: 8000000000000000 8000000000000000
PE[ 40] A/B: 8000000000000000 8000000000000000
PE[ 41] A/B: 8000000000000000 8000000000000000
PE[ 42] A/B: 8000000000000000 8000000000000000
PE[ 43] A/B: 8000000000000000 8000000000000000
PE[ 44] A/B: 8000000000000000 8000000000000000
PE[ 45] A/B: 8000000000000000 8000000000000000
PE[ 46] A/B: 8000000000000000 8000000000000000
PE[ 47] A/B: 8000000000000000 8000000000000000
PE[ 48] A/B: 8000000000000000 8000000000000000
PE[ 49] A/B: 8000000000000000 8000000000000000
PE[ 50] A/B: 8000000000000000 8000000000000000
PE[ 51] A/B: 8000000000000000 8000000000000000
PE[ 52] A/B: 8000000000000000 8000000000000000
PE[ 53] A/B: 8000000000000000 8000000000000000
PE[ 54] A/B: 8000000000000000 8000000000000000
PE[ 55] A/B: 8000000000000000 8000000000000000
PE[ 56] A/B: 8000000000000000 8000000000000000
PE[ 57] A/B: 8000000000000000 8000000000000000
PE[ 58] A/B: 8000000000000000 8000000000000000
PE[ 59] A/B: 8000000000000000 8000000000000000
PE[ 60] A/B: 8000000000000000 8000000000000000
PE[ 61] A/B: 8000000000000000 8000000000000000
PE[ 62] A/B: 8000000000000000 8000000000000000
PE[ 63] A/B: 8000000000000000 8000000000000000
PE[ 64] A/B: 8000000000000000 8000000000000000
PE[ 65] A/B: 8000000000000000 8000000000000000
PE[ 66] A/B: 8000000000000000 8000000000000000
PE[ 67] A/B: 8000000000000000 8000000000000000
PE[ 68] A/B: 8000000000000000 8000000000000000
PE[ 69] A/B: 8000000000000000 8000000000000000
PE[ 70] A/B: 8000000000000000 8000000000000000
PE[ 71] A/B: 8000000000000000 8000000000000000
PE[ 72] A/B: 8000000000000000 8000000000000000
PE[ 73] A/B: 8000000000000000 8000000000000000
PE[ 74] A/B: 8000000000000000 8000000000000000
PE[ 75] A/B: 8000000000000000 8000000000000000
PE[ 76] A/B: 8000000000000000 8000000000000000
PE[ 77] A/B: 8000000000000000 8000000000000000
PE[ 78] A/B: 8000000000000000 8000000000000000
PE[ 79] A/B: 8000000000000000 8000000000000000
PE[ 80] A/B: 8000000000000000 8000000000000000
PE[ 81] A/B: 8000000000000000 8000000000000000
PE[ 82] A/B: 8000000000000000 8000000000000000
PE[ 83] A/B: 8000000000000000 8000000000000000
PE[ 84] A/B: 8000000000000000 8000000000000000
PE[ 85] A/B: 8000000000000000 8000000000000000
PE[ 86] A/B: 8000000000000000 8000000000000000
PE[ 87] A/B: 8000000000000000 8000000000000000
PE[ 88] A/B: 8000000000000000 8000000000000000
PE[ 89] A/B: 8000000000000000 8000000000000000
PE[ 90] A/B: 8000000000000000 8000000000000000
PE[ 91] A/B: 8000000000000000 8000000000000000
PE[ 92] A/B: 8000000000000000 8000000000000000
PE[ 93] A/B: 8000000000000000 8000000000000000
PE[ 94] A/B: 8000000000000000 8000000000000000
PE[ 95] A/B: 8000000000000000 8000000000000000
PE[ 96] A/B: 8000000000000000 8000000000000000
PE[ 97] A/B: 8000000000000000 8000000000000000
PE[ 98] A/B: 8000000000000000 8000000000000000
PE[ 99] A/B: 8000000000000000 8000000000000000
PE[100] A/B: 8000000000000000 8000000000000000
PE[101] A/B: 8000000000000000 8000000000000000
PE[102] A/B: 8000000000000000 8000000000000000
PE[103] A/B: 8000000000000000 8000000000000000
PE[104] A/B: 8000000000000000 8000000000000000
PE[105] A/B: 8000000000000000 8000000000000000
PE[106] A/B: 8000000000000000 8000000000000000
PE[107] A/B: 8000000000000000 8000000000000000
PE[108] A/B: 8000000000000000 8000000000000000
PE[109] A/B: 8000000000000000 8000000000000000
PE[110] A/B: 8000000000000000 8000000000000000
PE[111] A/B: 8000000000000000 8000000000000000
PE[112] A/B: 8000000000000000 8000000000000000
PE[113] A/B: 8000000000000000 8000000000000000
PE[114] A/B: 8000000000000000 8000000000000000
PE[115] A/B: 8000000000000000 8000000000000000
PE[116] A/B: 8000000000000000 8000000000000000
PE[117] A/B: 8000000000000000 8000000000000000
PE[118] A/B: 8000000000000000 8000000000000000
PE[119] A/B: 8000000000000000 8000000000000000
PE[120] A/B: 8000000000000000 8000000000000000
PE[121] A/B: 8000000000000000 8000000000000000
PE[122] A/B: 8000000000000000 8000000000000000
PE[123] A/B: 8000000000000000 8000000000000000
PE[124] A/B: 8000000000000000 8000000000000000
PE[125] A/B: 8000000000000000 8000000000000000
PE[126] A/B: 8000000000000000 8000000000000000
PE[127] A/B: 8000000000000000 8000000000000000
PE[128] A/B: 8000000000000000 8000000000000000
PE[129] A/B: 8000000000000000 8000000000000000
PE[130] A/B: 8000000000000000 8000000000000000
PE[131] A/B: 8000000000000000 8000000000000000
PE[132] A/B: 8000000000000000 8000000000000000
PE[133] A/B: 8000000000000000 8000000000000000
PE[134] A/B: 8000000000000000 8000000000000000
PE[135] A/B: 8000000000000000 8000000000000000
PE[136] A/B: 8000000000000000 8000000000000000
PE[137] A/B: 8000000000000000 8000000000000000
PE[138] A/B: 8000000000000000 8000000000000000
PE[139] A/B: 8000000000000000 8000000000000000
PE[140] A/B: 8000000000000000 8000000000000000
PE[141] A/B: 8000000000000000 8000000000000000
PE[142] A/B: 8000000000000000 8000000000000000
PE[143] A/B: 8000000000000000 8000000000000000
PE[144] A/B: 8000000000000000 8000000000000000
PE[145] A/B: 8000000000000000 8000000000000000
PE[146] A/B: 8000000000000000 8000000000000000
PE[147] A/B: 8000000000000000 8000000000000000
PE[148] A/B: 8000000000000000 8000000000000000
PE[149] A/B: 8000000000000000 8000000000000000
PE[150] A/B: 8000000000000000 8000000000000000
PE[151] A/B: 8000000000000000 8000000000000000
PE[152] A/B: 8000000000000000 8000000000000000
PE[153] A/B: 8000000000000000 8000000000000000
PE[154] A/B: 8000000000000000 8000000000000000
PE[155] A/B: 8000000000000000 8000000000000000
PE[156] A/B: 8000000000000000 8000000000000000
PE[157] A/B: 8000000000000000 8000000000000000
PE[158] A/B: 8000000000000000 8000000000000000
PE[159] A/B: 8000000000000000 8000000000000000
PE[160] A/B: 8000000000000000 8000000000000000
PE[161] A/B: 8000000000000000 8000000000000000
PE[162] A/B: 8000000000000000 8000000000000000
PE[163] A/B: 8000000000000000 8000000000000000
PE[164] A/B: 8000000000000000 8000000000000000
PE[165] A/B: 8000000000000000 8000000000000000
PE[166] A/B: 8000000000000000 8000000000000000
PE[167] A/B: 8000000000000000 8000000000000000
PE[168] A/B: 8000000000000000 8000000000000000
PE[169] A/B: 8000000000000000 8000000000000000
PE[170] A/B: 8000000000000000 8000000000000000
PE[171] A/B: 8000000000000000 8000000000000000
PE[172] A/B: 8000000000000000 8000000000000000
PE[173] A/B: 8000000000000000 8000000000000000
PE[174] A/B: 8000000000000000 8000000000000000
PE[175] A/B: 8000000000000000 8000000000000000
PE[176] A/B: 8000000000000000 8000000000000000
PE[177] A/B: 8000000000000000 8000000000000000
PE[178] A/B: 8000000000000000 8000000000000000
PE[179] A/B: 8000000000000000 8000000000000000
PE[180] A/B: 8000000000000000 8000000000000000
PE[181] A/B: 8000000000000000 8000000000000000
PE[182] A/B: 8000000000000000 8000000000000000
PE[183] A/B: 8000000000000000 8000000000000000
PE[184] A/B: 8000000000000000 8000000000000000
PE[185] A/B: 8000000000000000 8000000000000000
PE[186] A/B: 8000000000000000 8000000000000000
PE[187] A/B: 8000000000000000 8000000000000000
PE[188] A/B: 8000000000000000 8000000000000000
PE[189] A/B: 8000000000000000 8000000000000000
PE[190] A/B: 8000000000000000 8000000000000000
PE[191] A/B: 8000000000000000 8000000000000000
PE[192] A/B: 8000000000000000 8000000000000000
PE[193] A/B: 8000000000000000 8000000000000000
PE[194] A/B: 8000000000000000 8000000000000000
PE[195] A/B: 8000000000000000 8000000000000000
PE[196] A/B: 8000000000000000 8000000000000000
PE[197] A/B: 8000000000000000 8000000000000000
PE[198] A/B: 8000000000000000 8000000000000000
PE[199] A/B: 8000000000000000 8000000000000000
PE[200] A/B: 8000000000000000 8000000000000000
PE[201] A/B: 8000000000000000 8000000000000000
PE[202] A/B: 8000000000000000 8000000000000000
PE[203] A/B: 8000000000000000 8000000000000000
PE[204] A/B: 8000000000000000 8000000000000000
PE[205] A/B: 8000000000000000 8000000000000000
PE[206] A/B: 8000000000000000 8000000000000000
PE[207] A/B: 8000000000000000 8000000000000000
PE[208] A/B: 8000000000000000 8000000000000000
PE[209] A/B: 8000000000000000 8000000000000000
PE[210] A/B: 8000000000000000 8000000000000000
PE[211] A/B: 8000000000000000 8000000000000000
PE[212] A/B: 8000000000000000 8000000000000000
PE[213] A/B: 8000000000000000 8000000000000000
PE[214] A/B: 8000000000000000 8000000000000000
PE[215] A/B: 8000000000000000 8000000000000000
PE[216] A/B: 8000000000000000 8000000000000000
PE[217] A/B: 8000000000000000 8000000000000000
PE[218] A/B: 8000000000000000 8000000000000000
PE[219] A/B: 8000000000000000 8000000000000000
PE[220] A/B: 8000000000000000 8000000000000000
PE[221] A/B: 8000000000000000 8000000000000000
PE[222] A/B: 8000000000000000 8000000000000000
PE[223] A/B: 8000000000000000 8000000000000000
PE[224] A/B: 8000000000000000 8000000000000000
PE[225] A/B: 8000000000000000 8000000000000000
PE[226] A/B: 8000000000000000 8000000000000000
PE[227] A/B: 8000000000000000 8000000000000000
PE[228] A/B: 8000000000000000 8000000000000000
PE[229] A/B: 8000000000000000 8000000000000000
PE[230] A/B: 8000000000000000 8000000000000000
PE[231] A/B: 8000000000000000 8000000000000000
PE[232] A/B: 8000000000000000 8000000000000000
PE[233] A/B: 8000000000000000 8000000000000000
PE[234] A/B: 8000000000000000 8000000000000000
PE[235] A/B: 8000000000000000 8000000000000000
PE[236] A/B: 8000000000000000 8000000000000000
PE[237] A/B: 8000000000000000 8000000000000000
PE[238] A/B: 8000000000000000 8000000000000000
PE[239] A/B: 8000000000000000 8000000000000000
PE[240] A/B: 8000000000000000 8000000000000000
PE[241] A/B: 8000000000000000 8000000000000000
PE[242] A/B: 8000000000000000 8000000000000000
PE[243] A/B: 8000000000000000 8000000000000000
PE[244] A/B: 8000000000000000 8000000000000000
PE[245] A/B: 8000000000000000 8000000000000000
PE[246] A/B: 8000000000000000 8000000000000000
PE[247] A/B: 8000000000000000 8000000000000000
PE[248] A/B: 8000000000000000 8000000000000000
PE[249] A/B: 8000000000000000 8000000000000000
PE[250] A/B: 8000000000000000 8000000000000000
PE[251] A/B: 8000000000000000 8000000000000000
PE[252] A/B: 8000000000000000 8000000000000000
PE[253] A/B: 8000000000000000 8000000000000000
PE[254] A/B: 8000000000000000 8000000000000000
PE[255] A/B: 8000802309000000 8800000ff3b47380
EEH: Reset without hotplug activity
EEH: Notify device drivers the completion of reset
NMI watchdog: BUG: soft lockup - CPU#19 stuck for 22s! [kworker/19:2:2695]
Modules linked in: kvm_hv kvm binfmt_misc vmx_crypto crc32c_vpmsum leds_powernv powernv_rng led_class powernv_op_panel rng_core ib_iser rdma_cm iw_cm ib_cm ib_core libiscsi scsi_transport_iscsi autofs4
CPU: 19 PID: 2695 Comm: kworker/19:2 Tainted: G      D W       4.8.0-11288-gb66484c #785
Workqueue: events wait_rcu_exp_gp
task: c0000007f4c10300 task.stack: c0000007f4e88000
NIP: c00000000015ff28 LR: c000000000137d94 CTR: c0000000000681e0
REGS: c0000007f4e8b890 TRAP: 0901   Tainted: G      D W        (4.8.0-11288-gb66484c)
MSR: 900000010280b033 <SF,HV,VEC,VSX,EE,FP,ME,IR,DR,RI,LE,TM[E]>  CR: 48002422  XER: 20000000
CFAR: c00000000015ff30 SOFTE: 1 
GPR00: c000000000137d94 c0000007f4e8bb10 c000000000dbc100 0000000000000020 
GPR04: c0000000001374a0 c000000000ce1280 0000000000000000 c000000000dee9a8 
GPR08: c00000000fe04c00 0000000000000001 c0000007fb4e3680 0000000000000000 
GPR12: c0000000000681e0 c00000000fe04c00 
NIP [c00000000015ff28] smp_call_function_single+0x98/0x180
LR [c000000000137d94] wait_rcu_exp_gp+0x294/0xaf0
Call Trace:
[c0000007f4e8bb10] [c00000000015ff58] smp_call_function_single+0xc8/0x180 (unreliable)
[c0000007f4e8bb50] [c000000000137d94] wait_rcu_exp_gp+0x294/0xaf0
[c0000007f4e8bc50] [c0000000000dbab0] process_one_work+0x2a0/0x5a0
[c0000007f4e8bce0] [c0000000000dbe58] worker_thread+0xa8/0x6a0
[c0000007f4e8bd80] [c0000000000e41d0] kthread+0x120/0x130
[c0000007f4e8be30] [c00000000000bf68] ret_from_kernel_thread+0x5c/0x74
Instruction dump:
409e0078 e9280030 3d42fff0 394a7580 7d4a4a14 812a0018 71290001 41820020 
60000000 60420000 7c210b78 7c421378 <812a0018> 71290001 4082fff0 7c2004ac 
NMI watchdog: BUG: soft lockup - CPU#32 stuck for 22s! [swapper/32:0]
Modules linked in: kvm_hv kvm binfmt_misc vmx_crypto crc32c_vpmsum leds_powernv powernv_rng led_class powernv_op_panel rng_core ib_iser rdma_cm iw_cm ib_cm ib_core libiscsi scsi_transport_iscsi autofs4
CPU: 32 PID: 0 Comm: swapper/32 Tainted: G      D W    L  4.8.0-11288-gb66484c #785
task: c0000017f91fce00 task.stack: c0000017f927c000
NIP: c00000000012a698 LR: c00000000012a840 CTR: 0000000000000000
REGS: c000001fffe8b520 TRAP: 0901   Tainted: G      D W    L   (4.8.0-11288-gb66484c)
MSR: 9000000000009033 <SF,HV,EE,ME,IR,DR,RI,LE>  CR: 44004882  XER: 00000000
CFAR: c00000000012a6a4 SOFTE: 1 
GPR00: c00000000012a840 c000001fffe8b7a0 c000000000dbc100 c000000ff7ed1400 
GPR04: 0000000000000011 0000000000000000 0000000000000000 0000000000000000 
GPR08: c000000ff7ed1400 0000000000040000 c0000007f90000b0 c0000007f9000028 
GPR12: 0000000000000040 c00000000fe08000 
NIP [c00000000012a698] __synchronize_hardirq+0x38/0xa0
LR [c00000000012a840] synchronize_irq+0x30/0xd0
Call Trace:
[c000001fffe8b7a0] [c000000000339548] remove_proc_subtree+0x168/0x1c0 (unreliable)
[c000001fffe8b7e0] [c00000000012a840] synchronize_irq+0x30/0xd0
[c000001fffe8b840] [c00000000012b37c] __free_irq+0x12c/0x380
[c000001fffe8b8d0] [c00000000012b6ec] free_irq+0x7c/0xb0
[c000001fffe8b900] [c000000000072748] opal_event_shutdown+0x88/0xd0
[c000001fffe8b940] [c00000000006db8c] opal_shutdown+0x1c/0x90
[c000001fffe8b970] [c000000000069d00] pnv_shutdown+0x20/0x40
[c000001fffe8b990] [c000000000026a18] machine_restart+0x38/0x90
[c000001fffe8ba00] [c0000000000e7538] emergency_restart+0x28/0x40
[c000001fffe8ba20] [c000000000568c44] sysrq_handle_reboot+0x24/0x40
[c000001fffe8ba40] [c000000000569490] __handle_sysrq+0xe0/0x250
[c000001fffe8bae0] [c00000000057f1a0] hvc_poll+0x230/0x380
[c000001fffe8bb80] [c0000000005803cc] hvc_handle_interrupt+0x1c/0x50
[c000001fffe8bba0] [c00000000012a2b0] __handle_irq_event_percpu+0x90/0x2f0
[c000001fffe8bc60] [c00000000012a540] handle_irq_event_percpu+0x30/0x90
[c000001fffe8bca0] [c00000000012a5fc] handle_irq_event+0x5c/0xc0
[c000001fffe8bcd0] [c00000000012f5b4] handle_level_irq+0xd4/0x1e0
[c000001fffe8bd00] [c000000000128e44] generic_handle_irq+0x44/0x70
[c000001fffe8bd20] [c000000000072600] opal_handle_events+0xd0/0x120
[c000001fffe8bd70] [c0000000000726a8] opal_interrupt+0x38/0x50
[c000001fffe8bda0] [c00000000012a2b0] __handle_irq_event_percpu+0x90/0x2f0
[c000001fffe8be60] [c00000000012a540] handle_irq_event_percpu+0x30/0x90
[c000001fffe8bea0] [c00000000012a5fc] handle_irq_event+0x5c/0xc0
[c000001fffe8bed0] [c00000000012f9f0] handle_fasteoi_irq+0xe0/0x290
[c000001fffe8bf00] [c000000000128e44] generic_handle_irq+0x44/0x70
[c000001fffe8bf20] [c00000000001641c] __do_irq+0x8c/0x210
[c000001fffe8bf90] [c000000000027ed4] call_do_irq+0x14/0x24
[c0000017f927fac0] [c000000000016634] do_IRQ+0x94/0x140
[c0000017f927fb10] [c000000000008e8c] hardware_interrupt_common+0x10c/0x180
--- interrupt: 501 at arch_local_irq_restore+0x5c/0x90
    LR = arch_local_irq_restore+0x40/0x90
[c0000017f927fe00] [c000000000155e44] tick_broadcast_oneshot_control+0x54/0x70 (unreliable)
[c0000017f927fe20] [c0000000007b6da4] cpuidle_enter_state+0x1b4/0x440
[c0000017f927fe80] [c000000000114da4] call_cpuidle+0x44/0x90
[c0000017f927fea0] [c000000000115138] cpu_startup_entry+0x1f8/0x320
[c0000017f927ff20] [c000000000043540] start_secondary+0x320/0x340
[c0000017f927ff90] [c00000000000b56c] start_secondary_prolog+0x10/0x14
Instruction dump:
7c0802a6 fba1ffe8 fbc1fff0 3ba3018c fbe1fff8 7c7e1b78 f8010010 f821ffc1 
48000010 60420000 7c210b78 7c421378 <e93e0128> 81290000 75290004 4082ffec