Google Chrome hard CPU lockup

Discuss usability issues, general maintenance, and general support issues for a grsecurity-enabled system.

Google Chrome hard CPU lockup

Postby shadowdaemon » Fri Jun 24, 2016 9:41 am

Hi,

recently I've been seeing various problems with Google Chrome. Originally Chrome was triggering brute force prevention which then provoked a kernel panic. I was asked to apply a patch (warn-fix.diff) which fixed the problem when running Chrome in a Qemu test VM. However I've noticed that the host system with this patch applied is now locking up almost immediately upon starting Chrome.

Also this only appears to be a problem when RBAC is enabled. Here is the netconsole log which has been copied from KDE konsole, I had some problems redirecting the netcat output to a file.

#uname -a
Linux seraphim 4.5.5-hardened-r2 #2 SMP Sun Jun 12 08:59:46 AEST 2016 x86_64 Intel(R) Core(TM) i5-2410M CPU @ 2.30GHz GenuineIntel GNU/Linux

grsec: (bowser:U:/opt/google/chrome/chrome-sandbox) exec of /opt/google/chrome/chrome-sandbox (/opt/google/chrome/chrome-sandbox --adjust-oom-score 5431 300 ) by /opt/google/chrome/chrome-sandbox[Chrome_ProcessL:5432] uid/euid:1236/1236 gid/egid:100/100, parent /opt/google/chrome/chrome[Chrome_ProcessL:5400] uid/euid:1236/1236 gid/egid:100/100
grsec: (bowser:U:/opt/google/chrome/chrome-sandbox) exec of /opt/google/chrome/chrome-sandbox (/opt/google/chrome/chrome-sandbox --adjust-oom-score 5436 300 ) by /opt/google/chrome/chrome-sandbox[Chrome_ProcessL:5437] uid/euid:1236/1236 gid/egid:100/100, parent /opt/google/chrome/chrome[Chrome_ProcessL:5400] uid/euid:1236/1236 gid/egid:100/100
grsec: (bowser:U:/opt/google/chrome/chrome-sandbox) exec of /opt/google/chrome/chrome-sandbox (/opt/google/chrome/chrome-sandbox --adjust-oom-score 5440 300 ) by /opt/google/chrome/chrome-sandbox[Chrome_ProcessL:5441] uid/euid:1236/1236 gid/egid:100/100, parent /opt/google/chrome/chrome[Chrome_ProcessL:5400] uid/euid:1236/1236 gid/egid:100/100
grsec: (bowser:U:/opt/google/chrome/chrome-sandbox) exec of /opt/google/chrome/chrome-sandbox (/opt/google/chrome/chrome-sandbox --adjust-oom-score 5444 300 ) by /opt/google/chrome/chrome-sandbox[Chrome_ProcessL:5445] uid/euid:1236/1236 gid/egid:100/100, parent /opt/google/chrome/chrome[Chrome_ProcessL:5400] uid/euid:1236/1236 gid/egid:100/100
grsec: (bowser:U:/opt/google/chrome/chrome-sandbox) exec of /opt/google/chrome/chrome-sandbox (/opt/google/chrome/chrome-sandbox --adjust-oom-score 5447 300 ) by /opt/google/chrome/chrome-sandbox[Chrome_ProcessL:5449] uid/euid:1236/1236 gid/egid:100/100, parent /opt/google/chrome/chrome[Chrome_ProcessL:5400] uid/euid:1236/1236 gid/egid:100/100
grsec: (bowser:U:/opt/google/chrome/chrome-sandbox) exec of /opt/google/chrome/chrome-sandbox (/opt/google/chrome/chrome-sandbox --adjust-oom-score 5451 300 ) by /opt/google/chrome/chrome-sandbox[Chrome_ProcessL:5453] uid/euid:1236/1236 gid/egid:100/100, parent /opt/google/chrome/chrome[Chrome_ProcessL:5400] uid/euid:1236/1236 gid/egid:100/100
NMI watchdog: Watchdog detected hard LOCKUP on cpu 0
Modules linked in: netconsole 9p kvm_intel kvm irqbypass snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_pcm usb_storage 9pnet rndis_host cdc_ether usbnet mii b43 bcma led_class mac80211 cfg80211 rfkill broadcom bcm_phy_lib tg3 ptp pps_core [last unloaded: 9p]
CPU: 0 PID: 5440 Comm: chrome Not tainted 4.5.5-hardened-r2 #4
Hardware name: Acer Aspire 5741 /Aspire 5741 , BIOS V1.05 04/06/2010
0000000000000000 67fad64cd8227e53 0000000000000086 0000000000000000
ffffffff81430a40 ffffffff8114b87e 67fad64cd8227e53 0000000000000000
0000000000000000 ffffffff8116d309 ffff88006f009c38 ffffffff81189302
Call Trace:
<NMI> [<ffffffff81430a40>] ? dump_stack+0x5a/0xaa
[<ffffffff8114b87e>] ? print_modules+0x5e/0xd0
[<ffffffff8116d309>] ? watchdog_overflow_callback+0xe9/0x110
[<ffffffff81189302>] ? __perf_event_overflow+0xa2/0x200
[<ffffffff810215f9>] ? intel_pmu_handle_irq+0x1d9/0x500
[<ffffffff82213527>] ? x86_64_start_kernel+0x158/0x18a
[<ffffffff81438e1d>] ? seq_buf_vprintf+0x3d/0x110
[<ffffffff82213527>] ? x86_64_start_kernel+0x158/0x18a
[<ffffffff8143ef82>] ? vsnprintf+0x292/0x5b0
[<ffffffff817ce04d>] ? menu_select+0x14d/0x410
[<ffffffff81438e1d>] ? seq_buf_vprintf+0x3d/0x110
[<ffffffff81435c06>] ? nmi_vprintk+0x46/0x80
[<ffffffff811906ed>] ? printk+0x75/0xa4
[<ffffffff817ce04d>] ? menu_select+0x14d/0x410
[<ffffffff81006e24>] ? show_regs+0x54/0x1b0
[<ffffffff81018c72>] ? perf_event_nmi_handler+0x42/0x70
[<ffffffff81007c9c>] ? nmi_handle+0x6c/0xd0
[<ffffffff81007e22>] ? do_nmi+0x122/0x400
[<ffffffff81a45c72>] ? end_repeat_nmi+0x1a/0x54
[<ffffffff8115a4b0>] ? pids_cancel.constprop.7+0x60/0x60
[<ffffffff811144de>] ? queued_write_lock_slowpath+0x5e/0xc0
[<ffffffff811144de>] ? queued_write_lock_slowpath+0x5e/0xc0
[<ffffffff811144de>] ? queued_write_lock_slowpath+0x5e/0xc0
<<EOE>> [<ffffffff81113e5f>] ? do_raw_write_lock+0x3f/0xd0
[<ffffffff810c8186>] ? copy_process.part.39+0x1986/0x1fc0
[<ffffffff810c89bb>] ? _do_fork+0xeb/0x3a0
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
Sending NMI to other CPUs:
NMI backtrace for cpu 1
CPU: 1 PID: 5436 Comm: chrome Not tainted 4.5.5-hardened-r2 #4
Hardware name: Acer Aspire 5741 /Aspire 5741 , BIOS V1.05 04/06/2010
task: ffff880062b45000 ti: ffff880062b45960 task.ti: ffff880062b45960
RIP: 0010:[<ffffffff81112309>] [<ffffffff81112309>] queued_spin_lock_slowpath+0x139/0x1f0
RSP: 0018:ffffc900025bbd90 EFLAGS: 00000046
RAX: 0000000000000000 RBX: 0000000000100101 RCX: ffff88006f451c80
RDX: ffff88006f4d1c80 RSI: 0000000000000101 RDI: 0000000000080000
RBP: ffffffff82004084 R08: 0000000000000000 R09: ffff8800626804a0
R10: 0000000000000000 R11: 000000000000155a R12: 00000000003d0f00
R13: ffff880062680480 R14: 0000035beecf19d0 R15: ffff880056b82800
FS: 0000035c17e9a900(0000) GS:ffff88006f440000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000035beecf0df8 CR3: 0000000001a56000 CR4: 00000000000006f0
Stack:
ffff88006f451c80 e6c6ae49a8653ee5 ffffffff82004080 ffffc900025bbeb8
ffffffff81114520 8f939bf70e2c4baa e6c6ae49a8653ee5 ffffffff82004080
ffffffff81113e5f 6fb5dcb136b88e3c ffffffff810c8186 ffffffffffff4111
Call Trace:
[<ffffffff81114520>] ? queued_write_lock_slowpath+0xa0/0xc0
[<ffffffff81113e5f>] ? do_raw_write_lock+0x3f/0xd0
[<ffffffff810c8186>] ? copy_process.part.39+0x1986/0x1fc0
[<ffffffff810c89bb>] ? _do_fork+0xeb/0x3a0
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
Code: 0c 83 e8 01 83 e2 30 48 98 48 81 c2 80 1c 01 00 48 03 14 c5 40 b4 f4 81 48 8b 04 24 48 89 02 8b 41 08 85 c0 75 09 f3 90 8b 41 08 <85> c0 74 f7 4c 8b 01 4d 85 c0 74 08 41 0f 18 08 eb 02 f3 90 8b
NMI backtrace for cpu 2
CPU: 2 PID: 5463 Comm: chrome Not tainted 4.5.5-hardened-r2 #4
Hardware name: Acer Aspire 5741 /Aspire 5741 , BIOS V1.05 04/06/2010
task: ffff8800572be400 ti: ffff8800572bed60 task.ti: ffff8800572bed60
RIP: 0010:[<ffffffff81112375>] [<ffffffff81112375>] queued_spin_lock_slowpath+0x1a5/0x1f0
RSP: 0018:ffffc9000268bbc0 EFLAGS: 00000202
RAX: 0000000000080101 RBX: 0000000000000001 RCX: 0000000000000001
RDX: 0000000000000101 RSI: 0000000000000101 RDI: ffffffff82004084
RBP: ffffffff82004084 R08: 0000000000000014 R09: ffffffff81e38f9c
R10: 0000035beedb89d0 R11: 0000000000000246 R12: 0000000000000014
R13: 000000000000001b R14: ffff8800572be400 R15: 0000000000000001
FS: 0000035beedb8700(0000) GS:ffff88006f480000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000035beedb6d30 CR3: 0000000001a58000 CR4: 00000000000006f0
Stack:
ffffffffffff4111 41196ed7c6a3b9e2 ffffffff82004080 0000000000000201
ffffffff81114461 ffffffffffff4111 41196ed7c6a3b9e2 000000000000000d
ffffc9000268bdf0 ffffffff81427782 ffffffffffff4111 ffffffffffff4111
Call Trace:
[<ffffffff81114461>] ? queued_read_lock_slowpath+0xc1/0xe0
[<ffffffff81427782>] ? gr_log_varargs+0x172/0x28e0
[<ffffffff8142ed43>] ? gr_log_resource+0xc3/0xf0
[<ffffffff81420af9>] ? gr_learn_resource+0x39/0x230
[<ffffffff810f8cca>] ? can_nice+0xba/0x130
[<ffffffff810dd9cd>] ? set_one_prio+0x9d/0xc0
[<ffffffff810dfbf5>] ? sys_setpriority+0x125/0x240
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
Code: 0f b6 04 24 4d 85 c0 88 45 00 74 3e c7 04 24 01 00 00 00 8b 04 24 41 89 40 08 e9 1b ff ff ff 83 fa 01 75 07 e9 18 ff ff ff f3 90 <8b> 45 00 84 c0 75 f7 b8 01 00 00 00 66 89 04 24 0f b7 04 24 66
NMI backtrace for cpu 3
CPU: 3 PID: 5444 Comm: chrome Not tainted 4.5.5-hardened-r2 #4
Hardware name: Acer Aspire 5741 /Aspire 5741 , BIOS V1.05 04/06/2010
task: ffff8800620d6e00 ti: ffff8800620d7760 task.ti: ffff8800620d7760
RIP: 0010:[<ffffffff81112320>] [<ffffffff81112320>] queued_spin_lock_slowpath+0x150/0x1f0
RSP: 0018:ffffc900025fbd90 EFLAGS: 00000002
RAX: 0000000000000000 RBX: 0000000000000101 RCX: ffff88006f4d1c80
RDX: 0000000000080101 RSI: 0000000000000101 RDI: 0000000000100000
RBP: ffffffff82004084 R08: 0000000000000000 R09: ffff8800574a01a0
R10: 0000000000000000 R11: 0000000000001559 R12: 00000000003d0f00
R13: ffff8800574a0180 R14: 0000035beec579d0 R15: ffff8800572bee00
FS: 0000035c17e9a900(0000) GS:ffff88006f4c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000035beec56df8 CR3: 0000000001a5a000 CR4: 00000000000006f0
Stack:
000000000000000a 64aa388e0dfa7059 ffffffff82004080 ffffc900025fbeb8
ffffffff81114520 8f939bf70e2c4baa 64aa388e0dfa7059 ffffffff82004080
ffffffff81113e5f 6fb5dcb136b88e3c ffffffff810c8186 ffffffffffff4111
Call Trace:
[<ffffffff81114520>] ? queued_write_lock_slowpath+0xa0/0xc0
[<ffffffff81113e5f>] ? do_raw_write_lock+0x3f/0xd0
[<ffffffff810c8186>] ? copy_process.part.39+0x1986/0x1fc0
[<ffffffff810c89bb>] ? _do_fork+0xeb/0x3a0
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
Code: 81 48 8b 04 24 48 89 02 8b 41 08 85 c0 75 09 f3 90 8b 41 08 85 c0 74 f7 4c 8b 01 4d 85 c0 74 08 41 0f 18 08 eb 02 f3 90 8b 55 00 <66> 85 d2 75 f6 be 01 00 00 00 eb 11 89 d0 f0 0f b1 75 00 39 c2
NMI watchdog: Watchdog detected hard LOCKUP on cpu 1
Modules linked in: netconsole 9p kvm_intel kvm irqbypass snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_pcm usb_storage 9pnet rndis_host cdc_ether usbnet mii b43 bcma led_class mac80211 cfg80211 rfkill broadcom bcm_phy_lib tg3 ptp pps_core [last unloaded: 9p]
CPU: 1 PID: 5436 Comm: chrome Not tainted 4.5.5-hardened-r2 #4
Hardware name: Acer Aspire 5741 /Aspire 5741 , BIOS V1.05 04/06/2010
0000000000000000 e6c6ae49a8653ee5 0000000000000086 0000000000000000
ffffffff81430a40 ffffffff8114b87e e6c6ae49a8653ee5 0000000000000000
0000000000000000 ffffffff8116d309 ffff88006c880000 ffffffff81189302
Call Trace:
<NMI> [<ffffffff81430a40>] ? dump_stack+0x5a/0xaa
[<ffffffff8114b87e>] ? print_modules+0x5e/0xd0
[<ffffffff8116d309>] ? watchdog_overflow_callback+0xe9/0x110
[<ffffffff81189302>] ? __perf_event_overflow+0xa2/0x200
[<ffffffff810215f9>] ? intel_pmu_handle_irq+0x1d9/0x500
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
[<ffffffff81438e1d>] ? seq_buf_vprintf+0x3d/0x110
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
[<ffffffff8143ef82>] ? vsnprintf+0x292/0x5b0
[<ffffffff8111231e>] ? queued_spin_lock_slowpath+0x14e/0x1f0
[<ffffffff81438e1d>] ? seq_buf_vprintf+0x3d/0x110
[<ffffffff81435c06>] ? nmi_vprintk+0x46/0x80
[<ffffffff811906ed>] ? printk+0x75/0xa4
[<ffffffff8111231e>] ? queued_spin_lock_slowpath+0x14e/0x1f0
[<ffffffff81006e24>] ? show_regs+0x54/0x1b0
[<ffffffff81018c72>] ? perf_event_nmi_handler+0x42/0x70
[<ffffffff81007c9c>] ? nmi_handle+0x6c/0xd0
[<ffffffff81007e22>] ? do_nmi+0x122/0x400
[<ffffffff81a45c72>] ? end_repeat_nmi+0x1a/0x54
[<ffffffff81112306>] ? queued_spin_lock_slowpath+0x136/0x1f0
[<ffffffff81112306>] ? queued_spin_lock_slowpath+0x136/0x1f0
[<ffffffff81112306>] ? queued_spin_lock_slowpath+0x136/0x1f0
<<EOE>> [<ffffffff81114520>] ? queued_write_lock_slowpath+0xa0/0xc0
[<ffffffff81113e5f>] ? do_raw_write_lock+0x3f/0xd0
[<ffffffff810c8186>] ? copy_process.part.39+0x1986/0x1fc0
[<ffffffff810c89bb>] ? _do_fork+0xeb/0x3a0
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
NMI watchdog: Watchdog detected hard LOCKUP on cpu 3
Modules linked in: netconsole 9p kvm_intel kvm irqbypass snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_pcm usb_storage 9pnet rndis_host cdc_ether usbnet mii b43 bcma led_class mac80211 cfg80211 rfkill broadcom bcm_phy_lib tg3 ptp pps_core [last unloaded: 9p]
CPU: 3 PID: 5444 Comm: chrome Not tainted 4.5.5-hardened-r2 #4
Hardware name: Acer Aspire 5741 /Aspire 5741 , BIOS V1.05 04/06/2010
0000000000000000 64aa388e0dfa7059 0000000000000086 0000000000000000
ffffffff81430a40 ffffffff8114b87e 64aa388e0dfa7059 0000000000000000
0000000000000000 ffffffff8116d309 ffff88006ca00000 ffffffff81189302
Call Trace:
<NMI> [<ffffffff81430a40>] ? dump_stack+0x5a/0xaa
[<ffffffff8114b87e>] ? print_modules+0x5e/0xd0
[<ffffffff8116d309>] ? watchdog_overflow_callback+0xe9/0x110
[<ffffffff81189302>] ? __perf_event_overflow+0xa2/0x200
[<ffffffff810215f9>] ? intel_pmu_handle_irq+0x1d9/0x500
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
[<ffffffff81438e1d>] ? seq_buf_vprintf+0x3d/0x110
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
[<ffffffff8143ef82>] ? vsnprintf+0x292/0x5b0
[<ffffffff81112335>] ? queued_spin_lock_slowpath+0x165/0x1f0
[<ffffffff81438e1d>] ? seq_buf_vprintf+0x3d/0x110
[<ffffffff81435c06>] ? nmi_vprintk+0x46/0x80
[<ffffffff811906ed>] ? printk+0x75/0xa4
[<ffffffff81112335>] ? queued_spin_lock_slowpath+0x165/0x1f0
[<ffffffff81006e24>] ? show_regs+0x54/0x1b0
[<ffffffff81018c72>] ? perf_event_nmi_handler+0x42/0x70
[<ffffffff81007c9c>] ? nmi_handle+0x6c/0xd0
[<ffffffff81007f13>] ? do_nmi+0x213/0x400
[<ffffffff81a45c72>] ? end_repeat_nmi+0x1a/0x54
[<ffffffff8111231d>] ? queued_spin_lock_slowpath+0x14d/0x1f0
[<ffffffff8111231d>] ? queued_spin_lock_slowpath+0x14d/0x1f0
[<ffffffff8111231d>] ? queued_spin_lock_slowpath+0x14d/0x1f0
<<EOE>> [<ffffffff81114520>] ? queued_write_lock_slowpath+0xa0/0xc0
[<ffffffff81113e5f>] ? do_raw_write_lock+0x3f/0xd0
[<ffffffff810c8186>] ? copy_process.part.39+0x1986/0x1fc0
[<ffffffff810c89bb>] ? _do_fork+0xeb/0x3a0
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [chrome:5463]
Modules linked in: netconsole 9p kvm_intel kvm irqbypass snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_pcm usb_storage 9pnet rndis_host cdc_ether usbnet mii b43 bcma led_class mac80211 cfg80211 rfkill broadcom bcm_phy_lib tg3 ptp pps_core [last unloaded: 9p]
CPU: 2 PID: 5463 Comm: chrome Not tainted 4.5.5-hardened-r2 #4
Hardware name: Acer Aspire 5741 /Aspire 5741 , BIOS V1.05 04/06/2010
task: ffff8800572be400 ti: ffff8800572bed60 task.ti: ffff8800572bed60
RIP: 0010:[<ffffffff81112375>] [<ffffffff81112375>] queued_spin_lock_slowpath+0x1a5/0x1f0
RSP: 0018:ffffc9000268bbc0 EFLAGS: 00000202
RAX: 0000000000080101 RBX: 0000000000000001 RCX: 0000000000000001
RDX: 0000000000000101 RSI: 0000000000000101 RDI: ffffffff82004084
RBP: ffffffff82004084 R08: 0000000000000014 R09: ffffffff81e38f9c
R10: 0000035beedb89d0 R11: 0000000000000246 R12: 0000000000000014
R13: 000000000000001b R14: ffff8800572be400 R15: 0000000000000001
FS: 0000035beedb8700(0000) GS:ffff88006f480000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000035beedb6d30 CR3: 0000000001a58000 CR4: 00000000000006f0
Stack:
ffffffffffff4111 41196ed7c6a3b9e2 ffffffff82004080 0000000000000201
ffffffff81114461 ffffffffffff4111 41196ed7c6a3b9e2 000000000000000d
ffffc9000268bdf0 ffffffff81427782 ffffffffffff4111 ffffffffffff4111
Call Trace:
[<ffffffff81114461>] ? queued_read_lock_slowpath+0xc1/0xe0
[<ffffffff81427782>] ? gr_log_varargs+0x172/0x28e0
[<ffffffff8142ed43>] ? gr_log_resource+0xc3/0xf0
[<ffffffff81420af9>] ? gr_learn_resource+0x39/0x230
[<ffffffff810f8cca>] ? can_nice+0xba/0x130
[<ffffffff810dd9cd>] ? set_one_prio+0x9d/0xc0
[<ffffffff810dfbf5>] ? sys_setpriority+0x125/0x240
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
Code: 0f b6 04 24 4d 85 c0 88 45 00 74 3e c7 04 24 01 00 00 00 8b 04 24 41 89 40 08 e9 1b ff ff ff 83 fa 01 75 07 e9 18 ff ff ff f3 90 <8b> 45 00 84 c0 75 f7 b8 01 00 00 00 66 89 04 24 0f b7 04 24 66
Sending NMI to other CPUs:
NMI backtrace for cpu 0
CPU: 0 PID: 5440 Comm: chrome Not tainted 4.5.5-hardened-r2 #4
Hardware name: Acer Aspire 5741 /Aspire 5741 , BIOS V1.05 04/06/2010
task: ffff880056c41e00 ti: ffff880056c42760 task.ti: ffff880056c42760
RIP: 0010:[<ffffffff811144de>] [<ffffffff811144de>] queued_write_lock_slowpath+0x5e/0xc0
RSP: 0018:ffffc900025dbdb8 EFLAGS: 00000006
RAX: 0000000000000101 RBX: ffffffff82004080 RCX: ffffffff8115a4b0
RDX: 00000000000000ff RSI: 000000000000000b RDI: ffffffff82004080
RBP: ffffc900025dbeb8 R08: 61c8864680b583eb R09: ffff8800575b3320
R10: 0000000000000000 R11: 0000000000001558 R12: 00000000003d0f00
R13: ffff8800575b3300 R14: 0000035beecfc9d0 R15: ffff8800620d5a00
FS: 0000035c17e9a900(0000) GS:ffff88006f400000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000035beecfbdf8 CR3: 0000000001a54000 CR4: 00000000000006f0
Stack:
8f939bf70e2c4baa 67fad64cd8227e53 ffffffff82004080 ffffffff81113e5f
6fb5dcb136b88e3c ffffffff810c8186 ffffffffffff4111 ffffffffffff4111
ffffffffffff4111 ffffffffffff4111 ffffffffffff4111 ffffffffffff4111
Call Trace:
[<ffffffff81113e5f>] ? do_raw_write_lock+0x3f/0xd0
[<ffffffff810c8186>] ? copy_process.part.39+0x1986/0x1fc0
[<ffffffff810c89bb>] ? _do_fork+0xeb/0x3a0
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
Code: f0 0f b1 13 85 c0 74 31 ba 01 00 00 00 eb 02 f3 90 0f b6 03 84 c0 75 f7 f0 0f b0 13 84 c0 75 ef ba ff 00 00 00 eb 02 f3 90 8b 03 <83> f8 01 75 f7 f0 0f b1 13 83 f8 01 75 ee c6 44 24 07 00 0f b6
NMI backtrace for cpu 1
CPU: 1 PID: 5436 Comm: chrome Not tainted 4.5.5-hardened-r2 #4
Hardware name: Acer Aspire 5741 /Aspire 5741 , BIOS V1.05 04/06/2010
task: ffff880062b45000 ti: ffff880062b45960 task.ti: ffff880062b45960
RIP: 0010:[<ffffffff81112309>] [<ffffffff81112309>] queued_spin_lock_slowpath+0x139/0x1f0
RSP: 0018:ffffc900025bbd90 EFLAGS: 00000046
RAX: 0000000000000000 RBX: 0000000000100101 RCX: ffff88006f451c80
RDX: ffff88006f4d1c80 RSI: 0000000000000101 RDI: 0000000000080000
RBP: ffffffff82004084 R08: 0000000000000000 R09: ffff8800626804a0
R10: 0000000000000000 R11: 000000000000155a R12: 00000000003d0f00
R13: ffff880062680480 R14: 0000035beecf19d0 R15: ffff880056b82800
FS: 0000035c17e9a900(0000) GS:ffff88006f440000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000035beecf0df8 CR3: 0000000001a56000 CR4: 00000000000006f0
Stack:
ffff88006f451c80 e6c6ae49a8653ee5 ffffffff82004080 ffffc900025bbeb8
ffffffff81114520 8f939bf70e2c4baa e6c6ae49a8653ee5 ffffffff82004080
ffffffff81113e5f 6fb5dcb136b88e3c ffffffff810c8186 ffffffffffff4111
Call Trace:
[<ffffffff81114520>] ? queued_write_lock_slowpath+0xa0/0xc0
[<ffffffff81113e5f>] ? do_raw_write_lock+0x3f/0xd0
[<ffffffff810c8186>] ? copy_process.part.39+0x1986/0x1fc0
[<ffffffff810c89bb>] ? _do_fork+0xeb/0x3a0
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
Code: 0c 83 e8 01 83 e2 30 48 98 48 81 c2 80 1c 01 00 48 03 14 c5 40 b4 f4 81 48 8b 04 24 48 89 02 8b 41 08 85 c0 75 09 f3 90 8b 41 08 <85> c0 74 f7 4c 8b 01 4d 85 c0 74 08 41 0f 18 08 eb 02 f3 90 8b
NMI backtrace for cpu 3
CPU: 3 PID: 5444 Comm: chrome Not tainted 4.5.5-hardened-r2 #4
Hardware name: Acer Aspire 5741 /Aspire 5741 , BIOS V1.05 04/06/2010
task: ffff8800620d6e00 ti: ffff8800620d7760 task.ti: ffff8800620d7760
RIP: 0010:[<ffffffff81112320>] [<ffffffff81112320>] queued_spin_lock_slowpath+0x150/0x1f0
RSP: 0018:ffffc900025fbd90 EFLAGS: 00000002
RAX: 0000000000000000 RBX: 0000000000000101 RCX: ffff88006f4d1c80
RDX: 0000000000080101 RSI: 0000000000000101 RDI: 0000000000100000
RBP: ffffffff82004084 R08: 0000000000000000 R09: ffff8800574a01a0
R10: 0000000000000000 R11: 0000000000001559 R12: 00000000003d0f00
R13: ffff8800574a0180 R14: 0000035beec579d0 R15: ffff8800572bee00
FS: 0000035c17e9a900(0000) GS:ffff88006f4c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000035beec56df8 CR3: 0000000001a5a000 CR4: 00000000000006f0
Stack:
000000000000000a 64aa388e0dfa7059 ffffffff82004080 ffffc900025fbeb8
ffffffff81114520 8f939bf70e2c4baa 64aa388e0dfa7059 ffffffff82004080
ffffffff81113e5f 6fb5dcb136b88e3c ffffffff810c8186 ffffffffffff4111
Call Trace:
[<ffffffff81114520>] ? queued_write_lock_slowpath+0xa0/0xc0
[<ffffffff81113e5f>] ? do_raw_write_lock+0x3f/0xd0
[<ffffffff810c8186>] ? copy_process.part.39+0x1986/0x1fc0
[<ffffffff810c89bb>] ? _do_fork+0xeb/0x3a0
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
Code: 81 48 8b 04 24 48 89 02 8b 41 08 85 c0 75 09 f3 90 8b 41 08 85 c0 74 f7 4c 8b 01 4d 85 c0 74 08 41 0f 18 08 eb 02 f3 90 8b 55 00 <66> 85 d2 75 f6 be 01 00 00 00 eb 11 89 d0 f0 0f b1 75 00 39 c2
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [chrome:5463]
Modules linked in: netconsole 9p kvm_intel kvm irqbypass snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_pcm usb_storage 9pnet rndis_host cdc_ether usbnet mii b43 bcma led_class mac80211 cfg80211 rfkill broadcom bcm_phy_lib tg3 ptp pps_core [last unloaded: 9p]
CPU: 2 PID: 5463 Comm: chrome Tainted: G L 4.5.5-hardened-r2 #4
Hardware name: Acer Aspire 5741 /Aspire 5741 , BIOS V1.05 04/06/2010
task: ffff8800572be400 ti: ffff8800572bed60 task.ti: ffff8800572bed60
RIP: 0010:[<ffffffff81112375>] [<ffffffff81112375>] queued_spin_lock_slowpath+0x1a5/0x1f0
RSP: 0018:ffffc9000268bbc0 EFLAGS: 00000202
RAX: 0000000000080101 RBX: 0000000000000001 RCX: 0000000000000001
RDX: 0000000000000101 RSI: 0000000000000101 RDI: ffffffff82004084
RBP: ffffffff82004084 R08: 0000000000000014 R09: ffffffff81e38f9c
R10: 0000035beedb89d0 R11: 0000000000000246 R12: 0000000000000014
R13: 000000000000001b R14: ffff8800572be400 R15: 0000000000000001
FS: 0000035beedb8700(0000) GS:ffff88006f480000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000035beedb6d30 CR3: 0000000001a58000 CR4: 00000000000006f0
Stack:
ffffffffffff4111 41196ed7c6a3b9e2 ffffffff82004080 0000000000000201
ffffffff81114461 ffffffffffff4111 41196ed7c6a3b9e2 000000000000000d
ffffc9000268bdf0 ffffffff81427782 ffffffffffff4111 ffffffffffff4111
Call Trace:
[<ffffffff81114461>] ? queued_read_lock_slowpath+0xc1/0xe0
[<ffffffff81427782>] ? gr_log_varargs+0x172/0x28e0
[<ffffffff8142ed43>] ? gr_log_resource+0xc3/0xf0
[<ffffffff81420af9>] ? gr_learn_resource+0x39/0x230
[<ffffffff810f8cca>] ? can_nice+0xba/0x130
[<ffffffff810dd9cd>] ? set_one_prio+0x9d/0xc0
[<ffffffff810dfbf5>] ? sys_setpriority+0x125/0x240
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
Code: 0f b6 04 24 4d 85 c0 88 45 00 74 3e c7 04 24 01 00 00 00 8b 04 24 41 89 40 08 e9 1b ff ff ff 83 fa 01 75 07 e9 18 ff ff ff f3 90 <8b> 45 00 84 c0 75 f7 b8 01 00 00 00 66 89 04 24 0f b7 04 24 66
Sending NMI to other CPUs:
NMI backtrace for cpu 0
CPU: 0 PID: 5440 Comm: chrome Tainted: G L 4.5.5-hardened-r2 #4
Hardware name: Acer Aspire 5741 /Aspire 5741 , BIOS V1.05 04/06/2010
task: ffff880056c41e00 ti: ffff880056c42760 task.ti: ffff880056c42760
RIP: 0010:[<ffffffff811144da>] [<ffffffff811144da>] queued_write_lock_slowpath+0x5a/0xc0
RSP: 0018:ffffc900025dbdb8 EFLAGS: 00000006
RAX: 0000000000000101 RBX: ffffffff82004080 RCX: ffffffff8115a4b0
RDX: 00000000000000ff RSI: 000000000000000b RDI: ffffffff82004080
RBP: ffffc900025dbeb8 R08: 61c8864680b583eb R09: ffff8800575b3320
R10: 0000000000000000 R11: 0000000000001558 R12: 00000000003d0f00
R13: ffff8800575b3300 R14: 0000035beecfc9d0 R15: ffff8800620d5a00
FS: 0000035c17e9a900(0000) GS:ffff88006f400000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000035beecfbdf8 CR3: 0000000001a54000 CR4: 00000000000006f0
Stack:
8f939bf70e2c4baa 67fad64cd8227e53 ffffffff82004080 ffffffff81113e5f
6fb5dcb136b88e3c ffffffff810c8186 ffffffffffff4111 ffffffffffff4111
ffffffffffff4111 ffffffffffff4111 ffffffffffff4111 ffffffffffff4111
Call Trace:
[<ffffffff81113e5f>] ? do_raw_write_lock+0x3f/0xd0
[<ffffffff810c8186>] ? copy_process.part.39+0x1986/0x1fc0
[<ffffffff810c89bb>] ? _do_fork+0xeb/0x3a0
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
Code: ff 00 00 00 f0 0f b1 13 85 c0 74 31 ba 01 00 00 00 eb 02 f3 90 0f b6 03 84 c0 75 f7 f0 0f b0 13 84 c0 75 ef ba ff 00 00 00 eb 02 <f3> 90 8b 03 83 f8 01 75 f7 f0 0f b1 13 83 f8 01 75 ee c6 44 24
NMI backtrace for cpu 1
CPU: 1 PID: 5436 Comm: chrome Tainted: G L 4.5.5-hardened-r2 #4
Hardware name: Acer Aspire 5741 /Aspire 5741 , BIOS V1.05 04/06/2010
task: ffff880062b45000 ti: ffff880062b45960 task.ti: ffff880062b45960
RIP: 0010:[<ffffffff81112306>] [<ffffffff81112306>] queued_spin_lock_slowpath+0x136/0x1f0
RSP: 0018:ffffc900025bbd90 EFLAGS: 00000046
RAX: 0000000000000000 RBX: 0000000000100101 RCX: ffff88006f451c80
RDX: ffff88006f4d1c80 RSI: 0000000000000101 RDI: 0000000000080000
RBP: ffffffff82004084 R08: 0000000000000000 R09: ffff8800626804a0
R10: 0000000000000000 R11: 000000000000155a R12: 00000000003d0f00
R13: ffff880062680480 R14: 0000035beecf19d0 R15: ffff880056b82800
FS: 0000035c17e9a900(0000) GS:ffff88006f440000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000035beecf0df8 CR3: 0000000001a56000 CR4: 00000000000006f0
Stack:
ffff88006f451c80 e6c6ae49a8653ee5 ffffffff82004080 ffffc900025bbeb8
ffffffff81114520 8f939bf70e2c4baa e6c6ae49a8653ee5 ffffffff82004080
ffffffff81113e5f 6fb5dcb136b88e3c ffffffff810c8186 ffffffffffff4111
Call Trace:
[<ffffffff81114520>] ? queued_write_lock_slowpath+0xa0/0xc0
[<ffffffff81113e5f>] ? do_raw_write_lock+0x3f/0xd0
[<ffffffff810c8186>] ? copy_process.part.39+0x1986/0x1fc0
[<ffffffff810c89bb>] ? _do_fork+0xeb/0x3a0
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
Code: 48 c1 ea 0c 83 e8 01 83 e2 30 48 98 48 81 c2 80 1c 01 00 48 03 14 c5 40 b4 f4 81 48 8b 04 24 48 89 02 8b 41 08 85 c0 75 09 f3 90 <8b> 41 08 85 c0 74 f7 4c 8b 01 4d 85 c0 74 08 41 0f 18 08 eb 02
NMI backtrace for cpu 3
CPU: 3 PID: 5444 Comm: chrome Tainted: G L 4.5.5-hardened-r2 #4
Hardware name: Acer Aspire 5741 /Aspire 5741 , BIOS V1.05 04/06/2010
task: ffff8800620d6e00 ti: ffff8800620d7760 task.ti: ffff8800620d7760
RIP: 0010:[<ffffffff8111231d>] [<ffffffff8111231d>] queued_spin_lock_slowpath+0x14d/0x1f0
RSP: 0018:ffffc900025fbd90 EFLAGS: 00000002
RAX: 0000000000000000 RBX: 0000000000000101 RCX: ffff88006f4d1c80
RDX: 0000000000080101 RSI: 0000000000000101 RDI: 0000000000100000
RBP: ffffffff82004084 R08: 0000000000000000 R09: ffff8800574a01a0
R10: 0000000000000000 R11: 0000000000001559 R12: 00000000003d0f00
R13: ffff8800574a0180 R14: 0000035beec579d0 R15: ffff8800572bee00
FS: 0000035c17e9a900(0000) GS:ffff88006f4c0000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000035beec56df8 CR3: 0000000001a5a000 CR4: 00000000000006f0
Stack:
000000000000000a 64aa388e0dfa7059 ffffffff82004080 ffffc900025fbeb8
ffffffff81114520 8f939bf70e2c4baa 64aa388e0dfa7059 ffffffff82004080
ffffffff81113e5f 6fb5dcb136b88e3c ffffffff810c8186 ffffffffffff4111
Call Trace:
[<ffffffff81114520>] ? queued_write_lock_slowpath+0xa0/0xc0
[<ffffffff81113e5f>] ? do_raw_write_lock+0x3f/0xd0
[<ffffffff810c8186>] ? copy_process.part.39+0x1986/0x1fc0
[<ffffffff810c89bb>] ? _do_fork+0xeb/0x3a0
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
Code: 40 b4 f4 81 48 8b 04 24 48 89 02 8b 41 08 85 c0 75 09 f3 90 8b 41 08 85 c0 74 f7 4c 8b 01 4d 85 c0 74 08 41 0f 18 08 eb 02 f3 90 <8b> 55 00 66 85 d2 75 f6 be 01 00 00 00 eb 11 89 d0 f0 0f b1 75
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [chrome:5463]
Modules linked in: netconsole 9p kvm_intel kvm irqbypass snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_pcm usb_storage 9pnet rndis_host cdc_ether usbnet mii b43 bcma led_class mac80211 cfg80211 rfkill broadcom bcm_phy_lib tg3 ptp pps_core [last unloaded: 9p]
CPU: 2 PID: 5463 Comm: chrome Tainted: G L 4.5.5-hardened-r2 #4
Hardware name: Acer Aspire 5741 /Aspire 5741 , BIOS V1.05 04/06/2010
task: ffff8800572be400 ti: ffff8800572bed60 task.ti: ffff8800572bed60
RIP: 0010:[<ffffffff81112378>] [<ffffffff81112378>] queued_spin_lock_slowpath+0x1a8/0x1f0
RSP: 0018:ffffc9000268bbc0 EFLAGS: 00000202
RAX: 0000000000080101 RBX: 0000000000000001 RCX: 0000000000000001
RDX: 0000000000000101 RSI: 0000000000000101 RDI: ffffffff82004084
RBP: ffffffff82004084 R08: 0000000000000014 R09: ffffffff81e38f9c
R10: 0000035beedb89d0 R11: 0000000000000246 R12: 0000000000000014
R13: 000000000000001b R14: ffff8800572be400 R15: 0000000000000001
FS: 0000035beedb8700(0000) GS:ffff88006f480000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000035beedb6d30 CR3: 0000000001a58000 CR4: 00000000000006f0
Stack:
ffffffffffff4111 41196ed7c6a3b9e2 ffffffff82004080 0000000000000201
ffffffff81114461 ffffffffffff4111 41196ed7c6a3b9e2 000000000000000d
ffffc9000268bdf0 ffffffff81427782 ffffffffffff4111 ffffffffffff4111
Call Trace:
[<ffffffff81114461>] ? queued_read_lock_slowpath+0xc1/0xe0
[<ffffffff81427782>] ? gr_log_varargs+0x172/0x28e0
[<ffffffff8142ed43>] ? gr_log_resource+0xc3/0xf0
[<ffffffff81420af9>] ? gr_learn_resource+0x39/0x230
[<ffffffff810f8cca>] ? can_nice+0xba/0x130
[<ffffffff810dd9cd>] ? set_one_prio+0x9d/0xc0
[<ffffffff810dfbf5>] ? sys_setpriority+0x125/0x240
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
Code: 24 4d 85 c0 88 45 00 74 3e c7 04 24 01 00 00 00 8b 04 24 41 89 40 08 e9 1b ff ff ff 83 fa 01 75 07 e9 18 ff ff ff f3 90 8b 45 00 <84> c0 75 f7 b8 01 00 00 00 66 89 04 24 0f b7 04 24 66 89 45 00
Sending NMI to other CPUs:
NMI backtrace for cpu 0
CPU: 0 PID: 5440 Comm: chrome Tainted: G L 4.5.5-hardened-r2 #4
Hardware name: Acer Aspire 5741 /Aspire 5741 , BIOS V1.05 04/06/2010
task: ffff880056c41e00 ti: ffff880056c42760 task.ti: ffff880056c42760
RIP: 0010:[<ffffffff811144dc>] [<ffffffff811144dc>] queued_write_lock_slowpath+0x5c/0xc0
RSP: 0018:ffffc900025dbdb8 EFLAGS: 00000006
RAX: 0000000000000101 RBX: ffffffff82004080 RCX: ffffffff8115a4b0
RDX: 00000000000000ff RSI: 000000000000000b RDI: ffffffff82004080
RBP: ffffc900025dbeb8 R08: 61c8864680b583eb R09: ffff8800575b3320
R10: 0000000000000000 R11: 0000000000001558 R12: 00000000003d0f00
R13: ffff8800575b3300 R14: 0000035beecfc9d0 R15: ffff8800620d5a00
FS: 0000035c17e9a900(0000) GS:ffff88006f400000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000035beecfbdf8 CR3: 0000000001a54000 CR4: 00000000000006f0
Stack:
8f939bf70e2c4baa 67fad64cd8227e53 ffffffff82004080 ffffffff81113e5f
6fb5dcb136b88e3c ffffffff810c8186 ffffffffffff4111 ffffffffffff4111
ffffffffffff4111 ffffffffffff4111 ffffffffffff4111 ffffffffffff4111
Call Trace:
[<ffffffff81113e5f>] ? do_raw_write_lock+0x3f/0xd0
[<ffffffff810c8186>] ? copy_process.part.39+0x1986/0x1fc0
[<ffffffff810c89bb>] ? _do_fork+0xeb/0x3a0
[<ffffffff81a44120>] ? entry_SYSCALL_64_fastpath+0x12/0x78
Code: 00 00 f0 0f b1 13 85 c0 74 31 ba 01 00 00 00 eb 02 f3 90 0f b6 03 84 c0 75 f7 f0 0f b0 13 84 c0 75 ef ba ff 00 00 00 eb 02 f3 90 <8b> 03 83 f8 01 75 f7 f0 0f b1 13 83 f8 01 75 ee c6 44 24 07 00
NMI backtrace for cpu 1
CPU: 1 PID: 5436 Comm: chrome Tainted: G L 4.5.5-hardened-r2 #4
Hardware name: Acer Aspire 5741 /Aspire 5741 , BIOS V1.05 04/06/2010
task: ffff880062b45000 ti: ffff880062b45960 task.ti: ffff880062b45960
RIP: 0010:[<ffffffff81112309>] [<ffffffff81112309>] queued_spin_lock_slowpath+0x139/0x1f0
RSP: 0018:ffffc900025bbd90 EFLAGS: 00000046
RAX: 0000000000000000 RBX: 0000000000100101 RCX: ffff88006f451c80
RDX: ffff88006f4d1c80 RSI: 0000000000000101 RDI: 0000000000080000
RBP: ffffffff82004084 R08: 0000000000000000 R09: ffff8800626804a0
R10: 0000000000000000 R11: 000000000000155a R12: 00000000003d0f00
R13: ffff880062680480 R14: 0000035beecf19d0 R15: ffff880056b82800
FS: 0000035c17e9a900(0000) GS:ffff88006f440000(0000) knlGS:0000000000000000
shadowdaemon
 
Posts: 11
Joined: Thu May 09, 2013 9:42 pm

Re: Google Chrome hard CPU lockup

Postby shadowdaemon » Fri Jun 24, 2016 9:53 am

I should also mention that this only happens on my Acer laptop. My Toshiba laptop which has the same kernel config is fine, and my test Qemu VM (using same kernel, same RBAC rules, etc) is also fine.
shadowdaemon
 
Posts: 11
Joined: Thu May 09, 2013 9:42 pm

Re: Google Chrome hard CPU lockup

Postby shadowdaemon » Sun Jun 26, 2016 12:04 am

Rebuilt the kernel with two changes (CONFIG_FRAME_POINTER and CONFIG_DEBUG_ATOMIC_SLEEP) and now it's not happening at all.
Previously the kernel would lock up immediately on starting Chrome, so not sure what's going on.

"uname -a" in the previous post should be:
Linux aether 4.5.5-hardened-r2 #5 SMP Sat Jun 25 02:46:35 AEST 2016 x86_64 Intel(R) Core(TM) i3 CPU M 330 @ 2.13GHz GenuineIntel GNU/Linux
shadowdaemon
 
Posts: 11
Joined: Thu May 09, 2013 9:42 pm


Return to grsecurity support

cron