Kernel panic/PAX ?

Discuss usability issues, general maintenance, and general support issues for a grsecurity-enabled system.

Kernel panic/PAX ?

Postby Virsh » Wed Sep 28, 2016 10:49 pm

Hello!
Sorry if I write not in that section.
All I've managed to catch.
Code: Select all
[ 1584.738781] PAX: size overflow detected in function skb_headers_offset_update net/core/skbuff.c:1051 cicus.698_38 min, count: 10, decl: network_header; num: 0; context: sk_buff;
[ 1584.738929] Kernel panic - not syncing: Aiee, killing interrupt handler!
[ 1584.738963] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 4.7.4-grsec #1
[ 1584.738989] Hardware name:
[ 1584.739033]  0000000000000046 0aa23a283aa7b298 ffff88022f303978 ffffffff814c2f8b
[ 1584.739066]  0000000000000020 0aa23a283aa7b298 ffffffff81d105a0 0000000000000009
[ 1584.739100]  ffff88022f303a10 ffffffff811c23b2 ffff88022f303a20 0000000000000008
[ 1584.739131] Call Trace:
[ 1584.739141]  <IRQ>  [<ffffffff814c2f8b>] dump_stack+0x60/0xb5
[ 1584.739169]  [<ffffffff811c23b2>] panic+0xe6/0x290
[ 1584.739189]  [<ffffffff810e6939>] ? vprintk_default+0x29/0x60
[ 1584.739210]  [<ffffffff81083e1e>] do_exit+0x90e/0xb90
[ 1584.739230]  [<ffffffff81023ed2>] ? show_stack_log_lvl+0x102/0x180
[ 1584.739253]  [<ffffffff8108415e>] do_group_exit+0x5e/0xf0
[ 1584.739272]  [<ffffffff8126c8bf>] report_size_overflow+0x7f/0x90
[ 1584.739294]  [<ffffffff8187e520>] skb_headers_offset_update+0x140/0x1d0
[ 1584.739318]  [<ffffffff81881245>] skb_copy_expand+0x115/0x1e0
[ 1584.739352]  [<ffffffffc050063f>] ieee80211_rx_handlers+0x15ef/0x2600 [mac80211]
[ 1584.739379]  [<ffffffff814e1955>] ? find_next_bit+0x15/0x40
[ 1584.739400]  [<ffffffff814c2d4f>] ? cpumask_next_and+0x2f/0x60
[ 1584.739430]  [<ffffffffc0501f02>] ieee80211_prepare_and_rx_handle+0x632/0x1600 [mac80211]
[ 1584.739459]  [<ffffffff8188af98>] ? __build_skb+0x48/0x240
[ 1584.739490]  [<ffffffffc050351a>] ieee80211_rx_napi+0x64a/0xca0 [mac80211]
[ 1584.739516]  [<ffffffff815002af>] ? swiotlb_tbl_sync_single+0x7f/0xa0
[ 1584.739541]  [<ffffffffc0d20b1c>] ath_rx_tasklet+0xb2c/0xe80 [ath9k]
[ 1584.739564]  [<ffffffff815006d0>] ? swiotlb_tbl_unmap_single+0x130/0x130
[ 1584.739590]  [<ffffffffc0d1da9e>] ath9k_tasklet+0xee/0x2b0 [ath9k]
[ 1584.739613]  [<ffffffff81085659>] tasklet_action+0x209/0x230
[ 1584.739634]  [<ffffffff819bdb1e>] __do_softirq+0x11e/0x2d4
[ 1584.739655]  [<ffffffff81085c43>] irq_exit+0x93/0xb0
[ 1584.739673]  [<ffffffff819bd774>] do_IRQ+0x54/0x110
[ 1584.739694]  [<ffffffff819bbc4b>] common_interrupt+0x8b/0x8b
[ 1584.739714]  <EOI>  [<ffffffff8181b029>] ? cpuidle_enter_state+0x129/0x2b0
[ 1584.739744]  [<ffffffff8181b217>] cpuidle_enter+0x17/0x30
[ 1584.739764]  [<ffffffff810cd8d3>] call_cpuidle+0x23/0x50
[ 1584.739783]  [<ffffffff810cdd87>] cpu_startup_entry+0x2a7/0x350
[ 1584.739805]  [<ffffffff81046acf>] start_secondary+0x24f/0x2f0
[ 1584.739860] Kernel Offset: disabled
[ 1584.745922] ---[ end Kernel panic - not syncing: Aiee, killing interrupt handler!


What can be done to roll back the kernel? The panic occurs with a specific access point.
Virsh
 
Posts: 8
Joined: Wed Sep 28, 2016 10:39 pm

Re: Kernel panic/PAX ?

Postby PaX Team » Thu Sep 29, 2016 4:42 am

this looks like the same issue as viewtopic.php?f=3&t=4448, can you follow the instructions there and help us find out the actual runtime values involved? also as a temporary workaround you can boot with pax_size_overflow_report_only to disable the reaction mechanism (it'll affect all size overflow reports though).
PaX Team
 
Posts: 2310
Joined: Mon Mar 18, 2002 4:35 pm

Re: Kernel panic/PAX ?

Postby Virsh » Thu Sep 29, 2016 6:04 am

PaX Team wrote:this looks like the same issue as viewtopic.php?f=3&t=4448, can you follow the instructions there and help us find out the actual runtime values involved? also as a temporary workaround you can boot with pax_size_overflow_report_only to disable the reaction mechanism (it'll affect all size overflow reports though).

I found bug report https://bugs.gentoo.org/show_bug.cgi?id=584378 .
Write what you need to do a more detailed instruction. I'll try. How to boot from pax_size_overflow_report_only?
Virsh
 
Posts: 8
Joined: Wed Sep 28, 2016 10:39 pm

Re: Kernel panic/PAX ?

Postby PaX Team » Thu Sep 29, 2016 7:12 am

you should apply the patch i posted at https://bugs.gentoo.org/show_bug.cgi?id=584378#c1 and then show us the resulting logs (there will be "PAX: network_header...." messsages before the size overflow report, we'll need them too). pax_size_overflow_report_only is a kernel command line parameter, it's described in Documentation/kernel-parameters.txt.
PaX Team
 
Posts: 2310
Joined: Mon Mar 18, 2002 4:35 pm

Re: Kernel panic/PAX ?

Postby Virsh » Thu Sep 29, 2016 7:55 am

PaX Team wrote:you should apply the patch i posted at https://bugs.gentoo.org/show_bug.cgi?id=584378#c1 and then show us the resulting logs (there will be "PAX: network_header...." messsages before the size overflow report, we'll need them too). pax_size_overflow_report_only is a kernel command line parameter, it's described in Documentation/kernel-parameters.txt.

patch skbuff.c patchtest
patching file skbuff.c
Hunk #1 FAILED at 973.
1 out of 1 hunk FAILED -- saving rejects to file skbuff.c.rej

patch --ignore-whitespace skbuff.c patchtest
Code: Select all
patching file skbuff.c
Hunk #1 succeeded at 1048 (offset 75 lines).

Is this normal? To compile the kernel?
Virsh
 
Posts: 8
Joined: Wed Sep 28, 2016 10:39 pm

Re: Kernel panic/PAX ?

Postby PaX Team » Thu Sep 29, 2016 8:39 am

use patch -l as the diff is whitespace damaged, or just apply it by hand, it's just a single line to add.
PaX Team
 
Posts: 2310
Joined: Mon Mar 18, 2002 4:35 pm

Re: Kernel panic/PAX ?

Postby Virsh » Thu Sep 29, 2016 8:45 pm

PaX Team wrote:use patch -l as the diff is whitespace damaged, or just apply it by hand, it's just a single line to add.

I boot new kernel
dmesg
Code: Select all
PAX:network_header:0 off:0

At the time of kernel panic, I don't see anything new. Caught via netconsole
What did I do wrong?
Here are the reports:
http://pastebin.com/0N0bnwQz
http://pastebin.com/Tds4uFwx
------------------------------
Need full dmesg? But I don't know how to get it.
Virsh
 
Posts: 8
Joined: Wed Sep 28, 2016 10:39 pm

Re: Kernel panic/PAX ?

Postby PaX Team » Thu Sep 29, 2016 8:51 pm

pass pax_size_overflow_report_only to the kernel and it won't panic then you can grab dmesg easily.
PaX Team
 
Posts: 2310
Joined: Mon Mar 18, 2002 4:35 pm

Re: Kernel panic/PAX ?

Postby Virsh » Thu Sep 29, 2016 10:56 pm

PaX Team wrote:pass pax_size_overflow_report_only to the kernel and it won't panic then you can grab dmesg easily.

Ок.
http://pastebin.com/KALQTQ3K
Virsh
 
Posts: 8
Joined: Wed Sep 28, 2016 10:39 pm

Re: Kernel panic/PAX ?

Postby PaX Team » Fri Sep 30, 2016 4:28 am

thanks, the offending values are:
Code: Select all
[  307.843964] PAX: network_header:0 off:ffffffc4
[  307.843968] PAX: size overflow detected in function skb_headers_offset_update net/core/skbuff.c:1052 cicus.735_41 min, count: 22, decl: network_header; num: 0; context: sk_buff;
that is, the kernel is trying to reduce skb->network_header when it is already 0 which i believe is non-sensical and thus an upstream problem. as i already suggested in the gentoo bugzilla, you should report this to netdev@vger.kernel.org and work with upstream developers to figure out what's going on.
PaX Team
 
Posts: 2310
Joined: Mon Mar 18, 2002 4:35 pm

Re: Kernel panic/PAX ?

Postby Virsh » Fri Sep 30, 2016 5:55 am

I send mail, my mail - gmail. Delivery to the following recipient failed permanently.
Gmail banned vger.kernel.org ? =)
Virsh
 
Posts: 8
Joined: Wed Sep 28, 2016 10:39 pm

Re: Kernel panic/PAX ?

Postby PaX Team » Fri Sep 30, 2016 6:23 am

it's probably because kernel mailing lists don't accept html email, you'll have to send plain text (also CC emese/spender/me please).
PaX Team
 
Posts: 2310
Joined: Mon Mar 18, 2002 4:35 pm

Re: Kernel panic/PAX ?

Postby Virsh » Fri Sep 30, 2016 7:39 am

Ok. I send mail via mutt...
Virsh
 
Posts: 8
Joined: Wed Sep 28, 2016 10:39 pm

Re: Kernel panic/PAX ?

Postby Virsh » Thu Dec 08, 2016 12:43 pm

PaX Team wrote:it's probably because kernel mailing lists don't accept html email, you'll have to send plain text (also CC emese/spender/me please).

Hello ! Can you say anything has changed regarding this bug?
P.s. In my experience this occurs on routers Dlink.
Virsh
 
Posts: 8
Joined: Wed Sep 28, 2016 10:39 pm

Re: Kernel panic/PAX ?

Postby PaX Team » Sat Dec 10, 2016 6:56 am

i haven't heard back anything so i have still no idea what to do with this...
PaX Team
 
Posts: 2310
Joined: Mon Mar 18, 2002 4:35 pm


Return to grsecurity support