JVM segfaults with grsecurity-2.1.14-2.6.30.4 (latest patch)

[Grach]

The JVM (icedtea6-1.5 or icedtea6-bin-1.4, Hardened Gentoo) always does segmentation fault within a second after a start on kernels with grsecurity-2.1.14-2.6.30.4 (including latest 200908051916 patch). Just start any JVM application to reproduce. There are no problems with grsecurity-2.1.14-2.6.30.6 and plain linux-2.6.30.4. What should I do to help to fix it, and what details do you need?

[PaX Team]

What should I do to help to fix it, and what details do you need?

if it's a segfault then you should get a coredump and post the usual info (backtrace, registers, etc). also what about sun's JVM?

[specs]

Same here. openjdk-6-jre.
Application below is cgoban from http://www.gokgs.com/ (standard application on that site).
Java starts up, I get an opening screen, I choose "login" and after logging in a third screen comes up which crashes.

When logging in as guest it seem to work.

Code: Select all

$ /usr/lib/jvm/java-6/jre/bin/javaws /usr/games/cgoban.jnlp
Segmentation fault


From dmesg:

Code: Select all

grsec: signal 11 sent to /usr/lib/jvm/java-6-openjdk/jre/bin/javaws.real[javaws.
real:4081] uid/euid:1000/1000 gid/egid:1000/1000, parent /bin/bash[bash:4057] ui
d/euid:1000/1000 gid/egid:1000/1000
grsec: signal 11 sent to /usr/lib/jvm/java-6-openjdk/jre/bin/javaws.real[javaws.
real:4081] uid/euid:1000/1000 gid/egid:1000/1000, parent /bin/bash[bash:4057] ui
d/euid:1000/1000 gid/egid:1000/1000
grsec: denied resource overstep by requesting 4096 for RLIMIT_CORE against limit
 0 for /usr/lib/jvm/java-6-openjdk/jre/bin/javaws.real[javaws.real:4081] uid/eui
d:1000/1000 gid/egid:1000/1000, parent /bin/bash[bash:4057] uid/euid:1000/1000 g
id/egid:1000/1000


Code: Select all

# paxctl -v /usr/lib/jvm/java-6/jre/bin/ja*
PaX control v0.5
Copyright 2004,2005,2006,2007 PaX Team <pageexec@freemail.hu>

- PaX flags: -p-s-m-x-e-- [/usr/lib/jvm/java-6/jre/bin/java]
        PAGEEXEC is disabled
        SEGMEXEC is disabled
        MPROTECT is disabled
        RANDEXEC is disabled
        EMUTRAMP is disabled
file /usr/lib/jvm/java-6/jre/bin/javaws is not a valid ELF executable
- PaX flags: -p---m-x-e-- [/usr/lib/jvm/java-6/jre/bin/javaws.real]
        PAGEEXEC is disabled
        MPROTECT is disabled
        RANDEXEC is disabled
        EMUTRAMP is disabled


I tried even strace -f -e trace=open,old_mmap,mmap2,munmap,mprotect,mremap /usr/lib/jvm/java-6/jre/bin/javaws /usr/games/cgoban.jnlp
http://www.aoi-karin.net/grsec/cgoban.txt

Sunjava on another pc yields the same problem.
2.6.29.6-grsec-200907231934 works good, 2.6.30.4-grsec-200908041752 does not.

[fabled]

Looks like it's the same bug as we have:
http://grsecurity.net/pipermail/grsecur ... 00968.html

You can verify this by adding rt_sigaction to set of monitored syscalls in strace and checking if rt_sigaction is called with SA_SIGINFO flag set.
The application is using mprotect with PROT_NONE, and getting two SIGSEGV's so it looks like the same issue: the first SIGSEGV is intentional, and application wants to handle it; but kernel is broke by not being able to send signal with SA_SIGINFO, and thus sends second SIGSEGV to application. The problem seems to be in the signal frame generation code (fault happens after properly dispatching the first SIGSEGV to strace, but before control is returned to application signal handler).

[PaX Team]

The problem seems to be in the signal frame generation code (fault happens after properly dispatching the first SIGSEGV to strace, but before control is returned to application signal handler).

it's fixed in the latest test patch, thanks for the help .

[spender]

I've uploaded a new patch as well that includes all the PaX fixes up to test21 (the latest)

-Brad

[specs]

I just tested the 200908051916 with interdiff between test17 and test21.
It works.

[Grach]

Works fot me too. Thanks to everyone involved. I apologize for not giving any useful information. Backtrace and registers of JIT-compiled code was worthles, so I was stuck in trying to read the assembly.