Kernel panic with 2.6.24 under Xen, take two

Discuss and suggest new grsecurity features

Moderators: spender, PaX Team

Re: Kernel panic with 2.6.24 under Xen, take two

Postby cormander » Mon Mar 10, 2008 7:20 pm

Doing an interdiff on grsecurity-2.1.11-2.6.24.3-200803031942.patch and pax-linux-2.6.24.3-test33.patch

Most of it works, lots of offsets, but a few failures:

Code: Select all
Hunk #14 FAILED at 528.
1 out of 15 hunks FAILED -- saving rejects to file arch/x86/mm/fault_32.c.rej
Hunk #1 FAILED at 51.
Hunk #11 FAILED at 1524.
2 out of 11 hunks FAILED -- saving rejects to file fs/exec.c.rej
Hunk #2 FAILED at 155.
1 out of 3 hunks FAILED -- saving rejects to file fs/proc/task_mmu.c.rej
Hunk #1 FAILED at 21.
Hunk #6 FAILED at 211.
2 out of 11 hunks FAILED -- saving rejects to file mm/mprotect.c.rej


Similar failures for test 31 and 32. Different hunk #s but for the same files. Which exact PAX patch went into grsecurity-2.1.11-2.6.24.3-200803031942.patch ? That way I'll have to do less digging trying to undo those failed hunks.

Thanks for your help so far!
cormander
 
Posts: 154
Joined: Tue Jan 29, 2008 12:51 pm
Location: Utah

Re: Kernel panic with 2.6.24 under Xen, take two

Postby PaX Team » Tue Mar 11, 2008 5:13 am

cormander wrote:Similar failures for test 31 and 32. Different hunk #s but for the same files. Which exact PAX patch went into grsecurity-2.1.11-2.6.24.3-200803031942.patch ? That way I'll have to do less digging trying to undo those failed hunks.
normally it's the last PaX test patch, so it should have been test33 and indeed, they interdiff cleanly for me.
PaX Team
 
Posts: 2310
Joined: Mon Mar 18, 2002 4:35 pm

Re: Kernel panic with 2.6.24 under Xen, take two

Postby xstasi » Tue Mar 11, 2008 7:05 am

....another update!

I found the file that messes everything up!

Here's what i've done:

I took a vanilla 2.6.24.3, unpacked it, patched it with pax
Then i patched -R (thanks splitdiff) only the file arch/x86/kernel/vmlinux_32.lds.S, and adjusted a few lines here and there to make it compile.

Obviously, the kernel doesn't run, but... it crashes on another part of the code

I launched gdb on it, inspected the memory.. and yes, the .data segment is there! kallsyms is full of nifty strings and data :D

So, the conclusion is that the file arch/x86/kernel/vmlinux_32.lds.S is the one who's messing with the segments, and making the kernel resulting full of zeroes under xen.

Obeying Murphy's laws, lds.S code is the only code type i've never read in a kernel, so it will take more than a long time for me to find what's offending there.

By the way, i thought it would have been a good idea to report my discover

Maybe pageexec, who's surely more experienced than me, will find a quicker solution ;)

Cheers...

xstasi
xstasi
 
Posts: 13
Joined: Tue Feb 19, 2008 10:09 am

Re: Kernel panic with 2.6.24 under Xen, take two

Postby PaX Team » Tue Mar 11, 2008 9:17 am

xstasi wrote:I took a vanilla 2.6.24.3, unpacked it, patched it with pax
Then i patched -R (thanks splitdiff) only the file arch/x86/kernel/vmlinux_32.lds.S, and adjusted a few lines here and there to make it compile.

Obviously, the kernel doesn't run, but... it crashes on another part of the code
vmlinux_32.lds.S is the linker script that is used to produce vmlinux, you can't arbitrarily patch it out as it contains crucial changes related to certain PaX features (some outside of config options, say direct PAE boot without the temporary 2-level page tables), no wonder it didn't work ;-). it also means that whatever xen doesn't like in the ELF produced by this linker script may turn out to be something to be taken care of in xen itself, not PaX. as i said before, it's time to ask the xen devs about how their ELF loader works, what it expects, etc then we can figure out a solution.
Obeying Murphy's laws, lds.S code is the only code type i've never read in a kernel, so it will take more than a long time for me to find what's offending there.
check binutils/ld info pages about linker scripts, they explain the syntax and everything else you need to know about them. you can also cut your teeth on the default linker scripts used to link userland executables, they're installed by binutils somewhere under /usr/lib/binutils/i686-pc-linux-gnu/2.18/ldscripts/ or similar.
PaX Team
 
Posts: 2310
Joined: Mon Mar 18, 2002 4:35 pm

Re: Kernel panic with 2.6.24 under Xen, take two

Postby xstasi » Tue Mar 11, 2008 10:33 am

PaX Team wrote:vmlinux_32.lds.S is the linker script that is used to produce vmlinux, you can't arbitrarily patch it out as it contains crucial changes related to certain PaX features (some outside of config options, say direct PAE boot without the temporary 2-level page tables), no wonder it didn't work ;-).

I didn't expect it to work... I did that test just to make sure that the .lsd.S was the file to blame ;)

PaX Team wrote:it also means that whatever xen doesn't like in the ELF produced by this linker script may turn out to be something to be taken care of in xen itself, not PaX. as i said before, it's time to ask the xen devs about how their ELF loader works, what it expects, etc then we can figure out a solution.

Now I fully agree with you, and i've done a bug report, check it here:
http://bugzilla.xensource.com/bugzilla/ ... gi?id=1189

I will keep you informed!

Cheers
xstasi
 
Posts: 13
Joined: Tue Feb 19, 2008 10:09 am

Re: Kernel panic with 2.6.24 under Xen, take two

Postby cormander » Tue Mar 11, 2008 11:50 am

If anyone is interested, I took the grsecurity patch posted today and removed pax from it, and posted it here:

http://www.ravencore.com/grsec/

I have used the patch utility for years, but admittedly never used interdiff or otherwise. This diff was generated with the following steps:

Code: Select all
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.24.3.tar.gz
wget http://grsecurity.net/test/grsecurity-2.1.11-2.6.24.3-200803110829.patch
wget http://grsecurity.net/test/pax-linux-2.6.24.3-test34.patch
tar zxf linux-2.6.24.3.tar.gz
mv linux-2.6.24.3 linux-2.6.24.3-pax
patch -p0 -i pax-linux-2.6.24.3-test34.patch
tar zxf linux-2.6.24.3.tar.gz
patch -p0 -i grsecurity-2.1.11-2.6.24.3-200803110829.patch
mv linux-2.6.24.3 linux-2.6.24.3-grsec
diff -ruN linux-2.6.24.3-pax linux-2.6.24.3-grsec > temp.diff
tar zxf linux-2.6.24.3.tar.gz
cd linux-2.6.24.3
patch -N -p1 -i ../temp.diff
cd ..
mv linux-2.6.24.3 linux-2.6.24.3-grsec-nopax


Then I resolved the hunk failures in the following files (not a whole lot of changes):

Code: Select all
linux-2.6.24.3-grsec-nopax/arch/x86/Kconfig
linux-2.6.24.3-grsec-nopax/arch/x86/mm/fault_32.c
linux-2.6.24.3-grsec-nopax/fs/exec.c
linux-2.6.24.3-grsec-nopax/fs/proc/task_mmu.c
linux-2.6.24.3-grsec-nopax/kernel/sysctl.c
linux-2.6.24.3-grsec-nopax/mm/mprotect.c
linux-2.6.24.3-grsec-nopax/security/Kconfig


And then finally a diff on the vanilla and grsec-nopax directories

The vmlinuz image from this kernel boots under xen. I'm in the process of testing the TPE, RBAC, etc features :)
Last edited by cormander on Thu Mar 27, 2008 5:16 pm, edited 1 time in total.
cormander
 
Posts: 154
Joined: Tue Jan 29, 2008 12:51 pm
Location: Utah

Re: Kernel panic with 2.6.24 under Xen, take two

Postby ThomasR » Mon Mar 24, 2008 5:25 pm

Hi Cormander,

did you make some progress on testing the grsec-nopax patch? I would be very interested in getting grsec to run with Xen as well.

Btw: Does anyone have experience with running unmodified linux guests under Xen (in terms of stability and performance)?

Best regards,

Thomas
ThomasR
 
Posts: 2
Joined: Thu Mar 20, 2008 5:56 pm

Re: Kernel panic with 2.6.24 under Xen, take two

Postby cormander » Tue Mar 25, 2008 12:36 am

Hello,

The grsec-nopax works great for me. No problems since I started using it (minus one bug report I filed, but it wasn't related to it being a xen domU). I also updated it to the March 23rd test patch this morning:

http://www.ravencore.com/grsec/

Though there is now a March 24th patch. Hard to keep up with the latest tests :)

What I do to save time on building new kernels for my xen domU images is change the .config file to have the network, filesystem (ext3 & reiserfs for me), and netfilter drivers to be built in, and do just 'make vmlinux'. It builds in around 10 minutes (whereas compiling all modules takes several hours).

As far as running unmodified kernels as paravirtulized guests... haven't done it beyond running it for a few days while I compared to to the grsecurity (with pax) on xen domU. Seems stable enough. Hell, if the kernel works just fine with the grsec-nopax patch, can't imagine there would be any problems with no patch at all.

Performance wise, if the other domU guests and the dom0 host OS are idle; running benchmark tests on one of the domUs, it almost feels like a real machine. They really did a good job with the paravirtualized interface for system CPU speed and I/O .

Stability wise, its pretty solid. I don't think redhat or novell would support it the way they do if it wasn't. The main stability factor for xen is what verson of the hypervisor you're running; I have 3.0.2 on my dom0 host OS, and it's rock solid. Only problem with it is you can't unmount xen images you've manually mounted from the dom0 under xen's mapper driver... which isn't a big issue anyway. From what I read 3.0.2 doesn't have suspend / resume support, but for me that's not a big deal. I compiled 3.2.1 xen hypervisor and over the weekend the host running it went offline.. haven't went to the datacenter to figure out why yet. So the latest version could be a mite unstable. I haven't tried 3.1
Last edited by cormander on Thu Mar 27, 2008 5:17 pm, edited 1 time in total.
cormander
 
Posts: 154
Joined: Tue Jan 29, 2008 12:51 pm
Location: Utah

Re: Kernel panic with 2.6.24 under Xen, take two

Postby ThomasR » Tue Mar 25, 2008 7:00 am

Hi,

thanks for the quick answer! I'll compile a test kernel myself this week.

cormander wrote:As far as running unmodified kernels as paravirtulized guests... haven't done it beyond running it for a few days while I compared to to the grsecurity (with pax) on xen domU. Seems stable enough. Hell, if the kernel works just fine with the grsec-nopax patch, can't imagine there would be any problems with no patch at all.


What I actually meant is running a kernel without the DomU patch using Xen's hardware virtualization capabilities (if your hardware has VT or Pacifica support). If that would work nearly as well as using a paravirtualized guest, it should be possible to use the regular grsec+pax patch.

Best regards,

Thomas
ThomasR
 
Posts: 2
Joined: Thu Mar 20, 2008 5:56 pm

Re: Kernel panic with 2.6.24 under Xen, take two

Postby cormander » Tue Mar 25, 2008 3:29 pm

ThomasR wrote:What I actually meant is running a kernel without the DomU patch using Xen's hardware virtualization capabilities (if your hardware has VT or Pacifica support). If that would work nearly as well as using a paravirtualized guest, it should be possible to use the regular grsec+pax patch.


If I ever get any hardware that supports full virtualization (and I just might here in a few months), I'll be sure to run some benchmarks.

BTW I just posted another nopax patch (off of today's release):

http://www.ravencore.com/grsec/
cormander
 
Posts: 154
Joined: Tue Jan 29, 2008 12:51 pm
Location: Utah

Re: Kernel panic with 2.6.24 under Xen, take two

Postby ralph » Wed Jun 04, 2008 4:12 pm

im fiddling with this myself and cant get my kernels to boot as soon as i add grsec support to it.
these patches of yours, would you be able to put these out in .patch format, not rpms?
im using debian etch with my own 3.2.1 hypervisor and want to run my own customized 2.6.2x.x with grsec for security and flexibility.
ralph
 
Posts: 2
Joined: Wed Jun 04, 2008 4:09 pm

Re: Kernel panic with 2.6.24 under Xen, take two

Postby cormander » Wed Jun 04, 2008 4:36 pm

You can get the src.rpm file and use rpm2cpio to extract the patches out of it; it contains not only the nopax patch, but pax updates to the grsecurity patch (if you ever use it on base hardware) and CVE patches applicable to 2.6.24.7. I also have about 300 backports in there - and the next one I'm going to post in the next week or so has an upward of 1400 patches from the upstream kernel that apply and build cleanly.
cormander
 
Posts: 154
Joined: Tue Jan 29, 2008 12:51 pm
Location: Utah

Re: Kernel panic with 2.6.24 under Xen, take two

Postby cormander » Wed Jun 04, 2008 5:02 pm

Actually I think I'll just post my git repository that I'm using for revision control, and it has all the patches in there.

I'll update my signature with the URL for it when that's done.
cormander
 
Posts: 154
Joined: Tue Jan 29, 2008 12:51 pm
Location: Utah

Re: Kernel panic with 2.6.24 under Xen, take two

Postby ralph » Thu Jun 05, 2008 6:44 am

Thanks alot.

used alien to convert kernel-grsec-2.6.25.4-200806012125.src.rpm to .tgz and then patched my own kernel tree.
Just compiled and it worked at once.

Good work! :)
ralph
 
Posts: 2
Joined: Wed Jun 04, 2008 4:09 pm

Re: Kernel panic with 2.6.24 under Xen, take two

Postby spender » Sun May 02, 2010 8:48 am

As of the patches uploaded last night, 32bit xen domU should be compatible with grsecurity/PaX. Please let us know if you continue to have any issues.

-Brad
spender
 
Posts: 2185
Joined: Wed Feb 20, 2002 8:00 pm
Location: VA, USA

PreviousNext

Return to grsecurity development

cron