VMMIRROR bug?

[kamihacker]

Feb 29 22:02:01 secure su: PAM unable to lopen(/lib/security/pam_xauth.so)
Feb 29 22:02:01 secure su: PAM [dlerror: /lib/security/pam_xauth.so: cannot open shared object file: No such file or directory]

I got this errors although the files is in place (it's nor memory neither hard disk problems since I already checked that)

afterwards in the logs:

Mar 1 01:10:22 secure kernel: kernel BUG at <bad filename>:103!
Mar 1 01:10:22 secure kernel: invalid operand: 0000
Mar 1 01:10:22 secure kernel: CPU: 0
Mar 1 01:10:22 secure kernel: EIP: 0010:[<0002c7ba>] Not tainted
Mar 1 01:10:22 secure kernel: EFLAGS: 00010286
Mar 1 01:10:22 secure kernel: eax: 00000000 ebx: c1c1e320 ecx: c1c1e320 edx: 00000000
Mar 1 01:10:22 secure kernel: esi: c1c1e320 edi: 00000000 ebp: f5636400 esp: cc3a7e3c
Mar 1 01:10:22 secure kernel: ds: 0018 es: 0018 ss: 0018
Mar 1 01:10:22 secure kernel: Process ipop3d (pid: 19777, stackpage=cc3a7000)
Mar 1 01:10:22 secure kernel: Stack: c2214300 001399d9 e0a34a00 00014211 0000001f 00000000 00000001 00000400
Mar 1 01:10:22 secure kernel: c1c1e320 f655a434 00000000 0002546f cc3a7ec8 c1c1e320 00000000 00001000
Mar 1 01:10:22 secure kernel: 00001000 00000001 00000000 00000400 f655a380 000000d2 f655a580 00022c97
Mar 1 01:10:22 secure kernel: Call Trace: [<001399d9>] [<00014211>] [<00000400>] [<0002546f>] [<00001000>]
Mar 1 01:10:22 secure kernel: [<00001000>] [<00000400>] [<00022c97>] [<00025a98>] [<00025940>] [<00000400>]
Mar 1 01:10:22 secure kernel: [<00000400>] [<00033a26>] [<00000400>] [<000331bd>] [<00003ad3>] [<00000400>]

and then:

Mar 1 05:56:25 secure sshd(pam_unix)[18202]: session opened for user root by (uid=0)
Mar 1 05:56:46 secure kernel: PAX: VMMIRROR: fault bug2, 84c01000, 84c01000, 24c00000, b4b7b000, 54b7c000
Mar 1 06:00:00 secure logger: weblogs: (5987) starting.
Mar 1 06:00:07 secure logger: weblogs: (5987) done.


I got this, plus weird behaviors with httpd like

[Sun Feb 29 23:31:33 2004] [alert] Child 5024 returned a Fatal error... !Apache is exiting!

how can I verify if this is because of kernel problems, grsecurity or glibc not being compatible with kernel/grsecurity? Distribution is RedHat 7.3 and this are the libc components on the box

glibc-kernheaders-2.4-7.16
glibc-2.2.5-43
glibc-2.2.5-44
glibc-common-2.2.5-44
glibc-devel-2.2.5-44

any clues? I was thinking of installing the 2.4.25 kernel unpatched to see if it still failed, but I'd like to know what's going and what can I do.

apparently it's related to this function on mm/memory.c

I disabled this two features to see how it goes
CONFIG_GRKERNSEC_PAX_SEGMEXEC
CONFIG_GRKERNSEC_PAX_RANDEXEC

thanks in advance for any info that you could provide

[PaX Team]

Mar 1 05:56:46 secure kernel: PAX: VMMIRROR: fault bug2, 84c01000, 84c01000, 24c00000, b4b7b000, 54b7c000

how can I verify if this is because of kernel problems, grsecurity or glibc not being compatible with kernel/grsecurity?

this message above comes from PaX and something looks quite screwed up - this process has created a mapping (file or anonymous) of some 800MB... that doesn't look normal to me. in any case, this consistency check points to some error, you can help me find out more. first you should try out the latest PaX patch alone with your .config and see if you can still reproduce the problem. then i'll need the PaX specific options from your .config and also an strace log of the process(es) that trigger this bug (feel free to send them in email, they're probably too big to post here).

[kamihacker]

now I'm getting this on two P4 machines with 1 GB or over

Mar 3 04:10:04 secure su: PAM adding faulty module: /lib/security/pam_xauth.so
Mar 3 04:10:04 secure su: PAM unable to dlopen(/lib/security/pam_xauth.so)
Mar 3 04:10:04 secure su: PAM [dlerror: /lib/security/pam_xauth.so: cannot open shared object file: No such file or directory]

the files are there and I have already issued badblocks and memory checks on both of them

please send me an e-mail address to which I could send you the kernel configuration and all other data you're requesting

*edit*

just after I posted for the first time I got this on the second server:

Mar 3 04:30:34 secure kernel: CPU: 0
Mar 3 04:30:34 secure kernel: EIP: 0010:[<0002c7d9>] Not tainted
Mar 3 04:30:34 secure kernel: EFLAGS: 00010282
Mar 3 04:30:34 secure kernel: eax: 00000000 ebx: c1b61ec0 ecx: c1b61ec0 edx: 00000000
Mar 3 04:30:34 secure kernel: esi: f6b39a54 edi: 00000000 ebp: 00000000 esp: e6c5be7c
Mar 3 04:30:34 secure kernel: ds: 0018 es: 0018 ss: 0018
Mar 3 04:30:34 secure kernel: Process ipop3d (pid: 29858, stackpage=e6c5b000)
Mar 3 04:30:34 secure kernel: Stack: f6b399a0 00000001 0003e878 f6b399a0 0000001f 00000000 00000001 00000400
Mar 3 04:30:34 secure kernel: c1b61ec0 f6b39a54 00000000 0002547f e6c5bf08 c1b61ec0 00000000 00001000
Mar 3 04:30:34 secure kernel: 00001000 00000001 00000000 00000400 f6b399a0 00000000 f62b00e0 ffffffea
Mar 3 04:30:34 secure kernel: Call Trace: [<0003e878>] [<00000400>] [<0002547f>] [<00001000>] [<00001000>]
Mar 3 04:30:34 secure kernel: [<00000400>] [<00025aa8>] [<00025950>] [<00000400>] [<00000400>] [<00033a36>]
Mar 3 04:30:34 secure kernel: [<00000400>] [<000331cd>] [<00000282>] [<00033527>] [<00003ae3>] [<00000400>]
Mar 3 04:30:34 secure kernel:
Mar 3 04:30:34 secure kernel: Code: 0f 0b 69 00 8b 2c 56 c0 8b 0d f0 54 15 c0 89 d8 29 c8 69 c0
Mar 3 04:30:34 secure kernel: kernel BUG at <bad filename>:105!
Mar 3 04:30:34 secure kernel: invalid operand: 0000
Mar 3 04:30:34 secure kernel: CPU: 0
Mar 3 04:30:34 secure kernel: EIP: 0010:[<0002c7d9>] Not tainted
Mar 3 04:30:34 secure kernel: EFLAGS: 00010282
Mar 3 04:30:34 secure kernel: eax: 00000000 ebx: c1b61ec0 ecx: c1b61ec0 edx: 00000000
Mar 3 04:30:34 secure kernel: esi: f6b39a54 edi: 00000000 ebp: 00000000 esp: e6c5fe8c
Mar 3 04:30:34 secure kernel: ds: 0018 es: 0018 ss: 0018
Mar 3 04:30:34 secure kernel: Process ipop3d (pid: 21582, stackpage=e6c5f000)
Mar 3 04:30:34 secure kernel: Stack: e6c5fee8 e0a85be0 000256a1 e0a85be0 0000001f 00000000 00000001 00000400
Mar 3 04:30:34 secure kernel: c1b61ec0 f6b39a54 00000000 0002547f e6c5ff18 c1b61ec0 00000000 00001000
Mar 3 04:30:34 secure kernel: 00001000 00000001 00000000 00000400 f6b399a0 f6207f20 000332c6 f6b399a0
Mar 3 04:30:34 secure kernel: Call Trace: [<000256a1>] [<00000400>] [<0002547f>] [<00001000>] [<00001000>]
Mar 3 04:30:34 secure kernel: [<00000400>] [<000332c6>] [<00025aa8>] [<00025950>] [<00000400>] [<00000400>]
Mar 3 04:30:34 secure kernel: [<00033a36>] [<00000400>] [<00003afd>] [<00003ae3>] [<00000400>] [<00000246>]
Mar 3 04:30:34 secure kernel:
Mar 3 04:30:34 secure kernel: Code: 0f 0b 69 00 8b 2c 56 c0 8b 0d f0 54 15 c0 89 d8 29 c8 69 c0

I have to say this is running on a Ensim webhosting server, so the ipop3d is most probably issuing a chroot operation in order to get the mail for the user. It's very odd since I've been working this way on my machine (Cel 1.3 GHz 512 MB) and it has never failed, but now working on P4's with 1GB or over this is happening on a daily basis

*edit*

regards

[PaX Team]

please send me an e-mail address to which I could send you the kernel configuration and all other data you're requesting

pageexec at freemail.hu (it's listed on the PaX homepage as well).

It's very odd since I've been working this way on my machine (Cel 1.3 GHz 512 MB) and it has never failed, but now working on P4's with 1GB or over this is happening on a daily basis

i see you have KERNEXEC enabled, could you try to disable it and see if that fixes the problem? also a week ago or so i fixed some potential issues in the KERNEXEC code, it didn't make it into the last grsec release but you can get a diff contributed by Peter Mazinger from http://lists.virus.org/grsec-0403/msg00004.html or you can try the plain PaX patch itself. either way feedback is welcome (and i'd still like to get your .confg, System.map and that strace log where the VMMIRROR check is triggered).