Page 1 of 1

MMAP failure after patching 3.8.2 w/ 2.9.1 test

PostPosted: Fri Mar 08, 2013 4:10 pm
by browndav
Hi everyone,

I've run in to odd behavior with the V8 javascript engine (in Node.js) when running on Linux 3.8.2 with the grsecurity patch applied, but not activated. That is, I've patched the source code, but have ensured that the various grsecurity and pax options are disabled. The grsecurity patch I'm using on Linux 3.8.2 is here: http://grsecurity.net/test/grsecurity-2 ... 2205.patch

I've tracked the issue down to a (reproducible) failed mmap operation. These are the syscalls being made by Node.js (each line is the same call, taken from a separate run of the 'node' executable on an empty script file):

Code: Select all
$ ./strace node empty.js 2>&1 | tee bad5 | grep mmap2 | grep 3355 # Run for each line
mmap2(0x900de000, 33554432, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x900de000
mmap2(0xb1f4a000, 33554432, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xb1f4a000
mmap2(0xad649000, 33554432, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xad649000
mmap2(0x9c101000, 33554432, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x9c101000
mmap2(0xa4884000, 33554432, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xa4884000
mmap2(0xa5bcf000, 33554432, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0xa5bcf000
mmap2(0xbefc5000, 33554432, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = -1 EINVAL (Invalid argument)


The final call to mmap (with address hint 0xbefc5000) fails, which causes the v8 engine and node.js to abort.

This failure is actually reproducible. Here's a small test program that reproduces the issue 100% of the time on grsecurity-patched kernels (regardless of whether or not grsec/pax features are enabled in the .config), and 0% of the time on stock 3.8.2 kernels:

Code: Select all
#include <sys/mman.h>
#include <stdio.h>

int main()
{
  void *x = mmap(
    (void *) 0xbffce000, 33554432,
      PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0
  );

  if (x == MAP_FAILED) {
    printf("failure; rv: %x\n", x);
    return 1;
  }

  return 0;
}


I've been able to work around it by telling V8/Node.js to not ask for addresses above 0xb0000000 or so.

From reading the mmap manual page and the V8 source code comments, it appears that -- since MAP_FIXED wasn't provided -- the kernel should be able to fulfill this request at a different virtual address if there's something already in the way.

I've made sure that ulimit is unlimited across-the-board, and that all of the grsecurity features are turned off (both via the kernel configuration, and via paxctl).

Is there something I'm missing here? Is this a bug? Is V8 / Node.js making assumptions about memory layout that aren't valid?

Here's the Node.js patch that caused this issue to surface, FWIW: https://github.com/joyent/node/commit/7 ... 7b8f3aac3f

Re: MMAP failure after patching 3.8.2 w/ 2.9.1 test

PostPosted: Sat Mar 09, 2013 12:27 pm
by PaX Team
the requested region is invalid, its end address is 0xc0fc5000 which is above the 32 bit TASK_SIZE. looks like i should take MAP_FIXED into account in i386_mmap_check, i'll fix it in the next patch.

Re: MMAP failure after patching 3.8.2 w/ 2.9.1 test

PostPosted: Sun Mar 10, 2013 2:11 pm
by browndav
Thanks so much for the quick reply!

Just in case it's of use to anyone, below is a workaround (warning: tested only by me, and not thoroughly) that seems to allow node.js versions after v0.8.10 or so to run reliably on grsec-patched kernels. This can probably go away once the proper fix is in (i.e. i386_mmap_check takes MAP_FIXED into account):

Code: Select all
diff -urN node-v0.8.22-prev/deps/v8/src/platform-posix.cc node-v0.8.22/deps/v8/src/platform-posix.cc
--- node-v0.8.22-prev/deps/v8/src/platform-posix.cc     2013-03-08 03:03:51.111974790 -0800
+++ node-v0.8.22/deps/v8/src/platform-posix.cc  2013-03-08 03:07:20.917969646 -0800
@@ -121,8 +121,13 @@
     // no hint at all (Linux, Solaris, illumos and derivatives). The high hint
     // prevents the break from getting hemmed in at low values, ceding half of
     // the address space to the system heap.
+
+    // The range 0x20000000 - 0x60000000 is relatively unpopulated across a
+    // variety of ASLR modes (PAE kernel, NX compat mode, etc) and on MacOS X
+    // 10.6 and 10.7.
+
     raw_addr &= 0x3ffff000;
-    raw_addr += 0x80000000;
+    raw_addr += 0x60000000; /* Was: 0x80000000 */
 #endif
     return reinterpret_cast<void*>(raw_addr);
   }

Re: MMAP failure after patching 3.8.2 w/ 2.9.1 test

PostPosted: Sun Mar 10, 2013 6:11 pm
by spender
Hi,

This is now fixed in the latest patches. Let us know if you still have any problems.

-Brad