GRKERNSEC_EXECVE in 2.6.32 and newer

[jorgus]

Hi,

There was a useful feature in grsecurity 2.1.12 for 2.6.27 kernels. It was doing a great job with Apache and suexec. I cannot find it in grsecurity 2.9.1 for 2.6.32 & 3.2. Were there any technical obstacles that this feature was dropped?

After some digging I found out that the last patch including GRKERNSEC_EXECVE is grsecurity-2.2.2-2.6.32.43-201107132223.patch. It disappeared from grsecurity-2.2.2-2.6.32.44-201108111959.patch and has been gone since then

[spender]

Hi,

This was removed because it was merged upstream. See the relevant code in fs/exec.c:

Code: Select all

        /*
         * We move the actual failure in case of RLIMIT_NPROC excess from
         * set*uid() to execve() because too many poorly written programs
         * don't check setuid() return code.  Here we additionally recheck
         * whether NPROC limit is still exceeded.
         */
        if ((current->flags & PF_NPROC_EXCEEDED) &&
            atomic_read(&cred->user->processes) > rlimit(RLIMIT_NPROC)) {
                retval = -EAGAIN;
                goto out_ret;
        }


I backported this code to 2.6.32.60 as well. So you're not losing any functionality by not seeing the option, it's always active now.

-Brad

[jorgus]

Well, I don't think it works in 2.6.32.60 and 3.2.44 as it used to in 2.6.27.x with GRKERNSEC_EXECVE. Of course it's not grsecurity fault (though I did not bother to check it on vanilla kernel yet), but it may be worthwhile to revisit GRKERNSEC_EXECVE usefulness.

Check this out (tested on 3.2.44 with grsecurity-2.9.1-3.2.44-201304262227.patch and also confirmed on 2.6.32.60 with a pretty recent grsec patch):

Code: Select all

root@myhost:~# su - -s/bin/bash testuser
testuser@myhost:~$
testuser@myhost:~$ ulimit -u 1
testuser@myhost:~$ ls
-su: fork: retry: Resource temporarily unavailable
-su: fork: retry: Resource temporarily unavailable
-su: fork: retry: Resource temporarily unavailable
-su: fork: retry: Resource temporarily unavailable
-su: fork: Resource temporarily unavailable
testuser@myhost:~$


So far so good. So in another screen window on root account I do the same as above. The result is obviously identical. I leave the two su sessions open and I open a third screen window with root shell. I do:

Code: Select all

root@myhost:~# ps uxaw|egrep '^testuser'
testuser  15981  0.0  0.0   4448  1508 pts/10   S    21:10   0:00 su - -s/bin/bash testuser
testuser  15982  0.0  0.0   3068  1412 pts/10   S+   21:10   0:00 -su
testuser  16408  0.0  0.0   4452  1512 pts/9    S    21:10   0:00 su - -s/bin/bash testuser
testuser  16409  0.0  0.0   3068  1408 pts/9    S+   21:10   0:00 -su


The processes are there, no cheating. I go back to one of my testuser's shells and I type in:

Code: Select all

szjtest@myhost:~$ exec ls -ld .
drwxr-s--- 10 testuser testuser 4096 Mar  7 22:38 .
root@myhost:~#


The su session is over, I get back the root shell. The point is that with GRKERNSEC_EXECVE working the way I always relied on it the 'exec ls -ld' would not have been executed since the rlimits were checked in the execve call. That is how it did the great job for me in suexec under 2.6.27.60 - my improved suexec after changing the gid/uid set rlimits based on the environment variables passed. Then it called execv() to run the requested CGI script. If the new process violated the process count limit, the execv() would fail. In 3.2.44 users can proliferate as many CGI scripts as they like and the only limitation is that they will not be able to fork when above the limit (at least there the process limit is checked properly). It was simply not possible in 2.6.27.x with GRKERNSEC_EXECVE.

[UPDATE]
I found a forsaken 2.6.27.something with GRKERNSEC_EXECVE and confirmed, that it worked properly. You don't even need to launch 2 concurrent sessions as su seems to use up two processes for the user (as indicated in the ps output above).

Code: Select all

verylamehost:~# su - -s/bin/bash nobody
No directory, logging in with HOME=/
nobody@verylamehost:/$ ulimit -u 1
nobody@verylamehost:/$ ls
-su: fork: Resource temporarily unavailable
nobody@verylamehost:/$ exec ls
-su: /bin/ls: Resource temporarily unavailable
-su: /bin/ls: Success  [yeah, you wish!]
nobody@verylamehost:/$   [still nobody]


Since we noticed that su uses up two processes, we can make another test to be 100% sure there is no flaws in our thinking:

Code: Select all

verylamehost:~# su - -s/bin/bash nobody
No directory, logging in with HOME=/
nobody@verylamehost:/$ ulimit -u 2  [higher limit this time]
nobody@verylamehost:/$ ls
-su: fork: Resource temporarily unavailable
nobody@verylamehost:/$ exec ls   [ should work since we'll stay in the 2 process limit ]
bin  boot  dev  etc  home  lib  lost+found  mnt  opt  proc  root  sbin  srv  sys  tmp  usr  var
verylamehost:~#   [ nobody's shell is gone, we're back to root's shell ]


[spender]

Hi jorgus,

Thanks for that report. I'll investigate this and if I can confirm your findings will try to resolve this problem in the upstream code. What I didn't mention in the previous post was that in addition to the feature being upstreamed, it was changed (in a good way, modulo this potential problem) to focus more on the larger problem of setuid binaries failing to check for setuid() failure, with the expected serious results. So the limit check was offloaded from setuid() and moved to exec time (subject to that PF_NPROC_EXCEEDED flag).

-Brad

[spender]

Hi Jorgus,

Can you please apply the following patch:
http://grsecurity.net/~spender/nproc_fix.diff

I believe it will resolve your issue as described in the comment of the patch.

-Brad

[jorgus]

The patch seems to fix the problem. Excellent work, thanks a lot Are you planning to include it in the grsecurity patches as such or possibly reintroduce GRKERNSEC_EXECVE? I would enable it in all my kernels anyway, but can imagine it causing some regression for people relying on the default behaviour. It seems that this never worked for rlimit_nproc in vanilla kernels the way it did with GRKERNSEC_EXECVE.

[spender]

I think this edge case wasn't even considered by upstream and I doubt the fix will cause any regressions. I'll be including it in the next patches -- thanks for the report and testing!

-Brad

[spender]

Hi jorgus,

I'm following up on this as my fix has caused some compatibility problems with certain users. Could you try the following compatibility fix and confirm for me that your previous tests still operate the same as after my last fix?

http://grsecurity.net/~spender/nproc2.diff

Thanks,
-Brad

[jorgus]

Hi,

It still works fine for me with this patch. The obvious su+ls test still behaves as expected and CGI processes cannot be spawned above the nproc limit by suexec. Thanks!

[zImage]

Hm, I can't get apache+suexec to work as expected when combined with this patch. The program flow is as follows:

Code: Select all

step syscall     uid     comment
-1   fork()      root    apache master process spawns a connection handling child
 0   setuid()    apache  privileges revoked (child has open fds etc.)
 1   fork()      apache  apache child prepares to serve a cgi by spawning another child (to be replaced w/ exec)
 2   setrlimit() apache  the new cgi child sets rlimit_nproc in accordance with httpd.conf for this suexec user's vhost
 3   execve()    root    the cgi child is replaced by the suexec binary (which is suid root)
 4   setuid()    user    suexec changes its uid in accordance with this user's vhost SuexecUserGroup directive
 5   execve()    user    actual cgi binary is called (e.g. php-cgi)


With default kernel behaviour we are getting failure at step 5, because PF_NPROC_EXCEEDED flag is raised at step 4.

With the patch described in this forum thread we get failure at step 3, probably because PF_NPROC_EXCEEDED flag is raised at step 2. This is problematic, because at step 3 we are still not running as the final web user and limit is not enforced against this web user's process count.

Here's a strace excerpt:

Code: Select all

[pid  7523] clone(Process 7546 attached
...
[pid  7546] chdir("/home/user/www/") = 0
[pid  7546] setrlimit(RLIMIT_CPU, {rlim_cur=60, rlim_max=60}) = 0
[pid  7546] setrlimit(RLIMIT_NPROC, {rlim_cur=20, rlim_max=20}) = 0
[pid  7546] setrlimit(RLIMIT_AS, {rlim_cur=128000000, rlim_max=128000000}) = 0
[pid  7546] execve("suexec", ["suexec", "1110", "2099", "sleep.php"], [/* 26 vars */]) = -1 EAGAIN (Resource temporarily unavailable)
...


Our current solution is to just revert the patch.

[jorgus]

Hi,

I think the rlimits you set in Apache apply to the apache user. The usecase I was describing involved a modified suexec setting rlimits after setgid() and setuid() calls. You will not be able to reproduce that behaviour using a stock suexec.

[zImage]

Indeed, RLIMIT_NPROC apache directive is applied (setrlimit()) initially for the apache user, but then uid is changed through suexec, and the resource limits are inherited by the process images that's executed later: apache(setrlimit)->suexec(setuid)->php. The problem is, the change described in this thread brakes this standard apache resource limiting mechanism. One has to remove RLIMIT_NPROC from httpd.conf and then use custom coded suexec to apply resource limits from within.