Page 1 of 1

Help needed: strange RLIMIT_NPROC overstep

PostPosted: Sat Sep 13, 2003 11:40 pm
by erce
Hi everybody.
I am using grsecurity for quite some time. However, recently I have a problem with setting up user limits (via pam_limits.so) and exim (but I think that any other process would do the same). I tried to figure out where the problem is but so far have no clue. I am running kernel 2.4.21 with respective grsecurity patch, Debian woody stable.

I set a nproc limit (32) for a specific group. When a user from that group tries to do "echo test | mail -s test root" I get:
grsec: attempted resource overstep by requesting X for RLIMIT_NPROC against limit 32 by (exim:21618) UID(0) EUID(8), parent (init:1) UID(0) EUID(0)
grsec: faild fork with errno -11 by exim UID(0) EUID(8), parent (init:1) UID(0) EUID(8)

I figured out that X is the total number of running root's proceses (which is of course more than 32). But I do not know why the fork fails for exim if it has UID 0 end EUID 8 (which is mail) and the limit is set for group users. If I increase the limit to a number higher than the number of running root's proceses, the overstep is not there of course.

Does anybody have an idea why this happens, what is the cause and what is the solution (maybe I miss something obvious)?

Thanks very much in advance.
Rasto.

PostPosted: Fri Sep 19, 2003 2:53 pm
by devastor
Hi,

This looks a lot like a problem I had some time ago:
I had rlimit_nproc set for users-group and su set to use pam.
I was in users-group myself and when I tried to su to root it failed because
it couldn't start a shell because of the nproc-limit that got inherited to root.

I am not familiar with exim, but i think something like that might happen in your case too..
mail is setgid mail and it somehow calls exim directly or something like that and
the limit gets inherited..

I got around the problem by removing myself from users-group, but can't tell what
would help in your case..

Anyway, this has nothing to do with grsec as far as i can tell..

PostPosted: Fri Sep 19, 2003 3:16 pm
by spender
What version of grsecurity are you using?

-Brad

PostPosted: Mon Sep 22, 2003 7:43 am
by erce
I'm using grsecurity 1.9.11 for 2.4.21 kernel. I still did not solve the problem and even my linux guru friends are not able to help :-)

PostPosted: Mon Sep 22, 2003 8:26 am
by spender
That problem is fixed in 1.9.12. It was also a problem in Openwall.

-Brad