VirtualBox and CONFIG_PAX_RAP

Discuss usability issues, general maintenance, and general support issues for a grsecurity-enabled system.

VirtualBox and CONFIG_PAX_RAP

Postby Ghowe » Thu Oct 06, 2016 1:41 pm

Hello.
Was trying to run VirtualBox on Grsecurity hardened kernel (latest patch, 4.7.5 kernel).

Had some troubles with it. VirtualBox was freezing computer completely (with VIRT_HOST=y, VIRT_EPT=y, VIRT_VIRTUALBOX=y).
Eventually I found out, that VirtualBox freezes computer if CONFIG_PAX_RAP option is enabled.

Can you please tell me whether it is a bug or a feature?
Ghowe
 
Posts: 5
Joined: Sun Nov 15, 2015 11:28 pm

Re: VirtualBox and CONFIG_PAX_RAP

Postby PaX Team » Thu Oct 06, 2016 2:34 pm

RAP among others also catches bugs that violate C function pointer related rules, such as type mismatches between a function pointer and target functions. i've fixed up in-tree code for RAP but for out-of-tree code someone else will have to do it.
PaX Team
 
Posts: 2310
Joined: Mon Mar 18, 2002 4:35 pm

Re: VirtualBox and CONFIG_PAX_RAP

Postby Ghowe » Thu Oct 06, 2016 6:23 pm

You mean, bugreport about it should be submitted to VirtualBox team?

And one more thing.
Help for CONFIG_PAX_RAP says:
"If you have an amd64 processor that does not support SMEP then you must also enable a KERNEXEC code pointer instrumentation method (see PAX_KERNEXEC_PLUGIN)."
I read online, that PAX_KERNEXEC=y also makes VirtualBox to freeze computer.

Does it mean, that on amd64 CPU with no SMEP (I don't see it among flags) I won't be able to use both RAP and VirtualBox?
Ghowe
 
Posts: 5
Joined: Sun Nov 15, 2015 11:28 pm

Re: VirtualBox and CONFIG_PAX_RAP

Postby PaX Team » Thu Oct 06, 2016 6:47 pm

Ghowe wrote:You mean, bugreport about it should be submitted to VirtualBox team?
yes, but i'm not sure they will care about it since their code 'works', it's just RAP that happens to enforce a C rule while also doing its job.
Does it mean, that on amd64 CPU with no SMEP (I don't see it among flags) I won't be able to use both RAP and VirtualBox?
the KERNEXEC gcc plugin can be enabled without enabling the rest of KERNEXEC.
PaX Team
 
Posts: 2310
Joined: Mon Mar 18, 2002 4:35 pm


Return to grsecurity support