Not sure about this idea.  Some issues come to mind:

* symmetric key must be randomly generated, otherwise data is far, far 
too large for public key crypto  (use symmetric key and encrypt only it 
under public key)

* I guess this only solves off-line attack, since on-line root user can 
just examine memory directly without generating core file

* even using fast symmetric crypto, each data byte in the core must be 
encrypted.  The fastest crypto algs I know of take at least a couple of 
cycles *per byte* on typical hardware (RC4 falls into the "fastest" case 
I know).  This will have a huge impact on the time it takes to dump 
core.   (At least on machines without good hardware crypto acceleration.)

* large core files (think gigabytes potentially here) now have to be 
decrypted to be used -- double the disk space.  This could get to be 
problematic on some systems with limited disk storage.

I think we're going overboard here.  Realistically, most sites are going 
to lock down core files altogether if this is a problem.  (Individual 
users can also do it using ulimit for their own non-setid processes, right?)

In the oddball case of a crashing program, the site admin may have to 
choose between taking down a server later to enable sguid core dumps, or 
leaving the security level lax.  Most production admins would choose the 
former, and probably most developers would choose the latter.

Folks, lets remember, you can't get *any* core file from a sguid process 
right now.  By adding this feature, we're adding value.  I believe we 
should add the feature in a security safe way now (along with leaving it 
disabled by default, at least for now), and wait until we get some 
operational experience with it before we start trying to solve the final 
.5% of the world's problems. 

    -- Garrett

John Darrow wrote:
> On Wed, Jan 25, 2006 at 11:21:32AM +1030, Brett Lymn wrote:
>   
>> On Wed, Jan 25, 2006 at 09:34:01AM +0900, Curt Sampson wrote:
>>     
>>> What advantages do you see to making it depend on kern.securelevel? What
>>> threat model do you have here?
>>>       
>> That someone could tweak the knobs up and be able to harvest private
>> information from set*id cores.
>>     
>
> Private information harvesting is also a potential threat model for
> non-set*id cores.
>
> It seems to me that, rather than not allowing the creation of cores
> that might be vital to debugging a problem, it makes more sense to
> directly address this threat model.
>
> How about a sysctl node which specifies a public key to encrypt all
> cores with.  Allow it to be set anytime, but changed or removed only
> at securelevel <= 0.  Then, if facing a debugging situation like those
> mentioned earlier in this thread (e.g. a vendor needs a core file),
> you can set the key, enable set*id cores, and take the resulting core
> dump to a secured machine elsewhere to decrypt it, without having to
> take the server down even briefly to enable the core dumps.  (Given
> both service expectations and the potential for leak-related problems
> that only crop up after a long running time, not having to down the
> server is a major plus.)
>
> With this sort of scheme, even if an intruder does manage to get on
> as root and grab a copy of a core, they're faced with a big
> encrypted blob.
>
> (You could potentially even have this be a per-user setting, but that
> would probably be overkill.)
>
> The SOC work on libbpg could provide a good chunk of the necessary
> code for implementing a mechanism like this.
>
> jdarrow
>   


-- 
Garrett D'Amore, Principal Software Engineer
Tadpole Computer / Computing Technologies Division,
General Dynamics C4 Systems
http://www.tadpolecomputer.com/
Phone: 951 325-2134  Fax: 951 325-2191