Inserts restricted to a trigger

Is there are way to restrict direct access to a table for inserts but allow
a trigger on another table to perform an insert for that user?

I'm trying to implement an audit table without allowing user tampering with
the audit information.

Thanks in advance,

Miles Elam