If it is not directly under your control, or in a safe, yes.
Why do you need to have a gun in your desk drawer, cupboard or other unsecured location? What benefit does that provide?
EDIT: Maybe not completely irresponsible, but certainly not as responsible as you could be.