To say “Thanks for the invite.  Sorry I can’t make it.” 

You see, THAT doesn’t make me feel like a loser.  Instead, it makes me feel like somebody who deserves to be acknowledged.  Like somebody with feelings who wants to have dignity and worth. 

Respond to your invitations, people.