I know this is really minor, but could you swap the position of the Refuse and Accept buttons? This way, reading from left to right, which is natural for English-speakers, the Accept button is on the left and the Refuse button is on the right. The current setup feels very unintuitive, as I find my cursor gravitating to the Refuse button every time I want to accept an invite.
Confirmation dialogues in software almost always have the "Save"/"OK"/"Confirm"/etc. button on the right and the "Don't Save"/"Cancel" button on the left.
