In fact, you can join them on a day out, where you may do anything you please; it could be going for drinks, or a bowling session, or just a game of poker in your cafeteria. As long as you don't remind them about work that day, and become one of them, you will be more than welcome to join the group.