Manda and I started a gym membership a few months ago, and like any locker room full of semi-strangers sometimes you hear things and want to jump into the conversation.
So, random college sportsball player, khaki shorts are never "a go" for team dinner function, no matter how casual, especially if it's no jeans. Invest in some khaki pants.