Have you considered a mid grey flannel - maybe in Fox Brothers cloth. My Fox Brothers Gun club check CPO shirt is a favourite.
I love my denim (634) and heavy cotton chinos and reckon a flannel in similar cut would complement a workwear wardrobe - for those really special occasions.