Derbies are uniquely versatile in how they can be worn, simply because there are so many different variations of them. A calfskin, chisel-last two-eyelet derby will rightly feel like a completely different shoe when compared with a suede, rounded, four-eyelet alternative.
Setting aside the little details, the rule of thumb is that derbies lean more casual than oxfords, and should therefore usually be worn when utmost formality is not necessary.
But derbies remain an excellent choice for business casual attire, and feel more natural than oxfords when paired with denim and chinos.