Can I add a product with a bill of materials to an order and have it directly relieve stock of the constituent products?

No. An order and associated shipment will always relieve stock of the specific product listed on the shipment, and not implicitly explode references to the constituent products. To include products assembled based on a bill of materials on a shipment, create stock of the product using a build transaction and then ship the product.