Physiques have nothing to do with it, as everyone knows cyclists historically have been the biggest users of steroids and the don't look like bodybuilders.
They are probably used to varying degrees by a few players but really one needs: a longish offseason to be able to take them, doping testers that can be avoided, and a load of money to pay the best in the PED game to get them the gear and dose them etc. Which most players dont have.
Also rugby is one of a few sports that doesn't push its players into athletic tests at really early stages. There is no combine where you're meant to be able to bench press 225lb 30 times or whatever. You just have to be a skilled player.