A gas container is unreliable when it comes to storage. Go to any welding supply store and they will have a poster of a car that tried to haul an acetylene bottle which leaked then subsequently exploded. Propane bottles of any kind are subject to this; new, refilled or otherwise.
I would guess that 12 refilled bottles of propane are statistically safer than 12 new ones in a box from bass pro. I cannot tell you how many times I have found an empty bottle in a package of 12. If you refill your own you probably will eliminate the leakers anyway.
As far as legality is concerned I don't know who is going to arrest you for such a digression. For example it is an extreme federal offense to modify or alter a pollution control device on a diesel motor vehicle. Yet people remove the system and buy computers that fool the onboard computer and blatantly roll coal (they call it), in front of God and everybody. So any legal concern is; in my mind out the window, as I do not think that you will be arraigned in front of the federal magistrate for the offense of refilling a camping bottle, if it is indeed an offense. There are unenforced and unenforceable laws everywhere. And if there is a law, it is probably a law that prohibits the sale or exchange of refilled bottles driven by the industry that does not want to compete against a refiling company who is refilling the bottles that were put onto the market by the original manufacturer. This would definitely mess up their gig.
As far as insurance is concerned, I doubt that they would make an issue over a couple of refilled propane bottles. If anything they would pay for your damages and recover their money from the company that made the adaptor and told you you could do it. You would be foolish to offer the fact that you refilled the bottle that caused the damage in question, unless asked directly. At that point you could say yes and I did it with this little adaptor I bought at Bass Pro Shop.
This having been said, I don't think safety, or legality is a concern, but practicality is. I would do it in a pinch or routinely if it suited me. It is much easier to tap onto your rigs supply line and never worry about it again. Or just do as one poster said, get the smallest refillable cylinder and carry that with you.