I think it simply feels wrong in every way, to ride a 5 m2, when you could use a nimble 2.2 m2 (30 knots average wind)
Have seen one super experienced semi pro having troubles even with a 3.5 in just below 30 knots but gusty wind, "It did not work" was his answer, and he rides ANYTHING and big waves.
But it all cooks down to what you aim for - yes you can survive in high winds easy, but if you can not make your "loved" moves out there, it is too much, right?
I cant do much yet anyways, but those making 360s and riding waves backwinded and jibing wing LE down etc, dont like having a wing more than twice the needed size
The other problem is wings getting destroyed by the boards, when wind picks the foil up after a crash, or even on land, they can not stay put on the ground in that much wind and can destroy your wing when blown into it - but that is not a "wing size" issue, I know.
Especially new wingfoilers, who can hold a big wing in a lot of wind, often trashes their wing, cut the fronttube and bladder with the wing, so it explodes.
This happens SO much more often, when windy, as you get out of control so much faster if using a wing twice as big.
Again, I know it is not an issue when experienced, if you dont make any errors and got a supersmall board.
It does not feel good when you have troubles sheeting the wing, and it always try to stay in its "flapping" mode (most flaps in this much when when neutral), and you can only sheet in partly now and then.
The good thing is, even when the COE moves, you can just move yourself forward or rear on the board so it fits.
Getting lifted off the board when strapless, is another issue, but comes down to rider level one must expect, yes.
Apart from above, you are right, it IS possible for most, to ride with big wings in high winds (just not much fun IMO)