Personally we have found 11 to be just about the hardest number. The difference in scaling between 10 to 11 is about 11mill HP. Now for a boss like the Iron Juggernaut, that's fine. You pew pew the boss the whole fight. 11 mill over 5-6 mins is do-able. But, for a fight like Galakras where the boss is a burn at the end. That 11 mill is pretty tuff to make. Especially being flex so the extra members are not exceptional.
TL:DR: the boss HP Scaling is a bit over tuned for 1-2 extra players. (on some fights)
A bit of clarification here: Boss health and damage scaling are both completely linear and proportional to group size, but scale up more slowly than raid size does. So, all else being equal, if you add players who are roughly on par with the average gear/skill of your current group, you'll have a slightly easier time as a result.
Thok, for example, has 867 million health with a 10-player Flexible raid. If you add an 11th player, his health will go up to 953 million. I.e., you've increased your raid's damage output by somewhere between 14 and 20% depending on how many healers you were running, but the boss's health only went up by 10%. Adding that 11th player will also make Thok melee for about 4% more damage, which will barely be noticeable.
More than anything, it's great to have some clarity. Ion has a great skill for expressing things clearly, and it's just really good to understand exactly how Flex scaling works. In my opinion, one of the biggest things Blizzard can do to ensure that Flex doesn't become elitist and closed to many is to be completely candid about the scaling. Bringing an extra DPS doesn't seem so bad when all they have to do is 4% boss health as damage. It's good to hear that they err on the side of caution, too, but also that they don't want tuning to be too trivial.
Nonetheless, it is inevitably the case that players will want others of the same or similar gear and experience levels in their group when pugging. Nobody wants to carry people in PuG raids!