I am of two minds on this. On one hand, having a group of diverse people bringing different skills to the table would help out immensely, and there are a lot of things you simply can't do by yourself (like pull 24-hour security). On the other hand, more people means more food, more water, a bigger shelter, more noise if you are trying to be quiet, more of a trail to follow if you are moving, slower movement, etc etc etc. The list goes on and on.
In my opinion the perfect group would be 3-4 people of varying backgrounds.
I am with you on this jsquared but sometimes a small group of 3 to 4 people is not a viable option.
I belong to a group of ten like minded individuals. When you bring our families together you are talking 45 people. Lots of ways to address the logistics of maintaining a large group. Moving the group can be done by splitting into smaller teams that traveling at staggered intervals. Time, distance, spatial relationship , etc can be used to "hide" a large group.
Prepare, plan, practice, evaluate and then start the cycle all over again.