The best way of explaining I've seen talked about the airflow as 'slugs' of air each having mass and interia.
The exhaust port on a 4 stroke is going to see 1 high pressure 'blow' phase to 3 phases doing nothing 'wait', so with straight exit for each pipe there will be blow, wait, wait, wait, blow, wait, wait, wait,....
All that slowing down and speeding up takes energy.
But it would get worse because the 'tap' if you like gets turned off immediately and quickly after the exhaust stroke so the fast moving slug all of a sudden has a vacumn behind it trying to slow it down / reverse flow..
So what if the pipes join up after enough length in the primaries to fit a slug?
Well after the join there will now be another slug arrive at the join in time to avoid the vacumn forming so the slug will keep traveling down the exhaust nice and quick.
The join with a 4 cylinder will see: blow, blow, blow, blow,....
Sure that is an ideal situation and there are certainly simplifications, but it is better than 'put a big whacking pipe on each cylinder.'
Franks has done some pretty cool practical experimentation with burble manifolds etc so I trust he knows what he's talking about.
Another way of looking at it is that sound is energy so all that noise, popping and farting is energy that could be used to make the car go forward. So the ideal then would be a silent exhaust with no mufflers etc required, cause mufflers are absorbing energy from the exhaust.
This is the model I have in my head so it's only an approximation of what's going on, it will be changed next time I read something interesting on the subject.