It sounds like we mostly all have a pretty good understanding of this, and put similar thoughts in different ways. The balance of factors is complex and at our "level of expertise" we can estimate but alot ends up trial and error, 'here are my turbos, now I have to work with them'

Well here's how I think about it anyway. This is sorta complex so let's break this into 3 main components:
1) Compressor and boost Pressure/Volume/Airmass
2) Turbine and "Backpressure"/Spoolup Driving Force
3) The Interaction of the Boost/Volume/Airmass _VS_ the Backpressure
1) Compressor and Boost Pressure/Volume/Airmass:
--The compressor makes boost pressure in PSI, which leads to airflow (FROM high pressure, TO low pressure) pushing air into the low pressure cylinder on intake stroke of the Otto cycle engine.
--The more air pressure (deltaP) the more airflow. This is easy! More boost=more flow. Less resistance to flow (better IC/piping flow, better manifold/head flow) leads to MORE flow at the SAME boost. Bonus!! (Still, "less resistance" is ultimately limited by the SIZE of the engine's cylinders and rpm it is turning at. Think of a curve going up fast, then levelling off as it reaches theoretical max point...diminishing returns to be had by more efficient piping etc).
--The COMPRESSOR MAP shows how efficient it is at MAKING that boost, and relates what it is physically able to make in PSI to the VOLUME/MASS of air at that point. Compressor wheels/housings can be designed for high volume/low pressure, low volume/high pressure, etc...
--Small compressors are EASY to spin, spool up faster, get you on boost faster, more lowend torque...but reach their max flow rates fast too, so are limited in topend HP. They also get OUT of their efficient compressor map islands at high flow, so at max they are beating up air alot/HEATING it, so though on top they can MAKE HIGH BOOST, the air is getting so HOT you are getting MORE PSI WITHOUT MORE AIR MASS! This is bad, makes for detonation, LESS hp...you have "gone over the hump" of that turbo's ability. See the TD04-9b at over (5500rpm+11psi) or so. 9b can do you proud at 16-18psi at 2500-5300rpm, but over that you get over the hump.
--Large compressors are HARDER to spin, spool up slower, but (with good design/housing) at mid/upper rpm regimes can make the SAME BOOST at HIGHER EFFICIENCY/FLOW than the small 9b ("flow/Airmass increase" is ONLY due to reduced temperature=denser air at same PSI/deltaP)...AND can make MORE PSI at HIGHER RPM regimes so can make more hp on top.
SO:
A Compressor makes PSI. PSI makes AIRFLOW. _WHAT/HOW MUCH_ you are flowing (AIR DENSITY/MASS) relates to TEMPERATURE of that air: Larger/moreEfficeint Compressor=Cooler=Denser=more O2 molecules/mass flowed at SAME PSI.
This comes into play mostly at topend psi/flows, where the limitations of small compressor are exceeded, while the larger compressor is still efficient there...though it will be LESS efficient at LOWER psi/flows. Our 9b's are GREAT for stock cars...NOT as good for modded/high hp targets.
2) Turbine and "backpressure"/spoolup driving force:
What DRIVES that compressor? TURBINE BACKPRESSURE = exhaust pressure released from the Otto engine on exhaust valve opening, escaping past/driving the turbine blades. MORE backpressure deltaP=Faster Spool=Great lowend.
LESS backpressure=SlowerSpool=Worse Lowend, BUT GAIN topend WHEN the BALANCE between backpressure and compressor gets BACK into its efficiency zone!
3) The Interaction of the Boost/Volume/Airmass _VS_ the backpressure:
MUCH of the reason "Bigger Turbos Make More Power" is that they have RAISED the engine RPM and BOOST LEVEL at which the maximum efficiencies of the COMPRESSOR and TURBINES happens, WITH RELATION to the ENGINE size, rpm, flow etc. These are CURVES, which intersect...where the COMPRESSOR PSI/FLOW curve and the TURBINE PSI/FLOW curves "come together" in MAJOR way determine how the engine acts, how much ultimate HP it can make, how fast it comes on boost!
-----------------
So, you can make a high hp turbo that has WAY big topend but bad spoolup because the turbines are too big for reasonable spoolup. It becomes a balancing act between what your backpressure is at max hp rpm, and what your compressor flow/efficiency is at max hp rpm. We must try to make good driving force (turbine/backpressure) while making good compressor flow.
Many turbos can work fine!...where do all YOUR curves intersect?
(Pics from chicks accepted in the Lounge

) We better not get into camshaft overlap, duration, etc but these systems all interact and each has its "curve"...the more you can get these curves to be at their PEAKS TOGETHER at the rpm you want, without losing "too much" spoolup, the better you are
Sure, bigger turbos make more topend. Sorta easy, if you target a 2 thousand rpm window and don't CARE about the rest (race/drag car)...the harder part is finding YOUR balance, especially those of us who street/roadcourse drive when losing spoolup/low/midrange really hurts.
Jack T.