What are the major decisions that a platform needs to make in order to balance incentivizing development vs. maintaining quality and control over their 3rd party app marketplace?
Let's look at this on three scales, in which the right answer for a given team is somewhere between two unrealistic and absolutist extremes.
First decision scale: Allow freeform development or provide a limited toolkit.
A lot of platform vendors assume that everyone building things for them is a developer, because they developed something. These vendors plan for developer support that they can't build, or build stuff that goes unused.
If the people solving problems on a platform are making a living by selling software that they wrote, they're developers. The platform should not proscribe their toolchain choices. They need a freeform environment that lets them do anything, and they don't want safety lines.
They need thorough documentation far more than they need anything else. Seriously, just direct resources to development and tech writing.
If the people solving problems on a platform are making a living by selling or using software that someone else wrote, they are not developers. Call them consultants, integrators, PS, admins, engineers, or architects instead.
Consultants who develop have different needs than developers who consult. They may want to teach their customers to fish, or they may want to be fishmongers, but they're not often trying to create new seafood concepts.
They want an easy way to connect components together to increase value. They want an easy and popular language with lots of community support, libraries of common functions, and simple guardrails that keep things safe and reasonable.
Second decision scale: Allow content dependencies and component reuse or force monolithic apps.
The chaos of extensibility, DLL Hell, a rich ecosystem of shared responsibility and global namespace? A platform that enables connectivity and dependency opens the door to expansion, competition, and growth, at the cost of instability.
The control of stability, bloated monoliths, a statically linked walled garden of singleton solutions? A platform that encourages safety and stability is easy to depend on, at the cost of expensive, repetitive efforts to reach limited solutions.
To consider this difference with more realism and less extremism, compare the Win32 ecosystem of the aughts with the IOS ecosystem of the teens (and note that the former added monolithic containers while the latter added sharing interfaces).
Third decision scale: Allow partners to write closed source or force them to be open source.
The verbs in that are not accidental. A platform has to offer a least some support for closed source, by keeping the source away from the user. Perhaps it goes farther and supports licensing, or brokers purchase for the partners, or not.
Of course, a partner can always choose to post their source code. If the platform only supports an scripting language and the user can just read the JS or PY files, then the partner doesn't have a choice: it's open source.
This scale decides if the possible business models are based on selling software, or selling services. Another way to say that: partners in the ecosystem can grow exponentially at high risk, or linearly at lower risk.
I've matrixed these scales and I don't have great examples for all of the possible combinations, but I do have a suspicion that going above the Bill Gates Line needs freeform development... more thought on that later.