A plugin can misbehave at two very different moments, and most scanning covers only one of them. Install script scanning reads what a plugin declares it will run when you add it: preinstall, install and postinstall hooks, and anything those hooks pull down. Postinstall script scanning is well covered for that reason, and runtime script risk is the other half, the code that executes every time you invoke the plugin instead of once during setup. Knowing where install vs runtime scripts diverge is what tells you how much a clean scan is actually worth. Below is what plugin scanners check today, where that coverage stops, and three checks that cover the rest on your own machine.
What install script scanning covers
A scanner reads the manifest and the install hooks a plugin declares, then judges those hooks against patterns that have caused trouble before. Four things get looked at.
- preinstall, install and postinstall scripts declared in the manifest. These run with your privileges during setup, which is why postinstall script scanning gets the most attention.
- Network calls made from those hooks. Fetching a second-stage binary at install time is the pattern most warnings are built around.
- File writes outside the plugin directory. Anything touching your home directory, shell profile or editor config gets flagged.
- Bundle declarations. A plugin that ships no bundle statement is harder to audit, so it scores lower rather than passing quietly.
The mechanics behind those checks are described at /blog/how-install-script-scanning-works, and the specific patterns that trigger a warning are listed at /blog/dangerous-install-script-explained.
Runtime script risk: the half that keeps running
Everything above happens once. A plugin that behaves perfectly during setup can still act differently on every invocation, and that code is ordinary plugin source, which a scanner has no reason to treat as dangerous on its own.
- Reading credentials from the environment or from config files each time it runs.
- Outbound requests during normal use. These are indistinguishable from legitimate telemetry without reading the code.
- Files written into the project directory after the fact, rather than during setup.
- Shell commands built from input, where the risk depends on the input rather than on the script itself.
This is the gap. Install-time behaviour is declared, structured and easy to check. Runtime behaviour is just code doing code things, and reviewing it is a human job.
| Install scripts | Runtime scripts | |
|---|---|---|
| When it runs | Once, during setup | Every invocation |
| Where it lives | Manifest hooks | Plugin source |
| Who checks it | Static install script scanning | Rarely, and mostly by review |
| Privileges | Your user account | Your user account plus your credentials |
| Typical failure | Pulls a second stage from the network | Leaks or modifies things over time |
| Visible in a scan result | Yes | Usually not |
Install vs runtime scripts: where the boundary sits
The boundary is not the file. It is the moment the code runs, and one plugin can put the same action on either side of it.
- A plugin that downloads a helper at install time is doing an install-time action and gets judged there.
- The same plugin checking for updates on each run is doing a runtime action, and that path is rarely scanned.
- A hook that writes to your shell profile is install-time. A plugin that reads that profile each session is runtime.
- Most plugins do both, which is why a clean install scan tells you less than it appears to.
A clean install scan means the setup was honest. It does not mean the plugin stays honest once it is running.
Covering the runtime half yourself
You do not need to audit source line by line. Three checks catch most of what matters, and they scale because you only run them on plugins that already passed the install scan.
Three checks worth an afternoon
- Read the network calls. Grep the plugin source for outbound requests and look at where they go. Legitimate telemetry has a documented endpoint; anything else is worth a question.
- Check what it reads from the environment. A plugin that pulls tokens or keys by name should say so in its README. The documentation review at /blog/how-to-evaluate-plugin-documentation-quality covers what a good one looks like.
- Watch the first week. Run new plugins on a machine where you would notice an unexpected write, rather than everywhere at once. The CI angle is covered at /blog/ci-cd-plugin-scanning.
FAQ
- Does a clean install scan mean the plugin is safe? It means the setup behaved. Runtime behaviour is separate, and no static install scan can speak to it.
- Why is runtime script risk harder to score? Because the same code is either harmless or harmful depending on intent, and intent is not visible in a manifest. Reviews catch it, patterns do not.
- Should I avoid plugins with postinstall scripts? No. Plenty of legitimate plugins compile native code or fetch platform binaries there. Read what the hook does rather than rejecting it on presence alone.
- Where does the grade come from if runtime is not scanned? The score covers maintenance, documentation, ecosystem health and install-time behaviour. Independent scoring rather than self-reported claims is described at /blog/dsh-plugin-review-process.
Treat a clean install scan as the first gate rather than the last one. Every plugin on dshquality.com has a score page listing what was checked and what was not, so you can see which half of the picture you are looking at. The review process behind those grades is at /blog/dsh-plugin-review-process, and everything we publish is at /blog.