---
title: "Do agent skills and MCP servers change after you install them? What 95 scanned commits showed"
url: https://upskill.coach/blog/do-agent-skills-and-mcp-servers-change-after-you-install-them
last_updated: 2026-11-17
tokens_approx: 2531
publisher: Avatar 8 LLC (ops@avatar33.com)
---

> For AI agents: the docs index is at https://upskill.coach/llms.txt. Every page has a markdown version: add .md to its URL.

# Do agent skills and MCP servers change after you install them? What 95 scanned commits showed

> Short answer: yes, most of them, and quickly. On 2026-11-17 we compared each of the 95 GitHub repos behind the [Upskill library](https://upskill.coach/library) catalog with the exact commit our security scan had read. 65 (68.4%) had new commits, and for repos we scanned 28 or more days earlier it was 57 of 74 (77%). The repos that changed were a median 12 commits ahead, 16 were 100 or more ahead, and the median first new commit came 3.4 days after our scan. 41 changed agent-facing instructions and 36 changed dependencies. A scan, a review or a star count covers one commit. Pin the commit you checked, and read the diff before you update.

When people worry about agent skills, they usually worry about the first install. A post in our 2026-10-25 collection put it bluntly: "SKILLS.md are part of supply chain risk" ([@this_evangelos](https://x.com/this_evangelos/status/2113420978839814418)). Another, replying to a security news post, made the follow-up point: with no CVE, "scanners stay quiet, so the check is manual: run the version command on every machine with the agent... and list which MCP servers each one can reach" ([@NeuralAlphaAI](https://x.com/NeuralAlphaAI/status/2113258756087914655)). By our hand count, only 2 of the 198 posts in that collection talked about versions or the supply chain at all. This post measures the part that gets less attention: what happens after the first install.

## How we measured

The catalog our coach recommends from has 123 items. 95 of them come from a public GitHub repo and carry the commit our scanner read (shown on each item page, for example [QuantConnect MCP](https://upskill.coach/mcp/quantconnect-mcp), approved today). 19 are our own house skills, pinned by a file hash instead, and 9 have no repo to compare (hosted services and prompts). For each repo we called GitHub's compare API from the scanned commit to the current default branch (`repos/{owner}/{repo}/compare/{scanned}...{branch}`) and recorded how many commits it was ahead and which files changed. Scans ran from 2026-10-17 to 2026-11-17, so a repo had between a few hours and 31 days to change (median 31).

The script is in our app repo as `scripts/build-commit-drift.mjs` and writes aggregates only, no repo names.

## Results

| | Repos | Changed since scan | Share |
|---|---|---|---|
| All | 95 | 65 | 68.4% |
| Skills | 40 | 26 | 65% |
| Repos | 12 | 7 | 58.3% |
| MCP servers | 33 | 24 | 72.7% |
| Libraries and SDKs | 10 | 8 | 80% |
| Scanned under 14 days ago | 12 | 4 | 33.3% |
| Scanned 14 to 27 days ago | 9 | 4 | 44.4% |
| Scanned 28+ days ago | 74 | 57 | 77% |

- **How far they moved:** 6,126 new commits in total across the 65 repos. Median 12, maximum 851; 16 repos were 100 or more commits ahead.
- **How soon:** 35 of the 65 had their first new commit within 7 days of our scan, 44 within 14 days. 46 had a commit in the 7 days before 2026-11-17.
- **Still there:** none of the 95 repos was deleted or archived. 1 had moved to a new owner or name (GitHub redirects it).
- **Unchanged:** 30 repos (31.6%) were exactly at the scanned commit. Several were scanned in the last two weeks, so this mostly reflects time, not stability.

## What changed

We sorted the changed file paths into three groups. A repo counts once per group.

| Change | Repos | Of the 65 that changed |
|---|---|---|
| Agent-facing instructions (SKILL.md, AGENTS.md, CLAUDE.md, GEMINI.md, `.cursor/rules`, markdown under `skills/` or `prompts/`) | 41 | 63.1% |
| Dependencies (package.json, lockfiles, requirements, pyproject, Cargo, go.mod, Gemfile) | 36 | 55.4% |
| Install and build hooks (setup.py, install.sh, postinstall scripts, Makefile, Dockerfile, CI workflows) | 34 | 52.3% |

These are the same places our scanner looks hardest: [prompt injection in agent-facing files, risky dependencies and install hooks](https://upskill.coach/research/agent-skill-security). A changed SKILL.md is new instructions your agent will follow. A changed lockfile is new code it will run. We did not judge whether any of these changes were harmful. Almost all of them are ordinary maintenance. The point is that the scan result no longer covers them.

## Why "I checked it once" is not enough

- **Most installs follow the branch.** `npx skills add owner/repo`, `git clone` without a checkout, `uvx package` and `npx package@latest` all fetch whatever is current. That is what you get unless you pin.
- **Our own install lines do this too.** As of 2026-11-17, 0 of the 95 install lines in our catalog pin the scanned commit. Each item page shows the commit we scanned, and the docs say a newer commit is not covered ([security scanning](https://upskill.coach/docs/security-scanning)), but the copy-paste line installs the latest version. If you want the version we reviewed, check out that commit.
- **Hosted MCP servers can't be pinned at all.** A remote MCP server runs on the vendor's machine. Its code can change with no commit you can see, so the only things you can watch are its tool list and tool descriptions.

## How to pin and re-check

1. **Install from a commit.** `git clone https://github.com/<owner>/<repo> && git -C <repo> checkout <commit>`. For Python: `pip install git+https://github.com/<owner>/<repo>@<commit>`. For npm: `npm install github:<owner>/<repo>#<commit>`, and commit the lockfile.
2. **Write the commit down** next to the item in your agent's config, so you know which version you are running.
3. **Before updating, diff the parts that matter:** `git diff <old>..<new> -- '*.md' package.json '*lock*' requirements*.txt pyproject.toml setup.py install.sh`. New instructions, new dependencies and new install scripts are the places to read.
4. **Re-run the checks on the new commit:** `osv-scanner` for dependencies, `gitleaks` for committed secrets, and a grep of agent-facing files for "ignore previous", "do not tell" and credential paths. Our [vetting checklist](https://upskill.coach/blog/how-to-vet-an-mcp-server-or-agent-skill) has the full commands.
5. **For remote MCP servers,** save the `tools/list` response the first time, compare it on each new session, and stop if a tool appears or a description changes in a way you didn't expect.
6. **Ask the owner before upgrading** anything that runs code, holds API keys, sends messages or moves money. An upgrade is a new install.

## What we have not measured

This is 95 repos from our own curated catalog, not a random sample of skills on GitHub. Scan dates differ (71 of the 95 were scanned on 2026-10-17, the first day, and the rest one or two at a time since), so the overall rate mixes time windows; the by-age rows are the fairer comparison. The file groups come from file names, not a review of each diff. GitHub's compare API lists at most 300 changed files and 250 commits, and 16 repos hit the file limit, so the three group counts are lower bounds. We did not check tags or releases, and a repo whose default branch moved isn't the same as a version a package manager publishes. These numbers are from one run on 2026-11-17.

## Let a coach keep track

Agent Coach recommends only items that passed a security scan at a pinned commit, shows that commit on each item page, and keeps checking in with your agent on whether each item still earns its place. It is free. Browse the [library](https://upskill.coach/library), read the [developer docs](https://upskill.coach/docs) (API and MCP), or paste one line into your agent:

```
Read https://upskill.coach/start and follow it.
```

Related: [do GitHub stars mean a skill is safe](https://upskill.coach/blog/do-github-stars-mean-an-agent-skill-or-mcp-server-is-safe), [how to store API keys for MCP servers](https://upskill.coach/blog/how-should-an-ai-agent-store-and-use-api-keys-for-mcp-servers), [security research](https://upskill.coach/research/agent-skill-security).

## FAQ

### Do agent skills and MCP servers change after you install them?

Most do, and fast. On 2026-11-17 we compared the 95 GitHub repos behind our catalog with the commit we had scanned: 65 (68.4%) had new commits, including 57 of the 74 we scanned 28 or more days earlier. For the repos that changed, the median first new commit landed 3.4 days after our scan.

### Is a security scan of an MCP server still valid after an update?

No. A scan covers the exact commit it read. In our data, 41 of the 65 repos that moved on had changed agent-facing files (SKILL.md, AGENTS.md, CLAUDE.md, prompt or skill markdown) and 36 had changed dependency manifests or lockfiles, which are the parts a scan checks most closely.

### How do I pin an agent skill or MCP server to a version?

Install from a commit, not a branch: `git clone <repo> && git -C <dir> checkout <commit>`, `pip install git+https://github.com/<owner>/<repo>@<commit>`, or `npm install github:<owner>/<repo>#<commit>`. Keep the lockfile. Before moving to a newer commit, diff the agent-facing files and the dependency files.

### Can you pin a remote (hosted) MCP server?

Not by commit. A hosted MCP server runs on the vendor's machine and can change without a commit you can see. Record the tool list and tool descriptions it returns, compare them on each run, and give it the least access that does the job.

### Does Agent Coach pin the commits it scanned?

It records them: every item page shows the scanned commit, and the library JSON has `scanned_commit`. But as of 2026-11-17, 0 of the 95 install lines in our catalog pin that commit; they install the latest version. Until that changes, check out the scanned commit yourself, or read the diff first.

Questions or corrections: ops@avatar33.com.
