For AI agents: the docs index is at /llms.txt. Every page has a markdown version: add .md to the URL.

Agent Coach

Do agent skills and MCP servers change after you install them? What 95 scanned commits showed

Short answer: yes, most of them, and quickly. On 2026-11-17 we compared each of the 95 GitHub repos behind the Upskill library catalog with the exact commit our security scan had read. 65 (68.4%) had new commits, and for repos we scanned 28 or more days earlier it was 57 of 74 (77%). The repos that changed were a median 12 commits ahead, 16 were 100 or more ahead, and the median first new commit came 3.4 days after our scan. 41 changed agent-facing instructions and 36 changed dependencies. A scan, a review or a star count covers one commit. Pin the commit you checked, and read the diff before you update.

When people worry about agent skills, they usually worry about the first install. A post in our 2026-10-25 collection put it bluntly: "SKILLS.md are part of supply chain risk" (@this_evangelos). Another, replying to a security news post, made the follow-up point: with no CVE, "scanners stay quiet, so the check is manual: run the version command on every machine with the agent... and list which MCP servers each one can reach" (@NeuralAlphaAI). By our hand count, only 2 of the 198 posts in that collection talked about versions or the supply chain at all. This post measures the part that gets less attention: what happens after the first install.

How we measured

The catalog our coach recommends from has 123 items. 95 of them come from a public GitHub repo and carry the commit our scanner read (shown on each item page, for example QuantConnect MCP, approved today). 19 are our own house skills, pinned by a file hash instead, and 9 have no repo to compare (hosted services and prompts). For each repo we called GitHub's compare API from the scanned commit to the current default branch (repos/{owner}/{repo}/compare/{scanned}...{branch}) and recorded how many commits it was ahead and which files changed. Scans ran from 2026-10-17 to 2026-11-17, so a repo had between a few hours and 31 days to change (median 31).

The script is in our app repo as scripts/build-commit-drift.mjs and writes aggregates only, no repo names.

Results

ReposChanged since scanShare
All956568.4%
Skills402665%
Repos12758.3%
MCP servers332472.7%
Libraries and SDKs10880%
Scanned under 14 days ago12433.3%
Scanned 14 to 27 days ago9444.4%
Scanned 28+ days ago745777%
  • How far they moved: 6,126 new commits in total across the 65 repos. Median 12, maximum 851; 16 repos were 100 or more commits ahead.
  • How soon: 35 of the 65 had their first new commit within 7 days of our scan, 44 within 14 days. 46 had a commit in the 7 days before 2026-11-17.
  • Still there: none of the 95 repos was deleted or archived. 1 had moved to a new owner or name (GitHub redirects it).
  • Unchanged: 30 repos (31.6%) were exactly at the scanned commit. Several were scanned in the last two weeks, so this mostly reflects time, not stability.

What changed

We sorted the changed file paths into three groups. A repo counts once per group.

ChangeReposOf the 65 that changed
Agent-facing instructions (SKILL.md, AGENTS.md, CLAUDE.md, GEMINI.md, .cursor/rules, markdown under skills/ or prompts/)4163.1%
Dependencies (package.json, lockfiles, requirements, pyproject, Cargo, go.mod, Gemfile)3655.4%
Install and build hooks (setup.py, install.sh, postinstall scripts, Makefile, Dockerfile, CI workflows)3452.3%

These are the same places our scanner looks hardest: prompt injection in agent-facing files, risky dependencies and install hooks. A changed SKILL.md is new instructions your agent will follow. A changed lockfile is new code it will run. We did not judge whether any of these changes were harmful. Almost all of them are ordinary maintenance. The point is that the scan result no longer covers them.

Why "I checked it once" is not enough

  • Most installs follow the branch. npx skills add owner/repo, git clone without a checkout, uvx package and npx package@latest all fetch whatever is current. That is what you get unless you pin.
  • Our own install lines do this too. As of 2026-11-17, 0 of the 95 install lines in our catalog pin the scanned commit. Each item page shows the commit we scanned, and the docs say a newer commit is not covered (security scanning), but the copy-paste line installs the latest version. If you want the version we reviewed, check out that commit.
  • Hosted MCP servers can't be pinned at all. A remote MCP server runs on the vendor's machine. Its code can change with no commit you can see, so the only things you can watch are its tool list and tool descriptions.

How to pin and re-check

  1. Install from a commit. git clone https://github.com/<owner>/<repo> && git -C <repo> checkout <commit>. For Python: pip install git+https://github.com/<owner>/<repo>@<commit>. For npm: npm install github:<owner>/<repo>#<commit>, and commit the lockfile.
  2. Write the commit down next to the item in your agent's config, so you know which version you are running.
  3. Before updating, diff the parts that matter: git diff <old>..<new> -- '*.md' package.json '*lock*' requirements*.txt pyproject.toml setup.py install.sh. New instructions, new dependencies and new install scripts are the places to read.
  4. Re-run the checks on the new commit: osv-scanner for dependencies, gitleaks for committed secrets, and a grep of agent-facing files for "ignore previous", "do not tell" and credential paths. Our vetting checklist has the full commands.
  5. For remote MCP servers, save the tools/list response the first time, compare it on each new session, and stop if a tool appears or a description changes in a way you didn't expect.
  6. Ask the owner before upgrading anything that runs code, holds API keys, sends messages or moves money. An upgrade is a new install.

What we have not measured

This is 95 repos from our own curated catalog, not a random sample of skills on GitHub. Scan dates differ (71 of the 95 were scanned on 2026-10-17, the first day, and the rest one or two at a time since), so the overall rate mixes time windows; the by-age rows are the fairer comparison. The file groups come from file names, not a review of each diff. GitHub's compare API lists at most 300 changed files and 250 commits, and 16 repos hit the file limit, so the three group counts are lower bounds. We did not check tags or releases, and a repo whose default branch moved isn't the same as a version a package manager publishes. These numbers are from one run on 2026-11-17.

Let a coach keep track

Agent Coach recommends only items that passed a security scan at a pinned commit, shows that commit on each item page, and keeps checking in with your agent on whether each item still earns its place. It is free. Browse the library, read the developer docs (API and MCP), or paste one line into your agent:

Read https://upskill.coach/start and follow it.

Related: do GitHub stars mean a skill is safe, how to store API keys for MCP servers, security research.

FAQ

Do agent skills and MCP servers change after you install them?

Most do, and fast. On 2026-11-17 we compared the 95 GitHub repos behind our catalog with the commit we had scanned: 65 (68.4%) had new commits, including 57 of the 74 we scanned 28 or more days earlier. For the repos that changed, the median first new commit landed 3.4 days after our scan.

Is a security scan of an MCP server still valid after an update?

No. A scan covers the exact commit it read. In our data, 41 of the 65 repos that moved on had changed agent-facing files (SKILL.md, AGENTS.md, CLAUDE.md, prompt or skill markdown) and 36 had changed dependency manifests or lockfiles, which are the parts a scan checks most closely.

How do I pin an agent skill or MCP server to a version?

Install from a commit, not a branch: git clone <repo> && git -C <dir> checkout <commit>, pip install git+https://github.com/<owner>/<repo>@<commit>, or npm install github:<owner>/<repo>#<commit>. Keep the lockfile. Before moving to a newer commit, diff the agent-facing files and the dependency files.

Can you pin a remote (hosted) MCP server?

Not by commit. A hosted MCP server runs on the vendor's machine and can change without a commit you can see. Record the tool list and tool descriptions it returns, compare them on each run, and give it the least access that does the job.

Does Agent Coach pin the commits it scanned?

It records them: every item page shows the scanned commit, and the library JSON has scanned_commit. But as of 2026-11-17, 0 of the 95 install lines in our catalog pin that commit; they install the latest version. Until that changes, check out the scanned commit yourself, or read the diff first.

Questions or corrections: ops@avatar33.com.

Published 2026-11-17, updated 2026-11-17 by Avatar 8. Markdown copy: /blog/do-agent-skills-and-mcp-servers-change-after-you-install-them.md