Skip to content

add: [microsoft-win11-connection-endpoints] generator for Windows 11 connection endpoints - #379

Merged
adulau merged 1 commit into
MISP:mainfrom
elhoim:warninglists/add-win11-generator
Aug 30, 2026
Merged

add: [microsoft-win11-connection-endpoints] generator for Windows 11 connection endpoints#379
adulau merged 1 commit into
MISP:mainfrom
elhoim:warninglists/add-win11-generator

Conversation

@elhoim

@elhoim elhoim commented Aug 29, 2026

Copy link
Copy Markdown
Member

Summary

Adds a new warninglist, microsoft-win11-connection-endpoints, generated from the two official Microsoft Learn pages listing Windows 11 network endpoints:

Unlike the existing hand-maintained microsoft-win10-connection-endpoints list (frozen since 2022-08-16), this one is backed by a generator (tools/generate-microsoft-win11-endpoints.py) so it can be refreshed automatically going forward.

Source choice

Both Learn URLs were probed first with curl/BeautifulSoup. Both return HTTP 200 with fully server-rendered <table> markup already containing the endpoint data — no JS execution needed. Given that, plain requests + BeautifulSoup against the rendered HTML was chosen over falling back to the raw markdown mirror on MicrosoftDocs/windows-itpro-docs on GitHub, since the HTML is directly reliable and avoids an extra source/rate-limit dependency.

Normalisation

The "Destination" column cells needed cleanup, documented as a comment block at the top of the generator:

  • Rows with fewer than 4 cells (headers, malformed) are skipped.
  • Cells with <br>-separated multiple endpoints are split.
  • Inline link/prose text used in place of a real destination (no dot, contains spaces) is dropped.
  • URL scheme, path, and trailing port are stripped.
  • Leading */*. wildcards are normalised to a leading . to match this repo's existing convention (see microsoft-win10-connection-endpoints).
  • A trailing * (used inconsistently on the source page, no accompanying footnote found) is stripped as a rendering artifact.
  • Anything left with whitespace, backticks, pipes, angle brackets, or no dot is rejected.

Result

152 entries, type: string, same matching_attributes as the Windows 10 list (domain, hostname, domain|ip).

Test plan

  • Generator runs cleanly (exit 0), generators.log has no ERROR/WARNING lines
  • jsonschema -i lists/microsoft-win11-connection-endpoints/list.json schema.json — exit 0
  • tools/validate_values.py — exit 0
  • Manually spot-checked entries: no HTML fragments, no empty strings, no stray markdown/backticks/pipes
  • make_list_unique.py run; confirmed via git status it touched no other tracked list
  • jq -S normalized, exec bit removed
  • generate_all.sh updated next to the other Windows/Microsoft generators (not appended at the end)
  • README.md index line inserted in correct alphabetical position

Note: split by the parent session into two PRs (script+generate_all.sh, and list.json+README) — both halves are independently coherent.


🤖 Generated with Claude Code

https://claude.ai/code/session_0121gk7vaCXX9bMri7tC9XVd

…connection endpoints

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0121gk7vaCXX9bMri7tC9XVd
@adulau
adulau merged commit eb0c198 into MISP:main Aug 30, 2026
5 checks passed
@adulau

adulau commented Aug 30, 2026

Copy link
Copy Markdown
Member

Nice one. Thank you!

@elhoim
elhoim deleted the warninglists/add-win11-generator branch August 31, 2026 18:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants