**Environment**
XProtect Corporate, **2025 R3**, Management Server + Event Server + API Gateway on the same box.
**Summary**
We have a client that authenticates fine, and whose REST calls through the API Gateway work perfectly (`/api/rest/v1/sites`, `/hardware`, `/cameras` all return real data). The Events & State WebSocket connects, completes the upgrade, and our client logs `connectionState: subscribed`. No error, no disconnect. But **we never receive a single event frame**, no matter what we trigger on the server.
**What we’ve ruled out**
- **Not a “service down” problem.** Event Server, Log Server, and Data Collector were each independently restarted by our site’s Milestone admin, live, one at a time. None of the four changed anything.
- **Not the API Gateway being generally broken.** Verified with Milestone’s own official verification steps: a real service-account login through the gateway returns a genuine bearer token, and a real authenticated `/api/rest/v1/sites` call through the same gateway returns real data.
- **Not a client-side timing miss.** We ran four separate 5-minute watch windows (polling our own event buffer every 2s) across three different real triggers on the server (an initial generic test, an online/offline test, and a recording server stop/start done twice). Cross-checked our watch window against the Management Client’s own System Logs: the recording server’s stop (4:54:24 PM) and start (4:55:59 PM) both land *inside* one of our watch windows (16:50:52–16:56:01) to the second. All four windows: connection stayed `subscribed`, zero events received.
- **Not “the events aren’t actually firing.”** Confirmed independently in System Logs, Audit Logs, *and* Event Server’s own log file that these events are real and dispatched on the server side, at the same timestamps we were watching.
**What we found and fixed — but it wasn’t sufficient**
We found this in the API Gateway’s own log (`gateway.log`):
```
ERROR - VmsTokenProvider.HandleSecret: Secret not found, try doing a re-registration
ERROR - VmsTokenProvider.HandleAndValidateSecret: IDP secret is invalid.
Registration of API Gateway may be required
ERROR - DynamicProxyConfigurationService.ConfigureEventServerProxyWithRetryAsync:
Failed to configure event server route | Retrying after 60s
```
Our reading: the API Gateway has its own identity/secret for authenticating itself to the IdP (separate from any client-facing service account), and that secret was missing/invalid — which meant it could never complete `ConfigureEventServerProxyAsync`, the step that wires up the Event Server route the WebSocket depends on.
**Fix applied:** removed the stale Gateway Service registration (Management Client → Tools → Registered Services) and recycled the `VideoOS ApiGateway AppPool` in IIS. This time, the gateway log showed a successful re-registration (an earlier attempt at the same recycle had produced no new log activity at all — this one clearly took effect).
**Result after the fix:** we forced a fresh client reconnect (to avoid reusing any pre-fix token/session) and re-ran the watch-window test against two more real triggers. Same outcome as before: connects, stays `subscribed`, **zero events delivered**.
**Where that leaves us**
The break is confirmed to sit *downstream* of “API Gateway can’t register with the IdP” — that specific problem is fixed and confirmed to have taken effect. Something else in the Event Server → API Gateway → WebSocket relay still isn’t passing events through, even though:
- the gateway’s own IdP registration is healthy,
- REST traffic through the same gateway works,
- the WS handshake/subscription itself succeeds and stays up,
- and the events are demonstrably firing on the Event Server side.
**What we’re asking**
1. Beyond `VmsTokenProvider`/IDP-secret registration, is there a **separate** registration or subscription step in the Event Server ↔ API Gateway relay for the Events & State WebSocket specifically (as opposed to REST proxying), that a stale/incomplete state could still be blocking even after the IDP secret fix?
2. What should we be looking for in **Event Server’s own log** (as opposed to `gateway.log`) for this specific handoff — is there a known log line that confirms/denies whether Event Server is actually pushing to the API Gateway’s relay for a given subscription?
3. Is there a known issue in 2025 R3 in this area, or a diagnostic/reset step (beyond what’s already in the API Gateway install docs) that specifically targets the Event Server-side half of this relay rather than the Gateway/IdP-side half we already fixed?