Policy Verification and Troubleshooting
Objective
In this lesson you will verify SD‑WAN policy distribution and troubleshoot policy evaluation on edge devices. We focus on using show/debug commands to confirm a policy was pushed from the controller plane, that the control-plane connections are healthy, and that the policy is installed and actively influencing traffic. In production, fast verification and accurate debugging reduce Mean Time To Repair (MTTR) when a path selection or security policy behaves unexpectedly — for example, when SaaS traffic should prefer an MPLS path but is still flowing across Internet links.
Quick Recap
This lesson continues the topology used throughout the lab series. We do not add new devices; we will verify policies on the existing controllers and two edges.
ASCII Topology (exact IPs on every interface)
vManage, vSmart and vBond sit in the management/control cloud. Two edge routers (HQ and Branch) represent production appliances.
+-----------------+
| vManage |
| eth0: 10.0.0.10 |
+--------+--------+
|
| 10.0.0.0/24
|
+-----------------+-------+--------+-----------------+
| | | |
+------+------+ +------+-------+ +-----+-----+ +-------+------+
| vSmart | | vBond | | ISP/Net | | ISP/Net |
| eth0:10.0.0.11| | eth0:10.0.0.12| | 192.0.2.2 | | 203.0.113.2 |
+------+-------+ +------+--------+ +-----+-----+ +-------+------+
| | | |
| | | |
+------+-------+ +------+--------+ +------+-------+ +------+-------+
| Edge-HQ | | Edge-Branch | | Internet | | Internet |
| Catalyst8000 | | ISR1000 | | 192.0.2.2 | | 203.0.113.2 |
| Gi0/0:192.0.2.1/30| Gi0/0:203.0.113.1/30 | |
+-----------------+ +-----------------+ +--------------+ +--------------+
Device table and primary addresses
| Device | Role | Primary Interface | IP address |
|---|---|---|---|
| vManage | Management Plane | eth0 | 10.0.0.10/24 |
| vSmart | Control/Policy Plane | eth0 | 10.0.0.11/24 |
| vBond | Orchestration | eth0 | 10.0.0.12/24 |
| Edge-HQ | Data-plane Edge | GigabitEthernet0/0 | 192.0.2.1/30 |
| Edge-Branch | Data-plane Edge | GigabitEthernet0/0 | 203.0.113.1/30 |
Tip: All management-plane domain names in examples use lab.nhprep.com and passwords use Lab@123. Organization name is NHPREP.
Key Concepts
Before touching the CLI, understand these concepts that drive verification and troubleshooting:
-
Policy distribution and controllers: The management plane (vManage) composes intent and pushes policies to the control plane (vSmart). The control plane then distributes policy to edges. When you verify, you check both the controller and the edge to ensure policy reached its destination.
- In production: a mis-pushed policy often looks like a local configuration mismatch; verifying controller distribution narrows the problem quickly.
-
Control-plane health influences policy application: Edges keep a secure control connection to controllers. If the control session is down, edges will not receive incremental policy updates and may operate on stale rules.
- Protocol behavior: edges periodically re-establish control sessions; during an outage, a stale policy may persist until the session is reestablished.
-
Policy evaluation order matters: SD‑WAN policy evaluation follows match → sequence → action ordering. A more specific match or higher-priority policy can override a generic policy; verification needs to confirm which exact rule was installed on the data-plane device.
- Think of policies like a stack of filters — the first matching filter decides the outcome.
-
Telemetry and AIOps integrations (NWPI, TE, UMTS, ThousandEyes): Visibility tools correlate underlay (TE/UMTS) and overlay (NWPI) metrics to show where the problem occurs and why. Use these to correlate policy events with path anomalies in production.
-
Application-aware routing (AAR) vs. topology-based: Some policies are application-driven (App Route); others select TLOCs by topology. When troubleshooting, confirm whether the policy expects app-detection or simple prefix matching.
Step-by-step configuration
Step 1: Confirm policy exists on the management plane (vManage)
What we are doing: Check that the intended policy was created in the management plane and exported to the control plane. This ensures the source-of-truth contains the policy before we hunt on devices.
# On the vManage CLI or API shell (management node)
show policy list
What just happened: The command enumerates the policy objects that exist in vManage with their names and last-push timestamps. If the policy is absent here, no edge will ever receive it. This is the authoritative check.
Real-world note: In production, vManage is the single pane of intent — always confirm here first before troubleshooting edge behavior.
Verify:
show policy list
Policy Name Type Last Push Time
App-Prefer-MPLS AppRoute 2026-03-30 09:12:45
Prefix-Block-Internet RoutePolicy 2026-03-30 09:13:02
TLOC-Preference-HQ TLOC 2026-03-30 09:13:20
Step 2: Verify control-plane connectivity from an edge (Edge-HQ)
What we are doing: Ensure Edge-HQ has healthy control connections to the controllers (vSmart/vManage/vBond). Without these, policy updates cannot be delivered.
configure terminal
logging buffered 100000 debugging
end
write memory
# On Edge-HQ CLI
show control connections
What just happened: The configuration enabled buffered logging at debugging level so we can capture policy-evaluation messages during troubleshooting. The show control connections command lists established secure control sessions to controllers including remote IPs, state and time since last message — a direct indicator of control-plane health.
Real-world note: Control reboots or network changes frequently cause edges to lose control connections; logging and control checks are the first triage steps.
Verify:
show control connections
Node-ID: edge-hq.lab.nhprep.com
Local-Controller: 10.0.0.11
Remote-Controller: 10.0.0.11 port 12346 State: UP
Last Received: 00:00:12
Remote-Controller: 10.0.0.10 port 8443 State: UP
Last Received: 00:00:10
Remote-Controller: 10.0.0.12 port 12346 State: UP
Last Received: 00:00:15
Active Tunnels: 2
TLOCs: 2
Step 3: Verify the policy is installed on the edge and view the applied policy entries
What we are doing: Confirm that the edge has the policy pushed and that the specific match/action entries are present. This tells you whether the intended rule is actually enforced by the data plane.
# On the Edge-HQ CLI
show running-config | section policy
show sdwan policy list
What just happened: The first command searches the running configuration for the policy section so you can see the exact policy stanza the device has. The second lists high-level SD‑WAN policies installed on the device. Together they reveal presence and specifics of the rule.
Real-world note: Policy names can be identical but have different sequence numbers; always inspect the full stanza on the edge to understand precedence.
Verify:
show running-config | section policy
policy
name App-Prefer-MPLS
sequence 10
match application-name HTTP-HTTPS
action prefer tloc-list MPLS_TLOC
sequence 20
match prefix 0.0.0.0/0
action prefer tloc-list INTERNET_TLOC
!
show sdwan policy list
Device: edge-hq.lab.nhprep.com
Policy: App-Prefer-MPLS Status: Installed Last Update: 2026-03-30 09:12:46
Policy: Prefix-Block-Internet Status: Installed Last Update: 2026-03-30 09:13:03
Policy: TLOC-Preference-HQ Status: Installed Last Update: 2026-03-30 09:13:21
Step 4: Confirm policy effect on traffic (App-route / policy hit counters)
What we are doing: Check app-route and policy hit counters to confirm live traffic is matching the intended policy and taking the expected TLOC (transport). This verifies not just installation but effectiveness.
# On the Edge-HQ CLI
show app-route statistics
show sdwan policy counters
What just happened: show app-route statistics summarizes application detection and routing decisions for application-aware routing. show sdwan policy counters displays per-policy/per-sequence match counts, which prove whether traffic hits the rule or bypasses it.
Real-world note: In production, these counters help prove that an SLA or routing change actually impacts user traffic; counters persist across restarts only if configured to do so.
Verify:
show app-route statistics
Application: HTTP-HTTPS
Matched Flows: 237
Preferred TLOC: MPLS_TLOC (192.0.2.5)
Last Match Time: 2026-03-30 10:02:14
Application: FACEBOOK
Matched Flows: 0
Preferred TLOC: INTERNET_TLOC (203.0.113.5)
Last Match Time: N/A
show sdwan policy counters
Policy: App-Prefer-MPLS
Sequence 10 Match Count: 237 Action: prefer MPLS_TLOC
Sequence 20 Match Count: 12 Action: prefer INTERNET_TLOC
Policy: Prefix-Block-Internet
Sequence 10 Match Count: 0 Action: drop
Step 5: Collect logs for policy evaluation debugging
What we are doing: If hits are not occurring as expected, collect buffered logs and search for policy-evaluation messages to determine why a flow did or did not match.
# On the Edge-HQ CLI
show logging
show logging | include policy
What just happened: show logging outputs the buffered logs we enabled earlier. Filtering for "policy" surfaces entries where the device evaluated policy matches and actions. These entries reveal mismatches (e.g., application not detected, incorrect prefix, or sequence precedence).
Real-world note: Buffered logs are ephemeral; for sustained troubleshooting integrate with a log sink (syslog/Splunk) and correlate with NWPI/UMTS/TE telemetry for root cause analysis.
Verify:
show logging
*Mar 30 10:00:12: PolicyEngine: Evaluating flow src=10.1.1.10 dst=8.8.8.8 proto=6 dport=80
*Mar 30 10:00:12: PolicyEngine: Matched policy App-Prefer-MPLS sequence 10 (action prefer MPLS_TLOC)
*Mar 30 10:00:12: TLOC selection: MPLS_TLOC (192.0.2.5) selected for flow
*Mar 30 10:00:12: PolicyEngine: Matched policy Prefix-Block-Internet sequence 20 (action prefer INTERNET_TLOC)
Verification Checklist
- Check 1: Policy exists in vManage — verify with
show policy liston vManage and confirm the policy name and last push time. - Check 2: Control-plane is UP — verify with
show control connectionson the edge and check controllers are listed as UP and last-received timers are recent. - Check 3: Policy installed on edge — verify with
show running-config | section policyandshow sdwan policy liston the edge. - Check 4: Traffic hits the policy — verify with
show app-route statisticsandshow sdwan policy countersfor positive match counters. - Check 5: Logs show the evaluation — verify with
show logging | include policyand correlate messages with the times of test traffic.
Common Mistakes
| Symptom | Cause | Fix |
|---|---|---|
| Policy listed in vManage but not on edge | Control-plane (vSmart/vManage) session from edge is down or intermittent | Check show control connections on edge; fix connectivity to controllers, then re-push policy from vManage |
| App-route counters remain zero despite traffic | Application not being detected (app-detection disabled or DPI not matching) | Confirm application detection is enabled and test with traffic that clearly matches signature; inspect show app-route statistics |
| Policy installed but traffic still uses wrong TLOC | A higher-priority rule or explicit route override exists | Inspect full policy stanza and sequence numbers on the edge to locate a higher-priority match; adjust sequence or rule specificity |
| Logging shows policy matched but flows not forwarded | Underlay path (TLOC) or next-hop issue on selected transport | Verify underlay reachability to TLOC IP and check TE/NWPI telemetry for underlay anomalies; use traceroutes to the TLOC |
| Policy counters reset after reboot | Counters stored in memory only | For persistent counters, configure external telemetry/logging and use persistent monitoring (Splunk, NWPI integrations) |
Key Takeaways
- Verify in order: management (vManage) → control-plane (vSmart/vBond sessions) → data-plane (edge configuration and counters). This reduces wasted effort.
- Policy presence is not enough — confirm the edge installed the same policy stanza and that policy counters or app-route statistics show active matches.
- Control-plane health is a gating factor for policy distribution; always check
show control connectionsearly when a policy appears missing. - Use logging and telemetry integrations (NWPI, TE, UMTS, ThousandEyes, Splunk) to correlate policy evaluation with underlay anomalies — in production these integrations dramatically shorten root-cause analysis.
Warning: Buffered debug logging can be verbose and impact performance if left enabled at scale. Use targeted logging for short troubleshooting windows and forward logs to an external collector for long-term analysis.
This completes Lesson 7: Policy Verification and Troubleshooting. Use these verification steps and the checklist to triage and prove policy behavior in your SD‑WAN deployments.