Verifying and Monitoring Segmentation
Objective
In this lesson you will verify SGT assignment and SGACL enforcement, and learn how to monitor segmentation policy hits across the fabric. This matters in production because enforcement is only useful if identity (SGT) is correctly applied to traffic and the correct policies are actually matching — otherwise lateral movement or incorrect access can go undetected. Real-world scenario: after onboarding users and devices into TrustSec and pushing SGACLs to enforcement points, operations must confirm SGTs are present on traffic seen at the data-center edge and that SGACL counters reflect expected policy hits during application access.
Quick Recap
This is Lesson 7 of 7 for Lab 46: End-to-End Segmentation — TrustSec/SGT. We continue with the same topology used in earlier lessons. No new devices are added in this lesson.
ASCII topology (interfaces and IPs are shown on every link):
[Host-A] [Host-B]
IP: 10.0.10.10/24 IP: 10.0.20.10/24
MAC: 0001.0001.0001 MAC: 0002.0002.0002
| |
| Gi0/1 | Gi0/2
| |
[SW1 - Catalyst 9500]-----------------[R1 - Border Router]
Mgmt: 192.168.1.10/24 Gi0/0: 10.0.10.254/24
VLAN10 SVI: 10.0.10.1/24 Gi0/1: 10.0.20.254/24
VLAN20 SVI: 10.0.20.1/24
Device table
| Device | Hostname | Management IP |
|---|---|---|
| Switch | SW1 (Cat 9500) | 192.168.1.10 |
| Router | R1 (Border) | (in-band only) |
Note: Host IPs, SVI IPs and interface names match the lab topology used in earlier lessons.
Key Concepts
Before using the CLI, understand these points so verification makes sense:
- SGT (Security Group Tag): a packet tag or metadata that identifies the security group of the source. On access-layer switches this is applied at ingress (based on 802.1X, MAB, or static mapping) and must be present when packets reach enforcement points (switch ASIC or firewall).
- Protocol behavior: when a host authenticates via 802.1X and ISE returns an SGT, the switch applies the SGT to the MAC/IP entry and forwards packets with that identity for enforcement.
- SGACL (Security Group ACL): a policy that permits/denies traffic based on SGT-to-SGT relationships. Enforcement is performed by the forwarding device (switch or firewall) using hardware TCAM or firewall policy.
- Packet flow: packet ingress -> SGT lookup -> SGACL lookup -> permit/deny decision -> counters increment.
- IP-SGT mapping and SXP: when a device (like a firewall) does not support inline tagging, an IP-to-SGT mapping can be learned from ISE and propagated by SXP to network devices. Verification requires checking the SXP bindings.
- Policy hit counters: counters (ACL or SGACL) are the primary operational signal that policies are being used. They are used in troubleshooting to confirm traffic is matching intended rules.
Real-world analogy: Think of SGTs as "colored badges" assigned to users; SGACLs check the badge color before admitting someone through a door. Counters are the door's clicker to count who actually passed.
Step-by-step configuration and verification
Step 1: Verify SGT assignment on SW1
What we are doing: Retrieve the local SGT assignment and cache on the access switch so we can confirm Host-A's identity (SGT) is applied. This confirms that authentication/authorization completed and the switch has linked the SGT to the host MAC/IP.
SW1# show cts role-based
What just happened: The command reads the switch's CTS (TrustSec) role-based table showing which endpoints have SGTs assigned and which roles are currently active. This output displays MAC/IP bindings and their associated SGT. On an operational switch, when Host-A authenticates, the switch caches the IP-to-SGT binding so enforcement and SXP propagation can use it.
Real-world note: If SGT does not appear here after successful 802.1X, check ISE authentication logs and switch RADIUS configuration.
Verify:
SW1# show cts role-based
Role-based SGT table
Endpoint: 10.0.10.10 MAC: 0001.0001.0001 SGT: 101 Source: local-auth
Endpoint: 10.0.20.10 MAC: 0002.0002.0002 SGT: 200 Source: local-auth
Number of entries: 2
- Expected: Host-A (10.0.10.10) appears with SGT 101 (example SGT). If missing, the SGT is not applied and SGACL enforcement will not operate as intended.
Step 2: Verify IP-to-SGT mappings (SXP bindings) on R1
What we are doing: Check SXP (SGT Exchange Protocol) bindings on the border/router to make sure devices that enforce without inline tags have the correct IP-to-SGT mappings. This ensures remote enforcement points can map IPs to SGT for policy decisions.
R1# show sxp bindings
What just happened: The command shows IP-to-SGT bindings learned via SXP. Routers/switches that receive SXP messages will populate local tables so that downstream enforcement devices (or the router itself) can make SGT-based decisions. If SXP bindings are stale or absent, SGACL enforcement based on IP-SGT mapping will fail.
Real-world note: In deployments with third-party firewalls that can't tag SGT inline, SXP helps the network tag traffic based on ISE mappings.
Verify:
R1# show sxp bindings
SXP Bindings
IP Address SGT Source
10.0.10.10 101 ISE-Export
10.0.20.10 200 ISE-Export
Total bindings: 2
- Expected: The binding for 10.0.10.10 -> SGT 101 exists and is sourced from ISE export.
Step 3: Verify SGACL enforcement counters on SW1 (ACL hit counts)
What we are doing: Inspect the SGACL/ACL counters that implement TrustSec enforcement on the switch to see if traffic from Host-A to an SGACL-protected resource is hitting the expected rules. This verifies behavioral enforcement and provides counters for monitoring.
SW1# show ip access-lists
What just happened: On many platforms, SGACLs manifest as specialized ACLs or as system policies with counters. The command lists access-lists and their per-line counters. When traffic matches an SGACL, the corresponding access-list line counter increments, which proves enforcement.
Real-world note: SGACLs should be applied in hardware and counters help validate hardware enforcement vs. software fallback.
Verify:
SW1# show ip access-lists
Extended IP access list SGACL-DB-WEB
10 permit ip 10.0.10.0 0.0.0.255 172.16.100.0 0.0.0.255 (hitcnt=485)
20 deny ip any any (hitcnt=0)
Extended IP access list SGACL-DC-ANY
10 permit ip 10.0.20.0 0.0.0.255 172.16.200.0 0.0.0.255 (hitcnt=120)
- Expected: The line corresponding to Host-A's traffic (or its subnet) shows non-zero hit counts (e.g., hitcnt=485), proving matching and enforcement occurred.
Step 4: Verify enforcement using hardware counters specific to CTS (role-based counters)
What we are doing: Display CTS enforcement hardware counters to confirm the device is using SGT information in hardware and to see policy hit statistics tracked by the TrustSec module.
SW1# show cts counters
What just happened: The switch returns counters for CTS enforcement: the number of packets permitted or denied by SGACLs, and hits per SGT-rule. This shows whether enforcement is happening in silicon (fast-path) and helps locate whether hits are occurring on intended rules.
Real-world note: If counters remain zero while traffic is flowing, the policy may be misapplied or compiled out of hardware due to TCAM exhaustion.
Verify:
SW1# show cts counters
CTS Enforcement Counters
SGT 101 -> SGT 300 : permit (hits=485)
SGT 200 -> SGT 300 : deny (hits=0)
Total permit hits: 485
Total deny hits: 0
- Expected: The SGT pair that should be allowed shows permit hits; if you expect denies on a flow, the deny counter should increment.
Step 5: Monitor segmentation policy hits over time (logging and summaries)
What we are doing: Use logging and summary counters to observe trends — confirming whether access attempts continue and whether any unexpected denies occur. This is an operational step to move from point-in-time checks to ongoing monitoring.
SW1# terminal length 0
SW1# show logging
SW1# show ip access-lists | include hitcnt
What just happened: The logging command scrolls the system log buffer where SGACL denies or ISE failures may appear. The piped ACL display filters for hit counts to summarize policy usage. Together these help identify anomalies and audit enforcement over time.
Real-world note: Correlate counters with ISE logs to understand why denies occur — for example, stale IP-SGT mapping or misclassified posture.
Verify:
SW1# show logging
Syslog logging: enabled (0 messages dropped, 0 messages rate-limited, 0 flushes, 0 overruns)
Console logging: level debugging
Monitor logging: level informational, 0 messages logged
Buffer logging: level informational, 10 messages logged
*Mar 1 10:12:00: %CTS-6-ENFORCE: Packet from 10.0.20.10 (0002.0002.0002) denied by SGACL rule 20
*Mar 1 10:12:05: %CTS-6-ENFORCE: Packet from 10.0.10.10 (0001.0001.0001) permitted by SGACL rule 10
SW1# show ip access-lists | include hitcnt
10 permit ip 10.0.10.0 0.0.0.255 172.16.100.0 0.0.0.255 (hitcnt=485)
- Expected: Logging contains entries for permit/deny matches and hitcnt reflects ongoing matches.
Verification Checklist
- Check 1: Host-A has an SGT assigned. Verify with
show cts role-basedand confirm the entry for 10.0.10.10 lists SGT 101. - Check 2: SXP bindings on border device include the host IP -> SGT mapping. Verify with
show sxp bindings. - Check 3: SGACL/ACL hit counters show non-zero hits for expected traffic. Verify with
show ip access-listsandshow cts counters. - Check 4: System logs show SGACL permit/deny events for the host. Verify with
show logging.
Common Mistakes
| Symptom | Cause | Fix |
|---|---|---|
Host does not appear in show cts role-based | Authentication/authorization failed, or switch did not get SGT from ISE | Verify 802.1X/RADIUS settings, confirm ISE returned SGT in the authorization only, re-run auth |
| Hit counters remain zero despite traffic | SGACL not applied to the correct interface/VRF or compiled out of hardware (TCAM full) | Check ACL/SGACL application points, inspect TCAM utilization; move rules to hardware-friendly forms |
Wrong SGT in show sxp bindings | ISE exported an incorrect mapping or stale mapping persisted | Force refresh from ISE, clear stale SXP binding, validate ISE identity-source attributes |
| Logs show permitted but expected denies | Policy logic reversed or ACL order incorrect | Verify SGACL logic and rule order; check that the SGT direction (source->destination) matches rule design |
Key Takeaways
- Always verify that SGTs are present in the access switch cache (
show cts role-based) before concluding enforcement will work — tagging is a prerequisite for SGACLs. - SGACL enforcement must be validated with counters (ACL hitcnt and CTS counters); counters are the operational evidence that policies are applied and hit in hardware.
- IP-to-SGT mappings via SXP are critical where inline tagging is not possible; verify SXP bindings on border devices to ensure consistent enforcement.
- Logging and periodic counter checks turn point-in-time validation into ongoing monitoring; correlate switch counters with ISE logs to troubleshoot mapping and policy issues.
Tip: In production, integrate SGACL counters and CTS logs into a central monitoring system so policy regressions or spikes in denied traffic are alerted automatically.
If you completed the verification steps and checks above, you have confirmed that SGTs are applied, SGACLs are enforced, and policy hits are being monitored — three pillars required to operationalize TrustSec segmentation in a production environment.