NHPREP — Hands-On CCNA, CCNP, CCIE Labs & Networking Courses

Objective

In this lesson you will configure dial-out (device-initiated) model-driven telemetry on Catalyst 9000 devices using the gRPC transport. We will make the leaf switches (c9k-leaf1 and c9k-leaf2) initiate telemetry connections to a central collector (c9k-spine) and stream interface operational-state data. In production, device-initiated telemetry is used to scale monitoring: devices push high-frequency, structured YANG data to collectors so operations teams can monitor health and performance without repeatedly polling each device.

Real-world scenario: NHPREP operates a leaf-spine fabric where each leaf must stream interface counters and operational state to a central telemetry pipeline (the TIG_MDT stack running on the collector) for real-time dashboards and alerting.

Quick Recap

This lab continues from Lesson 1. We will use the same topology and IP addresses from the reference. No new device IPs are added.

ASCII topology (VLAN1 management network). Exact IPs are shown on each management interface:

        +-----------------+                 +------------------+
        |   c9k-spine     |                 |   c9k-leaf2      |
        |  (collector)    |                 |                  |
        |  IP: 198.18.1.21|-----------------|  IP: 198.18.1.32 |
        +-----------------+   VLAN1 (mgmt)  +------------------+
                 |
                 |
                 | VLAN1 (mgmt)
                 |
        +-----------------+
        |   c9k-leaf1     |
        |  IP: 198.18.1.31|
        +-----------------+

Device login credentials for examples are developer / C1sco12345 when accessing devices directly; configuration uses admin-level commands.

Key Concepts

Dial-out vs Dial-in: In dial-out, the device initiates a transport connection to a collector and streams telemetry (push model). In dial-in, an external system connects to the device (e.g., NETCONF/gNMI) and requests data (pull model). Dial-out is preferred for high-frequency telemetry because it reduces collector polling load and firewall complexity.
gRPC Transport: gRPC provides an efficient, multiplexed, binary transport (over HTTP/2) used for telemetry dial-out. Devices open a persistent gRPC connection to the collector and stream telemetry messages encoded according to the configured encoding (e.g., self-describing GPB).
Subscription Components: A telemetry subscription ties together (a) the sensor path/YANG data to collect, (b) an encoding and sampling/interval, and (c) a destination (destination-group) where the device will push data. The device maintains state for each subscription and reports status.
Protocol behavior and packet flow: When a subscription is active, the device performs a TCP handshake to the collector, negotiates TLS/mTLS if configured, and streams telemetry messages asynchronously. Periodic subscriptions cause the device to sample data and send messages at the configured interval. The device also sends heartbeat/status messages and will attempt reconnect on failure.
Operational considerations: Telemetry increases CPU and network usage depending on sampling rates and number of subscriptions. In production pick sensible intervals, use filtering to limit fields, and secure transport (mTLS + DNS) for authentication.

Tip: Use the TIG_MDT stack on your collector to decode self-describing GPB telemetry and feed Grafana. In production, collectors validate TLS client certs (mTLS) — we demonstrate basic dial-out first, then add security in later lessons.

Step-by-step configuration

Step 1: Verify collector reachability from the leaf

What we are doing: Ensure the leaf devices can reach the collector (c9k-spine at 198.18.1.21) over the management network. Telemetry dial-out requires IP reachability and potentially firewall rules allowing outbound gRPC/TCP.

c9k-leaf1# ping 198.18.1.21

What just happened: The ping tests basic IP connectivity. If ping fails, telemetry will never establish because the device cannot initiate the TCP connection to the collector. This is the first, critical connectivity check.

Real-world note: In production, collectors often sit behind load balancers and firewall rules. Ensure outbound telemetry ports are allowed from devices to collectors.

Verify:

c9k-leaf1# ping 198.18.1.21
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 198.18.1.21, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/4/10 ms

Step 2: Configure destination-group pointing to the collector on c9k-leaf1

What we are doing: Create a telemetry destination-group that points to the collector IP. The destination-group tells the device where to initiate the gRPC connection for dial-out. This binding is required so subscriptions can reference the destination.

c9k-leaf1# configure terminal
c9k-leaf1(config)# telemetry model-driven
c9k-leaf1(config-telemetry)# destination-group TG1
c9k-leaf1(config-telemetry-dest)# destination 198.18.1.21
c9k-leaf1(config-telemetry-dest)# transport grpc
c9k-leaf1(config-telemetry-dest)# exit
c9k-leaf1(config-telemetry)# exit
c9k-leaf1(config)# end

What just happened:

telemetry model-driven enters the model-driven telemetry configuration context.
destination-group TG1 creates a logical destination-group named TG1.
destination 198.18.1.21 records the collector IP within TG1.
transport grpc configures the gRPC transport type for the destination. This instructs the device to initiate gRPC connections to 198.18.1.21 when subscriptions reference TG1.

Real-world note: For production, add port and TLS/mTLS parameters to the destination so the collector can authenticate devices (covered in a later lesson).

Verify:

c9k-leaf1# show telemetry model-driven destination-group
Destination-group: TG1
  Destination: 198.18.1.21
  Transport: grpc
  State: down
  Last state change time: 00:00:00

Step 3: Create a periodic subscription on c9k-leaf1 for interface operational data

What we are doing: Define a subscription named SUB-INT that samples interface oper-data (interfaces YANG) periodically and sends it to destination-group TG1. This subscription determines WHAT data the device will push and HOW OFTEN.

c9k-leaf1# configure terminal
c9k-leaf1(config)# telemetry model-driven
c9k-leaf1(config-telemetry)# subscription SUB-INT
c9k-leaf1(config-telemetry-sub)# sensor-path Cisco-IOS-XE-interfaces-oper:interfaces/interface
c9k-leaf1(config-telemetry-sub)# encoding self-describing-gpb
c9k-leaf1(config-telemetry-sub)# sample-interval 10000
c9k-leaf1(config-telemetry-sub)# destination-id TG1
c9k-leaf1(config-telemetry-sub)# exit
c9k-leaf1(config-telemetry)# exit
c9k-leaf1(config)# end

What just happened:

subscription SUB-INT creates a subscription object.
sensor-path selects the YANG model path for interface operational state (the device will collect interface entries).
encoding self-describing-gpb chooses the efficient GPB encoding with schema embedded (useful for TIG stack).
sample-interval 10000 sets a 10,000 ms (10 s) periodic interval for sampling.
destination-id TG1 tells the device to send subscription data to the previously configured destination-group TG1. When active, the device will sample interface state every 10 seconds and push the data.

Real-world note: For high cardinality or high-frequency metrics, test the CPU and link utilization — sampling many paths at small intervals can overwhelm devices.

Verify:

c9k-leaf1# show telemetry model-driven subscription all
Subscription Name: SUB-INT
  Sensor Path: Cisco-IOS-XE-interfaces-oper:interfaces/interface
  Encoding: self-describing-gpb
  Periodic: sample-interval 10000 ms
  Destination: TG1
  State: configured
  Last telemetry transmit: never

Step 4: Activate and confirm the dial-out session on c9k-leaf1

What we are doing: Allow the device to establish the gRPC connection and verify the session state. After creating the subscription and destination, the device will try to open the transport to the collector.

c9k-leaf1# show telemetry model-driven session all
Session ID: 1
  Subscription: SUB-INT
  Destination: 198.18.1.21
  Transport: grpc
  State: establishing
  Last connection attempt: 00:00:03 ago

c9k-leaf1# show telemetry model-driven destination-group
Destination-group: TG1
  Destination: 198.18.1.21
  Transport: grpc
  State: up
  Last state change time: 00:00:12

What just happened: The show telemetry model-driven session all output shows the device attempting to establish a session for the SUB-INT subscription. The destination-group state moving to up indicates the TCP/gRPC connection was successfully established. Once up, telemetry messages will flow according to the subscription interval.

Real-world note: If the session remains in establishing or down for long, check collector listener, firewall, and TLS parameters.

Verify:

c9k-leaf1# show telemetry model-driven subscription all
Subscription Name: SUB-INT
  Sensor Path: Cisco-IOS-XE-interfaces-oper:interfaces/interface
  Encoding: self-describing-gpb
  Periodic: sample-interval 10000 ms
  Destination: TG1
  State: active
  Last telemetry transmit: 00:00:05 ago

c9k-leaf1# show telemetry model-driven destination-group
Destination-group: TG1
  Destination: 198.18.1.21
  Transport: grpc
  State: up
  Last state change time: 00:00:12

Step 5: Repeat configuration on c9k-leaf2

What we are doing: Apply the same destination-group and subscription on c9k-leaf2 so both leaves stream telemetry to the collector. This demonstrates scaling the same subscription across multiple devices.

c9k-leaf2# configure terminal
c9k-leaf2(config)# telemetry model-driven
c9k-leaf2(config-telemetry)# destination-group TG1
c9k-leaf2(config-telemetry-dest)# destination 198.18.1.21
c9k-leaf2(config-telemetry-dest)# transport grpc
c9k-leaf2(config-telemetry-dest)# exit
c9k-leaf2(config-telemetry)# subscription SUB-INT
c9k-leaf2(config-telemetry-sub)# sensor-path Cisco-IOS-XE-interfaces-oper:interfaces/interface
c9k-leaf2(config-telemetry-sub)# encoding self-describing-gpb
c9k-leaf2(config-telemetry-sub)# sample-interval 10000
c9k-leaf2(config-telemetry-sub)# destination-id TG1
c9k-leaf2(config-telemetry-sub)# exit
c9k-leaf2(config-telemetry)# exit
c9k-leaf2(config)# end

What just happened: Leaf2 now has an identical telemetry configuration and will initiate its own gRPC session to 198.18.1.21. Each device independently manages its session and will retransmit on failure.

Real-world note: Use consistent subscription names and sensor-paths across devices for easier collector-side parsing and downstream indexing.

Verify:

c9k-leaf2# show telemetry model-driven subscription all
Subscription Name: SUB-INT
  Sensor Path: Cisco-IOS-XE-interfaces-oper:interfaces/interface
  Encoding: self-describing-gpb
  Periodic: sample-interval 10000 ms
  Destination: TG1
  State: active
  Last telemetry transmit: 00:00:07 ago

c9k-leaf2# show telemetry model-driven destination-group
Destination-group: TG1
  Destination: 198.18.1.21
  Transport: grpc
  State: up
  Last state change time: 00:00:15

Verification Checklist

Check 1: Collector reachability from each leaf. Verify with ping 198.18.1.21 on c9k-leaf1 and c9k-leaf2 — expect 100% success.
Check 2: Destination-group created and in up state. Verify with show telemetry model-driven destination-group — expect State: up and correct destination IP.
Check 3: Subscription is active and telemetry messages are being transmitted. Verify with show telemetry model-driven subscription all — expect State: active and recent Last telemetry transmit timestamps.

Common Mistakes

Symptom	Cause	Fix
Destination-group stays `down`	Collector not listening on the expected transport/port or firewall blocking outbound traffic	Verify collector listener (TIG_MDT) is running; open outbound gRPC/TCP ports; ensure IP 198.18.1.21 is reachable
Subscription remains `configured` but not `active`	Destination-group misconfigured or destination-id mismatch	Ensure subscription's `destination-id` matches the destination-group name; verify `show telemetry model-driven destination-group`
High CPU or interface utilization after enabling telemetry	Too-short sample-interval or overly broad sensor-path	Increase `sample-interval`, narrow sensor-path, or offload collection to a dedicated telemetry interface/VLAN
Collector receives unreadable messages	Encoding mismatch (collector expects different encoding)	Configure the same encoding on both device and collector, e.g., `self-describing-gpb` for default TIG stack

Key Takeaways

Dial-out (device-initiated) telemetry using gRPC scales monitoring by allowing devices to push structured YANG data to central collectors rather than collectors polling devices.
A telemetry subscription ties a sensor path, encoding, interval, and destination-group together. The destination-group defines the collector endpoint and transport (gRPC).
Verify connectivity and destination-group state first. The device must be able to open the transport (TCP/gRPC) to the collector for telemetry to work.
In production use secured transport (mTLS + DNS) and sensible sampling rates to avoid overwhelming devices and networks. Use the TIG_MDT stack on the collector to decode self-describing GPB streams and feed dashboards.

Warning: Telemetry can generate significant traffic and CPU load if misconfigured. Start with conservative sampling intervals and gradually increase frequency while monitoring device resource utilization.

End of Lesson 4: Dial-Out Telemetry. In the next lesson we will secure telemetry using TLS/mTLS and integrate with the TIG_MDT collector pipeline.