NHPREP — Hands-On CCNA, CCNP, CCIE Labs & Networking Courses

Objective

In this lesson you will implement a simple GitOps workflow for network engineering: prepare routers for pull-based automation, host network configuration in a Git repository, perform AI-style validation of intended changes, and apply validated configuration to devices. This matters in production because pull-based GitOps reduces change risk (the device pulls an approved artifact), enables audit trails (git commits + CI validation), and lets teams enforce policy consistently across many routers. Real-world scenario: a network operations team stores standardized interface and BGP templates in Git; an automated validator checks for disallowed commands and styles, then approved commits are pulled by a management host and applied to edge routers.

Quick Recap

This lab builds on the base topology used earlier (two routers). For this lesson we add a Management Host and a Git Server.

ASCII topology (exact IPs shown on every interface):

R1 and R2 are routers; MgmtHost and GitServer are Linux hosts on the management VLAN.

                +-------------+            +-------------+
                |   GitServer |            |  MgmtHost   |
                | IP:10.0.0.11|            | IP:10.0.0.10|
                +------+------+            +------+------+
                       |                          |
                       |  VLAN 10 / 10.0.0.0/24   |
                       |                          |
                +------+--------------------------+------+
                |                                       |
         +------+-----+                           +-----+------+
         |    R1      |                           |     R2     |
         | Gi0/0:     |                           | Gi0/0:     |
         | 10.0.0.1/24|                           | 10.0.0.2/24|
         +------------+                           +------------+

Device table

Device	Role	Management IP
R1	Edge Router	10.0.0.1/24
R2	Edge Router	10.0.0.2/24
MgmtHost	Pull Agent / Operator workstation	10.0.0.10/24
GitServer	Bare Git repo server	10.0.0.11/24

All examples use domain: lab.nhprep.com and password: Lab@123

Key Concepts

GitOps (pull-based automation): The device or a management agent pulls an artifact (configuration) from Git, rather than the controller pushing changes. This improves traceability and enables atomic rollbacks via commits.
- Packet/operation flow: operator commits -> CI/validator runs -> artifact is available in Git -> pull agent fetches artifact -> agent validates and applies to device.
SSH Key-based authentication and SCP/SFTP: Key-based auth allows non-interactive, secure file transfer and command execution from automation hosts. Enabling the router as an SCP/SFTP target or allowing SSH control is required for automated apply.
Validation (AI-style): A validation step inspects configs for policy violations (for example: forbidding 'no ip routing' or cleartext SNMP community strings). In production, this validation is provided by CI or an AI-review tool; here we simulate a validator script that enforces rules.
configure replace: On IOS routers, configure replace atomically replaces the running configuration from a file (forwards or rolls back as needed). This is useful for applying a validated full configuration artifact.
Auditing and immutability: Git commits are the single source of truth. In production, changes are accepted only from approved commits (signed or CI-validated), which supports audits and compliance.

Tip: Think of Git as the "contract" for your network devices — the device or its agent only accepts a contract that passed validation. This mirrors how containers are deployed from validated images.

Step-by-step configuration

Step 1: Configure management interfaces on R1 and R2

What we are doing: Assign management IPs to the routers' management-facing interfaces so the Git server and management host can reach them. Management IPs are the basis for all automation and SSH communication.

R1# configure terminal
R1(config)# interface GigabitEthernet0/0
R1(config-if)# description MGMT-to-Management-VLAN
R1(config-if)# ip address 10.0.0.1 255.255.255.0
R1(config-if)# no shutdown
R1(config-if)# exit
R1(config)# exit
R1# write memory

R2# configure terminal
R2(config)# interface GigabitEthernet0/0
R2(config-if)# description MGMT-to-Management-VLAN
R2(config-if)# ip address 10.0.0.2 255.255.255.0
R2(config-if)# no shutdown
R2(config-if)# exit
R2(config)# exit
R2# write memory

What just happened: Each router's Gi0/0 was configured with a static IPv4 address on the management network. no shutdown brings the interface up. write memory persists the configuration to NVRAM. With IPs in place, SSH and SCP from the management host will be routable.

Real-world note: In data centers, management networks are typically isolated and protected by access control lists and jump servers. You should never expose management interfaces directly to the internet.

Verify:

R1# show ip interface brief
Interface              IP-Address      OK? Method Status                Protocol
GigabitEthernet0/0     10.0.0.1        YES manual up                    up
GigabitEthernet0/1     unassigned      YES unset  administratively down down
Loopback0              unassigned      YES unset  up                    up

R2# show ip interface brief
Interface              IP-Address      OK? Method Status                Protocol
GigabitEthernet0/0     10.0.0.2        YES manual up                    up
GigabitEthernet0/1     unassigned      YES unset  administratively down down
Loopback0              unassigned      YES unset  up                    up

Step 2: Enable SSH, create automation user, and enable SCP on routers

What we are doing: Create an automation user (gitops), set the domain name, generate RSA keys for SSH, enable SSH version 2, and enable the router's SCP server so the management host can transfer files. This is required for secure, non-interactive automation.

R1# configure terminal
R1(config)# hostname R1
R1(config)# ip domain-name lab.nhprep.com
R1(config)# username gitops privilege 15 secret Lab@123
R1(config)# ip ssh version 2
R1(config)# crypto key generate rsa modulus 2048
R1(config)# ip scp server enable
R1(config)# exit
R1# write memory

R2# configure terminal
R2(config)# hostname R2
R2(config)# ip domain-name lab.nhprep.com
R2(config)# username gitops privilege 15 secret Lab@123
R2(config)# ip ssh version 2
R2(config)# crypto key generate rsa modulus 2048
R2(config)# ip scp server enable
R2(config)# exit
R2# write memory

What just happened: username gitops ... creates an automation account with full privileges (privilege 15) and secure secret. ip ssh version 2 and crypto key generate rsa enable SSHv2 and create host keys required for SSH. ip scp server enable allows the device to accept SCP transfers (so artifacts can be uploaded). These steps let the management host securely push validated configs.

Real-world note: In production, prefer certificate-based authentication (e.g., TACACS+/AAA) for central control of privileges. Key-based access removes the need to store plain passwords.

Verify:

R1# show running-config | section username
username gitops privilege 15 secret 5 $1$PdQ9$EXAMPLESECRETSTRING

R1# show ip ssh
SSH Enabled - version 2.0
Authentication timeout: 120 secs; Authentication retries: 3
R1# show ip scp server
SCP server is enabled

Step 3: Install the automation SSH public key on the routers

What we are doing: Add the management host's public SSH key into the router's ip ssh pubkey-chain for the gitops user so the pull agent can perform key-based authentication (non-interactive). This avoids storing a password in scripts and allows secure automated connections.

R1# configure terminal
R1(config)# ip ssh pubkey-chain
R1(cfg-ssh-pubkey)# username gitops
R1(cfg-ssh-pubkey-user)# key-string
R1(cfg-ssh-pubkey-data)# ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCexamplePublicKeyDataForLab
R1(cfg-ssh-pubkey-data)# exit
R1(cfg-ssh-pubkey-user)# exit
R1(cfg-ssh-pubkey)# exit
R1(config)# exit
R1# write memory

R2# configure terminal
R2(config)# ip ssh pubkey-chain
R2(cfg-ssh-pubkey)# username gitops
R2(cfg-ssh-pubkey-user)# key-string
R2(cfg-ssh-pubkey-data)# ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCexamplePublicKeyDataForLab
R2(cfg-ssh-pubkey-data)# exit
R2(cfg-ssh-pubkey-user)# exit
R2(cfg-ssh-pubkey)# exit
R2(config)# exit
R2# write memory

What just happened: The ip ssh pubkey-chain stanza registers the public key for the gitops user. When the management host connects with its matching private key, the router authenticates without prompting for a password. This is the recommended automation authentication method.

Real-world note: Keep private keys on a secure, access-controlled machine (like an automation controller or vault); protect their permissions (600) and rotate keys periodically.

Verify:

R1# show running-config | section ip ssh pubkey-chain
ip ssh pubkey-chain
 username gitops
  key-string
   ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCexamplePublicKeyDataForLab
R2# show running-config | section ip ssh pubkey-chain
ip ssh pubkey-chain
 username gitops
  key-string
   ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCexamplePublicKeyDataForLab

Step 4: Initialize the Git repository on the GitServer and add device config artifacts

What we are doing: Create a bare Git repository on the Git server and add per-device intended configuration artifacts (templates). These artifacts are the canonical source of truth for routers R1 and R2.

# On GitServer (10.0.0.11)
git init --bare /srv/gitops/network-configs.git

# On MgmtHost (10.0.0.10) clone the bare repo
git clone git@10.0.0.11:/srv/gitops/network-configs.git
cd network-configs

# Create R1 and R2 config artifacts
cat > R1-config.txt << EOF
hostname R1
interface GigabitEthernet0/0
 description MGMT-to-Management-VLAN
 ip address 10.0.0.1 255.255.255.0
 no shutdown
!
EOF

cat > R2-config.txt << EOF
hostname R2
interface GigabitEthernet0/0
 description MGMT-to-Management-VLAN
 ip address 10.0.0.2 255.255.255.0
 no shutdown
!
EOF

# Commit and push
git add R1-config.txt R2-config.txt
git commit -m "Add base management configs for R1 and R2"
git push origin master

What just happened: A bare repository was created on the Git server. The management host cloned, added configuration artifacts representing the desired running configuration for each router, committed them, and pushed to the Git server. These artifacts are now the authoritative configurations to be validated and applied.

Real-world note: In production you would place templates (e.g., Jinja2) and parameterize them for per-site differences. You would also enable branch protections and require protected merges via CI.

Verify:

# On GitServer
ls -la /srv/gitops/network-configs.git
total 36
drwxr-xr-x  8 git git 4096 Apr  2 10:00 .
drwxr-xr-x  3 root root 4096 Apr  2 09:59 ..
-rw-r--r--  1 git git   23 Apr  2 10:00 HEAD
drwxr-xr-x  2 git git 4096 Apr  2 10:00 objects
drwxr-xr-x  3 git git 4096 Apr  2 10:00 refs
-rw-r--r--  1 git git  123 Apr  2 10:00 config

# On MgmtHost verify commit history
cd network-configs
git log --oneline
e3b0c442 Add base management configs for R1 and R2

Step 5: Pull, validate (AI-style), and apply validated config to routers

What we are doing: On the MgmtHost simulate the GitOps pull agent: clone/pull the repository, run a validator script to reject dangerous commands, then upload the validated artifact to each router via SCP and apply with configure replace to atomically change running-config. This step shows the full pull-validate-apply cycle.

# On MgmtHost: pull latest from Git
cd ~/network-configs
git pull origin master

# Simple validator script (simulates AI validation; enforces policy)
cat > validator.sh << 'EOF'
#!/bin/bash
# Rule: forbid 'no ip routing' and forbid 'snmp-server community public'
for file in *.txt; do
  if grep -q -E "no ip routing|snmp-server community public" "$file"; then
    echo "Validation FAILED for $file"
    exit 1
  fi
done
echo "Validation PASSED"
EOF
chmod +x validator.sh

# Run validator
./validator.sh

# Upload validated file to routers using scp (key-based auth via gitops user)
scp R1-config.txt gitops@10.0.0.1:flash:R1-config.txt
scp R2-config.txt gitops@10.0.0.2:flash:R2-config.txt

# Apply on the routers using SSH to invoke configure replace
ssh gitops@10.0.0.1 "configure replace flash:R1-config.txt force"
ssh gitops@10.0.0.2 "configure replace flash:R2-config.txt force"

What just happened: The pull agent updated its local copy and ran a simple validator script that rejects files containing forbidden commands. After passing validation, the agent uploaded artifacts to each router's flash via SCP and used configure replace to atomically replace the running configuration from the uploaded file (force bypasses confirmation in lab contexts). This models a pull-based automated apply after validation.

Real-world note: The validator here is a small policy check. In production your validator could be a CI pipeline invoking policy-as-code, linter tools, and even an AI model that checks intent. The configure replace approach provides a clear atomic swap and an easy rollback path (keep previous config in startup-config or via commit history).

Verify:

# On R1: confirm the file exists in flash and check hostname
R1# dir flash:
Directory of flash:/

    1024  -rw-     1234  Apr 02 10:05:00 +00:00  R1-config.txt
    2048  -rw-     4096  Apr 02 09:50:00 +00:00  some-other-file.txt
1205568 bytes total (512000 bytes free)

R1# show running-config | include hostname
hostname R1

# On R2: confirm file and hostname
R2# dir flash:
Directory of flash:/

    1024  -rw-     1210  Apr 02 10:05:10 +00:00  R2-config.txt
1205568 bytes total (512000 bytes free)

R2# show running-config | include hostname
hostname R2

# On MgmtHost validate output
./validator.sh
Validation PASSED

Verification Checklist

Check 1: Management network IPs reachable — ping each router from MgmtHost (ping 10.0.0.1, ping 10.0.0.2) and expect successful responses.
Check 2: SSH key-based login works — on MgmtHost run ssh gitops@10.0.0.1 and expect a shell without password prompt (key auth).
Check 3: Git artifacts present on GitServer — verify git log on cloned repo shows the commit(s) you created.
Check 4: Validator passes for committed artifacts — ./validator.sh should output Validation PASSED.
Check 5: Config applied — after configure replace, show running-config | include hostname should reflect the new hostname or configuration elements from the artifact.

Common Mistakes

Symptom	Cause	Fix
SSH still prompts for password from MgmtHost	Public key not installed or file permissions on private key are wrong	Ensure the public key is present in `ip ssh pubkey-chain` for the username; set private key on MgmtHost to 600 and use correct ssh key when connecting
SCP fails with "permission denied"	`ip scp server enable` not configured or username lacks privileges	Enable `ip scp server enable` on router and ensure `username gitops` exists with privilege 15 or appropriate roles
Validation script rejects valid config	Validator has overly strict regex or incorrect rules	Review validator logic; test validator locally against sample files before automating
`configure replace` prompts for confirmation or fails	Wrong file path or syntax (`flash:` path missing)	Verify file exists with `dir flash:` and use correct `configure replace flash:filename force` syntax
Git push fails with permission denied	SSH key not installed on GitServer or Git user misconfigured	Ensure GitServer has the MgmtHost's public key in authorized_keys for the git user and that repo permissions allow push

Key Takeaways

GitOps shifts the network change model from ad-hoc pushes to an auditable, pull-based model where the device or agent pulls validated artifacts; this reduces drift and improves traceability.
Prepare devices for automation: management IPs, SSH (key-based) auth, and SCP/SFTP support are fundamental prerequisites.
Validation is critical — even simple policy checks prevent dangerous misconfigurations. In production, validators are part of CI and may incorporate linting and AI-assisted checks.
configure replace is a powerful mechanism for atomic config swaps on IOS; combined with Git commits and SCP transfer it forms a clean apply/rollback workflow.

Final note: In a production environment implement stricter controls: branch protections, signed commits, hardened automation hosts, centralized AAA/TACACS+, and thorough testing pipelines. The patterns shown here scale into those practices and are the foundation of enterprise GitOps for networking.

GitOps for Network Engineering

Objective

Quick Recap

Key Concepts

Step-by-step configuration

Step 1: Configure management interfaces on R1 and R2

Step 2: Enable SSH, create automation user, and enable SCP on routers

Step 3: Install the automation SSH public key on the routers

Step 4: Initialize the Git repository on the GitServer and add device config artifacts

Step 5: Pull, validate (AI-style), and apply validated config to routers

Verification Checklist

Common Mistakes

Key Takeaways