Chapter 3:

Network Maintenance
Tasks and Best
Practices

CCNP TSHOOT: Maintaining and Troubleshooting IP Networks

Course v7 Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 1

Chapter 3 Objectives
This chapter covers the following topics:
 Structured network maintenance
 Network maintenance processes and procedures
 Network maintenance services and tools
 Integrating troubleshooting into the network maintenance
process

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 2

© 2006, Cisco Systems, Inc. All rights reserved. 1

Presentation_ID.scr
 Structured
Network
Maintenance

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 3

Structured Network Maintenance

 Depending on the size and type of organization, some or all
of the following may be included in the maintenance tasks:
• Device Installation and Maintenance
• Failure Response
• Monitoring Network Performance
• Auditing Business Procedures
• Implementing Security Procedures and Security Auditing

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 4

© 2006, Cisco Systems, Inc. All rights reserved. 2

Presentation_ID.scr
 Structured Network Maintenance
 The structured approach to network maintenance has some
clear benefits over the interrupt-driven approach
• Reduced Network Downtime
• Cost-effectiveness
• Better Alignment with Business Objectives
• Higher Network Security

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 5

Network
Maintenance
Processes and
Procedures

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 6

© 2006, Cisco Systems, Inc. All rights reserved. 3

Presentation_ID.scr
 Network Maintenance Processes and
Procedures
 Steps to establish procedures that fit an organization’s
needs
• Identify network maintenance tasks.
• Recognize and describe the advantages of scheduled maintenance.
• Evaluate the key decision factors that affect change control
procedures
• Describe the essential elements of network documentation and its
function.
• Plan for efficient disaster recovery.
• Describe the importance of network monitoring and performance
measurement as an integral element of a proactive network
maintenance strategy.

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 7

Common Maintenance Tasks

 Accommodating adds, moves, and changes
 Installation and configuration of new devices
 Replacement of failed devices
 Backup of device configurations and software
 Troubleshooting link and device failures
 Software upgrading or patching
 Network monitoring
 Performance measurement and capacity planning
 Writing and updating documentation

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 8

© 2006, Cisco Systems, Inc. All rights reserved. 4

Presentation_ID.scr
 Network Maintenance Planning
 Scheduling maintenance
 Formalizing change-control procedures
 Establishing network documentation procedures
 Establishing effective communication
 Defining templates/procedures/conventions
 Planning for disaster recovery

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 9

Scheduling Maintenance Advantages

 Network downtime is reduced.
 Long-term maintenance tasks will not be neglected or
forgotten.
 Predictable lead times for change requests.
 Disruptive maintenance tasks can be scheduled during
assigned maintenance windows,reducing downtime during
production hours.

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 10

© 2006, Cisco Systems, Inc. All rights reserved. 5

Presentation_ID.scr
 Formalizing Change-Control Procedures
 Which types of change require authorization and who is
responsible for authorizing them?
 Which changes have to be done during a maintenance window
and which changes can be done immediately?
 What kind of preparation needs to be done before executing a
change?
 What kind of verification needs to be done to confirm that the
change was effective?
 What other actions (such as updating documentation) need to be
taken after a successful change?
 What actions should be taken when a change has unexpected
results or causes problems?
 What conditions allow skipping some of the normal change
procedures and which elements of the procedures should still be
followed?
Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 11

Establishing Network Documentation

Procedures
 Network drawings: Diagrams of the physical and logical
structure of the network
 Connection documentation: Lists of all relevant physical
connections, such as patches, connections to service
providers, and power circuits
 Equipment lists: Lists of all devices, part numbers, serial
numbers, installed software versions, software licenses (if
applicable), warranty/service information
 IP address administration: Lists of the IP subnets scheme
and all IP addresses in use
 Configurations: A set of all current device configurations or
even an archive that contains all previous configurations
 Design documentation: A document describing the
motivation behind certain implementation choices
Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 12

© 2006, Cisco Systems, Inc. All rights reserved. 6

Presentation_ID.scr
 Establishing Effective Communication
 Who is making changes and when?
 How does the change affect others?
 What are the results of tests that were done, and what
conclusions can be drawn?

If actions, test results, and conclusions are not communicated

between team members, the process in the hands of one
team member can be disruptive to the process handled by
another team member. You do not want to create new
problems while solving others.
Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 13

Defining Templates/Procedures/Conventions

 Are logging and debug time stamps set to local time or

coordinated universal time (UTC)?
 Should access lists end with an explicit “deny any”?
 In an IP subnet, is the first or the last valid IP address
allocated to the local gateway?

In many cases, you can configure a device in several different

ways to achieve the same results. However, using different
methods of achieving the same results in the same network
can easily lead to confusion, especially during
troubleshooting. Under pressure, valuable time can be
wasted in verifying configurations that are assumed incorrect
simply because they are configured differently.
Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 14

© 2006, Cisco Systems, Inc. All rights reserved. 7

Presentation_ID.scr
 Planning for Disaster Recovery
 To replace a failed device, you need the following items:
• Replacement hardware
• The current software version for the device
• The current configuration for the device
• The tools to transfer the software and configuration to the device
• Licenses (if applicable)
• Knowledge of the procedures to install software, configurations, and
licenses

In short, the key factors to a successful disaster recovery are

defining and documenting recovery procedures and making
sure that you always have the necessary elements available
in case a disaster strikes
Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 15

Network
Maintenance
Services and
Tools

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 16

© 2006, Cisco Systems, Inc. All rights reserved. 8

Presentation_ID.scr
 Fundamental Tools and Applications
in a Network Maintenance Toolkit

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 17

Network Time Services

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 18

© 2006, Cisco Systems, Inc. All rights reserved. 9

Presentation_ID.scr
 Logging Services
 Logging messages can be sent to one or more of the
following:
• Console (default)
• Monitor (vty/AUX)
• Buffer (volatile memory)
• Syslog server
• Flash memory (nonvolatile memory)
• Simple Network Management Protocol (SNMP) network management
server (as an SNMP trap)

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 19

Logging Services

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 20

© 2006, Cisco Systems, Inc. All rights reserved. 10

Presentation_ID.scr
 Logging severity levels
 (0) Emergency
 (1) Alert
 (2) Critical
 (3) Error !
 (4) Warning logging buffered level
!
 (5) Notification logging console level
 (6) Informational !
logging xxx.xxx.xxx.xxx
 (7) Debugging logging trap level

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 21

Performing Backup and Restore

FTP Configuration Backup

Store Username and Password for FTP Access

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 22

© 2006, Cisco Systems, Inc. All rights reserved. 11

Presentation_ID.scr
 Performing Backup and Restore
Configure SSH and SCP Backup

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 23

Performing Backup and Restore

Setting up an archive configuration

show archive Command

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 24

© 2006, Cisco Systems, Inc. All rights reserved. 12

Presentation_ID.scr
 Restore archived configuration

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 25

Integrating
Troubleshooting
into the Network
Maintenance
Process

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 26

© 2006, Cisco Systems, Inc. All rights reserved. 13

Presentation_ID.scr
 Integrating Troubleshooting into the Network
Maintenance Process

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 27

Network Documentation
 Network documentation is expected to include the following
items:
• Network diagrams
• Labeling interfaces and cables
• Device interconnections specifications
• Hardware and software inventory
• Addressing scheme
• Device configurations
• Design documentation

 Documentation that is wrong or outdated is often worse

than having no documentation at all.

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 28

© 2006, Cisco Systems, Inc. All rights reserved. 14

Presentation_ID.scr
 Baseline Documentation
 Basic performance statistics: Interface load for critical network links,
CPU load, and memory usage of routers and switches are essential
statistics to gather. These values can be polled and collected on a
regular basis using SNMP and graphed for visual inspection.

 Accounting of network traffic: Remote Monitoring (RMON), Network

Based Application Recognition (NBAR), or NetFlow statistics can be
used to profile different types of traffic on the network.

 Measurements of network performance characteristics: The IP SLA

feature in Cisco IOS can be used to measure critical performance
indicators such as delay and jitter across the network infrastructure.

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 29

Communication
 Communication is an essential part of the troubleshooting
process

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 30

© 2006, Cisco Systems, Inc. All rights reserved. 15

Presentation_ID.scr
 Change Control
 Change control is one of the most fundamental processes
in network maintenance.
 There is not anything different between making a change as
part of the maintenance process or as part of
troubleshooting.
 Within change-control procedures, there is always an
aspect of balancing urgency, necessity, impact, and risk.
 The troubleshooting process can benefit tremendously from
having well-defined and well-documented change
processes.

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 31

Chapter 3 Summary
 Structured network maintenance
 Network maintenance processes and procedures
 Network maintenance services and tools
 Integrating troubleshooting into the network maintenance
process

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 32

© 2006, Cisco Systems, Inc. All rights reserved. 16

Presentation_ID.scr
 Chapter 3 Labs
 Lab 3-1 Assembling Maintenance and Troubleshooting
Tools

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 33

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 34

© 2006, Cisco Systems, Inc. All rights reserved. 17

Presentation_ID.scr
 Acknowledgment
• Some of the texts and images are from Troubleshooting and Maintaining Cisco
IP Networks (TSHOOT) Foundation Learning Guide by Amir Ranjbar
(158720455X)
• Copyright © 2015 – 2016 Cisco Systems, Inc.
• Special Thanks to Bruno Silva

Chapter #
© 2007 – 2016, Cisco Systems, Inc. All rights reserved. Cisco Public 35

© 2006, Cisco Systems, Inc. All rights reserved. 18

Presentation_ID.scr