The Auvik collector is equipped with a Linux shell that can capture data about your network, devices, or collector to help Auvik diagnose issues.
If you run any of the following commands, send the resulting information to Auvik support for data analysis, or include it on a support ticket that needs follow-up or resolution.
- Enable collector shell
- Diagnose discovery scanning
- Diagnose discovery services
- Capture and send collector logs
How to enable the collector shell
Depending on the collector you have running, you may need to complete different actions for enabling the shell.
Virtual collector (OVA or scripted)
- From the virtual machine window for the collector, click onto the console.
- Select Option
6and enteryto enter the bash shell from the collector menu. - Type
sudo /usr/share/agent/bin/agent_shell
Windows service
- From the start menu, search for Command Prompt (cmd).
- Right-click and select Run as administrator.
- From the command prompt window, change the directory to the install location. For example:
cd c:\auvik - Type
AuvikAgentService -shell
How to diagnose discovery scanning
Where are all the devices?
Things to be answered with these diagnostics: What does the health check ping say about this device? Does it find the device? If so, what is the ping latency? If it doesn’t find the device, it's unlikely the Auvik collector will find the device no matter what we do.
Pick a device with a known IP address that you know is currently online. Next, try a scan of the subnet from the diagnostic shell. This will serve as a baseline for scan performance.
For example:
auvik > scan 10.0.40.0 24 Scan result for 10.0.40.0/24 10.0.40.1 10.0.40.110 10.0.40.123 10.0.40.198 auvik > ping 10.0.40.110 10.0.40.110 responded to ping auvik > portOpen 10.0.40.110 80 10.0.40.110:80 port is not open auvik > portScan 10.0.40.110 80 No ports found to be open
Try repeating the scan a couple of times to see if hosts magically start appearing. Contact Auvik support for further analysis of what you’re seeing.
Is a scan killing the network?
There are two parameters that control how fast Auvik sends out ICMP packets:
ScanService.max-simultaneous-ping
ScanService.simultaneous-send-delay
Both default to 10, meaning we send at most 10 packets within any particular 10ms window. Increasing the delay or decreasing the maximum number of sends will slow the scan down.
How to diagnose discovery services
SNMP
The most common answers have to do with devices not being configured correctly, or incorrect credentials being used. The diagnostic shell has a few debug commands that can help to understand SNMP issues:
snmpcred sets the credentials that will be used by the other commands.
For SNMP v1/2c:
|
|
configures snmp v2c community authentication with community string "private" |
|
|
configures snmp v1 community authentication with community string "private" |
For SNMP v3, with the auth protocol of MD5:
|
|
configures snmp v3 USM authentication with authentication hash algorithm MD5 and authentication passphrase "md5password", and privacy cipher DES and privacy passphrase "des56password" |
|
|
configures snmp v3 USM authentication with authentication hash algorithm MD5 and authentication passphrase "md5password", and privacy cipher 3DES-EDE and privacy passphrase "desedepassword" |
|
|
configures snmp v3 USM authentication with authentication hash algorithm MD5 and authentication passphrase "md5password", and privacy cipher AES-192 and privacy passphrase "aes192password" |
|
|
configures snmp v3 USM authentication with authentication hash algorithm MD5 and authentication passphrase "md5password", and privacy cipher AES-256 and privacy passphrase "aes265password" |
For SNMP v3, with the auth protocol of SHA:
|
|
configures snmp v3 USM authentication with authentication hash algorithm SHA and authentication passphrase "shapassword", and privacy cipher DES and privacy passphrase "despassword" |
snmpcred user V3User auth sha shapassword priv desede desedepassword |
configures snmp v3 USM authentication with authentication hash algorithm SHA and authentication passphrase "md5password", and privacy cipher 3DES-EDE and privacy passphrase "desedepassword" |
snmpcred user V3User auth sha shapassword priv aes192 aes192password |
configures snmp v3 USM authentication with authentication hash algorithm SHA and authentication passphrase "md5password", and privacy cipher AES-192 and privacy passphrase "aes192password" |
snmpcred user V3User auth sha shapassword priv aes256 aes265password |
configures snmp v3 USM authentication with authentication hash algorithm SHA and authentication passphrase "md5password", and privacy cipher AES-256 and privacy passphrase "aes256password" |
*Note about setting Auvik shell credentials for Cisco devices running SNMPv3 privacy above AES-128: Cisco uses what they call 3-DES key extension as part of the protocol; therefore to account for this, you'll need to add a "w3des" suffix to the privacy parameter e.g - when using AES-256, you would specify: priv aes256w3des aespassword
If snmpcred isn't invoked at all, or is invoked without any arguments, the default configuration is used: snmp v2c community authentication with community string "public".
snmpget attempts to fetch one or more OIDs from the host at a specified IP address using an SNMP GET request. If only an IP address is specified it fetches the OID sysDescr.0. To fetch other OIDs, any number of numeric OID identifiers can be specified following the IP address. Note that it doesn’t support symbolic OID names like the net-snmp tools do.
snmpgetnext is like snmpget except it uses an SNMP GETNEXT request.
snmpgetbulk is like snmpget except it uses an SNMP GETBULK request.
Note that SNMP has no "authentication failed" response, so a failed authentication is generally indistinguishable from "host down or not present".
For example:
auvik> snmpcred user V3User auth md5 md5password priv des des56password
user {
security_name: "V3User"
auth_protocol: SNMP_AUTH_MD5
auth_passphrase: "md5password"
privacy_protocol: SNMP_PRIVACY_DES
privacy_passphrase: "des56password"
}
auvik> snmpget 10.0.40.1
[1.3.6.1.2.1.1.1.0]: snmpString(Cisco IOS Software, 2801 Software (C2801-ADVENTERPRISEK9-M), Version 12.4(17), RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2007 by Cisco Systems, Inc.
Compiled Fri 07-Sep-07 17:15 by prod_rel_team)
auvik> snmpget 10.0.40.1 1.3.6.1.2.1.1.5.0
[1.3.6.1.2.1.1.5.0]: snmpString(CISCO_2801.auvik.local)
auvik> snmpget 10.0.40.2
snmp request returned no result
auvik>
snmpwalk attempts to retrieve all OIDS from a specified host that have a common prefix. If only an IP address is specific, snmpwalk will fetch all OIDs starting with 1.3.6.1.2.1.1. To fetch more or different OIDs, a different prefix can be specified as an additional parameter.
For example:
auvik> snmpwalk 10.0.40.3 1.3.6.1.2.1.1 [1.3.6.1.2.1.1.1.0]: snmpString(SonicWALL NSA 2400 (SonicOS Enhanced 5.8.1.14-68o)) [1.3.6.1.2.1.1.2.0]: oid([1.3.6.1.4.1.8741.1]) [1.3.6.1.2.1.1.3.0]: long(0x348e7c13e90) [1.3.6.1.2.1.1.4.0]: snmpString(Alex Hoff) [1.3.6.1.2.1.1.5.0]: snmpString(Auvik's SonicWall 2400) [1.3.6.1.2.1.1.6.0]: snmpString() [1.3.6.1.2.1.1.7.0]: int(0x4f)
SSH
The diagnostic shell contains a simple SSH tester. This tester opens a connection to the specified remote system, sends the command to be executed, and displays all output received in the next five seconds.
Note: please change nanook and password to the correct SSH login credentials.
For example:
auvik> ssh 10.0.40.1 nanook password show ver ssh session established and authenticated ssh channel created CISCO_2801> show ver Cisco IOS Software, 2801 Software (C2801-ADVENTERPRISEK9-M), Version 12.4(17), RELEASE SOFTWARE (fc1) Technical Support: http://www.cisco.com/techsupport Copyright (c) 1986-2007 by Cisco Systems, Inc. Compiled Fri 07-Sep-07 17:15 by prod_rel_team ROM: System Bootstrap, Version 12.4(13r)T, RELEASE SOFTWARE (fc1) CISCO_2801 uptime is 48 weeks, 3 hours, 26 minutes System returned to ROM by Reload Command System image file is "flash:c2801-adventerprisek9-mz.124-17.bin"
HTTP
The diagnostic shell contains a simple HTTP tester. It will attempt to fetch any URL, although note that the given URL must have at least a slash after the hostname to be considered valid by the remote system.
For example:
auvik> http http://10.0.40.1/
http_message {
response {
status: 401
entity {
content_type {
main_type: "application"
sub_type: "octet-stream"
}
data: "401 Unauthorized\r\n"
}
headers {
key: "Date"
value: "Mon, 30 May 2016 15:06:03 GMT"
}
headers {
key: "Server"
value: "cisco-IOS"
}
headers {
key: "Connection"
value: "close"
}
headers {
key: "Accept-Ranges"
value: "none"
}
headers {
key: "WWW-Authenticate"
value: "Basic realm=\"level_15_access\""
}
protocol: HTTP_1_1
}
}
How to capture and send collector logs
It's possible to use the collector shell to inspect files in the log directory.
The most important command is copylogs, which creates a zip file of all the log files (anything whose name contains .log) and uploads it to the auvik-agent-logs bucket in AWS S3.
auvik> copylogs ActorTable.log AuvikAgent.log AuvikAgent.log.0.gz AuvikAgent.log.1.gz AuvikAgent.log.2.gz AuvikWatchdog.log HealthCheck.log logs uploaded as https://s3.amazonaws.com/auvik-agent-logs/nanook/52e6555f-fe26-4274-a194-57076c96f23d/2016.10-master-/logs-20161028T170258.zip
The URL for the log zip file can then be added to the relevant support ticket.
