How can we help?

How do I debug using the Auvik collector?

Follow

The Auvik collector is equipped with a Linux shell that can capture data about your network, devices, or collector to help Auvik diagnose issues.

If you run any of the following commands, send the resulting information to Auvik support for data analysis, or include it on a support ticket that needs follow-up or resolution.

How to enable the collector shell

Depending on the collector you have running, you may need to complete different actions for enabling the shell.

Virtual collector (OVA or scripted)

  1. From the virtual machine window for the collector, click onto the console.
  2. Select Option 6 and enter y to enter the bash shell from the collector menu.
  3. Type sudo /usr/share/agent/bin/agent_shell

Windows service

  1. From the start menu, search for Command Prompt (cmd).
  2. Right-click and select Run as administrator.
  3. From the command prompt window, change the directory to the install location. For example: cd c:\auvik
  4. Type AuvikAgentService -shell

How to diagnose discovery scanning

Where are all the devices?

Things to be answered with these diagnostics: What does the health check ping say about this device? Does it find the device? If so, what is the ping latency? If it doesn’t find the device, it's unlikely the Auvik collector will find the device no matter what we do.

Pick a device with a known IP address that you know is currently online. Next, try a scan of the subnet from the diagnostic shell. This will serve as a baseline for scan performance.

For example:

auvik > scan 10.0.40.0 24
       Scan result for 10.0.40.0/24
       10.0.40.1
       10.0.40.110
       10.0.40.123
       10.0.40.198
auvik > ping 10.0.40.110
       10.0.40.110 responded to ping
auvik > portOpen 10.0.40.110 80
       10.0.40.110:80 port is not open
auvik > portScan 10.0.40.110 80
       No ports found to be open

Try repeating the scan a couple of times to see if hosts magically start appearing. Contact Auvik support for further analysis of what you’re seeing.

Is a scan killing the network?

There are two parameters that control how fast Auvik sends out ICMP packets:

ScanService.max-simultaneous-ping

ScanService.simultaneous-send-delay

Both default to 10, meaning we send at most 10 packets within any particular 10ms window. Increasing the delay or decreasing the maximum number of sends will slow the scan down.

How to diagnose discovery services

SNMP

The most common answers have to do with devices not being configured correctly, or incorrect credentials being used. The diagnostic shell has a few debug commands that can help to understand SNMP issues:

snmpcred sets the credentials that will be used by the other commands.

For SNMP v1/2c:

snmpcred community private

configures snmp v2c community authentication with community string "private"

snmpcred version 1 community private

configures snmp v1 community authentication with community string "private"

For SNMP v3, with the auth protocol of MD5:

snmpcred user V3User auth md5 md5password priv des des56password

configures snmp v3 USM authentication with authentication hash algorithm MD5 and authentication passphrase "md5password", and privacy cipher DES and privacy passphrase "des56password"

snmpcred user V3User auth md5 md5password priv desede desedepassword

configures snmp v3 USM authentication with authentication hash algorithm MD5 and authentication passphrase "md5password", and privacy cipher 3DES-EDE and privacy passphrase "desedepassword"

snmpcred user V3User auth md5 md5password priv aes192 aes192password

configures snmp v3 USM authentication with authentication hash algorithm MD5 and authentication passphrase "md5password", and privacy cipher AES-192 and privacy passphrase "aes192password"

snmpcred user V3User auth md5 md5password priv aes256 aes265password

configures snmp v3 USM authentication with authentication hash algorithm MD5 and authentication passphrase "md5password", and privacy cipher AES-256 and privacy passphrase "aes265password"

For SNMP v3, with the auth protocol of SHA:

snmpcred user V3User auth sha shapassword priv des despassword

configures snmp v3 USM authentication with authentication hash algorithm SHA and authentication passphrase "shapassword", and privacy cipher DES and privacy passphrase "despassword"

snmpcred user V3User auth sha shapassword priv desede desedepassword

configures snmp v3 USM authentication with authentication hash algorithm SHA and authentication passphrase "md5password", and privacy cipher 3DES-EDE and privacy passphrase "desedepassword"

snmpcred user V3User auth sha shapassword priv aes192 aes192password

configures snmp v3 USM authentication with authentication hash algorithm SHA and authentication passphrase "md5password", and privacy cipher AES-192 and privacy passphrase "aes192password"

snmpcred user V3User auth sha shapassword priv aes256 aes265password

configures snmp v3 USM authentication with authentication hash algorithm SHA and authentication passphrase "md5password", and privacy cipher AES-256 and privacy passphrase "aes265password"

If snmpcred isn't invoked at all, or is invoked without any arguments, the default configuration is used: snmp v2c community authentication with community string "public".

snmpget attempts to fetch one or more OIDs from the host at a specified IP address using an SNMP GET request. If only an IP address is specified it fetches the OID sysDescr.0. To fetch other OIDs, any number of numeric OID identifiers can be specified following the IP address. Note that it doesn’t support symbolic OID names like the net-snmp tools do.

snmpgetnext is like snmpget except it uses an SNMP GETNEXT request.

snmpgetbulk is like snmpget except it uses an SNMP GETBULK request.

Note that SNMP has no "authentication failed" response, so a failed authentication is generally indistinguishable from "host down or not present".

For example:

auvik> snmpcred user V3User auth md5 md5password priv des des56password
user {
security_name: "V3User"
auth_protocol: SNMP_AUTH_MD5
auth_passphrase: "md5password"
privacy_protocol: SNMP_PRIVACY_DES
privacy_passphrase: "des56password"
}
auvik> snmpget 10.0.40.1
[1.3.6.1.2.1.1.1.0]: snmpString(Cisco IOS Software, 2801 Software (C2801-ADVENTERPRISEK9-M), Version 12.4(17), RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2007 by Cisco Systems, Inc.
Compiled Fri 07-Sep-07 17:15 by prod_rel_team)
auvik> snmpget 10.0.40.1 1.3.6.1.2.1.1.5.0
[1.3.6.1.2.1.1.5.0]: snmpString(CISCO_2801.auvik.local)
auvik> snmpget 10.0.40.2
snmp request returned no result
auvik>

snmpwalk attempts to retrieve all OIDS from a specified host that have a common prefix. If only an IP address is specific, snmpwalk will fetch all OIDs starting with 1.3.6.1.2.1.1. To fetch more or different OIDs, a different prefix can be specified as an additional parameter.

For example:

auvik> snmpwalk 10.0.40.3 1.3.6.1.2.1.1
[1.3.6.1.2.1.1.1.0]: snmpString(SonicWALL NSA 2400 (SonicOS Enhanced 5.8.1.14-68o))
[1.3.6.1.2.1.1.2.0]: oid([1.3.6.1.4.1.8741.1])
[1.3.6.1.2.1.1.3.0]: long(0x348e7c13e90)
[1.3.6.1.2.1.1.4.0]: snmpString(Alex Hoff)
[1.3.6.1.2.1.1.5.0]: snmpString(Auvik's SonicWall 2400)
[1.3.6.1.2.1.1.6.0]: snmpString()
[1.3.6.1.2.1.1.7.0]: int(0x4f)

SSH

The diagnostic shell contains a simple SSH tester. This tester opens a connection to the specified remote system, sends the command to be executed, and displays all output received in the next five seconds.

For example:

auvik> ssh 10.0.40.1 nanook password show ver
ssh session established and authenticated
ssh channel created
CISCO_2801> show ver
Cisco IOS Software, 2801 Software (C2801-ADVENTERPRISEK9-M), Version 12.4(17), RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2007 by Cisco Systems, Inc.
Compiled Fri 07-Sep-07 17:15 by prod_rel_team
ROM: System Bootstrap, Version 12.4(13r)T, RELEASE SOFTWARE (fc1)
CISCO_2801 uptime is 48 weeks, 3 hours, 26 minutes
System returned to ROM by Reload Command
System image file is "flash:c2801-adventerprisek9-mz.124-17.bin"

HTTP

The diagnostic shell contains a simple HTTP tester. It will attempt to fetch any URL, although note that the given URL must have at least a slash after the hostname to be considered valid by the remote system.

For example:

auvik> http http://10.0.40.1/
http_message {
 response {
   status: 401
   entity {
     content_type {
       main_type: "application"
       sub_type: "octet-stream"
     }
     data: "401 Unauthorized\r\n"
   }
   headers {
     key: "Date"
     value: "Mon, 30 May 2016 15:06:03 GMT"
   }
   headers {
     key: "Server"
     value: "cisco-IOS"
   }
   headers {
     key: "Connection"
     value: "close"
   }
   headers {
     key: "Accept-Ranges"
     value: "none"
   }
   headers {
     key: "WWW-Authenticate"
     value: "Basic realm=\"level_15_access\""
   }
   protocol: HTTP_1_1
 }
}

How to capture and send collector logs

It's possible to use the collector shell to inspect files in the log directory.

The most important command is copylogs, which creates a zip file of all the log files (anything whose name contains .log) and uploads it to the auvik-agent-logs bucket in AWS S3.

auvik> copylogs
        ActorTable.log
        AuvikAgent.log
        AuvikAgent.log.0.gz
        AuvikAgent.log.1.gz
        AuvikAgent.log.2.gz
        AuvikWatchdog.log
        HealthCheck.log
        logs uploaded as https://s3.amazonaws.com/auvik-agent-logs/nanook/52e6555f-fe26-4274-a194-57076c96f23d/2016.10-master-/logs-20161028T170258.zip

The URL for the log zip file can then be added to the relevant support ticket.

Have more questions? Submit a request

Comments

Powered by Zendesk