browse
Overview
On some or all computers in your Umbrella deployment, the Umbrella roaming client is not registering correctly. You notice this because the Umbrella roaming client tray icon is in a red state on a machine, or the Umbrella roaming client is not showing in the dashboard as expected. In addition, you are currently, or may have at one point, used an HTTP proxy on your network. If a client is roaming, they may have come online behind an HTTP proxy.
You currently run an HTTP Proxy: The Umbrella roaming client uses the "SYSTEM" user, since it runs as a system service, and you may have pushed out configuration settings through GPO, or another RMM tool, which gave user-specific HTTP proxy settings to users in your organization. In addition, you have default deny rules in your gateway firewall which don't allow HTTP/HTTPS traffic unless it's going through the proxy.
You ran an HTTP Proxy in the past: The "SYSTEM" user might have previously had HTTP proxy settings set at some point in the past, and now that proxy no longer exists. Because the proxy no longer exists, the SYSTEM user receives a timeout when trying to access resources over HTTP/HTTPS.
Here's a helpful tool to check the SYSTEM user permissions: https://www.apreltech.com/Free/How_to_run_as_system_user. Use the utility to launch "C:\Program Files\Internet Explorer\iexplore.exe" and go to https://api.opendns.com/v2/OnPrem.Asset and look for "1005 Missing API Key" without any security prompts. If there are prompts, ensure that UDP/TCP 443 are open to api.opendns.com, crl4.digicert.com, ocsp.digicert.com and that the DigiCert Root CA is present on the system (https://global-root.digicert.com/).
If your firewall restricts unauthenticated accounts (like the SYSTEM account) from accessing the necessary sites, our registration domains may need to be exempted. Since the roaming client calls out registration from the SYSTEM user account, this must be opened up for our prerequisites to unauthenticated access.
Symptoms
The most common symptom is a failure to register to the Dashboard or sync with our API. This will cause the client to either never register (and therefore not enter a protected mode), or cease updating the Dashboard.
If you see these symptoms, restart the roaming client service then send a diagnostic report log to support right after restarting. The client includes a proxy check which only runs at its startup.
Identify if there's a proxy
First, check the logs:
If, in the logs, you see a log entry similar to this:
2014-03-14 09:39:43 [2252] [INFO ] Trying to get Device ID from API...
2014-03-14 09:39:43 [2252] [DEBUG] Sending GET to https://api.opendns.com/v3/organizations/XXXXX/roamingdevices/lookup?api-key=XXXXXXXXXXXXXXXXXXXXXXXXXX&deviceKey=XXX&userId=XXXXXXXX&fingerprint=XXXXXXXXXXXXXXXXXXX
2014-03-14 09:39:43 [2252] [ERROR] Error creating DeviceID: The remote name could not be resolved: 'api.opendns.com'
this:
2014-12-08 09:25:27 [5700] [ERROR] POST failed: The operation has timed out
or this:
2015-03-05 12:41:11 [4084] [ERROR] Error creating DeviceID: Unable to connect to the remote server
It's likely that proxy settings have something to do with this.
Scenario 1
The log shows it can't resolve api.opendns.com.
The remote name could not be resolved: 'api.opendns.com'
This can be caused by several issues:
- DNS Resolution is failing on your Network Connection—Make sure our DNS servers are allowed in your firewall if you block third-party DNS servers.
- You have a proxy server that is not allowing the connection to be made—If you have a proxy server, it's probably best to allow list *.opendns.com so it doesn't interfere with the registration or API syncing.
- You have Port 443/TCP limited to specific IP scopes—If you are using very strict firewall rules, you might have to open 443/TCP to all outbound connections temporarily. The Umbrella roaming client needs to fetch the SSL certificate from GeoTrust IP/domain space.
You can read our specific firewall needs in our Roaming Client Prerequisites article, which specifically call out HTTP proxies.
Scenarios 2 and 3
The log shows it can't resolve proxyserver.exampledomain.com
2014-03-14 09:39:43 [2252] [ERROR] Error creating DeviceID: The remote name could not be resolved: 'proxy1.exampledomain.com'
The log shows us that it's trying to send information to our api (api.opendns.com), but the resulting error says it tried to connect to 'proxy1.exampledomain.com`
The reason this is happening is because the Umbrella roaming client uses the computer's "SYSTEM" user's proxy settings using the .NET framework call of WebRequest.DefaultWebProxy. This is commonly set via GPO, and unless this key is specifically deleted, can stay in the registry long term. If this proxy server no longer exists, or needs to be updated to a different host,
you can modify the settings via the methods below.
Scenario 4
The log shows this message:
Error creating DeviceID: The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.
however, no firewall blocks are in place to UDP/TCP crl4.digicert.com or ocsp.digicert.com. If the computer is taken to a different network (say mobile hotspot), the issue continues.
Run this:
netsh winhttp show proxy
to determine if there is a proxy still set here. This proxy settings location is utilized by the Microsoft Cryptography API v2 as part of the .NET framework certificate validation. If this proxy is in place, it may cause this validation to fail. For more information, refer to https://support.microsoft.com/en-us/kb/2623724.
Scenario 4 may also be caused by PCI compliance settings and .NET if TLS 1.0 is disabled. See this article for more information.
Add/Remove Proxy Settings
Warning: This setting is normally set using GPO or some kind of script. It's not common that this is set on an individual computer. We recommend only using this method if you want to test on an individual computer, or you believe this setting is unique to this computer. Please skip down to the next method for information about using GPO for an entire group.
PsExec and Internet Explorer (IE 10 or later)
- Download and extract PsExec from http://download.sysinternals.com/files/PSTools.zip.
- Load an Administrative Command Prompt (Right Click > Run as Administrator).
- Use "cd" to change to the extracted PSTools directory.
cd C:\Path\To\PSTools\ - Run the following command to open Internet Explorer as the "SYSTEM" user:
PsExec.exe /i /s "C:\Program Files\Internet Explorer\iexplore.exe" - Open Internet Options from the Settings menu (cog) in Internet Explorer.
- In the Connections tab, click Local Area Network (LAN) Settings and change or delete proxy settings as needed.
- Click OK, then Apply, and close Internet Explorer.
The Umbrella roaming client should register within about three minutes.
Alternative (Registry)
- Check the key at HKEY_USERS\S-1-5-18\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Connections\DefaultConnectionSettings for proxy settings. If one is present, this would result in the proxy display above under the system user IE connection settings.
- To remove in the registry, follow these steps to copy the proxy-less settings from the current user:
- Open up the key at HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Connections\DefaultConnectionSettings and copy the value
- Copy it into the same key under HKEY_USERS\S-1-5-18 (the SYSTEM user)
- Restart the roaming client to validate if registration is successful
PsExec and MMC (IE9 or earlier)
- Download and extract PsExec from http://download.sysinternals.com/files/PSTools.zip.
- Load an Administrative Command Prompt (Right Click > Run as Administrator).
- Use "cd" to change to the extracted PSTools directory.
- Run the following command:
PsExec.exe /i /s mmc - Once the MMC loads, load the Group Policy Object snap-in.
- Navigate to the Local Connection Settings and change or delete the Proxy server information as needed.
- Click OK on all menus and the Umbrella roaming client should then register.
Edit Registry
Depending on your version of Windows Server, using the above-mentioned steps using the MMC console and selecting the correct group should work fine.
If your Windows Server version does not have those settings, you can modify or delete this setting via the registry in order to purge this setting at a global level (multiple computers).
HKEY_USERS
.DEFAULT\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Connections\DefaultConnectionSettings
The following is an example for an older version of IE for legacy applications. Removing the proxy values from the GPO or local settings would allow the Umbrella roaming client to connect and register as the system user (subject to these IE proxy settings).
If either of these methodologies does not work, please open a Support ticket through the dashboard and refer to this article.