UniFi - Troubleshooting Offline Cloud Key and Other Stability Issues


Overview


This article will explain how to troubleshoot, resolve and prevent Cloud Key failures and stability issues.

NOTES & REQUIREMENTS:
This article applies to the UniFi Cloud Key, which should be updated to the latest stable firmware/controller version before troubleshooting further. 

Table of Contents


  1. Introduction
  2. Symptoms
  3. Resolution
    1. Reboot
    2. Reset Cloud Key
    3. Restore from Working Backup
  4. Potential Causes & Prevention
    1. Power Outages/Interruptions
    2. Disk Space 
    3. Maximum Database Size Limit
  5. Related Articles

Introduction


Back to Top

The UniFi Cloud Key offers perhaps the most convenient and simple UniFi management method available. Setup is easy and doesn’t require a desktop/server on-site to be running at all times to keep online. The Cloud Key still requires some maintenance and use of best practices to ensure proper function. This article will explain how to fix Cloud Key stability issues, troubleshoot and resolve root causes when repeated issues are seen.


Symptoms


Back to Top

All of the following symptoms can indicate stability issues with your Cloud Key:

  1. Flashing white LED indicating the UniFi Cloud Key is disconnected.
  2. Cloud Key may show as offline in UniFi Management Dashboard (https://unifi.ubnt.com)offlineck.PNG
  3. Cloud Key UI may not be accessible via direct URL through browser
    • i.e. https://[cloudkeyip]:8443/manage/
  4. UniFi service shows as not running through Web UI (https://, Select option Configure > Maintenance > Service > Stop UniFi will be grayed out indicating it is already offline)
  5. UniFi Service will not start by clicking the button "Start UniFi".

Resolution


Back to Top

1. Reboot


As a first step, some Cloud Key issues can be resolved with a simple reboot. If you have not done this, do so now. This can be done by following these steps: 

  1. Open the Cloud Key management portal in a browser by typing the IP address of the Cloud Key as URL: i.e. https://192.168.1.15
  2. You may receive a Privacy Error. Click continue.
    • In Chrome, click Advanced > "Proceed to [IP of Cloud Key](unsafe)"
  3. Click Configure
  4. Log in with UniFi Cloud Key credentials used at setup. See our Related Articles below for credential questions.
  5. Select the Maintenance tab.
  6. Under Maintenance > Actions, click the button to Reboot.

If this does not resolve the issue, then proceed to the next paragraph. 

2. Reset Cloud Key


Nearly every UniFi Cloud Key failure, can be quickly resolved by a simple Reset and Restore of a backup file. The Cloud Key has made it easy to maintain backups in case of any failures. Auto Backup, if enabled, will create periodic snapshots that can be restored to in the case of any failures.

These are stored on the SD card and even if the Cloud Key is unable to be accessed manually, a reset of the device will permit a restore of the Auto Backup files on the SD card. Auto Backup is configured in the UniFi Controller under Settings > Auto Backup.

To ensure that you have enough intermittent backups should anything happen, it's recommended to have automated Weekly backups.

Steps: How to Reset your Cloud Key


ATTENTION: Prior to resetting your Cloud Key make sure you have backups available to restore from. To be safe, make sure to download from Cloud Key either through UI or via SSH prior to Resetting. Resetting the Cloud Key will lose all configuration. If no backup file is available, all settings will need to be reconfigured.

 

Reset Method 1: WebUI

  1. Open the Cloud Key management portal in the browser by typing the IP address of the Cloud Key as a URL: i.e. https://192.168.1.15.
  2. You may receive a Privacy Error, in Chrome, click Advanced > Proceed to [IP of Cloud Key](unsafe).
  3. Click Configure.
  4. Log in with UniFi Cloud Key credentials used at setup. See our Related Articles below for credential questions.
  5. Select the Maintenance tab.
  6. Click Reset to Defaults.

rebootck1.PNG

If access to the Cloud Key's management portal is not possible, use one of the following alternatives:

Reset Method 2: Hardware Button

  1. The reset button is at one end of the Cloud Key. See each model's Quick Start Guide for exact location.
  2. Insert a paper clip or similar object into the small hole until the button depresses.
  3. Hold for five seconds, then release.
  4. The LEDs should indicate the Cloud Key is restarting and being reset to default state.
  5. Once completed, the Cloud Key should show a steady white LED if this was done properly.

Reset Method 3: SSH

  1. Use an SSH client like Putty (Windows) or Terminal (macOS).
  2. Connect to the IP address of the Cloud Key.
  3. Authenticate using SSH credentials as previously configured. See this article for help on Cloud Key password disambiguation.
  4. Once you log in successfully, enter the following command then hit enter:
    • ubnt-systool reset2defaults
  5. The Cloud Key will then reset to default. 

How to Restore from Last Working Backup


Once your reset has completed successfully your Cloud Key should be in a factory default state- which will eliminate any database corruption/corrupted configuration of the Cloud Key. Now you are ready to restore from your last known working backup. Here are directions on how to do this:

Restore Method 1: Via UniFi Setup Wizard

  1. Open the Cloud Key management portal in the browser by typing the IP address of the Cloud Key as a URL: i.e. https://192.168.1.15.
  2. You may receive a Privacy Error, in Chrome, click Advanced > Proceed to [IP of Cloud Key](unsafe).
  3. Select Manage.
  4. The UniFi Setup Wizard will present existing Auto Backup files stored on the SD card.
  5. Select which backup to restore and click Restore.

setupwiz.png

Restore Method 2: Via WebUI

  1. Open the Cloud Key management portal in the browser by typing the IP address of the Cloud Key as a URL: i.e. https://192.168.1.15.
  2. You may receive a Privacy Error, in Chrome, click Advanced > Proceed to [IP of Cloud Key](unsafe).
  3. Click Configure.
  4. Login with default UniFi credentials:
    • Username: ubnt
    • Password: ubnt
  5. Go to the Maintenance tab.
  6. Click the Stop UniFi button, once the Start UniFi button becomes highlighted (appears clickable), proceed to the next step. restoreck.png
  7. Under Auto Backup select the most recent working backup file and click Restore.selectbackup.png
  8. If restoring from a local file, click Choose File.restorefile.png

Potential Causes and Prevention


Back to Top

Since Cloud Key gen1 was released, a number of features have been added and improved via firmware to improve stability and prevent common issues. Because of this, it is always recommended that you upgrade your Cloud Key firmware to the most recent stable version, especially if you repeatedly encounter any stability issues.

If Cloud Key stability issues are not resolved with a reset/restore, or repeatedly come back, it is particularly important to identify and resolve the root cause. The majority of such reported issues are found to be a result of one of the following causes:

Cause 1: Power Outages/Interruptions


User Tip: This cause is associated with Cloud Key Gen1. The Gen2 model has a "graceful shutdown" feature that should prevent sudden power interruptions.

The Cloud Key stores data and UniFi controller configuration in a MongoDB database. Even in small environments, there are ongoing database functions being performed at any given moment. If these database operations are interrupted as a result of a sudden power interruption, this can cause database corruptions which will ultimately hinder the proper function of the Cloud Key. In some cases, after a power outage, the Cloud Key may be unable to start the UniFi service properly due to database errors resulting from the interruption.

If you find your Cloud Key is losing power frequently despite no power loss, you may also have a bad Ethernet Cable being used to power the device via PoE- replace the cable to resolve this.

Symptoms

  • Stability issues arise after a known power outage/interrupted power.
  • Persistent Cloud Key stability issues; testing with different ethernet cable fails to reproduce.

How to Prevent

  1. Power the UniFi Switch or other PoE supplying device up to a power source with battery backup to ensure that power remains uninterrupted in short power outages.
  2. Plug a small charged battery like those used for charging mobile phones into the Cloud Key via the micro USB port- this will prevent the Cloud Key from being shut off in a power outage from interrupted PoE.
  3. Replace faulty Ethernet cable with new cable if the Cloud Key loses power in cases where there is no power outage.
  4. Shutdown/Reboot Cloud Key gracefully using the button in UI rather than unplugging/plugging back in. 

Cause 2: Disk Space


The Cloud Key was built for compact form, ease of install and affordability. With this comes less disk storage than you’d find on a server that would otherwise be running the UniFi Controller substance. In most environments, this will not become an issue and requires little consideration. However, in larger environments or configurations that have opted to retain more detailed data and a larger number of backups, disk space can become an issue. For this reason, it is good to monitor the disk usage on the Cloud Key regularly just to ensure that the disk space isn’t over-utilized.

storagespace.PNG

Symptoms

  • Storage Space Shows as close to 100% utilized
  • Cloud Key will shut down the controller service without warning as disk space is exhausted/MongoDB is unable to write to the database further

How to Prevent

  • Periodically check the Cloud Key disk utilization through the WebUI > Configure > Main.
  • Limit the number of backups stored on the Cloud Key. If longer retention of backups/data is needed, download these files then remove from Cloud Key storage.
  • Decrease the length of data retention in the UniFi Controller under Settings > Maintenance > Data Retention.
  • If you frequently exhaust disk space, add a larger microSD card to your Cloud Key. 
  • Perform regular compacts on Database under Settings > Maintenance > Compact Database.
  • Remove excessive backups under Settings > Auto Backup

Cause 3: Maximum Database Limit


All UniFi controllers operate with the use of a MongoDB database. What’s unique about the Cloud Key over most other controllers is that it has an ARMv7 32-bit CPU, and as such can only run 32-bit MongoDB. 32-bit MongoDB instances have a limitation over 64-bit mongo instances in that they can only support database sizes of approximately 2GB. See more info here.

As complexity and scale of the UniFi management increases, the database will grow in size. As the Cloud Key is geared towards smaller sites and portability/compactness, in most cases this will not ever be anything to worry about. However, given the 32-bit limit on size on the Cloud Key, if used in the largest environments (more than a couple dozen APs/UniFi devices), this limit will quickly be reached and prevent healthy function.

If you are unsure of whether your controller should be using a Cloud Key or not, review the following guidelines: UniFi - Cloud Key's Device Management Limit

Symptoms

  • server.log indicates error identical or similar to the following:
    • <inform_stat-10> ERROR dev - Error in DeviceManager.processStat() com.mongodb.WriteConcernException: { "serverUsed" : "127.0.0.1:27117" , "err" : "can't map file memory - mongo requires 64 bit build for larger datasets"`
  • The database is close to 2 GB in size
  • Restoring from a backup may work until the limit is reached again.

How to Prevent

  • Make sure your UniFi Deployment is not too large for the UniFi Cloud Key to handle. See here.
  • If the scale of the UniFi deployment on Cloud Key is growing too large, consider migrating to a software install of the UniFi Controller software on a server with a 64-bit OS.
  • Verify the size of your backups is not approaching the 2GB limit under Settings > Auto Backup.
  • Reduce Data Retention Settings, then Compact Database under Settings > Maintenance > Compact Database.

Other Causes


If you encounter an issue not covered by one of the above scenarios, get help on our Community. This article will continue to grow thanks to user feedback. 

However, if you still don't find a solution you may be dealing with hardware issues. These are very rare but can be encountered. In such cases, usually, the Cloud Key will fail to power on or the SD card will not permit additional data to be written to the Cloud Key. If the issue is apparently a hardware issue and your device is under warranty, fill out the RMA form to arrange a replacement: https://www.ubnt.com/support/rma/ 


Related Articles


Back to Top

UniFi - Accounts and Passwords for Controller, Cloud Key and Other Devices

UniFi SDN - Repairing Database Issues on the UniFi Controller

UniFi - How to Manage & Upgrade your Cloud Key

UniFi - How to Manually Change the Cloud Key's Controller Version via SSH

UniFi - Understanding PoE and How UniFi Devices are Powered


We're sorry to hear that!