Troubleshooting

Common troubleshooting and log inspection

Check the Logs

If things are not behaving as expected, a good first step is always to check the logs. There may be messages logged by Guacamole or guacd which describe exactly what is failing, or which will at least point you in the right direction toward resolving a problem.

Auto Docker Install method

If you installed Keeper Connection Manager using the Auto Docker Install method, you can use the kcm-setup.run script to check log files.

To see all recent logs across all containers:

sudo ./kcm-setup.run logs

To tail the logs in real time, use the -f flag

sudo ./kcm-setup.run logs -f

Container-specific logs can be monitored to troubleshoot specific issues, as outlined below.

guacd Logs

Command: sudo ./kcm-setup.run logs guacd
Use When:
- Troubleshooting connection issues
- Debugging protocol-specific errors (RDP, VNC, SSH)
- Analyzing session details and performance metrics
- Investigating connection attempts and results
- Viewing errors and stack traces related to guacd

guacamole Logs

Command: sudo ./kcm-setup.run logs guacamole
Use When:
- Debugging user authentication issues
- Monitoring user session activities
- Troubleshooting web request handling and responses
- Investigating configuration loading and parsing results
- Viewing errors and stack traces related to the web application
- Checking HTTP request and response logs

db Logs

Command: sudo ./kcm-setup.run logs db
Use When:
- Troubleshooting database connection issues
- Debugging query execution problems
- Monitoring transaction management
- Investigating database startup and shutdown logs
- Viewing user authentication results for the database
- Analyzing performance metrics
- Checking backup and restore operations
- Viewing errors and stack traces related to the database

ssl Logs

Command: sudo ./kcm-setup.run logs ssl
Use When:
- Troubleshooting SSL/TLS connection issues
- Debugging SSL certificate loading and verification
- Investigating secure connection establishment and termination
- Checking HTTP to HTTPS redirection logs
- Viewing handshake and encryption details
- Viewing errors and stack traces related to SSL/TLS
- Analyzing configuration results
- Checking client and server connection details

To look at individual log files, you can specify the type of log output:

sudo ./kcm-setup.run logs guacd
sudo ./kcm-setup.run logs guacamole
sudo ./kcm-setup.run logs db
sudo ./kcm-setup.run logs ssl

If you want to start digging around the Docker images, the below commands will get you started:

sudo docker-compose -p "kcm" -f /etc/kcm-setup/docker-compose.yml ps
sudo docker ps
sudo docker exec -it <container ID> /bin/bash
etc...

Custom Docker Install method

When you have installed using the Custom Docker Install method, you can use the built-in docker and docker-compose logging commands.

For example, when using docker-compose:

sudo docker-compose logs guacd
sudo docker-compose logs guacamole
sudo docker-compose logs db
sudo docker-compose logs ssl

Using the regular docker command, locate the container ID through either "docker ps":

sudo docker ps

CONTAINER ID   IMAGE ...
2aca97c20f5c   keeper/guacamole-ssl-nginx ...
73538e4af0b7   keeper/guacamole-db-mysql ...
44ab1b150bc9   keeper/guacd ...
85e79b62f058   keeper/guacamole ...

Then you can reference the specific container to get the logs:

sudo docker logs 44ab1b150bc9
sudo docker logs 85e79b62f058
sudo docker logs 73538e4af0b7
sudo docker logs 2aca97c20f5c

Connection Issues

If Keeper Connection Manager is hanging during startup, or unable to connect to any hosts, one thing to quickly check is if the Linux machine is capable of generating enough entropy for random number generation. To resolve this, we recommend installing the haveged package.

These packages can be installed using the commands below (CentOS / RHEL):

sudo yum install epel-release
sudo yum install haveged
sudo systemctl start haveged
sudo systemctl enable haveged

Verbose Logging

LOG_LEVEL

This variable is optional and specifies the lowest level of log message that should be displayed. In order of increasing verbosity, valid values are: "error", "warn", "info", "debug", "trace".

The default log level is "info".

Installation Issues

Packages cannot be downloaded from the Keeper Connection Manager repository

Verify that your server can access the internet

Depending on the configuration of your network and server, internet access may be unavailable. If this is the case, the site hosting the repository will not be reachable and the packages will not be able to be downloaded. You will need to adjust the configuration of your network/server to ensure the repository can be accessed. Alternatively, if your organization maintains its own internal set of repository mirrors, you can configure your repository to mirror ours and point your server at your mirror instead.

Web application reachability and behavior

Cannot connect to http://SERVER:8080/ after installation

Verify that the firewall is not blocking access

CentOS and RHEL come with the "firewalld" firewall by default, typically allowing inbound connections only to port 22 for SSH. If firewalld is enabled on your system, and you wish to access to port 8080, you will need to either temporarily disable the firewall (if you are only testing whether things work as expected) or allow direct access to port 8080 (if another server on your network will be providing SSL termination, and this server is not publicly visible).

To disable the firewall temporarily:

$ sudo systemctl stop firewalld

To allow direct access to port 8080:

$ sudo firewall-cmd --zone=public --add-port=8080/tcp --permanent
$ sudo firewall-cmd --reload

Verify that the server in question is reachable

If your server is on a private network, it may be the case that the machine you are testing from is simply unable to reach that network. You may need to move to a machine within the same network to perform your tests, or establish a temporary tunnel or bridge between the networks to cause the server to become reachable.

Verify that Tomcat is indeed running

The Apache Guacamole web application is run using Apache Tomcat. Installing Tomcat is part of the Keeper Connection Manager install procedures, and the Tomcat service must be running for the Guacamole web application installed as part of Keeper Connection Manager to become reachable. If unsure whether Tomcat is running, the easiest way to check is using systemctl:

$ sudo systemctl status tomcat

If Tomcat is not running, it will need to be started for the web application to be accessible via port 8080. The service should also be set to automatically start on boot, such that the service will come back online after a server restart.

To start Tomcat manually:

$ sudo systemctl start tomcat

To configure Tomcat to start automatically when the server next boots:

$ sudo systemctl enable tomcat

Verify that the web application (guacamole.war) is deployed

While we only strictly support Apache Tomcat, the packages provided by Keeper Connection Manager do not assume that you will be using Apache Tomcat. The file containing the web application, guacamole.war, therefore must be manually deployed after Tomcat is installed. If the web application is not deployed, Tomcat will accept connections to port 8080 but will report errors when you attempt to access Guacamole.

The web application is deployed by creating a symbolic link to /opt/keeper/share/guacamole/guacamole.war within /var/lib/tomcat/webapps, as documented by the installation instructions. If deployed correctly, a directory listing of /var/lib/tomcat/webapps should show a symbolic link to guacamole.war, even if the name of the link differs from the file it points to. For example, if deployed as "guacamole":

$ ls -l /var/lib/tomcat/webapps
lrwxrwxrwx. 1 root root 34 Mar 29 20:20 guacamole.war -> /opt/keeper/share/guacamole/guacamole.war

Or, if deployed to the root path:

$ ls -l /var/lib/tomcat/webapps
lrwxrwxrwx. 1 root root 34 Mar 29 20:20 ROOT.war -> /opt/keeper/share/guacamole/guacamole.war

Cannot connect to https://SERVER/ after setting up SSL termination

Verify that the firewall is not blocking access

CentOS and RHEL come with the "firewalld" firewall by default, typically allowing inbound connections only to port 22 for SSH. If firewalld is enabled on your system, and you are using the same system to provide SSL termination, you will need allow direct access to the standard HTTPS port (443). To allow direct access to port 443:

$ sudo firewall-cmd --zone=public --add-service=https --permanent
$ sudo firewall-cmd --reload

Note that there is no need to expose direct access to port 8080 if the server will be serving as SSL termination for your deployment. Port 8080 would need to be available only locally, to be used strictly internally by the proxy.

Check whether SELinux is denying proxy connections to Tomcat

By default, the SELinux system will block both the Apache HTTP server and Nginx from establishing outbound network connections, including TCP connections to an internal instance of Apache Tomcat. If this is the case, you may see AVC denials within the SELinux audit logs (/var/log/audit/audit.log), and SELinux must be configured to allow these connections by setting the httpd_can_network_connect boolean to "1":

$ sudo setsebool -P httpd_can_network_connect 1

Verify that the reverse proxy is running

Following packaging best practices, neither the packages providing the Apache HTTP server nor the packages providing Nginx will start their corresponding services by default. They must be manually started and manually configured to start automatically on boot. If this has not been done, there will be no service listening on port 443, and connections will either timeout or be refused entirely.

To start the Apache HTTP server and configure it to start automatically on boot:

$ sudo systemctl start httpd
$ sudo systemctl enable httpd

To start NGINX and configure it to start automatically on boot:

$ sudo systemctl start nginx
$ sudo systemctl enable nginx

Login page does not display

Verify that the "tomcat" user Is part of the "guacamole" group

The Keeper Connection Manager packages create a "guacamole" group for controlling access to files that should be readable only by the Apache Guacamole web application. Configuration files specific to Guacamole, like /etc/guacamole/guacamole.properties, have their permissions set such that only root and members of the "guacamole" group have read access. As it is Apache Tomcat which runs the Guacamole web application, and the Tomcat service runs as the "tomcat" user, the "tomcat" user must be a member of the "guacamole" group:

$ sudo usermod -aG guacamole tomcat

If this is not done, Guacamole will not be able to read its own configuration files, and web application startup will fail.

Check whether SELinux is denying access to the database

By default, the SELinux system will block Apache Tomcat from establishing outbound network connections to databases, including connections to local database instances. If this is the case, you may see AVC denials within the SELinux audit logs (/var/log/audit/audit.log), and SELinux must be configured to allow these connections by setting the tomcat_can_network_connect_db boolean to "1":

$ sudo setsebool -P tomcat_can_network_connect_db 1

The database initialization scripts may not have not been run

Apache Guacamole's database must be manually initialized using the SQL scripts provided. If these scripts have not been run, the tables required by Guacamole will not exist, and its attempts to query these tables will fail. If using MySQL, verify that you have indeed run the MySQL initialization scripts as documented. If using PostgreSQL, verify that you have run the PostgreSQL initialization scripts as documented, and that you did so prior to granting the required database permissions.

(If using PostgreSQL) Check whether PostgreSQL is configured to expect "Ident" authentication

If PostgreSQL has been installed locally, on the same server as Apache Guacamole, the default configuration of PostgreSQL will prevent Guacamole from authenticating with a username and password. The mechanisms used by PostgreSQL for authentication depend on the source of the database connection. By default, PostgreSQL is configured to expect "Ident" authentication rather than a username and password, which will result in Guacamole being unable to connect to PostgreSQL.

Check the contents of /var/lib/pgsql/data/pg_hba.conf, looking for one or more lines which associate IPv4 or IPv6 loopback addresses with "Ident":

host    all     all     127.0.0.1/32    ident
host    all     all     ::1/128         ident

If found, the keyword ident should be changed to md5 to allow username/password authentication from these addresses:

host    all     all     127.0.0.1/32    md5
host    all     all     ::1/128         md5

PostgreSQL will then need to be restarted to apply these changes:

$ sudo systemctl restart postgresql

(If using MySQL) Check whether MySQL is configured with default anonymous users

If MySQL or MariaDB are installed locally, on the same server as Apache Guacamole, the default configuration may prevent Guacamole from authenticating due to the way that MySQL and MariaDB handle authentication. If an anonymous user is defined for the same hostname/address, authentication using a non-anonymous user and password from the same hostname/address will fail.

This can be checked by querying the MySQL user table directly:

SELECT Host, User FROM mysql.user;

Any users with empty usernames in the results of the above query are anonymous users which may block authentication from succeeding:

+---------------------+----------------+
| Host                | User           |
+---------------------+----------------+
| %                   | guacamole_user |
| 127.0.0.1           | root           |
| ::1                 | root           |
| the.server.hostname |                |
| the.server.hostname | root           |
| localhost           |                |
| localhost           | root           |
+---------------------+----------------+

Dropping those users should allow non-anonymous authentication from those same hosts to succeed:

DROP USER ''@'localhost';
DROP USER ''@'the.server.hostname';

Verify the necessary database permissions have been granted to the database user

Apache Guacamole requires SELECT, INSERT, UPDATE, and DELETE privileges on all tables within its database. For PostgreSQL, it additionally requires SELECT and USAGE privileges on all sequences in its database.

If using MySQL, verify that you have indeed run the GRANT statement and FLUSH PRIVILEGES as documented. It is important to remember that MySQL caches database privileges and will not apply changes to privileges unless FLUSH PRIVILEGES is run.
If using PostgreSQL, verify that you have run both GRANT statements as documented, and that you did so prior to granting the required database permissions. It is important to remember that PostgreSQL can only grant permissions related to tables and sequences that exist. If GRANT statements are run before Guacamole's database tables exist, they will not have any effect.

Check for trailing whitespace after database-related properties

While the Java .properties files do not assign meaning to whitespace that occurs before a property value, everything after the first non-whitespace character of a property value is part of the property value, including further whitespace. If you are confident that all database-related properties within /etc/guacamole/guacamole.properties look correct, check that you have not accidentally inserted whitespace at the end of what otherwise appears to be a correct value.

Administration

LDAP users are not listed when integrating LDAP with a database

Check that your (administrative) Guacamole user account has permission to list LDAP users

Apache Guacamole's LDAP support will not bypass the permissions enforced by your LDAP directory. This means that your Guacamole user will only be able to see LDAP users within Guacamole's admin interface if your corresponding LDAP user has this permission to list these users, regardless of whether Guacamole's "search" LDAP user has permission and regardless of whether your database user has general permission to manage users.

If unsure whether your user has permission, a good way to check is to execute an LDAP query against your directory, binding using the DN and password associated with your LDAP account. The "ldapsearch" utility is a standard tool which can perform such a search:

$ ldapsearch -x -LLL -H ldap://YOUR_LDAP_SERVER -D "YOUR_LDAP_DN" -W -b USER_BASE_DN "(objectClass=*)"

where YOUR_LDAP_SERVER is the hostname or IP address of your LDAP server, YOUR_LDAP_DN is the full DN of your account within LDAP, and USER_BASE_DN is the base DN of all relevant users within your LDAP directory (as specified with the ldap-user-base-dn property in /etc/guacamole/guacamole.properties).

Check that you are using LDAP credentials to sign in to Guacamole, not database credentials.

If your user account within the database has an associated password, and you specify that password instead of the password associated with your LDAP account, you will successfully authenticate against the database only. This is a valid authentication result, but means that your session will lack access to LDAP objects. To be able to access objects within LDAP, including users, you must authenticate with credentials that your LDAP directory will accept.

Check for trailing whitespace after LDAP-related properties

While the Java .properties files do not assign meaning to whitespace that occurs before a property value, everything after the first non-whitespace character of a property value is part of the property value, including further whitespace. If you are confident that all LDAP-related properties within /etc/guacamole/guacamole.properties look correct, check that you have not accidentally inserted whitespace at the end of what otherwise appears to be a correct value. The presence of whitespace at the end of LDAP-related properties may result in queries failing and may cause Guacamole to incorrectly interpret the results of queries that appear to succeed.

The password for "guacadmin" (or the current user) cannot be changed

Use the "Preferences" interface for changing your own password, not the administrative user editor

Apache Guacamole has two mechanisms available for changing a user's password: the user editor, for changing the passwords of other users (without having to know their current passwords), and the "Preferences" screen, for changing your own password after verifying that you know your current password. As only the "Preferences" screen verifies that you know your current password, this is the only mechanism which Guacamole will allow if you are making changes to your own user. If you attempt to change your own password using the user editor, permission will be denied. For administrators, the "Preferences" screen can be accessed by going to "Settings" and then selecting "Preferences".

Check that the user in question has permission to change their own password

User accounts defined within the Apache Guacamole database do not necessarily have permission to change their own passwords. Permission to update your own password must be explicitly granted by the administrator. If you are an administrator and you want a newly-created user to be able to change their own password, check the box next to the "Change own password" permission within the user editor.

Remote desktop reachability and behavior

Cannot connect to Windows using RDP

Try explicitly specifying the security method

Recent versions of Windows require NLA (Network Level Authentication) by default, an RDP-specific security method which uses higher-grade encryption and which enforces authentication before allocating desktop session resources. If connections to your Windows RDP servers are failing, try selecting the "NLA" or "TLS" security mode within the failing connection(s) parameters. If you do not also specify the username, password, and (if applicable) domain, Guacamole will prompt you for these details upon connecting.

Similarly, older versions of Windows may not support NLA or TLS by default, and Guacamole's initial security negotiation may fail unless the older "RDP" security method is explicitly selected. In older versions of Guacamole, the "RDP" security method was the default.

Try disabling validation of the RDP server certificate

Both the NLA (Network Level Authentication) and TLS security modes involve client verification of the RDP server's certificate. In the case of most Windows servers, this certificate will be self-signed and cannot be verified. Lacking the ability to verify the certificate against a known certificate authority, Guacamole will refuse to complete the RDP connection. Individual RDP connections can be configured to bypass this verification by checking the "Ignore server certificate" option within the connection parameters.

Check whether Windows firewall rules are blocking RDP connections from guacd

The Windows firewall may be configured to block inbound RDP connections, either locally or through GPOs. If the Windows machine in question has historically been accessed using RDP, it may still be the case that RDP access is blocked for all but a specific subnet in which the Guacamole server does not reside. If this is the case, the Windows firewall should be reconfigured to allow the Guacamole server (or its subnet) to connect via RDP. The Windows server does not need to be publicly accessible.

File transfer using RDP drive redirection is not working

Check that the drive path points to a directory that is writable by the "guacd" user or group

The Keeper Connection Manager packages create a "guacd" user and group for controlling access to files that should be accessible only by guacd. If you are configuring RDP drive redirection, and this user (or group) is not given write access to the directory chosen as the RDP drive path, guacd will not be able to read/write files within that directory as it emulates the redirected drive. Appropriate permissions and ownership should be assigned to any directory intended for use as an RDP connection's drive path:

$ sudo chown -R root:guacd /path/to/directory
$ sudo chmod 770 /path/to/directory

File Permission Errors

When the packages are installed, the folder /var/lib/guacamole will be created by default for each service.

The "guacd" service is run as the guacd user and the guacd user is the only user that needs read/write permission on the subfolders within the /var/lib/guacamole subfolders, such as the "recordings" and "drives" folder.

The "guacamole" process runs as "guacamole" user, and needs read access to the "recordings" subfolder and files.

Issues Connecting via KSM when Using HTTP Proxy

See CATALINA_OPTS

PreviousCustom Extensions NextExporting Connections

Last updated 3 months ago