Troubleshooting
Common troubleshooting and log inspection
Last updated
Common troubleshooting and log inspection
Last updated
If things are not behaving as expected, a good first step is always to check the logs. There may be messages logged by Guacamole or guacd which describe exactly what is failing, or which will at least point you in the right direction toward resolving a problem.
If you installed Keeper Connection Manager using the Auto Docker Install method, you can use the kcm-setup.run script to check log files.
To see all recent logs across all containers:
To tail the logs in real time, use the -f flag
Container-specific logs can be monitored to troubleshoot specific issues, as outlined below.
Command: sudo ./kcm-setup.run logs guacd
Use When:
Troubleshooting connection issues
Debugging protocol-specific errors (RDP, VNC, SSH)
Analyzing session details and performance metrics
Investigating connection attempts and results
Viewing errors and stack traces related to guacd
Command: sudo ./kcm-setup.run logs guacamole
Use When:
Debugging user authentication issues
Monitoring user session activities
Troubleshooting web request handling and responses
Investigating configuration loading and parsing results
Viewing errors and stack traces related to the web application
Checking HTTP request and response logs
Command: sudo ./kcm-setup.run logs db
Use When:
Troubleshooting database connection issues
Debugging query execution problems
Monitoring transaction management
Investigating database startup and shutdown logs
Viewing user authentication results for the database
Analyzing performance metrics
Checking backup and restore operations
Viewing errors and stack traces related to the database
Command: sudo ./kcm-setup.run logs ssl
Use When:
Troubleshooting SSL/TLS connection issues
Debugging SSL certificate loading and verification
Investigating secure connection establishment and termination
Checking HTTP to HTTPS redirection logs
Viewing handshake and encryption details
Viewing errors and stack traces related to SSL/TLS
Analyzing configuration results
Checking client and server connection details
To look at individual log files, you can specify the type of log output:
If you want to start digging around the Docker images, the below commands will get you started:
When you have installed using the Custom Docker Install method, you can use the built-in docker and docker-compose logging commands.
For example, when using docker-compose:
Using the regular docker command, locate the container ID through either "docker ps":
Then you can reference the specific container to get the logs:
If Keeper Connection Manager is hanging during startup, or unable to connect to any hosts, one thing to quickly check is if the Linux machine is capable of generating enough entropy for random number generation. To resolve this, we recommend installing the haveged package.
These packages can be installed using the commands below (CentOS / RHEL):
LOG_LEVEL
This variable is optional and specifies the lowest level of log message that should be displayed. In order of increasing verbosity, valid values are: "error", "warn", "info", "debug", "trace".
The default log level is "info".
Verify that your server can access the internet
Depending on the configuration of your network and server, internet access may be unavailable. If this is the case, the site hosting the repository will not be reachable and the packages will not be able to be downloaded. You will need to adjust the configuration of your network/server to ensure the repository can be accessed. Alternatively, if your organization maintains its own internal set of repository mirrors, you can configure your repository to mirror ours and point your server at your mirror instead.
Verify that the firewall is not blocking access
CentOS and RHEL come with the "firewalld" firewall by default, typically allowing inbound connections only to port 22 for SSH. If firewalld is enabled on your system, and you wish to access to port 8080, you will need to either temporarily disable the firewall (if you are only testing whether things work as expected) or allow direct access to port 8080 (if another server on your network will be providing SSL termination, and this server is not publicly visible).
To disable the firewall temporarily:
To allow direct access to port 8080:
Verify that the server in question is reachable
If your server is on a private network, it may be the case that the machine you are testing from is simply unable to reach that network. You may need to move to a machine within the same network to perform your tests, or establish a temporary tunnel or bridge between the networks to cause the server to become reachable.
Verify that Tomcat is indeed running
The Apache Guacamole web application is run using Apache Tomcat. Installing Tomcat is part of the Keeper Connection Manager install procedures, and the Tomcat service must be running for the Guacamole web application installed as part of Keeper Connection Manager to become reachable. If unsure whether Tomcat is running, the easiest way to check is using systemctl:
If Tomcat is not running, it will need to be started for the web application to be accessible via port 8080. The service should also be set to automatically start on boot, such that the service will come back online after a server restart.
To start Tomcat manually:
To configure Tomcat to start automatically when the server next boots:
Verify that the web application (guacamole.war) is deployed
While we only strictly support Apache Tomcat, the packages provided by Keeper Connection Manager do not assume that you will be using Apache Tomcat. The file containing the web application, guacamole.war, therefore must be manually deployed after Tomcat is installed. If the web application is not deployed, Tomcat will accept connections to port 8080 but will report errors when you attempt to access Guacamole.
The web application is deployed by creating a symbolic link to /opt/keeper/share/guacamole/guacamole.war within /var/lib/tomcat/webapps, as documented by the installation instructions. If deployed correctly, a directory listing of /var/lib/tomcat/webapps should show a symbolic link to guacamole.war, even if the name of the link differs from the file it points to. For example, if deployed as "guacamole":
Or, if deployed to the root path:
Verify that the firewall is not blocking access
CentOS and RHEL come with the "firewalld" firewall by default, typically allowing inbound connections only to port 22 for SSH. If firewalld is enabled on your system, and you are using the same system to provide SSL termination, you will need allow direct access to the standard HTTPS port (443). To allow direct access to port 443:
Note that there is no need to expose direct access to port 8080 if the server will be serving as SSL termination for your deployment. Port 8080 would need to be available only locally, to be used strictly internally by the proxy.
Check whether SELinux is denying proxy connections to Tomcat
By default, the SELinux system will block both the Apache HTTP server and Nginx from establishing outbound network connections, including TCP connections to an internal instance of Apache Tomcat. If this is the case, you may see AVC denials within the SELinux audit logs (/var/log/audit/audit.log), and SELinux must be configured to allow these connections by setting the httpd_can_network_connect boolean to "1":
Verify that the reverse proxy is running
Following packaging best practices, neither the packages providing the Apache HTTP server nor the packages providing Nginx will start their corresponding services by default. They must be manually started and manually configured to start automatically on boot. If this has not been done, there will be no service listening on port 443, and connections will either timeout or be refused entirely.
To start the Apache HTTP server and configure it to start automatically on boot:
To start NGINX and configure it to start automatically on boot:
Verify that the "tomcat" user Is part of the "guacamole" group
The Keeper Connection Manager packages create a "guacamole" group for controlling access to files that should be readable only by the Apache Guacamole web application. Configuration files specific to Guacamole, like /etc/guacamole/guacamole.properties, have their permissions set such that only root and members of the "guacamole" group have read access. As it is Apache Tomcat which runs the Guacamole web application, and the Tomcat service runs as the "tomcat" user, the "tomcat" user must be a member of the "guacamole" group:
If this is not done, Guacamole will not be able to read its own configuration files, and web application startup will fail.
Check whether SELinux is denying access to the database
By default, the SELinux system will block Apache Tomcat from establishing outbound network connections to databases, including connections to local database instances. If this is the case, you may see AVC denials within the SELinux audit logs (/var/log/audit/audit.log), and SELinux must be configured to allow these connections by setting the tomcat_can_network_connect_db boolean to "1":
The database initialization scripts may not have not been run
Apache Guacamole's database must be manually initialized using the SQL scripts provided. If these scripts have not been run, the tables required by Guacamole will not exist, and its attempts to query these tables will fail. If using MySQL, verify that you have indeed run the MySQL initialization scripts as documented. If using PostgreSQL, verify that you have run the PostgreSQL initialization scripts as documented, and that you did so prior to granting the required database permissions.
(If using PostgreSQL) Check whether PostgreSQL is configured to expect "Ident" authentication
If PostgreSQL has been installed locally, on the same server as Apache Guacamole, the default configuration of PostgreSQL will prevent Guacamole from authenticating with a username and password. The mechanisms used by PostgreSQL for authentication depend on the source of the database connection. By default, PostgreSQL is configured to expect "Ident" authentication rather than a username and password, which will result in Guacamole being unable to connect to PostgreSQL.
Check the contents of /var/lib/pgsql/data/pg_hba.conf, looking for one or more lines which associate IPv4 or IPv6 loopback addresses with "Ident":
If found, the keyword ident should be changed to md5 to allow username/password authentication from these addresses:
PostgreSQL will then need to be restarted to apply these changes:
(If using MySQL) Check whether MySQL is configured with default anonymous users
If MySQL or MariaDB are installed locally, on the same server as Apache Guacamole, the default configuration may prevent Guacamole from authenticating due to the way that MySQL and MariaDB handle authentication. If an anonymous user is defined for the same hostname/address, authentication using a non-anonymous user and password from the same hostname/address will fail.
This can be checked by querying the MySQL user table directly:
Any users with empty usernames in the results of the above query are anonymous users which may block authentication from succeeding:
Dropping those users should allow non-anonymous authentication from those same hosts to succeed:
Verify the necessary database permissions have been granted to the database user
Apache Guacamole requires SELECT, INSERT, UPDATE, and DELETE privileges on all tables within its database. For PostgreSQL, it additionally requires SELECT and USAGE privileges on all sequences in its database.
If using MySQL, verify that you have indeed run the GRANT statement and FLUSH PRIVILEGES as documented. It is important to remember that MySQL caches database privileges and will not apply changes to privileges unless FLUSH PRIVILEGES is run.
If using PostgreSQL, verify that you have run both GRANT statements as documented, and that you did so prior to granting the required database permissions. It is important to remember that PostgreSQL can only grant permissions related to tables and sequences that exist. If GRANT statements are run before Guacamole's database tables exist, they will not have any effect.
Check for trailing whitespace after database-related properties
While the Java .properties files do not assign meaning to whitespace that occurs before a property value, everything after the first non-whitespace character of a property value is part of the property value, including further whitespace. If you are confident that all database-related properties within /etc/guacamole/guacamole.properties look correct, check that you have not accidentally inserted whitespace at the end of what otherwise appears to be a correct value.
Check that your (administrative) Guacamole user account has permission to list LDAP users
Apache Guacamole's LDAP support will not bypass the permissions enforced by your LDAP directory. This means that your Guacamole user will only be able to see LDAP users within Guacamole's admin interface if your corresponding LDAP user has this permission to list these users, regardless of whether Guacamole's "search" LDAP user has permission and regardless of whether your database user has general permission to manage users.
If unsure whether your user has permission, a good way to check is to execute an LDAP query against your directory, binding using the DN and password associated with your LDAP account. The "ldapsearch" utility is a standard tool which can perform such a search:
where YOUR_LDAP_SERVER is the hostname or IP address of your LDAP server, YOUR_LDAP_DN is the full DN of your account within LDAP, and USER_BASE_DN is the base DN of all relevant users within your LDAP directory (as specified with the ldap-user-base-dn property in /etc/guacamole/guacamole.properties).
Check that you are using LDAP credentials to sign in to Guacamole, not database credentials.
If your user account within the database has an associated password, and you specify that password instead of the password associated with your LDAP account, you will successfully authenticate against the database only. This is a valid authentication result, but means that your session will lack access to LDAP objects. To be able to access objects within LDAP, including users, you must authenticate with credentials that your LDAP directory will accept.
Check for trailing whitespace after LDAP-related properties
While the Java .properties files do not assign meaning to whitespace that occurs before a property value, everything after the first non-whitespace character of a property value is part of the property value, including further whitespace. If you are confident that all LDAP-related properties within /etc/guacamole/guacamole.properties look correct, check that you have not accidentally inserted whitespace at the end of what otherwise appears to be a correct value. The presence of whitespace at the end of LDAP-related properties may result in queries failing and may cause Guacamole to incorrectly interpret the results of queries that appear to succeed.
Use the "Preferences" interface for changing your own password, not the administrative user editor
Apache Guacamole has two mechanisms available for changing a user's password: the user editor, for changing the passwords of other users (without having to know their current passwords), and the "Preferences" screen, for changing your own password after verifying that you know your current password. As only the "Preferences" screen verifies that you know your current password, this is the only mechanism which Guacamole will allow if you are making changes to your own user. If you attempt to change your own password using the user editor, permission will be denied. For administrators, the "Preferences" screen can be accessed by going to "Settings" and then selecting "Preferences".
Check that the user in question has permission to change their own password
User accounts defined within the Apache Guacamole database do not necessarily have permission to change their own passwords. Permission to update your own password must be explicitly granted by the administrator. If you are an administrator and you want a newly-created user to be able to change their own password, check the box next to the "Change own password" permission within the user editor.
Try explicitly specifying the security method
Recent versions of Windows require NLA (Network Level Authentication) by default, an RDP-specific security method which uses higher-grade encryption and which enforces authentication before allocating desktop session resources. If connections to your Windows RDP servers are failing, try selecting the "NLA" or "TLS" security mode within the failing connection(s) parameters. If you do not also specify the username, password, and (if applicable) domain, Guacamole will prompt you for these details upon connecting.
Similarly, older versions of Windows may not support NLA or TLS by default, and Guacamole's initial security negotiation may fail unless the older "RDP" security method is explicitly selected. In older versions of Guacamole, the "RDP" security method was the default.
Try disabling validation of the RDP server certificate
Both the NLA (Network Level Authentication) and TLS security modes involve client verification of the RDP server's certificate. In the case of most Windows servers, this certificate will be self-signed and cannot be verified. Lacking the ability to verify the certificate against a known certificate authority, Guacamole will refuse to complete the RDP connection. Individual RDP connections can be configured to bypass this verification by checking the "Ignore server certificate" option within the connection parameters.
Check whether Windows firewall rules are blocking RDP connections from guacd
The Windows firewall may be configured to block inbound RDP connections, either locally or through GPOs. If the Windows machine in question has historically been accessed using RDP, it may still be the case that RDP access is blocked for all but a specific subnet in which the Guacamole server does not reside. If this is the case, the Windows firewall should be reconfigured to allow the Guacamole server (or its subnet) to connect via RDP. The Windows server does not need to be publicly accessible.
Check that the drive path points to a directory that is writable by the "guacd" user or group
The Keeper Connection Manager packages create a "guacd" user and group for controlling access to files that should be accessible only by guacd. If you are configuring RDP drive redirection, and this user (or group) is not given write access to the directory chosen as the RDP drive path, guacd will not be able to read/write files within that directory as it emulates the redirected drive. Appropriate permissions and ownership should be assigned to any directory intended for use as an RDP connection's drive path:
When the packages are installed, the folder /var/lib/guacamole will be created by default for each service.
The "guacd" service is run as the guacd user and the guacd user is the only user that needs read/write permission on the subfolders within the /var/lib/guacamole subfolders, such as the "recordings" and "drives" folder.
The "guacamole" process runs as "guacamole" user, and needs read access to the "recordings" subfolder and files.
See CATALINA_OPTS