Troubleshooting#

Server installation errors#

RHEL 8#

While installing the PEM server on RHEL 8, if you see this error:

[root@etpgxlt firstuser]# dnf install edb-pem
Updating Subscription Management repositories.
Last metadata expiration check: 0:01:33 ago on Wed 30 Mar 2022 01:28:16 AM EDT.
Error:
 Problem: problem with installed package python3-mod_wsgi-4.6.4-4.el8.s390x
  - package python39-mod_wsgi-4.7.1-4.module+el8.4.0+9822+20bf1249.s390x conflicts with python3-mod_wsgi provided by python3-mod_wsgi-4.6.4-4.el8.s390x
  - package python39-mod_wsgi-4.7.1-4.module+el8.4.0+9822+20bf1249.s390x conflicts with python3-mod_wsgi provided by python3-mod_wsgi-4.6.4-3.el8.s390x
  - package edb-pem-server-8.4.0-7.rhel8.s390x requires python39-mod_wsgi >= 4.7, but none of the providers can be installed
  - package edb-pem-8.4.0-7.rhel8.s390x requires edb-pem-server = 8.4.0-7.rhel8, but none of the providers can be installed
  - cannot install the best candidate for the job
(try to add --allowerasing to command line to replace conflicting packages or --skip-broken to skip uninstallable packages or --nobest to use not only best candidate packages)
[root@etpgxlt firstuser]#

Remove the python3-mod_wsgi package first:

dnf remove python3-mod_wsgi

Try installing the PEM server again.

On RHEL 8, if you see this error in the worker.log after configuring the PEM server:

Tue Nov 28 03:02:19 2023 WARNING: Error clearing zombies: ERROR:  failed to JIT module: Added modules have incompatible data layouts: E-m:e-i1:8:16-i8:8:16-i64:64-f128:64-a:8:16-n32:64 (module) vs E-m:e-i1:8:16-i8:8:16-i64:64-f128:64-v128:64-a:8:16-n32:64 (jit)
CONTEXT:  SQL statement "WITH running_agent_job AS (
        SELECT j.jobid, j.agent_id
        FROM pem.job j LEFT JOIN pem.joblog jl ON (j.jobid = jl.jlgjobid)
        WHERE jl.jlgstatus = r AND agent_id = $1 AND j.jobarid != agent_runtime_id
        ORDER BY j.jobid, jl.jlgjobid
        FOR UPDATE SKIP LOCKED
    ), joblog_status_update AS (
        UPDATE pem.joblog jl SET jlgstatus=d
        FROM running_agent_job r
        WHERE r.jobid = jl.jlgjobid AND jl.jlgstatus=r
        RETURNING r.agent_id, jl.jlgjobid, jl.jlgid
    ), jobsteplog_status_update AS (
        UPDATE pem.jobsteplog js SET jslstatus=d
        FROM joblog_status_update jl
        WHERE js.jsljlgid = jl.jlgid AND js.jslstatus=r
        RETURNING jl.agent_id, jl.jlgjobid AS job_id
    )
    UPDATE pem.job j SET jobprocessid=NULL, jobnextrun=NULL, jobarid=NULL
    FROM (SELECT DISTINCT agent_id, job_id FROM jobsteplog_status_update) js
    WHERE js.job_id = j.jobid"
PL/pgSQL function pem.clear_job_zombies(integer) line 8 at SQL statement

To resolve the error, set the jit parameter to off in postgresql.conf file of the backend database server:

jit=off

Restart the backend database server.

RHEL 7 ppc64le#

After installing the PEM server on RHEL 7 ppc64le, you might see the following cipher error in the worker log file:

WARNING: ConnectToPEM: unable to connect to PEM database: could not create SSL context: library has no ciphers

To resolve this error, if you intend to stay on PostgreSQL version 12, upgrade to the latest available version. Then, restart the PEM agent.

Reconfiguring the PEM server#

In some situations, you might need to uninstall the PEM server, reinstall it, and then configure the server again. To do so:

Remove the PEM server configuration and uninstall:

/usr/edb/pem/bin/configure-pem-server.sh –un

Remove the PEM packages:

yum erase edb-pem-server

Drop the pem database:

DROP DATABASE pem

Move the certificates from /root/.pem/ to another location:

mv /root/.pem/* <new_location>

Move the agent.cfg file from /usr/edb/pem/agent/etc/agent.cfg to another location:

mv /usr/edb/pem/agent/etc/agent.cfg <new_location>

Then, configure the PEM server again:

/usr/edb/pem/bin/configure-pem-server.sh

PEM web client not loading#

If the PEM web client isn’t loading, check the HTTPD log in the /var/log/httpd/error_log file.

If you see the following message in the log:

Truncated or oversized response headers received from daemon process edbpem: /usr/edb/pem/web/pem.wsgi

Add the following statement at the bottom of the Apache httpd.conf file located in the /etc/httpd/conf folder:

WSGIApplicationGroup %{GLOBAL}

Restart the HTTPD server after adding the statement:

sudo systemctl restart httpd.service

Error connecting to PostgreSQL server#

When connecting to a PostgreSQL server, you might get one of these error messages. Review the message carefully. Each error message attempts to incorporate the information you need to resolve the problem.

Connection to the server has been lost: This error message indicates that the connection attempt took longer than the specified threshold. There might be a problem with the connection properties provided on the Server dialog box, network connectivity issues, or the server might not be running.
Could not connect to Server: Connection refused: There are two possible reasons for this error: - The database server isn't running. Start the server. - The server isn't configured to accept TCP/IP requests on the address shown.

For security reasons, a PostgreSQL server "out of the box" doesn't listen on TCP/IP ports. Instead, you must enable it to listen for TCP/IP requests. Add tcpip = true to the postgresql.conf file for Versions 7.3.x and 7.4.x. Add listen_addresses='*' for Version 8.0.x and above. These additions make the server accept connections on any IP interface. For more information, refer to the PostgreSQL documentation about runtime configuration .

FATAL: no pg_hba.conf entry: If PEM displays this message when connecting, your server can be contacted correctly over the network, but it isn't configured to accept your connection. Your client wasn't detected as a legal user for the database.

To connect to a server, configure the pg_hba.conf file on the database server to accept connections from the host of the PEM client. Modify the pg_hba.conf file on the database server host, and add an entry in the form:

host template1 postgres 192.168.0.0/24 md5 for an IPV4 network
host template1 postgres ::ffff:192.168.0.0/120 md5 for an IPV6 network

For more information, see the PostgreSQL documentation about client authentication .

FATAL: password authentication failed: The password authentication failed for user error message indicates there might be a problem with the password you entered. Retry the password to confirm you entered it correctly. If the error message returns, make sure that you have the correct password, that you are authorized to access the server, and that the access was correctly configured in the server's postgresql.conf configuration file.

PEM web server status check#

Run this command to check whether the PEM webserver is up and running:

::: curl https://<SERVER_ADDR>:8443/pem/misc/ping -k -i __OUTPUT__ PING

Where SERVER_ADDR is the IP address of your PEM server. The output PING confirms the PEM web server is up and running.

Troubleshooting

Contents