Tatsuo Ishii [Wed, 17 Sep 2025 23:24:13 +0000 (08:24 +0900)]
Fix compiling issue on 32-bit environments.
It is reported that compiling src/parser/snprintf.c on 32-bit
environments fails due to undefined functions (isnan() and
ininf()). They come from math.h so include it. snprintf.c was
imported from PostgreSQL long time ago. If we look into the original
file (src/port/snprintf.c) it actually has "#include <math.h>"
already. Including math.h was provided as a pull request:
https://github.com/pgpool/pgpool2/pull/128
I also added a minor modification to the patch to reorder the
positions of include files.
Author: Gyorgy Sarvari <skandigraun@gmail.com>
Discussion: https://www.postgresql.org/message-id/
20250917.194736.
353755422175293639.ishii%40postgresql.org
Backpatch-through: v4.2
Taiki Koshino [Wed, 17 Sep 2025 07:11:57 +0000 (16:11 +0900)]
Update doc about "Setting up PostgreSQL standby" in "8.2. Pgpool-II + Watchdog Setup Example".
Add a note that when setting up a standby without online recovery, do not write primary_conninfo to postgresql.auto.conf.
Discussion: https://github.com/pgpool/pgpool2/issues/67
Backpatch-through: v4.2
Taiki Koshino [Wed, 10 Sep 2025 06:25:48 +0000 (15:25 +0900)]
Fix safer directory deletion in Bash in recovery_1st_stage.sample.
Quote variables in rm commands to avoid accidental deletion:
rm -rf "${DEST_NODE_PGDATA}"
If the variable is empty, rm could delete unexpected files or directories.
Tatsuo Ishii [Sun, 31 Aug 2025 06:49:15 +0000 (15:49 +0900)]
Allow to compile against gcc 15 (C23).
This commit includes multiple fixes to compile Pgpool-II in Fedora 42,
which uses gcc 15 (C23).
- Modify pool_type.h. "bool" is now standard in C99 and
above. PostgreSQL decided to require C99 to compile it. So we follow
the way, which is just including <stdbool.h>. Also we define
TRUE/FALSE to (bool) 1 and (bool) 0 respectively. They are used only
in Windows build in PostgreSQL but we still use them in some
places. Eventually we should replace it with true/false since we do
not support Windows.
- It is now required that function pointer arguments matches the
function prototype to be called. For example:
static pid_t worker_fork_a_child(ProcessType type, void (*func) (), void *params);
should be:
static pid_t worker_fork_a_child(ProcessType type, void (*func) (void *), void *params);
Also the prototype of pool_create_relcache() is changed,
- raw_expression_tree_walker() calls walker() in many places. Now
callers of walker() should cast the first argument of it using (Node
*). We replace the call:
return walker(((RangeVar *) node)->alias, context);
with:
return WALK(((RangeVar *) node)->alias, context);
where WALK is defined as:
#define WALK(n,c) walker((Node *) (n), c)
- Note: we have lots of warnings regarding OpenSSL while compiling
Pgpool-II in Fedora42. The version used in Fedora42:
$ openssl -version
OpenSSL 3.2.4 11 Feb 2025 (Library: OpenSSL 3.2.4 11 Feb 2025)
The fix is not included in this commit. We need
to look into it in the future.
Discussion: https://github.com/pgpool/pgpool2/issues/124
Backpatch-through: v4.6
Tatsuo Ishii [Fri, 5 Sep 2025 09:11:04 +0000 (18:11 +0900)]
Fix point less warning in query cache invalidation.
When memcached support is disabled, query cache invalidation by query
emitted point less warning. This makes 006.memcached regression test
failed.
PGPOOL SET CACHE DELETE '/*FORCE QUERY CACHE*/SELECT 1;';
WARNING: failed to delete query cache on memcached, memcached support is not enabled
Backpatch-through: v4.6
Tatsuo Ishii [Fri, 5 Sep 2025 08:12:12 +0000 (17:12 +0900)]
Fix query cache when pgpool is built without memcached.
When configure is not provided --with-memcached, compiler error
occured.
query_cache/pool_memqcache.c:542:17: 警告: 'free' called on pointer 'cih' with nonzero offset 24 -Wfree-nonheap-objec
t]
542 | free(ptr);
| ^~~~~~~~~
query_cache/pool_memqcache.c:2843:15: 備考: returned from 'pool_cache_item_header'
2843 | cih = pool_cache_item_header(cacheid);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is caused by the code block:
if (!pool_is_shmem_cache())
{
free(ptr);
}
The compiler thought that "ptr" could be the value returned by
pool_cache_item_header(), because the compiler does not understand
pool_is_shmem_cache() could return false only when memcached is
enabled. To fix this, surround the code block above with #ifdef
USE_MEMCACHED.
Reported-by: Bo Peng <pengbo@sraoss.co.jp>
Backpatch-through: v4.2
Tatsuo Ishii [Thu, 4 Sep 2025 05:59:26 +0000 (14:59 +0900)]
Fix query cache lock file handling.
Query cache module creates a lock file under logdir for concurrency
control. However, there were bugs in the handling:
1) Garbage file "QUERY_CACHE_LOCK_FILE" was created by pgpool main
process.
2) The lock file was not removed upon pgpool shutdown.
This commit fixes the bugs.
Author: Tatsuo Ishii <ishii@postgresql.org>
Reported-by: Bo Peng <pengbo@sraoss.co.jp>
Reviewed-by: Bo Peng <pengbo@sraoss.co.jp>
Backpatch-through: v4.4
Tatsuo Ishii [Sun, 31 Aug 2025 12:06:01 +0000 (21:06 +0900)]
Test: unbreak 039.log_backend_messages.
Commit
8ff2b9f6e mistakenly put synchronous commit parameters in
pgpool.conf. Unbreak the test by putting the parameters in
postgresql.conf. Also check if clustering mode is streaming
replication. Because that parameters causes suspends PostgreSQL if
clustering mode is other than streaming replication.
Backpatch-through: v4.6
Tatsuo Ishii [Sun, 31 Aug 2025 06:17:54 +0000 (15:17 +0900)]
Test: stabilize 039.log_backend_messages test.
In the test a query is sent to standby server right after rows are
inserted into primary server. Due to a replication lag, the inserted
rows could not be found on the standby in slower machines. This
commit tries to fix the issue by using synchronous replication with
remote_apply option.
Backpatch-through: v4.2
Tatsuo Ishii [Fri, 29 Aug 2025 07:15:55 +0000 (16:15 +0900)]
Test: fix ruby script in 010.rewrite_timestamp.
The ruby script used "File.exists", which is said to be obsoleted in
newer version of Ruby. Replace it with "File.exist".
Backpatch-through: v4.2
Tatsuo Ishii [Fri, 29 Aug 2025 06:48:57 +0000 (15:48 +0900)]
Test: tweak timeout in 034 and 075 tests.
034.promote_node and 075.detach_primary_left_down_node set the timeout
60 seconds for finishing follow primary script. It turns out that
these timeout values are not long enough, and sometimes caused false
errors. So make them from 60 seconds to 120 seconds.
Backpath-through: v4.2
Tatsuo Ishii [Wed, 27 Aug 2025 01:14:53 +0000 (10:14 +0900)]
Test: add ssl_ecdh_curve test to 023.ssl_connection.
023.ssl_connection did not cover the test for ssl_ecdh_curve. This
commit tests it using bad ssl_ecdh_curve parameter to see if connection
between frontend and pgpool fails.
Author: Tatsuo Ishii <ishii@postgresql.org>
Backpatch-through: v4.2
Taiki Koshino [Tue, 26 Aug 2025 04:37:16 +0000 (13:37 +0900)]
Doc: Fix releasenote link.
Tatsuo Ishii [Fri, 22 Aug 2025 00:01:06 +0000 (09:01 +0900)]
Doc: fix watchdog_setup manual.
It mistakenly stated that heartbeart is not setup in watchdog_setup.
Auther: Tatsuo Ishii <ishii@postgresql.org>
Backpatch-through: V4.2
Taiki Koshino [Tue, 19 Aug 2025 02:09:51 +0000 (11:09 +0900)]
Doc: Update doc version
Taiki Koshino [Tue, 19 Aug 2025 02:05:18 +0000 (11:05 +0900)]
Prepare 4.6.3.
Taiki Koshino [Tue, 19 Aug 2025 00:59:55 +0000 (09:59 +0900)]
Doc: add release notes.
Taiki Koshino [Fri, 15 Aug 2025 02:20:33 +0000 (11:20 +0900)]
Doc: Enhance "8.3. Replication Mode and Snapshot Isolation Mode Configuration Example" Document.
Updated the link to the example script and the explanation in the "8.3.3. Before you begin" section.
Taiki Koshino [Wed, 6 Aug 2025 02:28:48 +0000 (11:28 +0900)]
Doc: Update sample script links in "8.2. Pgpool-II + Watchdog Setup Example"
This is a follow-up to commit
34b07c2.
Updated the sample script links in section "8.2. Pgpool-II + Watchdog Setup Example" to point to the GitHub repository file URLs.
Taiki Koshino [Mon, 4 Aug 2025 01:01:59 +0000 (10:01 +0900)]
Doc: fix documentation for parameters that are not
reflected by reload.
"authentication_timeout" and "memqcache_oiddir" is not reflected by reload.
The documentation is changed to "This parameter can only be set at server start.".
Japanese doc too.
Backpatch-through: v4.2
Tatsuo Ishii [Fri, 25 Jul 2025 04:55:42 +0000 (13:55 +0900)]
Fix watchdog to print inappropriate NOTICE message.
read_ipc_socket_and_process() printed a notice message every time when
it wrote commands to IPC socket even if it was successful. Fix this to
print the notice message only when the write failed.
The reason why this bug was not recognized is, the message appears
only when log_min_messages is set to notice or higher.
Discussion: https://github.com/pgpool/pgpool2/issues/121
Backpatch-through: v4.2
Tatsuo Ishii [Thu, 24 Jul 2025 01:07:29 +0000 (10:07 +0900)]
Doc: unify watchdog leader terms (followup).
This is a followup commit to
222a88a751c75acfc809e06559621699255921f7.
This commit updates advaced.sgml.
Backpatch-trhough: v4.2
Tatsuo Ishii [Wed, 23 Jul 2025 06:51:56 +0000 (15:51 +0900)]
Doc: unify watchdog leader terms.
Previously terms "leader", "active" and "coordinator" were used to
mean the leader watchdog node. This brought enough confusion. So this
commit unifies them to "leader"."
Backpatch-trhough: v4.2
Tatsuo Ishii [Sat, 19 Jul 2025 06:48:37 +0000 (15:48 +0900)]
Fix resource leak while reading startup packet.
Per Coverity.
Backpatch-through: v4.2
Tatsuo Ishii [Sat, 19 Jul 2025 06:43:11 +0000 (15:43 +0900)]
Fix memory leak.
Fix resource leak in pool_push_pending_data pointed out by Coverity.
Backpatch-through: v4.2
Taiki Koshino [Wed, 16 Jul 2025 05:26:58 +0000 (14:26 +0900)]
Doc: Fix example script link at V4_6.
Modified the sample script in the section "8.2. Pgpool-II + Watchdog Setup Example"
Bo Peng [Mon, 30 Jun 2025 02:53:56 +0000 (11:53 +0900)]
Fix broken scram-sha-256 authentication on big-endian machies.
When scram-sha-256 authentication is performed, a hash function
pg_sha_256_final is used. It was imported from PostgreSQL and it uses
preprocessor define WORDS_BIGENDIAN to judge host machine's
endianness. Although WORDS_BIGENDIAN should be defined while
configure, this part was missed when pg_sha_256_final (and others) was
imported from PostgreSQL. As a result, scram-sha-256 worked only in
little endian machines. This commit fixes the issue by adding
AC_C_BIGENDIAN macro to configure.ac.
Author: Tatsuo Ishii
Reported-by: Christoph Berg
Reviewed-by: pranavkaruvally
Discussion: https://github.com/pgpool/pgpool2/issues/106
Backpatch-through: v4.2
Bo Peng [Mon, 23 Jun 2025 02:54:30 +0000 (11:54 +0900)]
Fix source code typos.
Tatsuo Ishii [Wed, 18 Jun 2025 08:04:13 +0000 (17:04 +0900)]
Doc: fix load balance explanation missed logical replication mode.
Backpatch-through: v4.2
Bo Peng [Wed, 18 Jun 2025 09:12:13 +0000 (18:12 +0900)]
Doc: enhance pcp_node_info document.
Clarify that each backend_application_nameX must match the value specified
in the application_name of primary_conninfo to correctly display
"replication_state" and "replication_sync_state".
Tatsuo Ishii [Wed, 18 Jun 2025 07:41:03 +0000 (16:41 +0900)]
Doc: fix load balance explanation missed Slony mode.
Since Slony mode will gone in master branch, fix is not applied to
master branch.
Backpatch-through: v4.6 to v4.2
Tatsuo Ishii [Sat, 14 Jun 2025 11:12:57 +0000 (20:12 +0900)]
Fix heartbeat device treatment.
wd_create_hb_recv_socket() and wd_create_hb_send_socket() called
setsockopt(2) with wrong argument.
struct ifreq i;
strlcpy(i.ifr_name, hb_if->if_name, sizeof(i.ifr_name));
if (setsockopt(sock, SOL_SOCKET, SO_BINDTODEVICE, &i, sizeof(i)) == -1)
:
This is not quite correct since the 4th argument should be
just a null terminated string (device name), not struct ifreq.
Discussion: [pgpool-hackers: 4602] heartbeat and SO_BINDTODEVICE
https://www.pgpool.net/pipermail/pgpool-hackers/2025-May/004603.html
Backpatch-through: v4.6
Tatsuo Ishii [Sat, 14 Jun 2025 07:15:59 +0000 (16:15 +0900)]
Fix resource leak in hearbeat receiver process.
Pointed out by Coverity.
Backpatch-through: v4.6
Tatsuo Ishii [Fri, 13 Jun 2025 01:08:12 +0000 (10:08 +0900)]
Enhance connecting process to backend.
In certain environment (especially k8s), DNS look up is unstable and
connecting to backend process fails. This occurs in call to
getaddrinfo() in connect_inet_domain_socket_by_port(). To enhance the
situation, retry up to 5 times (at each retry, sleep 1 second) if
getaddrinfo() fails with EAI_AGAIN. Note that if
connect_inet_domain_socket_by_port() is called with "retry" argument
is false, the retry will not happen. Health check calls
connect_inet_domain_socket_by_port() with the retry flag to false so
that retrying is controlled health check's own parameters.
Since up to now there's no similar issue reported, back patch to only
4.6 to make backpatching minimal.
Discussion: https://github.com/pgpool/pgpool2/issues/104
Backpatch-through: v4.6
Tatsuo Ishii [Sun, 8 Jun 2025 11:25:48 +0000 (20:25 +0900)]
Test: stabilize 029.cert_passphrase regression test.
When ssl_passphrase_command is not valid, the error message is
typically "bad decrypt" but it seems sometimes "wrong tag".
This is a backport of master
e30bb2ead.
Backpatch-through: v4.6 - v4.2.
Tatsuo Ishii [Mon, 9 Jun 2025 03:49:36 +0000 (12:49 +0900)]
Fix heartbeat_device treatment.
While processing pgpool.conf, heartbeat_device was mistakenly treated
and the first device was ignored. For example:
heartbeat_device0 = 'eth0'
the configuration process disregarded 'eth0' and acted as if no device
was set. Another example:
heartbeat_device0 = 'eth0;eth1'
"eth0" was simply ignored.
Reviewed-by: Bo Peng <pengbo@sraoss.co.jp>
Backpatch-through: v4.2
Tatsuo Ishii [Sun, 8 Jun 2025 06:32:00 +0000 (15:32 +0900)]
Doc: add section of kernel resources.
Pgpool-II uses System V shared memory and semaphores. It's better to
describe the requirements in the docs.
Backpatch-through: v4.2
Tatsuo Ishii [Sat, 7 Jun 2025 07:18:33 +0000 (16:18 +0900)]
Doc: add description for --with-ldap option of configure.
It was missed when LDAP support was introduced in v4.2
Backpatch-through: v4.2
Tatsuo Ishii [Thu, 5 Jun 2025 10:42:40 +0000 (19:42 +0900)]
Fix heartbeat receiver not working.
65dbbe7a0 added IPv6 support for heartbeat in 4.6. However it
mistakenly bound to only loopback addresses in heartbeat receive
process. Thus heartbeat messages from other watchdog heartbeat sender
were never received. To fix this add AI_PASSIVE flag to hints argument
to getaddrinfo(), which results in binding all network
interfaces. Note that before 4.6, heartbeat receive process uses
INADDR_ANY for bind(), which resulted in binding all network
interfaces too. So there's no big difference between 4.6 and pre-4.6.
Reviewed-by: Bo Peng <pengbo@sraoss.co.jp>
Backpatch-through: v4.6
Bo Peng [Thu, 5 Jun 2025 03:32:08 +0000 (12:32 +0900)]
Doc: fix command in "8.2. Pgpool-II + Watchdog Setup Example" to escape $PGDATA.
Tatsuo Ishii [Wed, 4 Jun 2025 11:10:34 +0000 (20:10 +0900)]
Doc: clarify supported platforms for Pgpool-II.
Backpatch-through: v4.2
Tatsuo Ishii [Tue, 3 Jun 2025 10:33:09 +0000 (19:33 +0900)]
Doc: enhance child_life_time document.
Backpatch-through: v4.2
Tatsuo Ishii [Mon, 2 Jun 2025 10:37:39 +0000 (19:37 +0900)]
Fix typo in pgpool.conf.
Backpatch-through: v4.3
Bo Peng [Thu, 29 May 2025 00:30:12 +0000 (09:30 +0900)]
Prepare 4.6.2
Bo Peng [Thu, 29 May 2025 00:28:04 +0000 (09:28 +0900)]
Doc: add release note.
Tatsuo Ishii [Tue, 27 May 2025 10:15:54 +0000 (19:15 +0900)]
Fix watchdog receive socket creation without IPv6.
When IPv6 network is not available, it was possible that watchdog
process won't start. Previously wd_create_recv_socket() issued
elog(ERROR) if creation or handling IPv6 socket failed. Unfortunately
at the time when wd_create_recv_socket() is called, the exception
stack is not established, and elog happily converts ERROR to FATAL,
which causes exiting watchdog process, thus exiting pgpool process.
To fix this, the elog(ERROR) calls are changed to elog(LOG).
Reported-by: Bo Peng (pengbo@sraoss.co.jp)
Discussion: https://github.com/pgpool/pgpool2/issues/99
Backpatch-through: v4.6
Tatsuo Ishii [Sat, 17 May 2025 06:24:23 +0000 (15:24 +0900)]
Suppress unnecessary information upon authentication failure.
Previously a message "password size does not match" was displayed when
client authentication failed. This could help an attacker to guess
password. Replace it just "password does not match".
Backpatch-through: v4.2
Tatsuo Ishii [Thu, 15 May 2025 09:03:50 +0000 (18:03 +0900)]
Allow pcp clients to connect to IPv6 addresses.
We have already allowed pcp server to connect to IPv6 addresses, but
pcp clients were not allowed to connect to them until today. This
commit allows pcp clients to connect to IPv6 addresses.
Discussion: [pgpool-general: 9481] Does pgpool 4.6.0 support pure ipv6 configuration?
https://www.pgpool.net/pipermail/pgpool-general/2025-May/009484.html
Backpatch-through: v4.6
Bo Peng [Thu, 15 May 2025 07:07:26 +0000 (16:07 +0900)]
Doc: Update release notes to include details of the vulnerability fix.
Bo Peng [Tue, 13 May 2025 09:29:54 +0000 (18:29 +0900)]
Doc: update release note.
Bo Peng [Tue, 13 May 2025 09:08:32 +0000 (18:08 +0900)]
Prepare 4.6.1
Bo Peng [Tue, 13 May 2025 09:06:36 +0000 (18:06 +0900)]
Doc: update release note.
Bo Peng [Tue, 13 May 2025 08:37:06 +0000 (17:37 +0900)]
Fix incorrect client authentication in some cases.
If enable_pool_hba = on, it's auth method is "password", no password
is registered in pool_passwd, and auth method in pg_hba.conf is
"scram-sha-256" or "md5", for the first time when a client connects to
pgpool, authentication is performed as expected. But if a client
connects to the cached connection, any password from the client is
accepted.
authenticate_frontend() asks password to the client and stores it in
frontend->password. When pgpool authenticate backend,
authenticate_frontend_SCRAM() or authenticate_frontend_md5() is called
depending on pg_hba.conf setting. authenticate_frontend_*() calls
get_auth_password() to get backend cached password but it mistakenly
returned frontend->password if pool_passwd does not have an entry for
the user. Then authenticate_frontend_*() tries to challenge based on
frontend->password. As a result, they compared frontend->password
itself, which always succeed. To fix this, when get_auth_password() is
called with reauth parameter being non 0, return backend->password.
Also if enable_pool_hba = off, in some cases a client is not asked
password for the first time, or when a client connects to cached
connection, even if it should be.
If pool_hba.conf is disabled, get_backend_connection() does not call
Client_authentication(), thus frontend->password is not set. Then
pool_do_reauth() calls do_clear_text_password(). It should have called
authenticate_frontend_clear_text() to get a password from the client,
but a mistake in a if statement prevented it. The mistake was fixed in
this commit.
Pgpool-II versions affected: v4.0 or later.
Also this commit does followings:
- Remove single PostgreSQL code path to simplify the authentication
code. As a result, following cases are no more Ok.
- Remove crypt authentication support for frontend and backend. The
feature had not been documented and never tested. Moreover crypt
authentication was removed long time ago in PostgreSQL (8.4, 2009).
- Add new regression test "040.client_auth". The test performs
exhaustive client authentication tests using a test specification
file formatted in CSV.
The csv files have 7 fields:
username: the username used for the test case
pool_hba.conf: takes "scram", "md5", "password", "pam", "ldap" or
"off". If "scram", "md5" , "password", "pam" or "ldap", the user
will have an entry in pool_hba.conf accordingly. If "off",
enable_pool_hba.conf will be off.
allow_clear_text_frontend_auth: takes "on" or "off".
pool_passwd: takes "AES", "md5" or "off". If "AES" or "md5" the
user's password will be stored in pool_passwd using ASE256 or md5
encryption method accordingly. If "off" is specified, no entry will
be created.
pg_hba.conf: almost same as pool_hba.conf except this is for
pg_hba.conf.
expected: takes "ok" or "fail". If ok, the authentication is
expected to be succeeded. If failed, the test is regarded as
failed. "fail" is opposite. The authentication is expected to be
failed. If succeeds, the test regarded as failed.
comment: arbitrary comment
By changing these fields, we can easily modify or add test
cases. The merit of this method is possible higher test
coverage. For human, it is easier to find uncovered test cases in a
table than in a program code.
Backpatch-through: v4.2
The patch was created by Tatsuo Ishii.
Taiki Koshino [Tue, 13 May 2025 06:00:43 +0000 (15:00 +0900)]
Doc: add release notes.
Tatsuo Ishii [Fri, 9 May 2025 01:55:38 +0000 (10:55 +0900)]
Doc: enhance query cache doc.
Pgpool refuses to cache a query calling functions returning TIMESTAMP
WITH TIMEZONE, TIME WITH TIMEZONE. If there are multiple functions
having same name and one of them returns TIMESTAMP WITH TIMEZONE, TIME
WITH TIMEZONE, pgpool refuses to cache even if one of them does not
return the data types. So add a note on this along with workaround.
Tatsuo Ishii [Thu, 8 May 2025 10:49:10 +0000 (19:49 +0900)]
Fix long standing bind bug with query cache.
When a named statement is prepared, it is possible to bind then
execute without a parse message. Problem is, table oids which are
necessary to invalidate query cache at execute or COMMIT was collected
only in parse messages process (Parse()). Thus if bind is executed
without parse after previous execute, no table oids were collected,
and pgpool failed to invalidate query cache.
Fix is collecting table oids at bind time too.
Add regression test to 006.memqcache.
Problem reported by and test program provided by Achilleas Mantzios
<a.mantzios@cloud.gatewaynet.com>.
Discussion: [pgpool-general: 9427] Clarification on query results cache visibility
https://www.pgpool.net/pipermail/pgpool-general/2025-April/009430.html
Backpatch-through: v4.2
Tatsuo Ishii [Thu, 1 May 2025 23:35:33 +0000 (08:35 +0900)]
Fix query cache invalidation bug.
When an execute message is received, pgpool checks its max number of
rows paramter. If it's not zero, pgpool sets "partial_fetch" flag to
instruct pool_handle_query_cache() to not create query cache. Problem
is, commit
2a99aa5d1 missed that even INSERT/UPDATE/DELETE sets the
execute message parameter to non 0 (mostly 1) and pgpool set the flag
for even none SELECTs. This resulted in failing to invalidate query
cache because if the flag is true, subsequent code in
pool_handle_query_cache() skips cache invalidation. It was an
oversight in this commit (my fault):
https://git.postgresql.org/gitweb/?p=pgpool2.git;a=commit;h=
2a99aa5d1910f1fd4855c8eb6751a26cbaa5e48d
To fix this change Execute() to check if the query is read only SELECT
before setting the flag.
Also add test to 006.memqcache.
Problem reported by and a test program provided by Achilleas Mantzios <a.mantzios@cloud.gatewaynet.com>.
Discussion: [pgpool-general: 9427] Clarification on query results cache visibility
https://www.pgpool.net/pipermail/pgpool-general/2025-April/009430.html
Backpatch-through: v4.2
Tatsuo Ishii [Mon, 5 May 2025 03:40:56 +0000 (12:40 +0900)]
Fix portability to OpenBSD.
- va_list is defined stdarg.h[0]
- pthread_t is defined in pthread.h / sys/types.h[1]
On OpenBSD sys/types.h doesn't suffice, so include pthread.h.
- LibreSSL has removed HMAC_CTX_init(), and has support for HMAC_CTX_new
since 2018. I've talked to Theo Buehler of LibreSSL and he said that he'd
prefer to simply remove the LIBRESSL_VERSION_NUMBER, but if desired by
upstream the LIBRESSL_VERSION_NUMBER should be 0x2070100fL.
- WIFEXITED is defined in sys/wait.h[2]
Author: Martijn van Duren (pgpool@list.imperialat.at)
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2025-May/004583.html
Backpatch-through: v4.2
Bo Peng [Fri, 2 May 2025 06:54:38 +0000 (15:54 +0900)]
Add major version information to the configuration file.
Bo Peng [Thu, 1 May 2025 02:36:55 +0000 (11:36 +0900)]
Fix json_writer did not properly encode special characters.
Pgpool would crash when the watchdog was enabled if wd_authkey contained special characters (e.g., a backslash).
The patch was originally created by Martijn van Duren and revised by Bo Peng.
Bo Peng [Wed, 30 Apr 2025 11:06:28 +0000 (20:06 +0900)]
Update pgpool.spec to prevent RPM build errors.
Tatsuo Ishii [Sun, 27 Apr 2025 13:11:20 +0000 (22:11 +0900)]
Fix IPv6 in heatbeat process.
From Pgpool-II 4.6.0, heartbeat process can handle IPv6 receiver
sockets. However, the process does not work normally if IPv6 is
disabled in the system. Like Pgpool-II main process and PostgreSQL, I
think it should work normally if IPv4 is available.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2025-April/004579.html
Backpatch-through: 4.6
Tatsuo Ishii [Thu, 24 Apr 2025 10:11:43 +0000 (19:11 +0900)]
Doc: enhance the description on connection_life_time
connection_life_time is a config value to determine the life time of
cached connections to PostgreSQL backend. Current document lacks a
description that the expiration calculation is actually done at the
time when the client disconnects to the process which holds the cached
connections.
Discussion: [pgpool-hackers: 4577] Doc: enhance the description on connection_life_time
https://www.pgpool.net/pipermail/pgpool-hackers/2025-April/004578.html
Backpatch-through: v4.2
Tatsuo Ishii [Tue, 1 Apr 2025 06:45:40 +0000 (15:45 +0900)]
Test: skip inaccessible Unix socket directories.
Commit
182b65bfc allows to use multiple Unix socket directories: /tmp
and /var/run/postgresql. However if the system does not have
accessible /var/run/postgresql, pgpool_setup fails unless
$PGSOCKET_DIR is explicitly set. Instead of failing, this commit
allows pgpool_setup to skip inaccessible directories.
Backpatch-through: v4.5
Taiki Koshino [Thu, 27 Mar 2025 05:43:53 +0000 (14:43 +0900)]
Allow regression tests to use multiple socket directories.
Author: Bo Peng
Tested by Taiki Koshino
Backpatch-through: V4.5
Tatsuo Ishii [Wed, 5 Mar 2025 10:55:11 +0000 (19:55 +0900)]
Doc: enhance the explanation on sr_check_user.
It must be a superuser or in the pg_monitor group.
Backpatch-through: v4.2.
Tatsuo Ishii [Tue, 4 Mar 2025 12:27:34 +0000 (21:27 +0900)]
Fix sr check and health check to reopen pool_passwd upon reload.
The streaming replication check and health check process forgot to
reopen pool_passwd upon reload. If sr_check_passwd or
health_check_passwd is empty string, the password is obtained from
pool_passwd. Thus those process read outdated content of pool_passwd
upon reload.
Backpatch-through: v4.2
Bo Peng [Thu, 27 Feb 2025 07:09:05 +0000 (16:09 +0900)]
Update sample script comment.
Bo Peng [Thu, 27 Feb 2025 06:08:38 +0000 (15:08 +0900)]
Prepare 4.6.0
Bo Peng [Thu, 27 Feb 2025 06:04:25 +0000 (15:04 +0900)]
Doc: update release date
Bo Peng [Thu, 27 Feb 2025 04:27:44 +0000 (13:27 +0900)]
Doc: add release notes.
Bo Peng [Wed, 26 Feb 2025 12:59:18 +0000 (21:59 +0900)]
Remove pg_basebackup from the sample follow primary script.
If pg_rewind fails, the safest way for users is to recover manually.
Bo Peng [Wed, 19 Feb 2025 09:58:43 +0000 (18:58 +0900)]
Doc: update copyright
Bo Peng [Wed, 19 Feb 2025 09:01:08 +0000 (18:01 +0900)]
Prepare RC1
Bo Peng [Wed, 19 Feb 2025 08:57:05 +0000 (17:57 +0900)]
Doc: update installation document to 4.6
Bo Peng [Tue, 11 Feb 2025 13:52:04 +0000 (22:52 +0900)]
Prepare 4.6beta1
Bo Peng [Tue, 11 Feb 2025 04:35:39 +0000 (13:35 +0900)]
Disable AM_MAINTAINER_MODE.
Tatsuo Ishii [Mon, 10 Feb 2025 09:28:51 +0000 (18:28 +0900)]
Fix too many log lines produced by streaming replication check.
The process started to call
get_pg_backend_status_from_leader_wd_node() which unconditionally emits
log message:
LOG: received the get data request from local pgpool-II on IPC interface
LOG: get data request from local pgpool-II node received on IPC interface is forwarded to leader watchdog node
every sr_check_period seconds, which is annoying. To fix this, an elog
line in process_IPC_data_request_from_leader() is downgraded from LOG
to DEBUG1.
Reported-by: Bo Peng.
Tatsuo Ishii [Mon, 10 Feb 2025 09:24:49 +0000 (18:24 +0900)]
Fix bug in heartbeat.
Following error message was recorded every wd_heartbeat_deadtime since
65dbbe7a0 was committed.
2025-02-10 10:50:37.990: heart_beat_receiver pid
1060625: ERROR: failed to get socket data from heartbeat receive socket list
2025-02-10 10:50:37.990: heart_beat_receiver pid
1060625: DETAIL: select() got timeout, exceed 30 sec(s)
The heartbeat receiver waits for heartbeart packet arrives in
select(2) until wd_heartbeat_deadtime is expired. I believe the logic
is wrong: it should wait forever until the packet arrives. In v4.5 or
earlier, the hearbeart receiver waits in recvfrom() without
timeout. So give NULL to select's timeout parameter so that it waits
forever. Since
65dbbe7a0 is only in master branch, no backpatch is
made.
Reported by: Peng Bo
Bo Peng [Mon, 10 Feb 2025 09:12:56 +0000 (18:12 +0900)]
Update sample scripts.
This commit includes:
- update sample scripts to PostgreSQL 17
- remove archive settings to disable archive mode
Bo Peng [Mon, 10 Feb 2025 09:12:03 +0000 (18:12 +0900)]
Doc: Update configuration example to 4.6 and PostgreSQL 17.
This commit includes:
- update configuration example to 4.6 and PostgreSQL 17
- update OS to Rocky Linux 9
Tatsuo Ishii [Mon, 3 Feb 2025 05:02:52 +0000 (14:02 +0900)]
Doc: the first cut of v4.6 release notes.
Bo Peng [Fri, 31 Jan 2025 00:43:57 +0000 (09:43 +0900)]
Fix per_node_error_log() error message that is printed with two colons.
Patch is created by Umar Hayat.
Tatsuo Ishii [Fri, 17 Jan 2025 05:22:05 +0000 (14:22 +0900)]
Doc: enhance client authentication chapter.
Add intro about pool_passwd. Previously there was only description on
pool_hba.conf in the overview page. A general guide to pool_passwd
will help users to understand this chapter.
Tatsuo Ishii [Tue, 14 Jan 2025 13:44:19 +0000 (22:44 +0900)]
Test: stabilize 032.dml_adaptive_loadbalance
Occasionally the test failed due to:
ERROR: relation "t2" does not exist
LINE 1: SELECT i, 'QUERY ID T1-1' FROM t2;
It seems the cause is that newly created table t2 takes sometime to
get replicated to standby. So insert "sleep 1" after the table
creation.
Backpatch-through: v4.2
Tatsuo Ishii [Sun, 12 Jan 2025 05:22:37 +0000 (14:22 +0900)]
Fix pool_signal.
Previously pool_signal did not set SA_RESTART flag. Thus any system
calls interrupted by a signal does not restart. Some of our code are
prepared so that they restart if a system call is interrupted by a
signal. But not sure all places are prepared too. So add the
flag. Note, PostgreSQL always uses the flag.
Bo Peng [Sun, 5 Jan 2025 12:53:37 +0000 (21:53 +0900)]
Update pgpool.spec.
Bo Peng [Sun, 5 Jan 2025 12:49:39 +0000 (21:49 +0900)]
Fix compiler warning:
warning: ‘delete_all_cache_on_memcached’ declared ‘static’ but never defined[-Wunused-function]
Bo Peng [Thu, 2 Jan 2025 07:56:56 +0000 (16:56 +0900)]
Update src/tools/pcp/.gitignore
Bo Peng [Mon, 16 Dec 2024 08:03:55 +0000 (17:03 +0900)]
Feature: Allow logging_collector related parameters to be changed by reloading the Pgpool-II configurations.
The following logging_collector related parameters can now be changed by reloading:
- log_truncate_on_rotation
- log_directory
- log_filename
- log_rotation_age
- log_rotation_size
- log_file_mode
Tatsuo Ishii [Wed, 11 Dec 2024 09:31:02 +0000 (18:31 +0900)]
Fix yet another query cache bug in streaming replication mode.
If query cache is enabled and query is operated in extended query mode
and pgpool is running in streaming replication mode, an execute
message could return incorrect results.
This could happen when an execute message comes with a non 0 row
number parameter. In this case it fetches up to the specified number of
rows and returns "PortalSuspended" message. Pgpool-II does not create
query cache for this. But if another execute message with 0 row
number parameter comes in, it fetches rest of rows (if any) and
creates query cache with the number of rows which the execute messages
fetched.
Obviously this causes unwanted results later on: another execute
messages returns result from query cache which has only number of rows
captured by the previous execute message with limited number of rows.
Another trouble is when multiple execute messages are sent
consecutively. In this case Pgpool-II returned exactly the same
results from query cache for each execute message. This is wrong since
the second or subsequent executes should return 0 rows.
To fix this, new boolean fields "atEnd" and "partial_fetch" are
introduced in the query context. They are initialized to false when a
query context is created (also initialized when bind message is
received). If an execute message with 0 row number is executed, atEnd
is set to true upon receiving CommandComplete message. If an execute
message with non 0 row number is executed, partial_fetch is set to
true and never uses the cache result, nor creates query cache.
When atEnd is true, pgpool will return CommandComplete message with
"SELECT 0" as a result of the execute message.
Also tests for this case is added to the 006.memqcache regression
test.
Backpatch-through: v4.2
Discussion: [pgpool-hackers: 4547] Bug in query cache
https://www.pgpool.net/pipermail/pgpool-hackers/2024-December/004548.html
Bo Peng [Mon, 9 Dec 2024 08:37:38 +0000 (17:37 +0900)]
Doc: fix the documentation typos.
Bo Peng [Mon, 9 Dec 2024 07:56:13 +0000 (16:56 +0900)]
Fixed an issue where pg_md5 and pg_enc would not update the password file if a file other than the default value was specified in the pool_passwd parameter.
This issue is reported by Sadhuprasad Patro.
Tatsuo Ishii [Thu, 5 Dec 2024 09:04:44 +0000 (18:04 +0900)]
Test: fix 006.memqcache regression test.
4dd7371c2 added test cases. SQL syntax used in the test was not
compatible with PostgreSQL 15 or earlier.
Backpatch-through: v4.2
Tatsuo Ishii [Wed, 4 Dec 2024 12:38:23 +0000 (21:38 +0900)]
Fix query cache bug in streaming replication mode.
When query cache is enabled and an execute message is sent from
frontend, pgpool injects query cache data into backend message buffer
if query cache data is available. inject_cached_message() is
responsible for the task. But it had an oversight if the message
stream from frontend includes more than one sets of bind or describe
message before a sync message. It tried to determine the frontend
message end by finding a bind complete or a row description message
from backend. But in the case, it is possible that these messages do
not indicate the message stream end because there are one more bind
complete or row description message. As a result the cached message is
inserted at inappropriate positron and pgpool mistakenly raised "kind
mismatch" error.
This commit changes the algorithm to detect the message stream end:
compare the number of messages from backend with the pending message
queue length. When a message is read from backend, the counter for the
number of message is counted up if the message is one of parse
complete, bind complete, close complete, command compete, portal
suspended or row description. For other message type the counter is
not counted up. If the counter reaches to the pending message queue
length, we are at the end of message stream and inject the cahced
messages.
Test cases for 006.memqcache are added.
Backpatch-through: v4.2.
Tatsuo Ishii [Mon, 2 Dec 2024 05:49:08 +0000 (14:49 +0900)]
Test: add check using netstat.
Sometimes we see regression errors like:
2024-12-01 13:55:55.508: watchdog pid 27340: FATAL: failed to create watchdog receive socket
2024-12-01 13:55:55.508: watchdog pid 27340: DETAIL: bind on "TCP:50002" failed with reason: "Address already in use"
Before starting each regression test, we use "clean_all" script to
kill all remaining process. I suspect that this is not enough to
release bound ports. So I add netstat command to check whether some
ports are remain bound.
For not this commit is master branch only.
Tatsuo Ishii [Sun, 1 Dec 2024 07:53:28 +0000 (16:53 +0900)]
Test: fix 039.log_backend_messages.
Commit
6d4106f9c forgot to add pgproto data which is necessary in the
test.
Tatsuo Ishii [Mon, 25 Nov 2024 09:09:59 +0000 (18:09 +0900)]
Feature: add log_backend_messages.
When enabled, log protocol messages from each backend. Possible
options are "none", "terse" and "verbose". "none" disables the feature
and is the default. "verbose" prints the log each time pgpool receives
a message from backend. "terse" is similar to verbose except it does
not print logs for repeated message to save log lines. If different
kind of message received, pgpool prints a log message including the
number of the message. One downside of "terse" is, the repeated
message will not be printed if the pgpool child process is killed
before different kind of message arrives.
For testing, 039.log_backend_messages is added.
Discussion: [pgpool-hackers: 4535] New feature: log_backend_messages
https://www.pgpool.net/pipermail/pgpool-hackers/2024-November/004536.html
Bo Peng [Wed, 27 Nov 2024 05:01:54 +0000 (14:01 +0900)]
Doc: add release notes.
Tatsuo Ishii [Mon, 18 Nov 2024 06:40:53 +0000 (15:40 +0900)]
Abort SSL negotiation if backend sends an error message.
In the client side implementation of SSL negotiation
(pool_ssl_negotiate_clientserver()), it was possible for a
man-in-the-middle attacker to send a long error message to confuse
Pgpool-II or client while in the SSL negotiation phase. This commit
rejects the negotiation immediately (issue a FATAL error) and exits
the session to prevent such an attack.
This resembles PostgreSQL's CVE-2024-10977.
Backpatch-through: v4.1