pg_recvlogical: Prevent flushed data from being re-sent.

Previously, when pg_recvlogical lost connection, reconnected, and restarted
replication, data that had already been flushed could be streamed again.
This happened because the replication start position used when restarting
replication was taken from the last standby status message, which could be
older than the position of the last flushed data. As a result, some flushed
data newer than the replication start position could exist and be re-sent.

This commit fixes the issue by ensuring all written data is flushed to disk
before restarting replication, and by using the last flushed position as
the replication start point. This prevents already flushed data from being
re-sent.

Additionally, previously when the --no-loop option was used, pg_recvlogical
could exit without flushing written data, potentially losing data. To fix
this issue, this commit also ensures all data is flushed to disk before
exiting due to --no-loop.

Author: Fujii Masao <masao.fujii@gmail.com>
Reviewed-by: Mircea Cadariu <cadariu.mircea@gmail.com>
Reviewed-by: Yilin Zhang <jiezhilove@126.com>
Reviewed-by: Dewei Dai <daidewei1970@163.com>
Reviewed-by: Chao Li <li.evan.chao@gmail.com>
Discussion: https://postgr.es/m/CAHGQGwFeTymZQ7RLvMU6WuDGar8bUQCazg=VOfA-9GeBkg-FzA@mail.gmail.com

diff --git a/src/bin/pg_basebackup/pg_recvlogical.c b/src/bin/pg_basebackup/pg_recvlogical.c
index fdbc40591ce8cfa9c0a471872370e6e8df4659d3..95624921c583ac4b95d991211d510631439ffe5d 100644 (file)
--- a/src/bin/pg_basebackup/pg_recvlogical.c
+++ b/src/bin/pg_basebackup/pg_recvlogical.c
@@ -191,6 +191,13 @@ OutputFsync(TimestampTz now)
 
    output_fsync_lsn = output_written_lsn;
 
+   /*
+    * Save the last flushed position as the replication start point. On
+    * reconnect, replication resumes from there to avoid re-sending flushed
+    * data.
+    */
+   startpos = output_fsync_lsn;
+
    if (fsync_interval <= 0)
        return true;
 
@@ -222,8 +229,6 @@ StreamLogicalLog(void)
    PQExpBuffer query;
    XLogRecPtr  cur_record_lsn;
 
-   output_written_lsn = InvalidXLogRecPtr;
-   output_fsync_lsn = InvalidXLogRecPtr;
    cur_record_lsn = InvalidXLogRecPtr;
 
    /*
@@ -1025,8 +1030,18 @@ main(int argc, char **argv)
             */
            exit(0);
        }
-       else if (noloop)
+
+       /*
+        * Ensure all written data is flushed to disk before exiting or
+        * starting a new replication.
+        */
+       if (outfd != -1)
+           OutputFsync(feGetCurrentTimestamp());
+
+       if (noloop)
+       {
            pg_fatal("disconnected");
+       }
        else
        {
            /* translator: check source for value for %d */