From 5525061c41cc4ee722e4876b1b99fab6d4a99d47 Mon Sep 17 00:00:00 2001 From: Craig Ringer Date: Tue, 21 Apr 2015 11:05:38 +0800 Subject: [PATCH] More detail on DDL rep, mention use of replicate_ddl_command --- doc/manual-ddl-replication.sgml | 206 +++++++++++++++++++++++++------- doc/manual-functions.sgml | 6 +- 2 files changed, 165 insertions(+), 47 deletions(-) diff --git a/doc/manual-ddl-replication.sgml b/doc/manual-ddl-replication.sgml index c1eb29af64..b09782a4fe 100644 --- a/doc/manual-ddl-replication.sgml +++ b/doc/manual-ddl-replication.sgml @@ -17,50 +17,104 @@ - To safely manipulate the database schema in a asynchronous - multimaster setup, all pending changes have to be replicated first. - Otherwise it is possible that a row being replicated contains data - for a row that has been dropped, or has no data for a row that is - marked NOT NULL. More complex cases also exit. To handle that - problem, &bdr; acquires a so called DDL lock the first time in a - transaction schema changes are made. + Before doing DDL on &bdr;, read + and . - Acquiring such a DDL locks requires contacting all nodes in a BDR - group, asking them to replicate all pending changes, and prevent - further changes from being made. Once all nodes are in that state, - the originator of the DDL lock is free to perform schema changes. + &bdr; is significantly different to standalone PostgreSQL when it + comes to DDL, and treating it as the same is a fast path to replication + problems. - - This means that schema changes, in contrary to data changes, can - only be performed while all configured nodes are reachable. If DDL - has to be performed while a node is down, it has to be removed from - the configuration (using ) - first. - + + + While DDL is in progress on any node in the system, statements that perform + writes (INSERT, UPDATE + DELETE, any DDL, etc) on that node or any other node will + ERROR even if the writes have nothing to do with the + objects currently being modified by the DDL in progress. + + - - This also makes DDL a heavier weight operation than on standalone - PostgreSQL. Performing DDL will cancel currently running - transactions and prevent new transactions from proceeding until - replication has caught up and the DDL has been applied. - - - While DDL is in progress on any node in the system, statements that perform - writes (INSERT, UPDATE - DELETE, any DDL, etc) on that node or any other node will - ERROR with SQLSTATE 55P03: - - ERROR: Database is locked against DDL operations - - This makes DDL a very heavyweight operation. Transactions performing DDL - should be short, should not be combined with lots of row changes, and - should avoid long running foreign key lookups, etc. - - - + + Executing DDL on BDR systems + + + A BDR group is not the same as a standalone PostgreSQL server. It + is based on asynchronous multi-master replication without a central + locking and transaction co-ordinator. This has important implications + when executing DDL. + + + + To safely manipulate the database schema in a asynchronous + multimaster setup, all pending changes have to be replicated first. + Otherwise it is possible that a row being replicated contains data + for a row that has been dropped, or has no data for a row that is + marked NOT NULL. More complex cases also exit. To handle + that problem, &bdr; acquires a so-called DDL lock the first time in a + transaction schema changes are made. + + + + Acquiring the global DDL lock requires contacting all nodes in a BDR + group, asking them to replicate all pending changes, and prevent + further changes from being made. Once all nodes are fully caught up, + the originator of the DDL lock is free to perform schema changes + and replicate them to the other nodes. While the global DDL lock + is held by a node, no nodes may perform any DDL or make any changes to + rows. + + + + This means that schema changes, in contrary to data changes, can + only be performed while all configured nodes are reachable. If DDL + has to be performed while a node is down, it has to be removed from + the configuration (using ) + first. + + + + DDL is a heavier weight operation than on standalone + PostgreSQL. Performing DDL on any node will cancel (abort) currently running + transactions on all nodes with an ERROR, + and will reject new DML (INSERT, + UPDATE and DELETE on all nodes with + an ERROR with SQLSTATE + 55P03: + + ERROR: Database is locked against DDL operations + + This continues until the DDL operation has replicated to all nodes, been + applied, and all nodes have confirmed to the DDL originator that the changes + have been applied. All DDL and DML will ERROR, + even if it does not affect the objects the currently in-progress DDL + is modifying. + + + + To minimise the impact of DDL, transactions performing DDL should be short, + should not be combined with lots of row changes, and should avoid long + running foreign key lookups. + + + + If DDL is holding the system up for too long, it is possible and safe to + cancel the DDL on the originating node like you would cancel any other + statement, e.g. with Control-C in + psql or with + pg_cancel_backend. + + + + Once the DDL operation has committed on the originating node, you cannot + cancel or abort it. You must wait for it to apply successfully on all + other nodes and for them to replay confirmation. This is why it is important + to keep DDL transactions short and fast. + + + Statement specific DDL replication concerns @@ -71,11 +125,53 @@ than one database. Others are disallowed. + + + Global DDL, like CREATE ROLE, CREATE USER + etc is not replicated and should be applied on each node + if the created objects will be referenced by a BDR-enabled database. + + + Not replicated DDL statements - The following DDL statements are not replicated: + Some DDL statements, mainly those that affect objects that are + PostgreSQL-instance-wide rather than database-sepecific, are not + replicated. They are applied on the node that executes them without taking + the global DDL lock and are not sent to other nodes. + + + + If you create non-replicated objects that create are to be referenced + by replicated objects (e.g. creating a role, not replicated, then creating + a table, replicated, that's owned by that role) you must ensure that the + non-replicated object is created on all &bdr; nodes. You can do this + manually, by running the statement on each node. Or you can use + to apply the statement + on the local node and manually enqueue it for replication on all nodes. + + + + Using bdr.bdr_replicate_ddl_command is the recommended + approach, e.g.: + + SELECT bdr.bdr_replicate_ddl_command('CREATE USER myuser;'); + + + + + + It is not necessary that the definition of objects like roles be the same + on all nodes, only that they exist. You can for example + CREATE ROLE somerole WITH NOLOGIN on most nodes, but + on one node you can create them WITH LOGIN. + + + + + The statements that are applied locally but not replicated are: @@ -303,10 +399,27 @@ - + Prohibited DDL statements + BDR prevents some DDL statements from running when it is active on a + database. This protects the consistency of the system by disallowing + statements that cannot be replicated correctly, or for which replication is + not yet supported. Statements that are supported with some restrictions + are covered in ; + commands that are entirely disallowed in &bdr; are covered below. + + + + Generally unsupported statements are prevented from being + executed, raising a feature_not_supported + (SQLSTATE 0A000) error. + + + + The following DDL commands are rejected by &bdr; when &bdr; is active on a + database, and will fail with an ERROR: @@ -690,15 +803,16 @@ - + DDL statements with restrictions - BDR prevents some DDL statements from running when - it is active on a database. This protects the - consistency of the system by disallowing statements - that cannot be replicated correctly, or for which - replication is not yet supported. + BDR prevents some DDL statements from running when it is active on a + database. This protects the consistency of the system by disallowing + statements that cannot be replicated correctly, or for which replication is + not yet supported. Entirely prohibited statements are covered above in + ; commands where + some subcommands or features are limited are covered below. diff --git a/doc/manual-functions.sgml b/doc/manual-functions.sgml index a2abdb91b7..6ff754ce67 100644 --- a/doc/manual-functions.sgml +++ b/doc/manual-functions.sgml @@ -172,7 +172,7 @@ - + &bdr;/&udr; @@ -187,6 +187,10 @@ extension on all peer nodes. This function is useful mainly for replicating DDL in an &udr; setup where the transparent DDL replication is not available. + The same limitations apply to this function as to DDL run directly + by the user, except that DDL not normally replicated by &bdr; + will be replicated if run with this function; see + . -- 2.39.5