From: Craig Ringer Date: Tue, 24 Mar 2015 17:18:45 +0000 (+0800) Subject: doc: Write some conflict docs, partly cribbed from wiki, partly new X-Git-Url: http://git.postgresql.org/gitweb/static/gitweb.js?a=commitdiff_plain;h=a9c88110b35993b50b46f17cdb0cd9c104268154;p=2ndquadrant_bdr.git doc: Write some conflict docs, partly cribbed from wiki, partly new --- diff --git a/doc/manual-conflicts.sgml b/doc/manual-conflicts.sgml index 57c1435976..89b57c4249 100644 --- a/doc/manual-conflicts.sgml +++ b/doc/manual-conflicts.sgml @@ -10,31 +10,306 @@ different nodes can result in data conflicts. - + + Some clustering systems use distributed lock mechanisms to prevent + concurrent access to data. These can perform reasonably when servers are + very close but cannot support geographically distributed applications as + very low latency is critical for acceptable performance. + + + + Distributed locking is essentially a pessimistic approach, whereas BDR + advocates an optimistic approach: avoid conflicts where possible but allow + some types of conflict to occur and and resolve them when they arise. + + + + How conflicts happen + + + Inter-node conflicts arise as a result of sequences of events that could not + happen if all the involved transactions happened concurrently on the same + node. Because the nodes only exchange changes after transactions commit, each + transaction is individually valid on the node it committed on but would not + be valid if run on another node that has done other work in the mean time. + Since &bdr; apply essentially replays the transaction on the other nodes, the + replay operation can fail if there is a conflict between a transaction being + applied and a transaction that was committed on the receiving node. + + + + The reason most conflicts can't happen when all transactions run on a single + node is that PostgreSQL has inter-transaction communication mechanisms + to prevent it - UNIQUE indexes, + SEQUENCEs, row and relation locking, + SERIALIZABLE dependency tracking, etc. All of these + mechanisms are ways to communicate between transactions to prevent + undesirable concurrency issues. + + + + &bdr; does not have a distributed transaction manager or lock manager. + That's part of why it performs well with latency and network partitions. As + a result, so transactions on different nodes execute entirely in + isolation from each other. Despite the usual perception that + "more isolation is good" you actually need to reduce isolation to prevent + conflicts. + + + + Types of conflict + + + Row conflicts + + + The most common conflicts are row conflicts where two operations affect a + row with the same key in ways they could not do on a single node. &bdr; can + detect most of those and apply last-update-wins conflict handling or invoke + a user-defined conflict handler. + + + + Row conflicts include: + + INSERT vs INSERT + INSERT vs UPDATE + UPDATE vs DELETE + INSERT vs DELETE + + + + + + + Constraint conflicts + + + Constraint conflicts can also occur, mainly with foreign keys. These are + usually transient issues that arise from transactions being applied in a + different order to the order they appeared to occur logically on the nodes + that originated them. + + + + While apply is strictly ordered for any given origin node, there is no + enforcemnet of ordering of transactions between two different nodes, so + it's possible for (e.g.) node1 to insert a row into T1, which is replayed to node2. + node2 inserts a row into T2 which has a foreign key reference to the row from T1. + On node3, if the transaction from node2 that inserts the row into T2 is received + before the transaction from node1 that inserts the row into T1, + the transaction from node2 will fail to apply. This failure will record a rollback + in and an ERROR + with details in the PostgreSQL error log on the applying node (node3). In + this case &bdr; will retry the transaction from node2 periodicially, so + once it's replayed the transaction from node1 that it depends on the + transaction will commit successfully. + + + + Constraint conflicts are generally transient and require no administrator action. + + + + + Constraint conflicts are the reason why &bdr; does not support exclusion + constraints. In a distributed asynchronous system it is not possible to + ensure that no set of rows that violates the constraint exists, because + all transactions on different nodes are fully isolated. Exclusion constraints + would lead to replay deadlocks where replay could not progress from any + node to any other node because of exclusion constraint violations. + + + + + + + Global data conflicts + + + Conflicts can also arise where nodes have global (PostgreSQL-system-wide) + data, like roles, that differs. This can result in operations - mainly + DDL - that can be run successfully and committed + on one node, but then fail to apply to other nodes. + + + + For example, node1 might have a user named + fred, but that user was not created on node2. + &bdr; does not replicate CREATE USER (see + ) so this situation can arise easily. + If fred on node1 creates a table, it will + be replicated with its owner set to fred. + When the DDL command is applied to node2 the DDL will fail + because there is no user named fred. + This failure will emit an ERROR in the + PostgreSQL logs on node2 and increment + .nr_rollbacks. + + + + Administrator intervention is required to resolve this conflict + by creating the user fred on node2. + (It need not have the same permissions, but must exist). + + + + + + Lock conflicts and deadlock aborts + + + Because &bdr; apply processes operate very like normal user sessions + they are subject to the usual rules around row and table locking. This + can sometimes lead to &bdr; apply processes waiting on locks held + by user transactions, or even by each other. + + + + Relevant locking includes; + + explicit table-level locking (LOCK TABLE ...) by user sessions + explicit row level locking (SELECT ... FOR UPDATE/FOR SHARE) by user sessions + locking from foreign keys + implicit locking because of row UPDATEs, INSERTs or DELETEs, either from local activity or apply from other servers + + + + + It is even possible for a &bdr; apply process to deadlock with a user + transaction, where the user transaction is waiting on a lock held + by the apply process and vice versa. Two apply processes may also + deadlock with each other. PostgreSQL's deadlock detector will + step in and terminate one of the problem transactions. If the &bdr; apply + worker's process is terminated it will simply retry and generally succeed. + + + + All these issues are transient and generally require no administrator + action. If an apply process is stuck for a long time behind a lock + on an idle user session the administrator may choose to terminate + the user session to get replication flowing again, but this is + no different to a user holding a long lock that impacts another + user session. + + + + Use of the + log_lock_waits facility in PostgreSQL can help identify locking + related replay stalls. + + + + + + Divergent conflicts + + + Divergent conflicts arise when data that should be the same on different + nodes differs unexpectedly. Divergent conflicts should not occur, but not + all such conflicts can be reliably prevented at time of writing. + + + + + Changing the PRIMARY KEY of a row can lead to a + divergent conflict if another node changes the key of the same row before + all nodes have replayed the change. Avoid changing primary keys, or + change them only on one designated node. + + + + + Divergent conflicts involving row data generally require administrator + action to manually adjust the data on one of the nodes to be consistent + with the other one while replication is temporarily disabled using . Such conflicts should not arise + so long as &bdr; is used as documented and settings or functions marked + as unsafe are avoided. + + + + + + + + + + + + Avoiding or tolerating conflicts + - The documentation on multi-master conflicts is under ongoing development and - will see significant enhancements in the next point release. + In most cases appropriate application design can be used to avoid conflicts + and/or the application can be made tolerant of conflicts. - - + + Conflicts can only happen if there are things happening at the same time on + multiple nodes, so the simplest way to avoid conflicts is to only ever write + to one node, or to only ever write to independent subsets of the database on + each node. For example, each node might have a separate schema, and while + they all exchange data with each other, writes are only ever performed on + the node that "owns" a given schema. + + + + For INSERT vs INSERT conflicts, use of + can completely prevent conflicts. + - - Types of conflict + + BDR users may sometimes find it useful to perform distributed locking at the + application level in cases where conflicts are not acceptable. + - Many conflict types are possible: - - INSERT/INSERT - INSERT/UPDATE - UPDATE/DELETE - INSERT/DELETE - Transient unhandled conflicts, e.g. foreign key constraint replay order conflicts - Divergent conflicts - + The best course of action is frequently to allow conflicts to occur and + design the application to work with &bdr;'s conflict resolution + mechansisms to cope with the conflict. See . - + + + + Conflict resolution + + + When &bdr; detects data row conflicts it ensures that the conflict is + handled the same way on all nodes, so that the resulting state of the table + is the same for every node. + + + + By default &bdr; uses a last-update-wins strategy + to resolve conflicts. The most recent update will be retained, + and the oldest update discarded in its entirety. No attempt to merge + the data in the conflicting rows is made. + + + + Users may override this behaviour with application-specific knowledge + in . + + + + + + + + User defined conflict handlers + + + &bdr; provides facilities for users to override the default last-update-wins + data row conflict resolution strategy. + + + + + + See also: + diff --git a/doc/manual-settings.sgml b/doc/manual-settings.sgml index 48b14e7e9a..ea00f0cb29 100644 --- a/doc/manual-settings.sgml +++ b/doc/manual-settings.sgml @@ -377,6 +377,27 @@ + + bdr.do_not_replicate (boolean) + + bdr.do_not_replicate configuration parameter + + + + + This parameter is intended for internal use only. Changes made in a + transaction with this parameter set will not be queued for replication + to other nodes. + + + + Inconsiderate usage of this option easily allows to break + replication setups. + + + + +