David E. Wheeler [Fri, 7 Dec 2012 01:05:01 +0000 (17:05 -0800)]
Try always committing deltas on 8.4 and higher.
If we always log deltas and kick queues, we *never* need a `makedelta` option.
I'm not sure if this will really work in practice, however, for these reasons:
* I'm not entirely sure how Bucardo will handle the dupes. Given the previous
presence of `makedelta`, I'm assuming fine, but even so I'm not sure we want
to *always* enable them: it could create a lot more traffic between nodes.
This could potentially be mediated by adding separate delta functions for
each table and sync, but that might be a PITA.
* There is a failing test in t/40-postgres.t -- the very last test, in fact.
Something related to customcols: Database D ends up with the rows values 30,
60, 90, and 210 rather than 2, 3, 7, and 30. I am unable to figure out why
this would be.
* There may be some other consequence of doing this that could break things
that I, in my ignorance, and completely unaware of.
* This leaves older versions of PostgreSQL behind. But maybe we don't care?
8.3 is mighty old (I suspect that today's release of 8.3.22 will be the
last).
However, given all that, if this could be made to work, it's pretty damned
simple! If not, please revert this and we can come up with something else.
(Ref issue #19).
David E. Wheeler [Fri, 7 Dec 2012 00:58:20 +0000 (16:58 -0800)]
Get `help add customcols` working.
David E. Wheeler [Fri, 7 Dec 2012 00:23:01 +0000 (16:23 -0800)]
Fix failing fullcopy test.
The code deleted here was added in
63a425f0ef2b018f03eef9dd7486a7c1a9b2a390.
However, in my testing on PostgreSQL 9.2, the sequence replication tests in
t/10-fullcopy.t failed. I tried changing `defined` to `exists` but it did not
help. So I'm thinking there must be a better way to deal with sequence
attributes in older versions of PostgreSQL, though I don't know what it might
be.
David E. Wheeler [Fri, 7 Dec 2012 00:12:43 +0000 (16:12 -0800)]
Fix test failuers on later Postgres releases.
In
2d7f462e2a3ccc0ee370e10aac8811c33fbde999, Greg changed the calls to `ALTER
SEQUENCE` so that they work for older versions of PostgreSQL. However, it
broke newer versions (I'm testing on 9.2.2). It looks like the `START` clause
was added in 8.4, so let's use it as we did before for 8.4 and higher and
leave it out for 8.3 and older.
David E. Wheeler [Thu, 6 Dec 2012 22:51:43 +0000 (14:51 -0800)]
Update TODOs.
David E. Wheeler [Thu, 6 Dec 2012 22:44:47 +0000 (14:44 -0800)]
Add failing makedelta test.
Towards getting `makedelta` working again (Issue #19).
There is a `wait_for_notice` commented out, and a failing assertion on line
88. Once makedelta is fixed, the test should pass and the commented-out test
should be uncommented.
David E. Wheeler [Tue, 27 Nov 2012 00:35:21 +0000 (16:35 -0800)]
Remove `update relgroup`.
It was never implemented. Closes #22.
Greg Sabino Mullane [Tue, 27 Nov 2012 00:16:06 +0000 (19:16 -0500)]
Bare minimum seems to be 8.2. Tried 8.1 and it has lots of issues: too many to worry about fixing.
Greg Sabino Mullane [Fri, 16 Nov 2012 15:49:35 +0000 (10:49 -0500)]
Facebook
Greg Sabino Mullane [Fri, 9 Nov 2012 14:22:23 +0000 (09:22 -0500)]
More testing workarounds to support non-truncate-trigger Postgres versions. WIP.
Greg Sabino Mullane [Thu, 8 Nov 2012 22:39:46 +0000 (17:39 -0500)]
Support older versions by simply skipping unknown sequence attributes.
Greg Sabino Mullane [Thu, 8 Nov 2012 16:04:07 +0000 (11:04 -0500)]
Minor changes to support older PG versions
Greg Sabino Mullane [Thu, 8 Nov 2012 12:57:33 +0000 (07:57 -0500)]
Make sure we support PGBINDIR inside bucardo for the psql variable too.
Greg Sabino Mullane [Thu, 8 Nov 2012 03:46:14 +0000 (22:46 -0500)]
Notes about testing different versions of Postgres
Greg Sabino Mullane [Thu, 8 Nov 2012 03:18:55 +0000 (22:18 -0500)]
We might allow synonyms, but we need to make sure the tests are looking for the returned literal 'relgroup', not 'herd'
David E. Wheeler [Wed, 7 Nov 2012 19:59:54 +0000 (11:59 -0800)]
Add support for multiple log destinations to chkconfig script.
Also sleep for 2 seconds between start and stop in the `restart` command.
David E. Wheeler [Wed, 7 Nov 2012 00:54:08 +0000 (16:54 -0800)]
Update Changes in bucardo.spec.
Greg Sabino Mullane [Tue, 6 Nov 2012 23:59:03 +0000 (18:59 -0500)]
Signature for version 4.99.6 (Version 5, beta 6)
Greg Sabino Mullane [Tue, 6 Nov 2012 22:38:34 +0000 (17:38 -0500)]
Add more logging for conflict handling activity.
Greg Sabino Mullane [Tue, 6 Nov 2012 22:38:09 +0000 (17:38 -0500)]
This test is still a work in progress.
Greg Sabino Mullane [Tue, 6 Nov 2012 20:19:08 +0000 (15:19 -0500)]
No tabs
Greg Sabino Mullane [Tue, 6 Nov 2012 20:18:58 +0000 (15:18 -0500)]
Skip patch files when checking for tabs.
Greg Sabino Mullane [Tue, 6 Nov 2012 20:18:45 +0000 (15:18 -0500)]
glog call forgot the level.
David E. Wheeler [Tue, 6 Nov 2012 19:33:11 +0000 (11:33 -0800)]
Keep all the custom code types in one place.
David E. Wheeler [Tue, 6 Nov 2012 19:16:52 +0000 (11:16 -0800)]
Ignore MANIFEST.bak.
David E. Wheeler [Tue, 6 Nov 2012 19:16:15 +0000 (11:16 -0800)]
Update manifest with `make manifest`.
Exclude .db and .bak files.
David E. Wheeler [Tue, 6 Nov 2012 19:09:12 +0000 (11:09 -0800)]
Fix failing test.
Just needed quotation marks.
David E. Wheeler [Tue, 6 Nov 2012 19:05:58 +0000 (11:05 -0800)]
Put custom code into its own package.
So that globals are not immediately available. Also set $_ to the same value
as the hash ref passed in.
Greg Sabino Mullane [Tue, 6 Nov 2012 03:25:02 +0000 (22:25 -0500)]
Spelling updates.
Greg Sabino Mullane [Mon, 5 Nov 2012 22:59:26 +0000 (17:59 -0500)]
Add our example customcode file to the manifest.
Greg Sabino Mullane [Mon, 5 Nov 2012 22:26:32 +0000 (17:26 -0500)]
More customcode work.
David E. Wheeler [Mon, 5 Nov 2012 20:11:11 +0000 (12:11 -0800)]
Allow custom code status to be set.
David E. Wheeler [Mon, 5 Nov 2012 20:05:43 +0000 (12:05 -0800)]
Use nicer code deref.
David E. Wheeler [Mon, 5 Nov 2012 19:52:11 +0000 (11:52 -0800)]
Improve compilation of custom code.
Poperly use the custom code source as the body of an `eval`ed subroutine
reference, rather than `eval` it on every call. This has two benefits:
* `shift` works properly in the custom code.
* We no longer need the `dummy` key, as the compilation should give us all we
need.
I've also `local`ed `$_` to prevent leakage. Frankly, we might want to have it
`eval`ed in a completely separate package with all globals reset. Or better
yet, use Safe.pm.
If we still want to execute the function before we approve it, we can put the
`dummy` back, but I don't think that gains us anything, as nothing really runs
in that case other than the check for `dummy`.
David E. Wheeler [Mon, 5 Nov 2012 19:13:15 +0000 (11:13 -0800)]
Document more keys in the hash passed to custom codes.
David E. Wheeler [Mon, 5 Nov 2012 18:36:40 +0000 (10:36 -0800)]
Add description of each customcode item that can be updated.
David E. Wheeler [Mon, 5 Nov 2012 18:32:41 +0000 (10:32 -0800)]
Basic test of `update customcode`.
Gets customcode exception test passing again.
David E. Wheeler [Mon, 5 Nov 2012 17:59:32 +0000 (09:59 -0800)]
Change customcode.status default to "active".
It is a text column, not a boolean, and Bucardo.pm looks for "active".
Greg Sabino Mullane [Mon, 5 Nov 2012 17:41:33 +0000 (12:41 -0500)]
Merge branch 'master' of ln6.endpoint.com:/var/lib/git/bucardo
Greg Sabino Mullane [Mon, 5 Nov 2012 17:40:53 +0000 (12:40 -0500)]
Work on supporting 'bucardo update customcode'
Still needs to possibly support more attributes.
Also may remove the priority, if it makes more sense just in the customcode_map
David E. Wheeler [Mon, 5 Nov 2012 17:20:10 +0000 (09:20 -0800)]
Fix serialization and deadlock error detection.
0406ecb added deadlock detection, but forgot to put the list of error states
to check after the call to `first {}` for both serialization and deadlock
detection. So fix that. The serialization test passes again.
David E. Wheeler [Mon, 5 Nov 2012 17:15:09 +0000 (09:15 -0800)]
Chomp schema lines before adding configs.
David E. Wheeler [Mon, 5 Nov 2012 17:01:41 +0000 (09:01 -0800)]
Duh, $count can be 0E0.
Greg Sabino Mullane [Mon, 5 Nov 2012 16:49:54 +0000 (11:49 -0500)]
Complete and working test for exception handling!
Greg Sabino Mullane [Mon, 5 Nov 2012 16:49:25 +0000 (11:49 -0500)]
Add status field to the customcode table, so we can easily toggle them off if needed.
Greg Sabino Mullane [Mon, 5 Nov 2012 15:12:03 +0000 (10:12 -0500)]
Tweak customcode exception example
Greg Sabino Mullane [Mon, 5 Nov 2012 03:57:26 +0000 (22:57 -0500)]
More cleanups of the exception custom handler code.
Greg Sabino Mullane [Sun, 4 Nov 2012 16:30:07 +0000 (11:30 -0500)]
Goodbye customcode.trigrules
Greg Sabino Mullane [Sun, 4 Nov 2012 16:25:38 +0000 (11:25 -0500)]
Spelling
Greg Sabino Mullane [Sun, 4 Nov 2012 03:39:32 +0000 (23:39 -0400)]
Clean up the customcode SQL so it no longer errors out when no matching goat-level customcodes.
Greg Sabino Mullane [Sun, 4 Nov 2012 03:25:58 +0000 (23:25 -0400)]
Chipping away on some cc exception work.
Greg Sabino Mullane [Sun, 4 Nov 2012 03:16:39 +0000 (23:16 -0400)]
Make a todo about showing deadlock information in payload.
Greg Sabino Mullane [Sun, 4 Nov 2012 02:28:34 +0000 (22:28 -0400)]
Sneaky tabs, how we hates them!
Greg Sabino Mullane [Sun, 4 Nov 2012 02:06:15 +0000 (22:06 -0400)]
Be more explicit if we know the reason for failing to install is lack of Pl/PerlU
Greg Sabino Mullane [Sat, 3 Nov 2012 23:33:02 +0000 (19:33 -0400)]
Duh. Should be < 0, not 1. Thanks David.
David E. Wheeler [Sat, 3 Nov 2012 22:42:31 +0000 (15:42 -0700)]
Bump version in RPM spec file.
Greg Sabino Mullane [Sat, 3 Nov 2012 19:32:36 +0000 (15:32 -0400)]
Add dist/ to MANIFEST
Greg Sabino Mullane [Sat, 3 Nov 2012 19:30:44 +0000 (15:30 -0400)]
Bump version to 4.99.6 in preparation for next beta.
No HTML files as 'make html' is making too many changes at the moment.
Greg Sabino Mullane [Sat, 3 Nov 2012 19:22:16 +0000 (15:22 -0400)]
Add some tests to the MANIFEST
Greg Sabino Mullane [Sat, 3 Nov 2012 19:19:21 +0000 (15:19 -0400)]
Add test skeleton for custom code exception handling.
Greg Sabino Mullane [Sat, 3 Nov 2012 19:11:54 +0000 (15:11 -0400)]
Uncuddle for consistency.
Greg Sabino Mullane [Sat, 3 Nov 2012 19:04:07 +0000 (15:04 -0400)]
Get rid of that pesky -w: causing problems with ancient versions of "env"
Greg Sabino Mullane [Sat, 3 Nov 2012 19:03:09 +0000 (15:03 -0400)]
Quick addition of kid retry on deadlock, similar to serialization. Also allow to set to -1 to prevent the kid from retrying at all. Although the controller may restart a new kid anyway, this is a start.
Greg Sabino Mullane [Sat, 3 Nov 2012 18:39:01 +0000 (14:39 -0400)]
Revert "Don't use /usr/bin/env."
This reverts commit
790ad8c1ecf28fb37ac8e0ac654cecedc8b88eaa.
No, we are not going to hardcode /usr/bin/perl, as this will break things for at least one major user of Bucardo. If the -w is the problem, remove that, but keep /usr/bin/env please.
David E. Wheeler [Sat, 3 Nov 2012 17:10:24 +0000 (10:10 -0700)]
Don't use /usr/bin/env.
Older versions do not recognize when options are passed to the program. IOW, `/usr/bin/env perl -w` dies.
David E. Wheeler [Fri, 2 Nov 2012 21:40:36 +0000 (14:40 -0700)]
Add test for serialization failure.
Should work on Postgres 8.4 and higher, and skip all on older versions. Works
by adding a sequence and a trigger to a target table, and configuring the
trigger to fire only when the `session_replica_role` is "replica". This means
that it fires when data is `COPY`ed into the table from the source.
The trigger calls a PL/pgSQL function that consults a sequence, and thows a
serialization failure exception when the next value from the sequence is odd.
It does not throw it when its value is even. The result is that the first
attempt to copy the data into the table throws the serialization exception,
but the second attempt does not.
The test checks that the serialization failure happens by listening for the
`syncsleep` notification. If it doesn't come, there was no serialization
failure, and the test fails. If there is, it then tests that the sync finished
anyway and that the new data was copied over.
Resolves #29.
In passing, I've also changed the formatting of the message about the duration
of the sleep to use a stringification of that time, rather than turning it
into an integer, since the new default is 0.5 (and using `%f` yieled too many
decimal places). And if the sleep time is 0, we don't call sleep and send
a slightly different message.
It might make sense to send the sleep nofication only if we're going to sleep,
and then to always send another message, say serialretry, just before trying
again. I think that would be cleaner, since it does not actually always sleep.
Thoughts?
David E. Wheeler [Fri, 2 Nov 2012 16:47:14 +0000 (09:47 -0700)]
No, really properly check for the old bucarodo_sequences table.
The query returns true if the new table is present, not if the old one
is present.
David E. Wheeler [Fri, 2 Nov 2012 16:45:53 +0000 (09:45 -0700)]
Properly check for old bucardo_sequenes table.
The logic was off before.
David E. Wheeler [Fri, 2 Nov 2012 16:39:05 +0000 (09:39 -0700)]
Properly check for presence of old bucardo_sequences table.
`$count` will be true if it exists, and false if it does not. The issue was
not to do with `execute()`, as I originally thought; @machack666 was right
about that. Resolves #31.
David E. Wheeler [Fri, 2 Nov 2012 00:33:27 +0000 (17:33 -0700)]
Require Pod::Usage and List::Util at runtime.
David E. Wheeler [Fri, 2 Nov 2012 00:16:32 +0000 (17:16 -0700)]
Fix some issues with `add sequnce`.
David E. Wheeler [Fri, 2 Nov 2012 00:02:40 +0000 (17:02 -0700)]
Fix `list_sequences()`.
David E. Wheeler [Thu, 1 Nov 2012 23:44:48 +0000 (16:44 -0700)]
Update Changes since 4.99.5.
David E. Wheeler [Thu, 1 Nov 2012 23:13:05 +0000 (16:13 -0700)]
Require List::Util and Pod::Usage in the spec file.
David E. Wheeler [Thu, 1 Nov 2012 23:07:11 +0000 (16:07 -0700)]
Show both users if try to connect as "postgres" and "bucardo".
Took me an hour to chase down a bad login because I kept telling it to use
user bucardo and it said it could not connect as user "postgres". It was only
when I dug deep enough that I discovered that it was sneakily changing the
username on me. So let's just show both user in the error message.
David E. Wheeler [Thu, 1 Nov 2012 22:41:57 +0000 (15:41 -0700)]
Use lexical subs, not globals.
Globals are not a very safe thing to use, as they persist beyond the execution
of the function in the global namespace. There is simply no need for that kind
of pollution and leakage.
David E. Wheeler [Thu, 1 Nov 2012 22:35:13 +0000 (15:35 -0700)]
Forgot to rename some standard_conflict columns.
No idea why tests were not failing.
David E. Wheeler [Thu, 1 Nov 2012 22:28:47 +0000 (15:28 -0700)]
Eliminate remaining `goto`s from PL/Perl.
The use of "goto" to jump into a construct has been deprecated since Perl
5.12. No need to fill up logs with the deprecation warning when there are
much cleaner way to do conditionals.
This patch is bigger than it looks: most changes are indentation. Run `git
diff -w` to see what it really looks like.
David E. Wheeler [Thu, 1 Nov 2012 22:14:21 +0000 (15:14 -0700)]
Eliminate `goto KICKFUNC`.
David E. Wheeler [Thu, 1 Nov 2012 21:58:28 +0000 (14:58 -0700)]
Update init script and RPM patch.
David E. Wheeler [Thu, 1 Nov 2012 21:54:39 +0000 (14:54 -0700)]
Sleep for 0.5s after serialization failure.
By default, and rather than the previous 10s time. Kevin Gritter tells us that
we could probably eliminate the sleep, as the chances of a conflict after a
serialization rollback are extremely low. Even 20ms would be generous, he
says. Going with 0.5s (500 ms) just because it's the same value as the default
`kid_sleep`.
David E. Wheeler [Thu, 1 Nov 2012 21:26:22 +0000 (14:26 -0700)]
Tweak formatting of config settings in Pod.
David E. Wheeler [Thu, 1 Nov 2012 20:37:18 +0000 (13:37 -0700)]
Replace --debug options with --log options.
`--debugdir`, `--debugsyslog`, and `--debugfile` have all been folded into
`--debug-destination`. This option may be specified multiple times, with the
values "syslog", "stderr", "stdout", "none", or a directory name. This means
you can log to a directory, syslog, and STDERR all at the same time if you
like. The old options are still around in name, but their values are used only
if `--log-destination` is not, and then they are adapted to it.
Implementation-wise, there is now a new method, _logto(), that returns a list
of code references, one for each destination. glog() simply passes the log
header and message to each code reference in turn. _logto() also notices when
the process has been forked and creates new code refernces as appropriate.
This is especially important when --log-separate is specified. If
--log-destination is passed "none", there will be no logging at all (even if
there are other calls to log-destination).
Relatedly:
* `--debugfilesep` has been renamed to `--log-separate`.
* `--debugname` has been renamed to `--log-extension`.
* `--cleandebugs` has been renamed to `log-clean`.
* If there are "stderr" or "stdout" destinations, the corrsponding file
handles are no longer closed once the MCP forks.
* Logging to the warning file no longer happens when the string "Warning"
appears in the log message, but when th elog leve is LOG_WARN.
David E. Wheeler [Wed, 31 Oct 2012 23:42:01 +0000 (16:42 -0700)]
Allow --no-debug and document --debug.
David E. Wheeler [Wed, 31 Oct 2012 23:34:26 +0000 (16:34 -0700)]
Implement `remove_sequence()`.
It's the same as `remove_table()`, so I just factored the common code into its
own function. As a result, `remove_table()` will no longer inadvertently
remove sequences.
I also fixed a bug where it would report removing tables that it had not,
in fact, removed.
Resolves #21.
David E. Wheeler [Wed, 31 Oct 2012 23:23:01 +0000 (16:23 -0700)]
Add `makedelta` to `add table` and `update table`.
While at it, fix `update_table()` so that it actually works! It was accessing
the hash of columns as an array, rather than a hash.
David E. Wheeler [Wed, 31 Oct 2012 23:14:18 +0000 (16:14 -0700)]
More sync makedelta stuff removed. All unused.
David E. Wheeler [Wed, 31 Oct 2012 23:07:00 +0000 (16:07 -0700)]
Remove mention of `makedelta` from the schema docs.
It was removed in
002c58cad7.
David E. Wheeler [Wed, 31 Oct 2012 23:00:18 +0000 (16:00 -0700)]
Remove double periods.
David E. Wheeler [Wed, 31 Oct 2012 22:58:46 +0000 (15:58 -0700)]
Document all the configuration settings.
Include a comment on how to re-generate the list from a new install as needed.
David E. Wheeler [Wed, 31 Oct 2012 00:14:26 +0000 (17:14 -0700)]
Document and display all the `update` actions.
Discovered a bug in Pod::Usage in the process, reported here:
https://rt.perl.org/rt3//Public/Bug/Display.html?id=115534. The workaround is
to force Pod::Usage to inherit from Pod::PlainText instead of Pod::Text when
using negation -- that is, for the output of `help add` and `help update`,
where we don't want to show all the subcommands.
Anyway, I think this covers all the casses we need. Resolves #25.
David E. Wheeler [Tue, 30 Oct 2012 19:32:55 +0000 (12:32 -0700)]
Handle aliases and unknown commands in `superhelp()`.
All the known commands and actions, including any aliases or mis-spellings,
need to go into `%names, so if new ones are added, document them in Pod and
put them there. Ref issue #25.
David E. Wheeler [Tue, 30 Oct 2012 18:57:15 +0000 (11:57 -0700)]
Replace `usage()` with `usage_exit()`.
Rather than passing around a usage statement, just pass around the key for
where to find it in the Pod. Then, when usage is needed, call `usage_exit()`
and *then* load Pod::Usage, find the section, show it, and exit. Bonus: no
more use of scalar ref `open`.
David E. Wheeler [Tue, 30 Oct 2012 17:59:13 +0000 (10:59 -0700)]
Fix output of `superhelp()`.
David E. Wheeler [Tue, 30 Oct 2012 17:52:11 +0000 (10:52 -0700)]
Add some more hyphens and alternates.
David E. Wheeler [Tue, 30 Oct 2012 17:34:18 +0000 (10:34 -0700)]
Use Getopt::Long for bucardorc options.
And document them.
David E. Wheeler [Tue, 30 Oct 2012 17:00:01 +0000 (10:00 -0700)]
Add some hyphens and short names for options.
Short names are the same (except for password) as offered by `psql`.
David E. Wheeler [Tue, 30 Oct 2012 16:30:00 +0000 (09:30 -0700)]
Add options table and move command-specific options to their sections.
Ref issue #25.
David E. Wheeler [Tue, 30 Oct 2012 03:53:20 +0000 (20:53 -0700)]
Fix help- and usage-based test failures.
The switch to Pod::Usage broke quite a few things. I failed to specify the
proper place to look for things when `usage()` was called from withing functions
actually doing stuff. So do that.
Also change the `=head3`s under `add` to include the word `add`. This allows
them to stand better on their own. Relatedly, add a simple list of types under
`=head2 add` so that there is a reference for the user to easily know what's
available to look up more. Ref GitHub issue #25.
Greg Sabino Mullane [Tue, 30 Oct 2012 02:13:41 +0000 (22:13 -0400)]
Start cleaning up the maekdelta system. Sync-level makedeltas make no sense right now.