Greg Sabino Mullane [Tue, 6 Nov 2012 03:25:02 +0000 (22:25 -0500)]
Spelling updates.
Greg Sabino Mullane [Mon, 5 Nov 2012 22:59:26 +0000 (17:59 -0500)]
Add our example customcode file to the manifest.
Greg Sabino Mullane [Mon, 5 Nov 2012 22:26:32 +0000 (17:26 -0500)]
More customcode work.
David E. Wheeler [Mon, 5 Nov 2012 20:11:11 +0000 (12:11 -0800)]
Allow custom code status to be set.
David E. Wheeler [Mon, 5 Nov 2012 20:05:43 +0000 (12:05 -0800)]
Use nicer code deref.
David E. Wheeler [Mon, 5 Nov 2012 19:52:11 +0000 (11:52 -0800)]
Improve compilation of custom code.
Poperly use the custom code source as the body of an `eval`ed subroutine
reference, rather than `eval` it on every call. This has two benefits:
* `shift` works properly in the custom code.
* We no longer need the `dummy` key, as the compilation should give us all we
need.
I've also `local`ed `$_` to prevent leakage. Frankly, we might want to have it
`eval`ed in a completely separate package with all globals reset. Or better
yet, use Safe.pm.
If we still want to execute the function before we approve it, we can put the
`dummy` back, but I don't think that gains us anything, as nothing really runs
in that case other than the check for `dummy`.
David E. Wheeler [Mon, 5 Nov 2012 19:13:15 +0000 (11:13 -0800)]
Document more keys in the hash passed to custom codes.
David E. Wheeler [Mon, 5 Nov 2012 18:36:40 +0000 (10:36 -0800)]
Add description of each customcode item that can be updated.
David E. Wheeler [Mon, 5 Nov 2012 18:32:41 +0000 (10:32 -0800)]
Basic test of `update customcode`.
Gets customcode exception test passing again.
David E. Wheeler [Mon, 5 Nov 2012 17:59:32 +0000 (09:59 -0800)]
Change customcode.status default to "active".
It is a text column, not a boolean, and Bucardo.pm looks for "active".
Greg Sabino Mullane [Mon, 5 Nov 2012 17:41:33 +0000 (12:41 -0500)]
Merge branch 'master' of ln6.endpoint.com:/var/lib/git/bucardo
Greg Sabino Mullane [Mon, 5 Nov 2012 17:40:53 +0000 (12:40 -0500)]
Work on supporting 'bucardo update customcode'
Still needs to possibly support more attributes.
Also may remove the priority, if it makes more sense just in the customcode_map
David E. Wheeler [Mon, 5 Nov 2012 17:20:10 +0000 (09:20 -0800)]
Fix serialization and deadlock error detection.
0406ecb added deadlock detection, but forgot to put the list of error states
to check after the call to `first {}` for both serialization and deadlock
detection. So fix that. The serialization test passes again.
David E. Wheeler [Mon, 5 Nov 2012 17:15:09 +0000 (09:15 -0800)]
Chomp schema lines before adding configs.
David E. Wheeler [Mon, 5 Nov 2012 17:01:41 +0000 (09:01 -0800)]
Duh, $count can be 0E0.
Greg Sabino Mullane [Mon, 5 Nov 2012 16:49:54 +0000 (11:49 -0500)]
Complete and working test for exception handling!
Greg Sabino Mullane [Mon, 5 Nov 2012 16:49:25 +0000 (11:49 -0500)]
Add status field to the customcode table, so we can easily toggle them off if needed.
Greg Sabino Mullane [Mon, 5 Nov 2012 15:12:03 +0000 (10:12 -0500)]
Tweak customcode exception example
Greg Sabino Mullane [Mon, 5 Nov 2012 03:57:26 +0000 (22:57 -0500)]
More cleanups of the exception custom handler code.
Greg Sabino Mullane [Sun, 4 Nov 2012 16:30:07 +0000 (11:30 -0500)]
Goodbye customcode.trigrules
Greg Sabino Mullane [Sun, 4 Nov 2012 16:25:38 +0000 (11:25 -0500)]
Spelling
Greg Sabino Mullane [Sun, 4 Nov 2012 03:39:32 +0000 (23:39 -0400)]
Clean up the customcode SQL so it no longer errors out when no matching goat-level customcodes.
Greg Sabino Mullane [Sun, 4 Nov 2012 03:25:58 +0000 (23:25 -0400)]
Chipping away on some cc exception work.
Greg Sabino Mullane [Sun, 4 Nov 2012 03:16:39 +0000 (23:16 -0400)]
Make a todo about showing deadlock information in payload.
Greg Sabino Mullane [Sun, 4 Nov 2012 02:28:34 +0000 (22:28 -0400)]
Sneaky tabs, how we hates them!
Greg Sabino Mullane [Sun, 4 Nov 2012 02:06:15 +0000 (22:06 -0400)]
Be more explicit if we know the reason for failing to install is lack of Pl/PerlU
Greg Sabino Mullane [Sat, 3 Nov 2012 23:33:02 +0000 (19:33 -0400)]
Duh. Should be < 0, not 1. Thanks David.
David E. Wheeler [Sat, 3 Nov 2012 22:42:31 +0000 (15:42 -0700)]
Bump version in RPM spec file.
Greg Sabino Mullane [Sat, 3 Nov 2012 19:32:36 +0000 (15:32 -0400)]
Add dist/ to MANIFEST
Greg Sabino Mullane [Sat, 3 Nov 2012 19:30:44 +0000 (15:30 -0400)]
Bump version to 4.99.6 in preparation for next beta.
No HTML files as 'make html' is making too many changes at the moment.
Greg Sabino Mullane [Sat, 3 Nov 2012 19:22:16 +0000 (15:22 -0400)]
Add some tests to the MANIFEST
Greg Sabino Mullane [Sat, 3 Nov 2012 19:19:21 +0000 (15:19 -0400)]
Add test skeleton for custom code exception handling.
Greg Sabino Mullane [Sat, 3 Nov 2012 19:11:54 +0000 (15:11 -0400)]
Uncuddle for consistency.
Greg Sabino Mullane [Sat, 3 Nov 2012 19:04:07 +0000 (15:04 -0400)]
Get rid of that pesky -w: causing problems with ancient versions of "env"
Greg Sabino Mullane [Sat, 3 Nov 2012 19:03:09 +0000 (15:03 -0400)]
Quick addition of kid retry on deadlock, similar to serialization. Also allow to set to -1 to prevent the kid from retrying at all. Although the controller may restart a new kid anyway, this is a start.
Greg Sabino Mullane [Sat, 3 Nov 2012 18:39:01 +0000 (14:39 -0400)]
Revert "Don't use /usr/bin/env."
This reverts commit
790ad8c1ecf28fb37ac8e0ac654cecedc8b88eaa.
No, we are not going to hardcode /usr/bin/perl, as this will break things for at least one major user of Bucardo. If the -w is the problem, remove that, but keep /usr/bin/env please.
David E. Wheeler [Sat, 3 Nov 2012 17:10:24 +0000 (10:10 -0700)]
Don't use /usr/bin/env.
Older versions do not recognize when options are passed to the program. IOW, `/usr/bin/env perl -w` dies.
David E. Wheeler [Fri, 2 Nov 2012 21:40:36 +0000 (14:40 -0700)]
Add test for serialization failure.
Should work on Postgres 8.4 and higher, and skip all on older versions. Works
by adding a sequence and a trigger to a target table, and configuring the
trigger to fire only when the `session_replica_role` is "replica". This means
that it fires when data is `COPY`ed into the table from the source.
The trigger calls a PL/pgSQL function that consults a sequence, and thows a
serialization failure exception when the next value from the sequence is odd.
It does not throw it when its value is even. The result is that the first
attempt to copy the data into the table throws the serialization exception,
but the second attempt does not.
The test checks that the serialization failure happens by listening for the
`syncsleep` notification. If it doesn't come, there was no serialization
failure, and the test fails. If there is, it then tests that the sync finished
anyway and that the new data was copied over.
Resolves #29.
In passing, I've also changed the formatting of the message about the duration
of the sleep to use a stringification of that time, rather than turning it
into an integer, since the new default is 0.5 (and using `%f` yieled too many
decimal places). And if the sleep time is 0, we don't call sleep and send
a slightly different message.
It might make sense to send the sleep nofication only if we're going to sleep,
and then to always send another message, say serialretry, just before trying
again. I think that would be cleaner, since it does not actually always sleep.
Thoughts?
David E. Wheeler [Fri, 2 Nov 2012 16:47:14 +0000 (09:47 -0700)]
No, really properly check for the old bucarodo_sequences table.
The query returns true if the new table is present, not if the old one
is present.
David E. Wheeler [Fri, 2 Nov 2012 16:45:53 +0000 (09:45 -0700)]
Properly check for old bucardo_sequenes table.
The logic was off before.
David E. Wheeler [Fri, 2 Nov 2012 16:39:05 +0000 (09:39 -0700)]
Properly check for presence of old bucardo_sequences table.
`$count` will be true if it exists, and false if it does not. The issue was
not to do with `execute()`, as I originally thought; @machack666 was right
about that. Resolves #31.
David E. Wheeler [Fri, 2 Nov 2012 00:33:27 +0000 (17:33 -0700)]
Require Pod::Usage and List::Util at runtime.
David E. Wheeler [Fri, 2 Nov 2012 00:16:32 +0000 (17:16 -0700)]
Fix some issues with `add sequnce`.
David E. Wheeler [Fri, 2 Nov 2012 00:02:40 +0000 (17:02 -0700)]
Fix `list_sequences()`.
David E. Wheeler [Thu, 1 Nov 2012 23:44:48 +0000 (16:44 -0700)]
Update Changes since 4.99.5.
David E. Wheeler [Thu, 1 Nov 2012 23:13:05 +0000 (16:13 -0700)]
Require List::Util and Pod::Usage in the spec file.
David E. Wheeler [Thu, 1 Nov 2012 23:07:11 +0000 (16:07 -0700)]
Show both users if try to connect as "postgres" and "bucardo".
Took me an hour to chase down a bad login because I kept telling it to use
user bucardo and it said it could not connect as user "postgres". It was only
when I dug deep enough that I discovered that it was sneakily changing the
username on me. So let's just show both user in the error message.
David E. Wheeler [Thu, 1 Nov 2012 22:41:57 +0000 (15:41 -0700)]
Use lexical subs, not globals.
Globals are not a very safe thing to use, as they persist beyond the execution
of the function in the global namespace. There is simply no need for that kind
of pollution and leakage.
David E. Wheeler [Thu, 1 Nov 2012 22:35:13 +0000 (15:35 -0700)]
Forgot to rename some standard_conflict columns.
No idea why tests were not failing.
David E. Wheeler [Thu, 1 Nov 2012 22:28:47 +0000 (15:28 -0700)]
Eliminate remaining `goto`s from PL/Perl.
The use of "goto" to jump into a construct has been deprecated since Perl
5.12. No need to fill up logs with the deprecation warning when there are
much cleaner way to do conditionals.
This patch is bigger than it looks: most changes are indentation. Run `git
diff -w` to see what it really looks like.
David E. Wheeler [Thu, 1 Nov 2012 22:14:21 +0000 (15:14 -0700)]
Eliminate `goto KICKFUNC`.
David E. Wheeler [Thu, 1 Nov 2012 21:58:28 +0000 (14:58 -0700)]
Update init script and RPM patch.
David E. Wheeler [Thu, 1 Nov 2012 21:54:39 +0000 (14:54 -0700)]
Sleep for 0.5s after serialization failure.
By default, and rather than the previous 10s time. Kevin Gritter tells us that
we could probably eliminate the sleep, as the chances of a conflict after a
serialization rollback are extremely low. Even 20ms would be generous, he
says. Going with 0.5s (500 ms) just because it's the same value as the default
`kid_sleep`.
David E. Wheeler [Thu, 1 Nov 2012 21:26:22 +0000 (14:26 -0700)]
Tweak formatting of config settings in Pod.
David E. Wheeler [Thu, 1 Nov 2012 20:37:18 +0000 (13:37 -0700)]
Replace --debug options with --log options.
`--debugdir`, `--debugsyslog`, and `--debugfile` have all been folded into
`--debug-destination`. This option may be specified multiple times, with the
values "syslog", "stderr", "stdout", "none", or a directory name. This means
you can log to a directory, syslog, and STDERR all at the same time if you
like. The old options are still around in name, but their values are used only
if `--log-destination` is not, and then they are adapted to it.
Implementation-wise, there is now a new method, _logto(), that returns a list
of code references, one for each destination. glog() simply passes the log
header and message to each code reference in turn. _logto() also notices when
the process has been forked and creates new code refernces as appropriate.
This is especially important when --log-separate is specified. If
--log-destination is passed "none", there will be no logging at all (even if
there are other calls to log-destination).
Relatedly:
* `--debugfilesep` has been renamed to `--log-separate`.
* `--debugname` has been renamed to `--log-extension`.
* `--cleandebugs` has been renamed to `log-clean`.
* If there are "stderr" or "stdout" destinations, the corrsponding file
handles are no longer closed once the MCP forks.
* Logging to the warning file no longer happens when the string "Warning"
appears in the log message, but when th elog leve is LOG_WARN.
David E. Wheeler [Wed, 31 Oct 2012 23:42:01 +0000 (16:42 -0700)]
Allow --no-debug and document --debug.
David E. Wheeler [Wed, 31 Oct 2012 23:34:26 +0000 (16:34 -0700)]
Implement `remove_sequence()`.
It's the same as `remove_table()`, so I just factored the common code into its
own function. As a result, `remove_table()` will no longer inadvertently
remove sequences.
I also fixed a bug where it would report removing tables that it had not,
in fact, removed.
Resolves #21.
David E. Wheeler [Wed, 31 Oct 2012 23:23:01 +0000 (16:23 -0700)]
Add `makedelta` to `add table` and `update table`.
While at it, fix `update_table()` so that it actually works! It was accessing
the hash of columns as an array, rather than a hash.
David E. Wheeler [Wed, 31 Oct 2012 23:14:18 +0000 (16:14 -0700)]
More sync makedelta stuff removed. All unused.
David E. Wheeler [Wed, 31 Oct 2012 23:07:00 +0000 (16:07 -0700)]
Remove mention of `makedelta` from the schema docs.
It was removed in
002c58cad7.
David E. Wheeler [Wed, 31 Oct 2012 23:00:18 +0000 (16:00 -0700)]
Remove double periods.
David E. Wheeler [Wed, 31 Oct 2012 22:58:46 +0000 (15:58 -0700)]
Document all the configuration settings.
Include a comment on how to re-generate the list from a new install as needed.
David E. Wheeler [Wed, 31 Oct 2012 00:14:26 +0000 (17:14 -0700)]
Document and display all the `update` actions.
Discovered a bug in Pod::Usage in the process, reported here:
https://rt.perl.org/rt3//Public/Bug/Display.html?id=115534. The workaround is
to force Pod::Usage to inherit from Pod::PlainText instead of Pod::Text when
using negation -- that is, for the output of `help add` and `help update`,
where we don't want to show all the subcommands.
Anyway, I think this covers all the casses we need. Resolves #25.
David E. Wheeler [Tue, 30 Oct 2012 19:32:55 +0000 (12:32 -0700)]
Handle aliases and unknown commands in `superhelp()`.
All the known commands and actions, including any aliases or mis-spellings,
need to go into `%names, so if new ones are added, document them in Pod and
put them there. Ref issue #25.
David E. Wheeler [Tue, 30 Oct 2012 18:57:15 +0000 (11:57 -0700)]
Replace `usage()` with `usage_exit()`.
Rather than passing around a usage statement, just pass around the key for
where to find it in the Pod. Then, when usage is needed, call `usage_exit()`
and *then* load Pod::Usage, find the section, show it, and exit. Bonus: no
more use of scalar ref `open`.
David E. Wheeler [Tue, 30 Oct 2012 17:59:13 +0000 (10:59 -0700)]
Fix output of `superhelp()`.
David E. Wheeler [Tue, 30 Oct 2012 17:52:11 +0000 (10:52 -0700)]
Add some more hyphens and alternates.
David E. Wheeler [Tue, 30 Oct 2012 17:34:18 +0000 (10:34 -0700)]
Use Getopt::Long for bucardorc options.
And document them.
David E. Wheeler [Tue, 30 Oct 2012 17:00:01 +0000 (10:00 -0700)]
Add some hyphens and short names for options.
Short names are the same (except for password) as offered by `psql`.
David E. Wheeler [Tue, 30 Oct 2012 16:30:00 +0000 (09:30 -0700)]
Add options table and move command-specific options to their sections.
Ref issue #25.
David E. Wheeler [Tue, 30 Oct 2012 03:53:20 +0000 (20:53 -0700)]
Fix help- and usage-based test failures.
The switch to Pod::Usage broke quite a few things. I failed to specify the
proper place to look for things when `usage()` was called from withing functions
actually doing stuff. So do that.
Also change the `=head3`s under `add` to include the word `add`. This allows
them to stand better on their own. Relatedly, add a simple list of types under
`=head2 add` so that there is a reference for the user to easily know what's
available to look up more. Ref GitHub issue #25.
Greg Sabino Mullane [Tue, 30 Oct 2012 02:13:41 +0000 (22:13 -0400)]
Start cleaning up the maekdelta system. Sync-level makedeltas make no sense right now.
David E. Wheeler [Tue, 30 Oct 2012 00:56:44 +0000 (17:56 -0700)]
Revert
d65b04310a.
Greg must have missed a spot or three, as it made the fullcopy tests fail. So
reverting for now.
David E. Wheeler [Tue, 30 Oct 2012 00:52:58 +0000 (17:52 -0700)]
Use Pod::Usage for `help` output.
Eliminates the need for redundant docs in the Pod and the code.
Relies on the use of scalar ref open, supported in Perl 5.6 and higher, IIRC. Pod::Usage is a core module since forever.
David E. Wheeler [Mon, 29 Oct 2012 22:58:12 +0000 (15:58 -0700)]
Store and use options when restarting after death.
Took me a while to hunt down, but on a server, if I killed the PostgreSQL back
ends, Bucardo eventually died. Once it noticed the back ends were gone, it
tried to restart itself via `exec`. Unfortunately, however, I was using a
start script that passes command-line options, and those options were not
preserved for the `exec` call. It died because it did not have permission to
open the log file, because `--debugdir` was no longer known to it!
So we try to preserve all the command-line options and pass them to
`start_mcp()`. The stripping of nouns from the options is not perfect, but
probably good enough.
We also remove the `dosyncs` stuff from `start_mcp()` and friends, since it
was never used: no parameters are supported for the `start` command, and
nothing was passed to `start_mcp()`. So the feature was never used. Removing
all that stuff, we can just pass in the command-line arguments and use them in
the `exec` call.
Resolves #28.
Greg Sabino Mullane [Mon, 29 Oct 2012 21:56:55 +0000 (17:56 -0400)]
Typo.
Greg Sabino Mullane [Mon, 29 Oct 2012 21:21:15 +0000 (17:21 -0400)]
Make variables that contain PIDs from connect_database more obvious.
Greg Sabino Mullane [Mon, 29 Oct 2012 20:46:17 +0000 (16:46 -0400)]
Minor comment cleanups
David E. Wheeler [Mon, 29 Oct 2012 19:26:07 +0000 (12:26 -0700)]
No longer check for "perl" in process names.
When attempting to kill a PID, that is. It looks like, as of the first big 5.0
commit, `$0` is assigned to with the word "Budardo", but not "perl". This was
preventing the VAC process from being killed. So just check for "Bucardo", not
"perl". Closes #27.
David E. Wheeler [Mon, 29 Oct 2012 18:18:12 +0000 (11:18 -0700)]
Set flag to run again, rather than send message.
Keeps the re-try after serialization failure in the child, without relying on
the MCP sending a message to get us going again. Fortunately, the $kicked
variable was already there for just this sort of thing.
David E. Wheeler [Mon, 29 Oct 2012 17:33:30 +0000 (10:33 -0700)]
Remove `getrows`.
It is not used. Custom code must fetch rows itself (`getdbh` is your friend).
David E. Wheeler [Mon, 29 Oct 2012 17:28:00 +0000 (10:28 -0700)]
More on what overdue and expired are for.
David E. Wheeler [Sat, 27 Oct 2012 00:47:36 +0000 (17:47 -0700)]
Rename "goat" to "relation".
All UI-based. Thanks to the separate table/sequence commands, there aren't too
many user-visible referenes to goats. The one place where it remains with
this commit (and the same goes for herd) is in the usage statement, which
references the `goat` table. I'd like to remove all references to tables
in usage statements altogether, but not in this commit. Resolves #24.
There was only one command that took a `goat` parameter: `add customcode`. It
was pretty seriously broken. It now mostly works, but the `relation` parameter
is ignored! This is because the customcode_map table needs an integer. So
should this be changed to "table"? And should it require the
$schema.$table_name format? Note the `XXX` added here. Ref GitHub #20.
David E. Wheeler [Fri, 26 Oct 2012 23:50:41 +0000 (16:50 -0700)]
Rename "herd" to "relgroup".
Per discussion, and just in the UI: command names, parameters, and messages
sent to the user. The internals still use the term "herd" for the most part.
First half of GitHub Issue #24.
David E. Wheeler [Fri, 26 Oct 2012 22:24:40 +0000 (15:24 -0700)]
An attemp to document `expired`.
The difference from `overdue` escapes me, since neither seems to have an affect on whether or why a sync is kicked.
David E. Wheeler [Fri, 26 Oct 2012 22:18:52 +0000 (15:18 -0700)]
Use ucfirst for all display items but name.
David E. Wheeler [Fri, 26 Oct 2012 22:10:23 +0000 (15:10 -0700)]
Document no-op `getrows` param.
And ucfirst the values in Stayalive/Kidsalive output.
David E. Wheeler [Fri, 26 Oct 2012 21:55:40 +0000 (14:55 -0700)]
I think customcode.about is a free-form description field.
David E. Wheeler [Fri, 26 Oct 2012 21:50:59 +0000 (14:50 -0700)]
Rename configs in the database on upgrade.
David E. Wheeler [Fri, 26 Oct 2012 21:45:37 +0000 (14:45 -0700)]
Rename `ping` columns to `autokick`.
David E. Wheeler [Fri, 26 Oct 2012 20:01:01 +0000 (13:01 -0700)]
Rename `standard_conflict` to `conflict_strategy`.
The latter is a better description of the purpose of the parameter. The former
is still supported by `add sync` and `update sync`, for backward
compatibility.
David E. Wheeler [Fri, 26 Oct 2012 18:31:28 +0000 (11:31 -0700)]
Move the `standad_conflict` parameter from table to sync.
Tables will eventually allow it to be overridden (like autokick), but
currently they don't, and in the meantime, we need to be able to set them for
syncs. Resolves GitHub issue #10.
David E. Wheeler [Fri, 26 Oct 2012 18:15:15 +0000 (11:15 -0700)]
Rename the "ping" parameter to "autokick".
This is to prevent confusion with the "ping" command. It's a better, more
descriptive name, anyway. Per IRC discussion.
David E. Wheeler [Fri, 26 Oct 2012 16:24:11 +0000 (09:24 -0700)]
Remove the `do_listen` parameter
Per discussion. It was just an alias for `ping`.
David E. Wheeler [Fri, 26 Oct 2012 00:50:57 +0000 (17:50 -0700)]
Cancel any async queries.
Guess there were some still running, so we need to cancel them.
David E. Wheeler [Fri, 26 Oct 2012 00:17:19 +0000 (17:17 -0700)]
After a serialization failure, we need to kick it.
Greg Sabino Mullane [Thu, 25 Oct 2012 23:52:10 +0000 (19:52 -0400)]
Begin experimenting with the payload system for syncsleep
David E. Wheeler [Thu, 25 Oct 2012 23:50:46 +0000 (16:50 -0700)]
Gah! Typo.
David E. Wheeler [Thu, 25 Oct 2012 23:49:38 +0000 (16:49 -0700)]
Indent `eval` block.
Whitespace-only change.
David E. Wheeler [Thu, 25 Oct 2012 23:46:48 +0000 (16:46 -0700)]
Log sync sleeps in CTL.
And also clean up unfinished statement handles. And fix a typo!