bucardo.git
11 years agoTry harder to collect and emit errors on validation failures.
David E. Wheeler [Wed, 16 Oct 2013 20:39:09 +0000 (13:39 -0700)]
Try harder to collect and emit errors on validation failures.

There's no need to wait for a NOTIFY, as `validate_sync()` runs synchronously.
So just check the return value for errors, and use excpetion handling to catch
other kinds of errors. This allows us to catch and emit all errors and also
allow execution to finish validating all syncs (should it be validating
multiples). Closes #65.

11 years agoImprove the auto-creation of the "bucardo" role.
David E. Wheeler [Tue, 15 Oct 2013 23:35:57 +0000 (16:35 -0700)]
Improve the auto-creation of the "bucardo" role.

When adding a database, that is. The old code led to confusing errors, where
the user would be told that Bucardo failed to connect as user "postgres" when
you had not, in fact, asked it to. So log more informat as it goes along,
noting failure to connect as "bucardo", trying to connect as "postgres", and
showing success or failure at doing so. This makes things much clearer to the
user what's going on -- and is less fussy, to boot.

Resolves #63.

11 years agoMore bypassing of track_rates
Greg Sabino Mullane [Tue, 15 Oct 2013 00:56:56 +0000 (20:56 -0400)]
More bypassing of track_rates

11 years agoSkip the not-working-now track_rates section.
Greg Sabino Mullane [Tue, 15 Oct 2013 00:55:05 +0000 (20:55 -0400)]
Skip the not-working-now track_rates section.

11 years agoMerge in patch to ensure objects from different databases but having the same
Joshua Tolley [Mon, 14 Oct 2013 14:52:23 +0000 (08:52 -0600)]
Merge in patch to ensure objects from different databases but having the same
names are treated distinctly.

11 years agoLog verbose instead of debug.
David E. Wheeler [Mon, 7 Oct 2013 18:45:39 +0000 (11:45 -0700)]
Log verbose instead of debug.

For serialization and deadlock failures.

11 years agoLog loglevel as a string.
David E. Wheeler [Mon, 7 Oct 2013 16:56:06 +0000 (09:56 -0700)]
Log loglevel as a string.

11 years agoMake it clearer which handles to log.
David E. Wheeler [Mon, 7 Oct 2013 16:23:07 +0000 (09:23 -0700)]
Make it clearer which handles to log.

11 years agoTopic, not lexical.
David E. Wheeler [Mon, 7 Oct 2013 16:21:55 +0000 (09:21 -0700)]
Topic, not lexical.

11 years agoLog only errors.
David E. Wheeler [Mon, 7 Oct 2013 16:20:29 +0000 (09:20 -0700)]
Log only errors.

11 years agoTry harder to find initdb when testing.
Greg Sabino Mullane [Sat, 5 Oct 2013 01:19:27 +0000 (21:19 -0400)]
Try harder to find initdb when testing.

11 years agoAdd some text fields to the dbgroup and herd tables for future use in "cloning"
Greg Sabino Mullane [Sat, 5 Oct 2013 01:19:03 +0000 (21:19 -0400)]
Add some text fields to the dbgroup and herd tables for future use in "cloning"

11 years agoMake the inactivate sub a bit smarter. May fix the recurring doublefree/coredump...
Greg Sabino Mullane [Fri, 4 Oct 2013 14:30:44 +0000 (10:30 -0400)]
Make the inactivate sub a bit smarter. May fix the recurring doublefree/coredump VAC issue.

11 years agoFix inaccurate comment.
David E. Wheeler [Wed, 2 Oct 2013 23:43:27 +0000 (16:43 -0700)]
Fix inaccurate comment.

11 years agoChanging a sync status does not load or unload it.
David E. Wheeler [Wed, 2 Oct 2013 22:34:22 +0000 (15:34 -0700)]
Changing a sync status does not load or unload it.

So document that fact. We may or may not want to change this in the future.
Ref #55.

11 years agoUpdate sync status on activate/deactivate.
David E. Wheeler [Wed, 2 Oct 2013 22:29:07 +0000 (15:29 -0700)]
Update sync status on activate/deactivate.

Resolves #55.

11 years agoDocument strict_checking for tables, too.
David E. Wheeler [Wed, 2 Oct 2013 21:57:50 +0000 (14:57 -0700)]
Document strict_checking for tables, too.

Allow it to be passed to `add table` as well as `update table`.

Ref #57.

11 years agoDocument strict_checking and add it to `add sync`.
David E. Wheeler [Wed, 2 Oct 2013 21:51:23 +0000 (14:51 -0700)]
Document strict_checking and add it to `add sync`.

Resolves #57.

11 years agoAdd the reload_config_timeout config.
David E. Wheeler [Wed, 2 Oct 2013 18:41:07 +0000 (11:41 -0700)]
Add the reload_config_timeout config.

And put it to use in the reload_config command. Also provide more context in
the error message when it does timeout. Closes #67.

11 years agoChange default reload_config timeout to 30s.
David E. Wheeler [Wed, 2 Oct 2013 18:25:34 +0000 (11:25 -0700)]
Change default reload_config timeout to 30s.

Ref #67.

11 years agoPurge unknown tables from bucardo_delta_targets.
David E. Wheeler [Wed, 2 Oct 2013 16:25:39 +0000 (09:25 -0700)]
Purge unknown tables from bucardo_delta_targets.

They seem to get left behind when a table is dropped. Maybe other reasons,
I honestly don't know.

12 years agoDon't show serialize/deadlock errors unless we are LOG_VERBOSE, per mailing list...
Greg Sabino Mullane [Wed, 11 Sep 2013 15:41:27 +0000 (11:41 -0400)]
Don't show serialize/deadlock errors unless we are LOG_VERBOSE, per mailing list discussion.

12 years agoSwitch time zone to GMT, as we did in Bucardo4
Greg Sabino Mullane [Thu, 29 Aug 2013 02:45:55 +0000 (22:45 -0400)]
Switch time zone to GMT, as we did in Bucardo4

12 years agoIn `status`, if last run is "empty", call it "good".
David E. Wheeler [Thu, 25 Jul 2013 15:33:03 +0000 (17:33 +0200)]
In `status`, if last run is "empty", call it "good".

Two reasons for this:

* An empty run is not a failure, but "Empty" is kind of distracting
  in the output.
* It is useful, if you just kicked a sync, to see that it ran in the output of
  `status`, even if it was empty.

12 years agoState can also be "Unknown".
David E. Wheeler [Thu, 25 Jul 2013 14:57:54 +0000 (16:57 +0200)]
State can also be "Unknown".

Though I am not sure how it gets in that state...

12 years agoNote state value for currently-running sync.
David E. Wheeler [Thu, 25 Jul 2013 14:55:16 +0000 (16:55 +0200)]
Note state value for currently-running sync.

12 years agoEnd syncs on deadlock and serialization failure.
David E. Wheeler [Thu, 25 Jul 2013 14:40:05 +0000 (16:40 +0200)]
End syncs on deadlock and serialization failure.

When I added the serialization failure-handling code, I neglected to notice
that it left orphaned `syncrun` rows with no `ended` value. This confused
later syncruns -- like the one that would run after the 0.5s sleep after a
serialization failure. The code was rolling back the main (Bucardo) database
connection, but the syncrun was already committed elsewhere (so that other
processes could see that a sync is in progress, of course; duh!).

So modify the code to instead call `end_syncrun()` and declare the sync
failed, but, of course, with a status recording the seriazliation failure or
deadlock.

12 years agoListen for autokicks on sources for the sync, not all sources.
David E. Wheeler [Wed, 24 Jul 2013 17:45:50 +0000 (19:45 +0200)]
Listen for autokicks on sources for the sync, not all sources.

I had a database in two dbgroups (and syncs), once as a target and once as a
source. On startup, `validate_sync()` was processing the sync where it was a
target, first, and that was the role saved for it in `$self->{sdb}`. Then,
when `validate_sync()` iterated over `$self->{sdb}`, it thought it was a role,
and so did not have the MCP listen for kicks.

Not only that, but sources from other, previously-validated syncs were
unnecessarily listening for kicks!

So change the code to iterate over the list of DBs for the current call to
`validate_sync()`, not all dbs for all syncs, when figuring out whereq to listen
to for kicks.

Note that the use of the same db object in multiple db groups and syncs *may*
cause issues in other places. It would be smart to audit all code that checks
the value in `{role}` for each hash in `$self->{sdb}` and figure out if it is
appropriate. Since the same db can have different roles in differnt syncs, it
might make sense to record all of its roles and to check all of them, rather
than just rely on the first role found.

12 years agoRevert "Make kick triggers consistently use the non-payload version."
David E. Wheeler [Wed, 24 Jul 2013 17:23:16 +0000 (19:23 +0200)]
Revert "Make kick triggers consistently use the non-payload version."

This reverts commit 25973f4b5ba797568467632b79e8129ffe007f49. I'm pretty sure
this was not the problem, since I have syncs that *do* work with this
approach.

The problem I'm having with a NOTIFY being ignored is that the database in
question is used in two db groups, once as a source, another as a target. As
it validates, it finds the target record before the source, and stores that
one away. Then the autokick is ignored, because Bucardo thinks the DB is a
target, not a source. Working on a fix for that now.

12 years agoCheck for empty state, too.
David E. Wheeler [Wed, 24 Jul 2013 16:16:22 +0000 (18:16 +0200)]
Check for empty state, too.

12 years agoRemove listen_payload in unlisten_all().
David E. Wheeler [Wed, 24 Jul 2013 16:16:04 +0000 (18:16 +0200)]
Remove listen_payload in unlisten_all().

12 years agoAdd daemon role to application name.
David E. Wheeler [Wed, 24 Jul 2013 16:15:44 +0000 (18:15 +0200)]
Add daemon role to application name.

12 years agoMake kick triggers consistently use the non-payload version.
Greg Sabino Mullane [Wed, 24 Jul 2013 14:46:47 +0000 (10:46 -0400)]
Make kick triggers consistently use the non-payload version.
May want to have MCP be able to recognize both versions for backwards compat.

12 years agoFix `reload config` timeout.
David E. Wheeler [Tue, 23 Jul 2013 22:46:14 +0000 (00:46 +0200)]
Fix `reload config` timeout.

First, allow `reload config 30` to work. Resolves #66.

Second, actually *capture* the timeout and wait that long, instead of just
five seconds every time.

12 years agoRemove comment line about spacing - no longer applies.
Greg Sabino Mullane [Thu, 18 Jul 2013 02:48:37 +0000 (22:48 -0400)]
Remove comment line about spacing - no longer applies.

12 years agoMake sure "bucardo help" doesn't try to connect to a database. Per github issue #64
Greg Sabino Mullane [Thu, 18 Jul 2013 02:44:45 +0000 (22:44 -0400)]
Make sure "bucardo help" doesn't try to connect to a database. Per github issue #64

12 years agoBump the version to 5.0.0.
Greg Sabino Mullane [Mon, 1 Jul 2013 16:00:43 +0000 (12:00 -0400)]
Bump the version to 5.0.0.

12 years agoSwitch default isolation level back to repeatable read
Greg Sabino Mullane [Fri, 28 Jun 2013 17:06:59 +0000 (13:06 -0400)]
Switch default isolation level back to repeatable read

12 years agoMatch process name case-insensitive
Greg Sabino Mullane [Fri, 21 Jun 2013 03:40:29 +0000 (23:40 -0400)]
Match process name case-insensitive

12 years agoProper debug for isolation levels
Greg Sabino Mullane [Fri, 21 Jun 2013 02:33:01 +0000 (22:33 -0400)]
Proper debug for isolation levels

12 years agoFixes for upgrade magic
Greg Sabino Mullane [Mon, 17 Jun 2013 03:45:11 +0000 (23:45 -0400)]
Fixes for upgrade magic

12 years agoLet's do github canonically
Greg Sabino Mullane [Wed, 29 May 2013 17:15:28 +0000 (13:15 -0400)]
Let's do github canonically

12 years agoMove some things from bugzilla to here
Greg Sabino Mullane [Mon, 27 May 2013 12:26:04 +0000 (08:26 -0400)]
Move some things from bugzilla to here

12 years agoSuper minor tweak
Greg Sabino Mullane [Sat, 18 May 2013 04:43:18 +0000 (00:43 -0400)]
Super minor tweak

12 years agoVAC handle pg_listener, per list discussion
Greg Sabino Mullane [Sat, 4 May 2013 15:19:28 +0000 (11:19 -0400)]
VAC handle pg_listener, per list discussion

12 years agoThis should have gone in with the last commit, as well
Joshua Tolley [Wed, 24 Apr 2013 15:15:18 +0000 (09:15 -0600)]
This should have gone in with the last commit, as well

12 years agoImprove output, and update tests accordingly
Joshua Tolley [Wed, 24 Apr 2013 15:14:48 +0000 (09:14 -0600)]
Improve output, and update tests accordingly

12 years agoAdd a bit more output to identify a database
Joshua Tolley [Tue, 23 Apr 2013 16:09:35 +0000 (10:09 -0600)]
Add a bit more output to identify a database

12 years agoAdd tests and code for update table db filtering
Joshua Tolley [Tue, 23 Apr 2013 16:06:00 +0000 (10:06 -0600)]
Add tests and code for update table db filtering

12 years agoClean up lots, and get list table working better
Joshua Tolley [Tue, 23 Apr 2013 15:51:25 +0000 (09:51 -0600)]
Clean up lots, and get list table working better

12 years agoClean up tests
Joshua Tolley [Tue, 23 Apr 2013 15:46:16 +0000 (09:46 -0600)]
Clean up tests

12 years agoMerge branch 'master' of ssh://bucardo.org/var/lib/git/bucardo into database-specific...
Joshua Tolley [Mon, 22 Apr 2013 22:11:59 +0000 (16:11 -0600)]
Merge branch 'master' of ssh://bucardo.org/var/lib/git/bucardo into database-specific-names

12 years agoFix customcols; all preexisting tests pass
Joshua Tolley [Mon, 22 Apr 2013 22:11:25 +0000 (16:11 -0600)]
Fix customcols; all preexisting tests pass

12 years agoeven more tests work now
Joshua Tolley [Mon, 22 Apr 2013 21:50:08 +0000 (15:50 -0600)]
even more tests work now

12 years agoFix herd adding logic, with caveats noted in test source
Joshua Tolley [Mon, 22 Apr 2013 19:49:07 +0000 (13:49 -0600)]
Fix herd adding logic, with caveats noted in test source

12 years agoConsolidate methods for finding goats
Joshua Tolley [Mon, 22 Apr 2013 16:01:20 +0000 (10:01 -0600)]
Consolidate methods for finding goats

12 years agoAllow for auto reconnection of databases for crash testing.
Greg Sabino Mullane [Wed, 17 Apr 2013 16:03:17 +0000 (12:03 -0400)]
Allow for auto reconnection of databases for crash testing.

12 years agoTypo
Joshua Tolley [Wed, 17 Apr 2013 15:35:25 +0000 (09:35 -0600)]
Typo

12 years agoMerge from master
Joshua Tolley [Tue, 16 Apr 2013 20:16:53 +0000 (14:16 -0600)]
Merge from master

12 years ago02-bctl-herd.t passes
Joshua Tolley [Tue, 16 Apr 2013 20:14:38 +0000 (14:14 -0600)]
02-bctl-herd.t passes

12 years agoThe regex looking at ps output when killing children didn't match the children
Mitchell Perilstein [Wed, 27 Mar 2013 17:51:43 +0000 (13:51 -0400)]
The regex looking at ps output when killing children didn't match the children

12 years agoAvoid warning if no $HOME.
Mitchell Perilstein [Fri, 29 Mar 2013 18:57:36 +0000 (14:57 -0400)]
Avoid warning if no $HOME.

12 years agoIncremental progress
Joshua Tolley [Tue, 16 Apr 2013 19:16:51 +0000 (13:16 -0600)]
Incremental progress

Refactored %GOAT, and several functions that refer to it
Added beginnings of a new test file

This doesn't work yet, but needs to live somewhere other than just my computer.

12 years agoShow correct error
Greg Sabino Mullane [Fri, 12 Apr 2013 02:50:22 +0000 (22:50 -0400)]
Show correct error

12 years agoAdd more explicit support for adding and removing sequences
Joshua Tolley [Fri, 29 Mar 2013 16:18:55 +0000 (10:18 -0600)]
Add more explicit support for adding and removing sequences

12 years agoAttempts to make VAC daemon quieter, per IRC discussions.
Greg Sabino Mullane [Wed, 20 Mar 2013 20:10:11 +0000 (16:10 -0400)]
Attempts to make VAC daemon quieter, per IRC discussions.

12 years agoMake sure regex matches so "Not needed" doesn't raise a warning.
Greg Sabino Mullane [Sun, 17 Mar 2013 16:08:28 +0000 (12:08 -0400)]
Make sure regex matches so "Not needed" doesn't raise a warning.

12 years agoAllow transaction isolation level to be changed
Greg Sabino Mullane [Fri, 1 Mar 2013 21:04:23 +0000 (16:04 -0500)]
Allow transaction isolation level to be changed

12 years agoBetter debugging sub
Greg Sabino Mullane [Sun, 24 Feb 2013 05:01:42 +0000 (00:01 -0500)]
Better debugging sub

12 years agoSignature for version 4.99.7 4.99.7
Greg Sabino Mullane [Thu, 21 Feb 2013 01:51:09 +0000 (20:51 -0500)]
Signature for version 4.99.7

12 years agoSometimes the bucardo_delta table can get duplicate entries. Make sure the purge...
Greg Sabino Mullane [Wed, 20 Feb 2013 15:28:25 +0000 (10:28 -0500)]
Sometimes the bucardo_delta table can get duplicate entries. Make sure the purge function is not tripped up by that.

12 years agoMinor Perl::Critic inspired tweaks
Greg Sabino Mullane [Tue, 19 Feb 2013 22:20:34 +0000 (17:20 -0500)]
Minor Perl::Critic inspired tweaks

12 years agoFix comment
Greg Sabino Mullane [Tue, 19 Feb 2013 21:55:12 +0000 (16:55 -0500)]
Fix comment

12 years agoShow filename for subroutines that do not pass.
Greg Sabino Mullane [Tue, 19 Feb 2013 21:54:55 +0000 (16:54 -0500)]
Show filename for subroutines that do not pass.

12 years agoTesting tweaks.
Greg Sabino Mullane [Tue, 19 Feb 2013 19:44:53 +0000 (14:44 -0500)]
Testing tweaks.

12 years agoMake sure logdest gets forced to an arrayref.
Greg Sabino Mullane [Tue, 19 Feb 2013 19:36:16 +0000 (14:36 -0500)]
Make sure logdest gets forced to an arrayref.

12 years agoTabs to space
Greg Sabino Mullane [Tue, 19 Feb 2013 15:30:58 +0000 (10:30 -0500)]
Tabs to space

12 years agoSkip generated files from some tests.
Greg Sabino Mullane [Tue, 19 Feb 2013 15:30:44 +0000 (10:30 -0500)]
Skip generated files from some tests.

12 years agoBump version to 4.99.7
Greg Sabino Mullane [Tue, 19 Feb 2013 15:27:08 +0000 (10:27 -0500)]
Bump version to 4.99.7

12 years agoWork around for Postgres bug in which notices are not picked up if unlisten * and...
Greg Sabino Mullane [Wed, 13 Feb 2013 04:25:43 +0000 (23:25 -0500)]
Work around for Postgres bug in which notices are not picked up if unlisten * and listens are put in the same transaction!

12 years agoMake sure logdest works if specified in an rc file.
Greg Sabino Mullane [Wed, 13 Feb 2013 04:24:43 +0000 (23:24 -0500)]
Make sure logdest works if specified in an rc file.
Have reload_config check that Bucardo is up and running first
Allow wait_for_notice to have a timeout

12 years agoMild cleanups. Allow foo:bar in addition to foo=bar in the RC files
Greg Sabino Mullane [Tue, 12 Feb 2013 20:40:41 +0000 (15:40 -0500)]
Mild cleanups. Allow foo:bar in addition to foo=bar in the RC files

12 years agoMake sure that when verifying columns, etc. across source databases, we only check...
Greg Sabino Mullane [Thu, 7 Feb 2013 15:46:00 +0000 (10:46 -0500)]
Make sure that when verifying columns, etc. across source databases, we only check source databases used in the current sync. Fixes some complaints from the mailing list.

12 years agoDon't run FK info query if no FKs
Greg Sabino Mullane [Wed, 6 Feb 2013 23:39:42 +0000 (18:39 -0500)]
Don't run FK info query if no FKs

12 years agoBump to 2013
Greg Sabino Mullane [Mon, 4 Feb 2013 16:01:30 +0000 (11:01 -0500)]
Bump to 2013

12 years agoTypo
Greg Sabino Mullane [Fri, 1 Feb 2013 22:17:14 +0000 (17:17 -0500)]
Typo

12 years agoMerge remote-tracking branch 'burbon/fix-update-dbgroup'
Greg Sabino Mullane [Fri, 1 Feb 2013 21:34:18 +0000 (16:34 -0500)]
Merge remote-tracking branch 'burbon/fix-update-dbgroup'

12 years agoMerge remote-tracking branch 'burbon/fix-add-all-tables'
Greg Sabino Mullane [Fri, 1 Feb 2013 21:32:31 +0000 (16:32 -0500)]
Merge remote-tracking branch 'burbon/fix-add-all-tables'

12 years agoFIX: bucardo update dbgroup do not affect db data
burbon [Thu, 31 Jan 2013 22:20:41 +0000 (23:20 +0100)]
FIX: bucardo update dbgroup do not affect db data

12 years agoremove unnecessary variable.
Martin Atukunda [Tue, 29 Jan 2013 21:04:50 +0000 (21:04 +0000)]
remove unnecessary variable.

12 years agouse $sth->rows() to get number of rows.
Martin Atukunda [Tue, 29 Jan 2013 09:35:29 +0000 (09:35 +0000)]
use $sth->rows() to get number of rows.

From the DBI manpage:

'A DBI statement handle execute always returns true regardless of the number of rows affected, even it's zero." It only returns the number of rows affected for non-"SELECT" statements.'

This patch ensures that we get the proper count by calling $sth->rows() instead of relying on $sth->execute.

12 years agoFIX: add all tables with table/schema limit
Błażej Cegiełka [Fri, 25 Jan 2013 02:11:07 +0000 (03:11 +0100)]
FIX: add all tables with table/schema limit
add no_autoabbrev to save '-n' (schema|n=s@ in second GetOptions) from binding to 'no-bucardorc' in first GetOptions

12 years agoFIX: add all tables with exclude table/schema
Błażej Cegiełka [Fri, 25 Jan 2013 00:33:35 +0000 (01:33 +0100)]
FIX: add all tables with exclude table/schema

12 years agoIdea from mailing list: combine kick triggers and funcs
Greg Sabino Mullane [Thu, 24 Jan 2013 08:10:36 +0000 (03:10 -0500)]
Idea from mailing list: combine kick triggers and funcs

12 years agoShow status, not details. PID is not in the syncrun table anymore.
Greg Sabino Mullane [Mon, 7 Jan 2013 05:03:14 +0000 (00:03 -0500)]
Show status, not details. PID is not in the syncrun table anymore.

12 years agoBetter output for "whydie" section.
Greg Sabino Mullane [Sat, 5 Jan 2013 04:42:00 +0000 (23:42 -0500)]
Better output for "whydie" section.

12 years agoMore incremental work for solving the FK conflict problem.
Greg Sabino Mullane [Thu, 3 Jan 2013 00:09:42 +0000 (19:09 -0500)]
More incremental work for solving the FK conflict problem.

12 years agoTrack which tables are referencing each other via foreign keys. This is needed for...
Greg Sabino Mullane [Fri, 28 Dec 2012 04:33:48 +0000 (23:33 -0500)]
Track which tables are referencing each other via foreign keys. This is needed for conflict resolution work.

12 years agoStore the dbgroup rather than the sync inside of bucardo_delta_targets. This makes...
Greg Sabino Mullane [Mon, 24 Dec 2012 21:48:29 +0000 (16:48 -0500)]
Store the dbgroup rather than the sync inside of bucardo_delta_targets. This makes more sense, as not only are the track_* tables storing the db name, but the main getdelta queries use that as well. Thus, we don't want to store syncs as dbgroups are more unique.

12 years agoWalked through to make sure all execute() calls are followed by a finish() or a fetch...
Greg Sabino Mullane [Mon, 24 Dec 2012 05:16:05 +0000 (00:16 -0500)]
Walked through to make sure all execute() calls are followed by a finish() or a fetchall()