-
Notifications
You must be signed in to change notification settings - Fork 59
indexing context: make it easier to log and report on indexing status #405
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
rjbs
wants to merge
42
commits into
andk:master
Choose a base branch
from
rjbs:index-context
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
42 commits
Select commit
Hold shift + click to select a range
a4ecf6d
package indexing: try to avoid partial updates
rjbs 0283479
PAUSE::Indexer::Context: a (currently zero-value) object for context
rjbs dc1264c
Indexer context: move indexing warnings to context
rjbs 9fafeea
Indexer context: move alerts to context
rjbs c298e3d
indexer: make more dist indexing steps check for skip
rjbs 7d93865
indexer: add Abort and Errors to control indexing
rjbs ec64c11
indexer: a big refactor to change how we abort indexing
rjbs 415fc4b
indexer constants: remove a bunch of dead constants
rjbs f177b3f
support v5.16: stop using indented heredocs
rjbs f35ccc3
support v5.16: remove postfix deref
rjbs 186ee6b
indexer context: call a method instead of copypasting
rjbs d5bb3a3
indexer: improve the no_distname_permissions error
rjbs 2876386
indexer: restructure the try/catch for dist indexing
rjbs a2a4d15
indexer: drop old warning code, use the new code
rjbs 54efa06
no_index rules: simplify code while refactoring
rjbs 28ff4b6
indexer: turn Abort into Abort::Dist
rjbs 0096a18
indexer: gather packages, then index them
rjbs 4905131
indexer: introduce an Abort::Package catch loop
rjbs 8dda205
indexer: factor our part of the package indexer
rjbs c01d16d
indexer: try to use "return early" instead of $ok more
rjbs 0ce0d54
indexer context: add package-level indexing status
rjbs 584c086
indexer: clean up a weirdly-formatted comment
rjbs 4d13f49
indexer: use ->abort_indexing_package
rjbs 24a2a61
replace use of index_status with context in reporting
rjbs 39bb0b5
indexer: reintroduce parameters to package-level errors
rjbs 5325b8d
indexer: improve what we provide as overall status
rjbs e97b634
indexer: word wrap the "nothing updated" text
rjbs ce6d391
PAUSE::Indexer::Errors: make sure Carp is loaded, we use it!
rjbs 05a0bef
PAUSE::Package: pass older_isa_regular_perl along
rjbs 46dad7b
PAUSE::Indexer::Context: make alert and error methods look the same
rjbs 1f56eb6
PAUSE::Indexer::Context: log package errors when setting them
rjbs 826ab37
PAUSE::package: remove dead, commented-out code
rjbs 9d94118
Mock::Dist: remove this unused code
rjbs 7996eec
PAUSE::Indexer::Errors: fix keys from $old
rjbs 8c9c5c0
PAUSE::Indexer::Context: make add_dist_error only accept refs
rjbs 393a5ea
PAUSE::Indexer::Context: pass hashrefs to abort_indexing_dist
rjbs 6758a60
PAUSE::dist: do not include private errors in report
rjbs 4662714
PAUSE::DBError: use block-style package
rjbs 344f6a0
PAUSE.pm: move email sending into PAUSE.pm
rjbs fac7463
PAUSE.pm: add "write mail to mbox" facility
rjbs a6e19d1
indexer: add tests for "nothing to index" email
rjbs 3fbb036
indexer: be more precise in "nothing to index" email text
rjbs File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
package PAUSE::Indexer::Abort::Dist; | ||
use v5.12.0; | ||
use Moo; | ||
|
||
has public => ( | ||
is => 'ro', | ||
default => 0, | ||
); | ||
|
||
has message => ( | ||
is => 'ro', | ||
required => 1, | ||
); | ||
|
||
no Moo; | ||
1; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
package PAUSE::Indexer::Abort::Package; | ||
use v5.12.0; | ||
use Moo; | ||
|
||
has public => ( | ||
is => 'ro', | ||
default => 0, | ||
); | ||
|
||
has message => ( | ||
is => 'ro', | ||
required => 1, | ||
); | ||
|
||
no Moo; | ||
1; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,174 @@ | ||
package PAUSE::Indexer::Context; | ||
use v5.12.0; | ||
use Moo; | ||
|
||
use PAUSE::Indexer::Abort::Dist; | ||
use PAUSE::Indexer::Abort::Package; | ||
use PAUSE::Indexer::Errors; | ||
use PAUSE::Logger '$Logger'; | ||
|
||
has package_warnings => ( | ||
is => 'bare', | ||
reader => '_package_warnings', | ||
default => sub { {} }, | ||
); | ||
|
||
sub add_package_warning { | ||
my ($self, $package_obj, $warning) = @_; | ||
|
||
my $package = $package_obj->{PACKAGE}; | ||
my $pmfile = $package_obj->pmfile->{PMFILE}; | ||
|
||
my $key = "$package\0$pmfile"; | ||
|
||
my $list = ($self->_package_warnings->{$key} //= []); | ||
push @$list, { | ||
package => $package, | ||
pmfile => $pmfile, | ||
text => $warning, | ||
}; | ||
|
||
$Logger->log([ | ||
"adding package warning to %s: %s", | ||
$package_obj->{PACKAGE}, | ||
$list->[-1], | ||
]); | ||
|
||
return; | ||
} | ||
|
||
has package_status => ( | ||
is => 'bare', | ||
reader => '_package_status', | ||
default => sub { {} }, | ||
); | ||
|
||
sub _set_package_error { | ||
my ($self, $package_obj, $status) = @_; | ||
|
||
$self->_package_status->{ $package_obj->{PACKAGE} } = { | ||
is_success => 0, | ||
filename => $package_obj->{PP}{infile}, | ||
version => $package_obj->{PP}{version}, | ||
header => $status->{header}, | ||
body => $status->{body}, | ||
package => $package_obj->{PACKAGE}, | ||
}; | ||
|
||
$Logger->log([ | ||
"set error status for %s to %s", | ||
$package_obj->{PACKAGE}, | ||
$status, | ||
]); | ||
|
||
return; | ||
} | ||
|
||
sub record_package_indexing { | ||
my ($self, $package_obj) = @_; | ||
|
||
$self->_package_status->{ $package_obj->{PACKAGE} } = { | ||
is_success => 1, | ||
filename => $package_obj->{PP}{infile}, | ||
version => $package_obj->{PP}{version}, | ||
header => "Indexed successfully", | ||
body => "The package was indexed successfully.", | ||
package => $package_obj->{PACKAGE}, | ||
}; | ||
|
||
$Logger->log([ | ||
"set OK status for %s", | ||
$package_obj->{PACKAGE}, | ||
]); | ||
|
||
return; | ||
} | ||
|
||
sub package_statuses { | ||
my ($self) = @_; | ||
|
||
my %stash = %{ $self->_package_status }; | ||
return @stash{ sort keys %stash }; | ||
} | ||
|
||
sub abort_indexing_package { | ||
my ($self, $package_obj, $error) = @_; | ||
|
||
$Logger->log("abort indexing $package_obj->{PACKAGE}"); | ||
|
||
$self->_set_package_error($package_obj, $error); | ||
|
||
die PAUSE::Indexer::Abort::Package->new({ | ||
message => $error->{header}, | ||
public => 1, | ||
}); | ||
} | ||
|
||
sub warnings_for_all_packages { | ||
my ($self) = @_; | ||
|
||
return map {; @$_ } values %{ $self->_package_warnings }; | ||
} | ||
|
||
sub warnings_for_package { | ||
my ($self, $package_name) = @_; | ||
|
||
return grep {; $_->{package} eq $package_name } | ||
$self->warnings_for_all_packages; | ||
} | ||
|
||
has alerts => ( | ||
is => 'bare', | ||
reader => '_alerts', | ||
default => sub { [] }, | ||
); | ||
|
||
sub add_alert { | ||
my ($self, $alert) = @_; | ||
$alert =~ s/\v+\z//; | ||
|
||
push @{ $self->_alerts }, $alert; | ||
return; | ||
} | ||
|
||
sub all_alerts { | ||
my ($self) = @_; | ||
return @{ $self->_alerts }; | ||
} | ||
|
||
has dist_errors => ( | ||
is => 'bare', | ||
reader => '_dist_errors', | ||
default => sub { [] }, | ||
); | ||
|
||
sub add_dist_error { | ||
my ($self, $error) = @_; | ||
|
||
$Logger->log_fatal([ "add_dist_error got bogus input: %s", $error ]) | ||
unless ref $error and $error->{header}; | ||
|
||
$Logger->log([ "adding dist error: %s", $error->{header} ]); | ||
push @{ $self->_dist_errors }, $error; | ||
|
||
return; | ||
} | ||
|
||
sub all_dist_errors { | ||
my ($self) = @_; | ||
return @{ $self->_dist_errors }; | ||
} | ||
|
||
rjbs marked this conversation as resolved.
Show resolved
Hide resolved
|
||
sub abort_indexing_dist { | ||
my ($self, $error) = @_; | ||
rjbs marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
$self->add_dist_error($error); | ||
|
||
die PAUSE::Indexer::Abort::Dist->new({ | ||
message => $error->{header}, | ||
public => $error->{public}, | ||
}); | ||
} | ||
|
||
no Moo; | ||
1; |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.