Maintenance: Enhance VCR helper with better failure messages

1ebddff95 added an RSpec metadata flag to simplify the use of VCR:

    describe 'super cool feature', :use_vcr do
      it 'totally works' { ... }
    end

Under the hood, this option automatically generates filenames
for the VCR cassettes it uses on each example it's applied to.
These filenames are based on the example descriptions;
for the sample block above, the resulting filename would be

    super_cool_feature_totally_works.yml

This introduces the risk of test regressions
that could be extremely confusing and hard to debug,
simply because someone changed the example description.

This commit injects custom error messages into RSpec
to elucidate this problem and avoid needless debugging.

=== Design challenges

This error message injection uses Ruby's Module#prepend
to monkey-patch methods defined in RSpec,
meaning that the changes are coupled to RSpec's implementation.
In short, if the implementation changes,
this custom error messaging could break.

Specifically, there are two different modes of failure in RSpec,
and the custom error was thus injected in two corresponding places:

* `.notify_failure` for normal test failure
  (i.e., when an expectation is not met); and
* `.handle_matcher` for exceptions raised during a test.
This commit is contained in:
Ryan Lue 2019-11-18 23:27:21 +08:00 committed by Thorsten Eckel
parent 9ce331be14
commit a5d4cd80b2

View file

@ -20,8 +20,8 @@ VCR.configure do |config|
end
end
RSpec.configure do |config|
config.around(:each, use_vcr: true) do |example|
module VCRHelper
def self.auto_record(example)
spec_path = Pathname.new(example.file_path).realpath
cassette_path = spec_path.relative_path_from(Rails.root.join('spec')).sub(/_spec\.rb$/, '')
cassette_name = "#{example.example_group.description} #{example.description}".gsub(/[^0-9A-Za-z.\-]+/, '_').downcase
@ -37,3 +37,57 @@ RSpec.configure do |config|
end
end
end
module RSpec
VCR_ADVISORY = <<~MSG.freeze
If this test is failing unexpectedly, the VCR cassette may be to blame.
This can happen when changing `describe`/`context` labels on some specs;
see commit message 1ebddff95 for details.
Check `git status` to see if a new VCR cassette has been generated.
If so, rename the old cassette to replace the new one and try again.
MSG
module Support
module VCRHelper
def self.inject_advisory(example)
# block argument is an #<RSpec::Expectations::ExpectationNotMetError>
define_method(:notify_failure) do |e|
super(e.exception(VCR_ADVISORY + e.message))
end
example.run
ensure
remove_method(:notify_failure)
end
end
singleton_class.send(:prepend, VCRHelper)
end
module Expectations
module VCRHelper
def self.inject_advisory(example)
define_method(:handle_matcher) do |*args|
super(*args)
rescue => e
raise e.exception(VCR_ADVISORY + e.message)
end
example.run
ensure
remove_method(:handle_matcher)
end
end
PositiveExpectationHandler.singleton_class.send(:prepend, VCRHelper)
NegativeExpectationHandler.singleton_class.send(:prepend, VCRHelper)
end
end
RSpec.configure do |config|
config.around(:each, use_vcr: true, &VCRHelper.method(:auto_record))
config.around(:each, use_vcr: true, &RSpec::Support::VCRHelper.method(:inject_advisory))
config.around(:each, use_vcr: true, &RSpec::Expectations::VCRHelper.method(:inject_advisory))
end