testing - Is there a way to make cucumber try a scenario again before moving on to the next scenario -
i'm trying clean our functional suite @ work , wondering if there way have cucumber repeat scenario , see if passes before moving on next scenario in feature? phantom headless webkit browser poltergeist driver.
basically our build keeps on failing because box gets overwhelmed test , during scenario page won't have enough time render whatever we're trying test. therefore, produces false positive. know of no way anticipate test hang build.
what nice have hook(one idea) happens after each scenario. if scenario passes great print results scenario , move on. however, if scenario fails try running again make sure isn't build getting dizzy. , print results scenario , move on next test.
does have idea on how implement that?
i'm thinking like
after |scenario| if scenario.failed? result = scenario.run_again # made function know fact doesn't exist (see http://cukes.info/api/cucumber/ruby/yardoc/cucumber/ast/scenario.html) if !result cucumber.wants_to_quit = true end end end
the initial solution saw was: how rerun failed scenarios using cucumber?
this fine, need make sure
cucumber @rerun.txt
actually corrected reports if test passed. like
cucumber @rerun.txt --format junit --out foo.xml
where foo.xml junit report said feature 1, 2 & 5 passing while 3 , 4 failing, 1, 2, 3, 4 & 5 passing though rerun.txt said rerun 3 , 4.
i use rerun extensively, , yes, output correct features rerun.txt file. have cucumber.yml file defines bunch of "profiles". note rerun profile:
<% rerun = file.file?('rerun.txt') ? io.read('rerun.txt') : "" rerun_opts = rerun.to_s.strip.empty? ? "--format #{env['cucumber_format'] || 'progress'} features" : "--format #{env['cucumber_format'] || 'pretty'} #{rerun}" %> <% standart_opts = "--format html --out report.html --format rerun --out rerun.txt --no-source --format pretty --require features --tags ~@wip" %> default: <%= standart_opts %> --no-source --format pretty --require features rerun: <%= rerun_opts %> --format junit --out junit_format_rerun --format html --out rerun.html --format rerun --out rerun.txt --no-source --require features core: <%= standart_opts %> --tags @core jenkins: <%= standart_opts %> --tags @jenkins
so happens here run cucumber. during initial run, it'll throw failed scenarios rerun.txt file. then, after, i'll rerun failed tests following command:
cucumber -p rerun
the downfall requires additional command (which can automate, of course) , clutters test metrics if have them in place.
Comments
Post a Comment