selenium - Ensure Maven build fails when tests fail to run -

i've noticed when running maven builds on jenkins number of jbehave tests run vary 1 run another. when analyzing logs see following snippet:

    failed run story stories/cancel.story java.lang.interruptedexception: stories/cancel.story     @ org.jbehave.core.embedder.storyrunner$runcontext.interruptifcancelled(storyrunner.java:616)     @ org.jbehave.core.embedder.storyrunner.runstepswhilekeepingstate(storyrunner.java:514)     @ org.jbehave.core.embedder.storyrunner.runscenariosteps(storyrunner.java:479)     @ org.jbehave.core.embedder.storyrunner.runstepswithlifecycle(storyrunner.java:445)     @ org.jbehave.core.embedder.storyrunner.runcancellable(storyrunner.java:305)     @ org.jbehave.core.embedder.storyrunner.run(storyrunner.java:220)     @ org.jbehave.core.embedder.storyrunner.run(storyrunner.java:181)     @ org.jbehave.core.embedder.storymanager$enqueuedstory.call(storymanager.java:235)     @ org.jbehave.core.embedder.storymanager$enqueuedstory.call(storymanager.java:207)     @ java.util.concurrent.futuretask.run(futuretask.java:262)     @ java.util.concurrent.threadpoolexecutor.runworker(threadpoolexecutor.java:1145)     @ java.util.concurrent.threadpoolexecutor$worker.run(threadpoolexecutor.java:615)     @ java.lang.thread.run(thread.java:745) 

the problem when tests skipped or fail run in way build still considered success.

is there maven surefire plugin configuration ensure whenever tests fail run build results in failure? here maven surefire build configurations

<build>     <plugins>         <plugin>             <groupid>org.apache.maven.plugins</groupid>             <artifactid>maven-surefire-plugin</artifactid>             <version>2.11</version>             <configuration>                 <skip>true</skip>             </configuration>         </plugin>         <plugin>             <artifactid>maven-failsafe-plugin</artifactid>             <version>2.11</version>             <configuration>                 <includes>                     <include>**/*testsuite.java</include>                 </includes>             </configuration>             <executions>                 <execution>                     <goals>                         <goal>integration-test</goal>                         <goal>verify</goal>                     </goals>                 </execution>             </executions>         </plugin>         <plugin>             <groupid>org.apache.maven.plugins</groupid>             <artifactid>maven-compiler-plugin</artifactid>             <version>3.1</version>         </plugin>         <plugin>             <groupid>org.apache.maven.plugins</groupid>             <artifactid>maven-javadoc-plugin</artifactid>             <version>2.9</version>         </plugin>         <plugin>             <groupid>net.thucydides.maven.plugins</groupid>             <artifactid>maven-thucydides-plugin</artifactid>             <version>${thucydides.version}</version>             <executions>                 <execution>                     <id>thucydides-reports</id>                     <phase>post-integration-test</phase>                     <goals>                         <goal>aggregate</goal>                     </goals>                 </execution>             </executions>             <dependencies>                 <dependency>                     <groupid>log4j</groupid>                     <artifactid>log4j</artifactid>                     <version>1.2.17</version>                 </dependency>             </dependencies>         </plugin>         <plugin>             <groupid>org.apache.maven.plugins</groupid>             <artifactid>maven-site-plugin</artifactid>             <version>3.2</version>             <configuration>                 <reportplugins>                     <plugin>                         <groupid>net.thucydides.maven.plugins</groupid>                         <artifactid>maven-thucydides-plugin</artifactid>                         <version>${thucydides.version}</version>                     </plugin>                 </reportplugins>             </configuration>         </plugin>     </plugins> </build> 

your maven-surefire-plugin set skip tests (with <skip>true</skip>), tests running maven-failsafe-plugin. plug-in supposed not stop on failure during integration-test, , fail on verify phase.

so if want question answered:

is there maven surefire plugin configuration ensure whenever tests fail run build results in failure?

that is: want maven-surefire-plugin run tests, , not maven-failsafe-plugin, answer is: remove

        <configuration>             <skip>true</skip>         </configuration> 

from pom. in case don't need maven-failsafe-plugin configuration, because make tests run twice.

but if goal maven-failsafe-plugin work, think may have 1 of following issues:

1. not running right goal. plug-in help states, should invoke as

   mvn verify 

2. an old plug-in, not compatible test framework using (current version 2.19.1)

3. or recommendation:

   for complex builds, may better separate executions integration-test , verify goals:

   <plugin>   <groupid>org.apache.maven.plugins</groupid>   <artifactid>maven-failsafe-plugin</artifactid>   <version>2.19.1</version>   <executions>     <execution>       <id>integration-test</id>       <goals>         <goal>integration-test</goal>       </goals>     </execution>     <execution>       <id>verify</id>       <goals>         <goal>verify</goal>       </goals>     </execution>   </executions> </plugin>