Continuing the discussion from Arquillian Core Suite/SubSuite support:
We need to figure out how to expose the Suite as an API.
There’s more than one way to skin a cat.
Feel free to comment, propose or ‘like/vote’ on the individual styles as you see fit.
What works for you?
@Suite(value = "org.jboss.arquillian.showcase.universe*|.*(core|warp|drone|graphene|persistence).*Suite", strategy = PackageResolverStrategy.class)
@RunWith(Arquillian.class)
public class ApplicationSuite {
@Deployment
public static WebArchive deploy() {
return Deployments.Client.full();
}
}
ApplicationSuite
- ChildTestClass
- ApplicationSubSuite
- Child2TestClass
The current proposal fairly closely matches how normal JUnit Suite works, by adding multiple children to a given Suite level recursively. One thing Arquillian has that normal JUnit doesn’t have is a dependency on the Suite level it self, e.g. Deployment definitions or hierarchical metadata.
Implemented in 1.1.8.Final-SNAPSHOT
public class ApplicationSuite {
@Deployment
public static WebArchive deploy() {
return Deployments.Client.full();
}
}
@BelongsTo(ApplicationSuite.class)
public class ApplicationSubSuite {
@Deployment(name = "backend")
public static WebArchive deploy() {
return Deployments.Client.full();
}
}
@BelongsTo(ApplicationSuite.clas)
@RunWith(Arquillian.class)
public class ChildTestClass {
@Test
public void t() {}
}
@BelongsTo(ApplicationSubSuite.clas)
@RunWith(Arquillian.class)
public class Child2TestClass {
@Test
public void t() {}
}
The Child could point to where the Metadata about it’s deployments are defined, if multiple Children in the current run point to the same metadata, they are considered to be a ‘Suite’. A Suite of one or many, just depends on whos pointing to what. Not 100% sure how to map that physically in a JUnit Suite UI style as I think we technically could get 3 Class requests and some how have to fool the IDE to see that as 1 node with 3 children.(But that’s another issue)
Implemented in 1.1.8.Final-SNAPSHOT
public class ApplicationSuite {
@Deployment
public static WebArchive deploy() {
return ShrinkWrap.create(WebArchive.class);
}
}
@RunWith(Arquillian.class)
public class ChildTestClass extends ApplicationSuite {
@Test
public void t() {}
}
@RunWith(Arquillian.class)
public class Child2TestClass extends ApplicationSuite {
@Test
public void t() {}
}
<arquillian>
<property name="suiteConfiguration">org.some.DeploymentConfiguration</property>
</arquillian>
public class DeploymentConfiguration {
@Deployment
public static WebArchive deploy() {
return Deployments.Client.full();
}
}
@RunWith(Arquillian.class)
public class ChildTestClass {
@Test
public void t() {}
}
@RunWith(Arquillian.class)
public class Child2TestClass {
@Test
public void t() {}
}
Allowing a Global Configuration of where to read the metadata, similar to how the Suite Extension works today.
Leave Deployment creation as is, but allow to configure specific deployments to act as ‘global’ deployments for reuse between multiple Children.
<arquillian>
<property name="reuseNamedArchives">app.war</property>
</arquillian>
@RunWith(Arquillian.class)
public class ChildTestClass {
@Deployment
public static WebArchive deploy() {
return ShrinkWrap.create(WebArchive.class, "app.war")
.addClass(Y.class);
}
@Test
public void t() {}
}
@RunWith(Arquillian.class)
public class Child2TestClass {
@Deployment
public static WebArchive deploy() {
return ShrinkWrap.create(WebArchive.class, "app.war")
.addClass(X.class);
}
@Test
public void t() {}
}
For me first approach is a bit complicated, in my case I needed to read other strategy to then understand exactly what this was about. Also an small mistake in reg exp makes something to work or not which makes devs starting to inspect what is happening if the test is not defined correctly if it is the regular expression, if they have added something in regular expression that it was not necessary…
Also I think that if you cannot run the child alone it can be a hassle when one test of the whole suite doesn’t pass.
The global and named approach I will discard because I like to have as much information as possible in test so with a quick overview I can know exactly what the test does or where belongs. So having to configure it in another place makes any developer to remember that maybe in arquillian.xml there is something that may be important. But reading both approaches I would prefer GlobalModule instead of Named because at Global at least you see that there is no @deployment and probably it is because the definition is on arquillian.xml.
And then we have Class inheritance. Maybe it is the most natural one choice but during my career I have seen a lot of tests that extends from some custom class because they have implemented there some custom operations or extensions. I know that this can be considered a bad practice in some places, but if we simply cut the wings to people who are doing this from his point of view will mean that we are not helping them at all.
So although it may be the most complicated ones for implementing Child reference would be the easiest one to explain and use it.
In my opinion having the ability to start a single test in the IDE is a must.
So SuiteScanner is out imo.
Named resusable components has the problem that it enforces different application names if you have multiple suites in one project and some applications (I know at least one;-)) relies on JNDI names of the form java:app//…
Global module suite will place constraints on the project layout and I personally don’t like that very much.
Class inheritence has the problems Alex stated and a single suite per project is the same problem that I didn’t like so much at Global module suite.
So I would agree with Alex and prefer Child reference.
Maybe the restriction of one test only belonging to one suite could be circumvented by placing the test methods into super classes that the concrete test classes extend.
The concrete test classes would do nothing more than inherit the test methods and point to the test suite they belong to:
public abstract class AbstractTest {
@Test test() {...}
}
@RunWith(Arquillian.class)
@BelongsTo(ApplicationSuite1.class)
public Test1 extends AbstractTest {}
@RunWith(Arquillian.class)
@BelongsTo(ApplicationSuite2.class)
public Test2 extends AbstractTest {}
Yeah, that’s the workaround I was thinking about as well.
lordofthejars and robertpanzer motivated it pretty well(including the workaround).
+1 for child inheritance.
Both Suite Scanner and Child Reference are implemented in the latest published 1.1.8.Final-SNAPSHOT version.
For Suites to work in Maven you need to force Surefire to use the JUnit 4.7 provider:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.18.1</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
4.4 seconds to run 238 integration tests… not too shabby
I am for child reference as well.
I do not know if “BelongsTo” annotation is the best one. Not saying it is chosen badly, just that the annotation alone does not quite say it has something to do with Arquillian suite. If I saw it for the first time, I would not know that “BelongsTo” has some relationship to suites at all …
Agree. Not sure if we should describe that this deployment scenario is read from somewhere else (and since many read from this they can reuse) or if we should be more explicit and describe it as this is part of the larger suite definition x…
Maybe just putting the word “suite” into that annotation in order to be more explicit could do the job.
@BelongsToSuite(MySuite.class)
@PartOfSuite(MySuite.class)
@SuiteAware(MySuite.class)
If the only intent of suite class is the deployment providing, that annotation could be maybe more deployment-centric. Something like @DeploymentProvider(MyDeployments.class) or anything like that, just an idea.
I also like these as they better communicate what the annotation is provoking
@BelongsToSuite(MySuite.class)
@PartOfSuite(MySuite.class)
@SuiteAware is not a favorite of mine because it suggests that it would also run without.
@DeploymentProvider would also not really hit the whole point if a suite would also have callbacks @BeforeSuite and @AfterSuite in the future or if a suite could contain rules.
Sure I meant Child Reference - it got mixed up. But I see it is already implemented - thanks.
I vote also for child reference. What about the following for referencing more than one suite:
@BelongsToSuite({MySuite.class, SecondSuite.class})
The point from robert is for us very usefull:
@BeforeSuite and @AfterSuite
When this works in combination with persistence extension api:
@UsingDataSet and @DataSource
then we can inject data for all tests on different datasources. Very cool feature…
So for this to work well the
@BeforeSuite and @AfterSuite
Annotation should not be restricted to use only once in suite. If annotated on more than one method you can use same/different datasets on different datasources. Very useful in our case, as we have a multi schema application…
Hello,
just for my interest. Are there some plans for merging the suite functionality to new release of core?
Thank you
Ondra