Unify Arquillian OpenShift/K8 projects

dev
#1

Hi, all:

This discussion is spawned off from one that started on an internal RHT list. The gist is:

We have work done by the Fabric8 team as an ARQ extension to test components running inside Kubernetes (including OpenShift) :

https://github.com/fabric8io/fabric8/tree/master/components/fabric8-arquillian

There is an Arquillian extension for testing components running inside Docker containers:

https://github.com/arquillian/arquillian-cube

And the Cloud Enablement team has been working on a bit called ce-arq (which I don’t have reference to, actually).

I’d like the ARQ, Fabric8, and Cloud Enablement teams to start work on defining scopes for each of these projects and figuring out:

  • Where there’s overlap
  • If there’s something we should do to mitigate overlap
  • If there’s benefit in merging featuresets into a common project (or set of projects)

The idea here is, sure to reduce overall work by not all addressing the same problem – but also it’ll really help with our user experience and testing story to have a toolset we’re all rallying around. At the moment simple questions like, “hey, how should I execute integration tests on my app deployed on OpenShift” are yielding different answers depending upon our biases. :slight_smile:

This thread to kick off the discussion, and I’ll leave it to the project leads to start to work through this and make recommendations.

Thanks!

S,
ALR

Moving this discussion to public forum.

#2

The Cloud Enablement work can be found here: https://github.com/jboss-openshift/ce-arq

Here’s where I think there is overlap:

  • Client management (i.e. creating the k8s/ose client). Currently, ce-arq is using the client provided by arquillian-cube/openshift
  • Environment setup/teardown. ce-arq provides support for per test case, whereas fabric8-arquillian supports per execution. fabric8 has a nice mechanism for managing “sessions” which would be nice to glob onto, especially for things like per suite (which isn’t really supported at all)

As for mitigating overlap:

  1. We should be using a shared client (i.e. all three projects get a handle to the client from a single source).
  2. Separate out the environment setup so that it too can be shared (e.g. so somebody (us) using ce-arq can use all the k8s assertion goodness that comes with fabric8), but still use their own mechanism for setup/teardown of the ose app under test.

As far as project structure goes, I’m all for consolidation, but that might be a bit much to ask for out of the gate. I think it would be nice if the projects were collocated, but I don’t know if that’s a requirement. I also don’t really have an opinion as to general project structure (one project, two, three, etc.). Arquillian has a pretty flexible extension mechanism, so I think it just comes down to how much effort folks want to put into reorganizing their code.

#3

Is this summary of ‘target usage’ accurate per effort?

  • Arquillian Cube

Any Application code that is running on OpenShift, but currently limited support for any type outside of a Pod(e.g. Service, Route, etc)

  • Arquillian CE

Mostly focused Around Applications running in WildFly/EAP on OpenShift. Has more support for Service/Routes etc.

  • Arquillian Fabric8

Focuses on the infrastructure level, Asserting state on the Pods themselves, not worrying too much about what is running inside the Pods. ‘Low’ level control of all OpenShift types, Pods, Services, Routes etc.

#4

Working on this with Fabric8 team.

#5

We have released the first version of the joined effort between Cube and Fabric 8. There is plenty more to be done, but I believe it’s a great step forward.

@rcernich We will take a look on how to bring those features you highlighted as an overlap to Core and Cube. Just need some time to wrap my head around it :slight_smile:

#6

According to Ioannis we have next steps to be done:

Here are some thoughts on things that could be added next:

  1. Allow the test to run in a pod inside kubernetes, rather than localy.
  2. Allow starting minikube, nanokube, minishift during startup (maybe using spacelify or something like that).
  3. Drone/Graphene integration (I am almost clueless about those, but I would love to be able to run something like the TodoBrowserTest against a kubernetes service).
  4. Closer integration with Openshift (also clueless on what the existing openshift module does, but it would be great to align unify).

I think that next step should be moving fabric8 model into Cube model. Basically it means to implement elements that are injectable into the test such as Pod, Service or ReplicationController.

This means that we should create these new classes inside cube-kubernetes module, implementing https://github.com/arquillian/arquillian-cube/blob/master/spi/src/main/java/org/arquillian/cube/spi/Cube.java#L6 interface. Notice that this interface has something called CubeMetadata (https://github.com/arquillian/arquillian-cube/blob/master/spi/src/main/java/org/arquillian/cube/spi/Cube.java#L54) where you can implement custom operations of specific Cube. For example in case of Docker (https://github.com/arquillian/arquillian-cube/blob/master/spi/src/main/java/org/arquillian/cube/spi/metadata/CanCopyToContainer.java) you can see already defined metadata points at https://github.com/arquillian/arquillian-cube/tree/master/spi/src/main/java/org/arquillian/cube/spi/metadata And then implementation https://github.com/arquillian/arquillian-cube/blob/7956e07c1761821f0fe62289a0dfe96d0f4c0888/docker/docker/src/main/java/org/arquillian/cube/docker/impl/client/metadata/CopyToContainer.java

Than the idea is take all these classes and register them in CubeRegistry https://github.com/arquillian/arquillian-cube/blob/7956e07c1761821f0fe62289a0dfe96d0f4c0888/spi/src/main/java/org/arquillian/cube/spi/CubeRegistry.java so any extension of Cube can take them freely.

#7

Just a heads up, so that we can make sure we are all on the same page.

A lot of work is currently been done in aligning the kubernetes with the openshift extension.

A nice analogy provided by @rcernich is that kubernetes extension takes a ‘containerless’ approach while the openshift extension takes a more ‘docker’ like approach, but it would be best to have the same approaches/features regardless of the environment (kubernetes/openshift).

So, there is concensus that we should ‘merge’ all functionality inside the kubernetes extension and keep openshift for ‘services’ specific to openshift (e.g. openshift requires a custom implementation of a namesapce service). Then if we see fit, we might even completely merge them if it makes sense.

The above pretty much sums up a discussion me, @jwendell and @rcernich had and wanted to share with the rest of the arquillian team, for input feedback etc.