Skip to content

rbm run tests

Jump to bottom
inazaruk edited this page Jan 19, 2012 · 42 revisions

rbm-run-tests

Go to home | documentation | ant tasks

This tasks runs tests on emulator or real device. It allows to generate junit reports into a single monolith file or into multiple files (one per each test suite). It also allows to install multiple .apk before running tests (and uninstalling them after tests are run).

User can also change testrunner and pass any custom arguments to testrunner to further tune its behavior to his or her needs.

This task also can fail the build if at least one test fails. This is turned off by default.

Prototype

<rbm-run-tests package="${tester.package}" 
               runner="${runner.classname}" 
               uninstall="${uninstall}" 
               failOnError="${failOnError}">
      <junit dir="${output.dir}" multiple="${use.multiple.files}" />
      <apks>
         <apk file="${apk.file1}" />
         <apk file="${apk.file2}" />
         <tester file="${apk.tester}" />
         <fileset refid="${apk.fileset}" />
      </apks>
      <args>
          <arg key="${arg.key1}" value="${arg.value1}" />
          <arg key="${arg.key2}" value="${arg.value2}" />
          <coverage enable="${enable.coverage}" />
          <logonly enable="${enable.logonly}" />
          <debug enable="${enable.debug}" />
          <class name="${test.classname1}" >
             <name name="${test.classname2}" />
             <name name="${test.classname3}" />
          </class>
      </args>
</rbm-run-tests>

Attributes

  • package - optional, specifies package name of tester application. Note that this attribute is optional, but if it's not specified, <tester file="..." /> must be specified inside <apks /> element. Otherwise task will not know from what package to run tests and will fail the build.

  • runner - optional, default: android.test.InstrumentationTestRunner. Specifies what testrunner should be used to run tests.

  • uninstall - optional, default: true. If set to true all .apk files are uninstalled before installing them again (to erase application data), and then uninstalled after tests have run (sequence: unisntall -> install -> test -> uninstall). If set to false, all .apk files are re-installed (i.e. installed over the old version of .apk file if such existed) with all old application data left intact; also after tests are run .apk files are not uninstalled (i.e. now sequence is: reinstall -> test).

  • failOnError - optional, default: false. If set to true and at least one test fails - build fails. If set to false and any number of tests fail, build continues to run.

Elements

The junit element

<junit dir="${output.dir}" multiple="${use.multiple.files}" />

Optional, can only appear once. Specifies where to save junit reports. If this element is not added, reports are not created.

  • dir - required, path where report file(s) are placed.
  • multiple - optional, default: true. If true, each test suite has its own report file. If false, all suite reports are consolidated into one file.

The apks element

<apks>
    <apk file="${apk.file1}" />
    <apk file="${apk.file2}" />
    <tester file="${apk.tester}" />
    <fileset refid="${apk.fileset}" />
</apks>

Specified what .apk files must be installed before running tests. There are three supported sub-elements:

  • apk - can appear multiple times, has form <apk file="${apk.file.path}" />.
  • fileset - can appear multiple times, usual ant fileset.
  • tester - can appear at most 1 time. Note that this element is required if task does not have package attribute. Works same way as apk but package name is extracted from this specific apk. Has form: <tester file="${tester.apk.file}.

The args element

<args>
    <arg key="${arg.key1}" value="${arg.value1}" />
    <arg key="${arg.key2}" value="${arg.value2}" />
    <coverage enable="${enable.coverage}" />
    <logonly enable="${enable.logonly}" />
    <debug enable="${enable.debug}" />
    <class name="${test.classname1}" >
        <name name="${test.classname2}" />
        <name name="${test.classname3}" />
    </class>
</args>

Specifies custom arguments that are passed to testrunner. Testrunner accepts arguments in format key=value.

  • arg - can appear multiple times, if argument with the same name appears multiple times, the last one takes precedence (this is actually implementation specific, so don't rely on this fact). Has form: <arg key="${key} value=${value}" />`

  • coverage - should appear at most once, has format: <coverage enable="${enable.coverage}" />. Enables coverage reports.

Same as <arg key="coverage" value="${enable.coverage}" />.

  • logonly - should appear at most once, has format <logonly enable="${enable.logonly}" />. If set to true, tests are only enumerated (and not run).

Same as <arg key="log" value="${enable.logonly}" />.

  • debug - should appear at most once, has format <debug enable="${enable.debug}" />. Asks testrunner to wait for debugger to be attached before running tests.

Same as <arg key="debug" value="${enable.debug}" />.

  • class- should appear at most once, has format <class name="${classname}" />. Specifies what test suite classes to run.

Supports nested elements, like:

<class name="${test.classname1}" >
   <name name="${test.classname2}" />
   <name name="${test.classname3}" />
</class>

If nested elements are specified, name attribute may be omitted.

Examples

Let's assume that ${test.apk.file} has package name tester.pkg. Then these examples have same effect:

<rbm-run-tests>
   <apks>
      <apk file="${tested.apk}" />
      <tester file="${tester.apk}" />
   </apks>
</rbm-run-tests>

<rbm-run-tests package="tester.pkg" >
   <apks>
      <apk file="${tested.apk}" />
      <apk file="${tester.apk}" />
   </apks>
</rbm-run-tests>

If you want to fail the build when at least one test fails (but after all tests have run):

<rbm-run-tests failOnError="true">
    <apks>
        <apk file="${tested.apk}" />
        <tester file="${tester.apk}" />
    </apks>
</rbm-run-tests>

If you want to troubleshoot application's data after tests, disable .apk unisntallation:

<rbm-run-tests uninstall="false" >
    <apks>
       <apk file="${tested.apk}" />
       <tester file="${tester.apk}" />
    </apks>
</rbm-run-tests>

If you want to specify set of .apk files in some other place and then pass them to testrunner:

<fileset dir="${bin.dir}" id="apk.files" >
    <include name="*.apk"/>
</fileset>
<rbm-run-tests package="com.robomorphine.tester.app" >
   <apks>
       <fileset refid="apk.files" />
   </apks>
</rbm-run-tests>

If you want to generate junit reports:

<rbm-run-tests>
    <junit dir="${project.bin.dir}/junit" multiple="true" />
    <apks>
        <apk file="${tested.apk}" />
        <tester file="${tester.apk}" />
    </apks>
</rbm-run-tests>

And finally if you want to fully customize which testrunner runs your tests and what arguments you want to send him:

<rbm-run-tests runner="android.test.InstrumentationTestRunner" >
   <apks>
      <apk file="${tested.apk}" />
       <tester file="${tester.apk}" />
   </apks>
   <args>
       <arg key="log" value="true" />
       <coverage enable="false" />
       <debug enable="false" />
   </args>
</rbm-run-tests>

If you're using standard testrunner, you can specify what test suites are executed:

<rbm-run-tests>
   <apks>
       <apk file="${tested.apk}" />
       <tester file="${tester.apk}" />
   </apks>
   <args>
      <class>
          <name name="com.robomorphine.tester.ExampleTestA" />
          <name name="com.robomorphine.tester.ExampleTestB" />
       </class>
   </args>
</rbm-run-tests>

Go to home | documentation | ant tasks

Clone this wiki locally