Sharing report results TeamCity to Octane

Hello,

I am not being successful in sending my xml report from TeamCity to ALM Octane. 

I have followed the documentation on https://admhelp.microfocus.com/octane/en/16.1.100-16.2.100/Online/Content/AdminGuide/how-setup-testing-integration.htm.

I'm using ALM Octane 16.1.100.152 and cucumber-jvm 7.0.0. The Octane plugin for TeamCity is 1.4.6.

 

My initial questions are regarding the report file as they might solve my problem:

  • configuration for sharing the report with Octane: considering I have the plugin installed in TeamCity, from the documentation it sounds that it suffices to place the xml report in an artifact folder whose name is in the configuration parameter gherkin_artifact_folder_for_octane and set Testing_framework=cucumber_jvm for the report to be shared with Octane by the plugin when the job is finished. If the gherkin test already exists in Octane, the result is then shown there, otherwise a new entry is created. Is that correct or is there another step that is needed for the result to be injected?

  • xml report format: Since the results in the generated xml report wasn't injected to Octane, I suppose the reason might be the format. From the documentation, it seems that the name of the report would be "_OctaneGherkinResults.xml".  That made me think that I need to convert my initial report to a different format for Octane to read it.

I saw on https://admhelp.microfocus.com/octane/en/16.1.100-16.2.100/Online/Content/UserGuide/how_automate_gherkin.htm#mt-item-4 that if I'm using octane-cucumber-jvm, the standard name of the report is OctaneGherkinResults.xml. This lead me to think I should use that library. However, on its GitHub page it says it's only for cucumber up to 4.8.1 and that  the newer tool is bdd2octane.

So I used bdd2octane to generate this other report. As before, TeamCity was set to save it in the artifacts folder "gherkin" and the configuration parameters gherkin_artifact_folder_for_octane was set to "gherkin" and Testing_framework to "cucumber_jvm".

However, Octane still does not find this test result. It does recognize that the build with the corresponding number was successful or not and when it started/ended. But the test results are not injected.

Hopefully someone can point me toward the right direction.

I'm adding the general format of the xml reports below in case it helps.

Thanks!

Denise

Here is the general format of the xml report generated after my test runs:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>

<testsuite errors=… failure… name… skpiied… tests… time…>

<testcase classname=… name=… time=…>

<system-out><![CDATA[Given… passed

When… ….passed

Then…  ….passed]]>

</system-out>

</testcase>

</testsuite>

 

And here is the general format of the xml report generated when running bdd2octane with the above as input:

<?xml version="1.0" ?>

<!--Generated by bdd2octane 1.1.2-->

<features version="1">

<feature name=… path=… started=… tag={this is the Octane tag used to identify this feature}>

<file>

<![CDATA[#Auto generated Octane revision tag

{here is the whole feature file}]]>

</file>

<scenarios>

<scenario name… outlineIndex=…>

<steps>

<step name=Given… duration=… status=…></step>

<step name=When… …></step>

<step…></step>

</steps>

</scenario>

</scenarios>

</feature>

</features>

Parents
  • Hello,

    1. yes this is the case when same gherkin test with same context(release,milestone,program or environment) of the pipeline the test will not create new run only update the status of the exist one, but it will add new run on previous run tab.

    2.The format "_OctaneGherkinResults.xml" is not required , what is do require is that the folder with result generate from bdd2Octane will copied into Team city artifacts
    You can add on bdd2Octane tool the place you like to output the results for example --resultFile="bdd2octane-reports\cucumber-jvm-result.xml"

    Then on team city job you can define on project General setting in Artifact paths:bdd2octane-reports => gherkin-results
    This should copy result of bdd2Octane into Team City artifact and the plugin should get XML files from there that contain results

    Thanks,

    Nissim

  • Thank you Nissim.


    I'm doing all the steps you mentioned and the report generated with bdd2octane also seems correct and is in the TeamCity artifact. The name of the artifact folder (in the case of your example "gherkin_results") is defined in the configuration parameter "gherkin_artifact_folder_for_octane". So I guess the issue lies elsewhere.


    Are there known reasons why the plugin might not send the report or why Octane might not get it?

    Best regards,

    Denise

  • There is no known reason why it's not sending.

    In order to check if test result report has been sent from Team City you can check on plugin logs, that are located on this URL
    http(s)://[Team city url]:[port/]nga/logs/ and search for "successfully pushed test results for" or finish event that have tests "FINISHED(tests=true)" 

    If it is sending you can check on Octane why test not injected by adding follow for pipeline, but it will show the error of test injection only on case that you inject BDD specification. Follow can be done by click on pipeline menu (3 dots) -> view then on top right you have follow button

    Thanks,

    Nissim

    
    
Reply
  • There is no known reason why it's not sending.

    In order to check if test result report has been sent from Team City you can check on plugin logs, that are located on this URL
    http(s)://[Team city url]:[port/]nga/logs/ and search for "successfully pushed test results for" or finish event that have tests "FINISHED(tests=true)" 

    If it is sending you can check on Octane why test not injected by adding follow for pipeline, but it will show the error of test injection only on case that you inject BDD specification. Follow can be done by click on pipeline menu (3 dots) -> view then on top right you have follow button

    Thanks,

    Nissim

    
    
Children