Fix failing tests

[jenlampton]

comment from @vstemen:

So far as I can tell, it is working fine but the automated tests using the testing module fail. They also fail on Drupal. Strangely, on Drupal, it immediately fails with

1 pass, 1 fail, and 0 exceptions

But on Backdrop, it reports a lot more tests and about half of them fail.

48 passes, 44 fails, 6 exceptions, and 19 debug messages

from @jenlampton:

I checked on the tests in the code, and it does look like there is only one test in this module. As a sanity check, are you sure you are running the same test group on the d7 instance and B? (Is there only one test box checked, and it's the one named "CSS Injector Functionality"?) I haven't tried to run the tests, but my guess is that maybe you are running more than one?

comment from @vstemen:

Yes, I checked the "CSS Injector" box only. It does say it is only running
1 test.

Processing test 1 of 1 - CSS Injector Functionality.

but after a fairly long delay of about a minute, it errors with

An error has occurred.
Please continue to the error page

An AJAX HTTP error occurred. HTTP Result Code: 504 Debugging information
follows. Path: /batch?id=9&op=do_nojs&op=do StatusText: Gateway Time-out
ResponseText: 504 Gateway Time-out 504 Gateway Time-out nginx/1.12.0 

Then when I go to the error page, I get

CSS Injector Functionality
48 passes, 43 fails, 6 exceptions, and 19 debug messages

and lists a whole bunch of FAIL and PASS result lines.

History

This issue has been copied over from backdrop-ops/contrib#256 (comment)

[jenlampton]

My guess is that there's a loop in the test somewhere that isn't terminating properly, since the error you're getting is a time-out. And that would explain why you're getting a lot more pass/fails than expected too.

My approach would be to comment things out and run the test till it completes successfully, then go back and see what's wrong with the parts that are commented out :)

[vstemen]

The problem appears to not be with css_injector. I ran another test using the
shell script and got similar results. With this test it took almost 4 minutes.

$ ./core/scripts/run-tests.sh  CommonFormatDateTestCase

Backdrop test run        
---------------          

Tests to be run:         
 - Format date (CommonFormatDateTestCase)          

Test run started:        
 Monday, April 23, 2018 - 12:05am                  

Test summary             
------------             

Common: Format date (CommonFormatDateTestCase) 45 passes, 55 fails, 0 exceptions, and 14 debug messages

Test run duration: 3 min 58 sec        

It seems to be something going on in the core setUp() function. In
css_injector, if I put a return immediately there is no delay, but if I return
after it calls
parent::setUp(array('css_injector', 'php'));
I get the long delay.

in css_injector.test I put a return right after the setUp() call and it takes
over a minute.

function setUp() {
  parent::setUp(array('css_injector', 'php'));
return; // XXX
  $privileged_user = $this->backdropCreateUser(array('administer css injection', 'use PHP for settings'));
  $this->backdropLogin($privileged_user);
}

And it Outputs

Backdrop test run        
---------------          

Tests to be run:         
 - CSS Injector Functionality (CSSInjectorTest)    

Test run started:        
 Monday, April 23, 2018 - 12:19am                  

Test summary             
------------             

CSS Injector: CSS Injector Functionality (CSSInjectorTest) 0 passes, 1 fail, and 0 exceptions          
Test run duration: 1 min 9 sec                     

Notice also, if I move the return down and run additional instructions, the
number of fails increases.

function setUp() {
  parent::setUp(array('css_injector', 'php'));
  $privileged_user = $this->backdropCreateUser(array('administer css injection', 'use PHP for settings'));
return; // XXX
  $this->backdropLogin($privileged_user);
}

Notice, here I moved the return after the backdropCreateUser() call and it
changed the result to 3 fails.

Backdrop test run        
---------------          

Tests to be run:         
 - CSS Injector Functionality (CSSInjectorTest)    

Test run started:        
 Monday, April 23, 2018 - 12:30am                  

Test summary             
------------             

CSS Injector: CSS Injector Functionality (CSSInjectorTest) 0 passes, 3 fails, and 0 exceptions         
Test run duration: 1 min 6 sec                     

In these tests, I also returned immediately in testCSSInjectionUI().

I wonder if it could be something about my environment or nginx web server setup
but Backdrop otherwise seems to be working fine.

[jenlampton]

The problem is php module. That doesn't exist for Backdrop, so you can't include it in the setUp() function.

…

On Sun, Apr 22, 2018, 10:53 PM Vincent Stemen ***@***.***> wrote: The problem appears to not be with css_injector. I ran another test using the shell script and got similar results. With this test it took almost 4 minutes. $ ./core/scripts/run-tests.sh CommonFormatDateTestCase Backdrop test run --------------- Tests to be run: - Format date (CommonFormatDateTestCase) Test run started: Monday, April 23, 2018 - 12:05am Test summary ------------ Common: Format date (CommonFormatDateTestCase) 45 passes, 55 fails, 0 exceptions, and 14 debug messages Test run duration: 3 min 58 sec It seems to be something going on in the core setUp() function. In css_injector, if I put a return immediately there is no delay, but if I return after it calls parent::setUp(array('css_injector', 'php')); I get the long delay. in css_injector.test I put a return right after the setUp() call and it takes over a minute. function setUp() { parent::setUp(array('css_injector', 'php')); return; // XXX $privileged_user = $this->backdropCreateUser(array('administer css injection', 'use PHP for settings')); $this->backdropLogin($privileged_user); } And it Outputs Backdrop test run --------------- Tests to be run: - CSS Injector Functionality (CSSInjectorTest) Test run started: Monday, April 23, 2018 - 12:19am Test summary ------------ CSS Injector: CSS Injector Functionality (CSSInjectorTest) 0 passes, 1 fail, and 0 exceptions Test run duration: 1 min 9 sec Notice also, if I move the return down and run additional instructions, the number of fails increases. function setUp() { parent::setUp(array('css_injector', 'php')); $privileged_user = $this->backdropCreateUser(array('administer css injection', 'use PHP for settings')); return; // XXX $this->backdropLogin($privileged_user); } Notice, here I moved the return after the backdropCreateUser() call and it changed the result to 3 fails. Backdrop test run --------------- Tests to be run: - CSS Injector Functionality (CSSInjectorTest) Test run started: Monday, April 23, 2018 - 12:30am Test summary ------------ CSS Injector: CSS Injector Functionality (CSSInjectorTest) 0 passes, 3 fails, and 0 exceptions Test run duration: 1 min 6 sec In these tests, I also returned immediately in testCSSInjectionUI(). I wonder if it could be something about my environment or web server setup but Backdrop otherwise seems to be working fine. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAYSR-mSxViXig07Rvc_HdPADVAGAjQ1ks5trWxxgaJpZM4Ten42> .

[vstemen]

Oh. I missed that. I hadn't looked that closely at the code since I didn't
know much about the testing API. I watched a Drupal 7 video about it today and
gain a better understanding :-).

I removed the php module portion of the tests and got a lot better results.

72 passes, 3 fails, 0 exceptions, and 25 debug messages

I still don't know why the 3 failures are happening.

Also, for some reason, I get very different results when I run the tests using
the command line script.

 $ ./core/scripts/run-tests.sh  CSSInjectorTest

 CSS Injector: CSS Injector Functionality (CSSInjectorTest) 27 passes, 30
 fails, 15 exceptions, and 13 debug messages            

 Test run duration: 1 min 28 sec                    

I will commit the modified .test file, probably sometime this evening for
review.

[vstemen]

OK. I pushed a commit with the test disabled that used the php module.

It seems to have improved a lot, but I seem to keep getting different results.
Now, after removing the test code that used the php module, I get

Results
0 passes, 1 fail, and 0 exceptions

Message                 Group               Filename            Line    Function                                Status
FAIL: The test did not complete 
due to a fatal error.   Completion check    css_injector.test   32      CSSInjectorTest->testCSSInjectionUI()   Fail

But if I reload the http://test1.test/admin/config/development/testing/results/1
page, I instantly get

Results
54 passes, 1 fail, 0 exceptions, and 18 debug messages

with a all green pass lines except the one failure which is different than the
one above.

FAIL: Rule 2 file css_injector_2.css was found on page user Other   css_injector.test   89  CSSInjectorTest->testCSSInjectionUI()   Fail

If I click on the Verbose message link right above the failure line, it shows

ID #10
GET request to: user
Ending URL: http://test1.test/accounts/jdevupzt

along with some other information.

I don't understand why, when I reload the page after the test completes, it
shows completely different results.

I cleared the cache and reloaded the testing page prior to running the test.
I'm still not sure if I am doing this correctly.

[vstemen]

I seem to consistently be getting just a couple failures now when running the
automated tests via the web interface.

When I tested on Drupal I actually got even more failures, so the tests are
apparently just broken. So far as I can tell, everything is working though, so
I will just have to re-visit it later if I get the chance. Perhaps one of the
original maintainers or developers will be interested in looking into it.

If we don't discover any other issues right away, I will go ahead and do
a release with the final cleanups that I have done.

[vstemen]

Weird. I reloaded the results page again and it changed the status to
17 passes, 0 fails, 0 exceptions, and 6 debug messages