I'm using Capybara, the selenium-webdriver gem, and chromedriver in order to drive my javascript enabled tests.
The problem is that about 50% of our builds fail due to a Net::ReadTimeout error. At first this was manifesting as a 'could not find element' error, but after I upped Capybara's default max wait time to 30 seconds, I started seeing the timeout.
I examined the screenshots of when the timeout happens, it's stuck on a 'Successfully logged in' modal that we show briefly before using the Javascript function, location.reload(), to reload the page.
I've ran the test locally and can sometimes reproduce it, also randomly. Sometimes it zips by this modal and does the reload so fast you can barely see it, and other times it just hangs forever.
I don't feel like it's an asset compilation issue, since the site has already loaded at that point in order for the user to access the login form.
Wondering if anyone has seen this before and knows a solution.
The specific code:
visit login_path
page.within '#sign-in-pane__body' do
fill_in 'Email', with: user.email
click_button 'Submit'
end
expect(page).to have_content 'Enter Password'
page.within '#sign-in-pane__body' do
fill_in 'Password', with: user.password
click_button 'Submit'
end
expect(page).to have_text 'Home page landing text'
The hang up happens between click_button 'Submit' and expecting the home page text.
The flow of the logic causing the timeout is the user submits the login form, we wait for the server to render a .js.erb template that triggers a JS event upon successful login. When that trigger happens we show a modal saying that login was successful, then execute a location.reload().
It turned out this wasn't exclusive to doing a location.reload() in JS. It sometimes happened just visiting a page.
The solution for me was to create an HTTP client for the selenium driver and specify a longer timeout:
Capybara.register_driver :chrome do |app|
client = Selenium::WebDriver::Remote::Http::Default.new
client.read_timeout = 120
Capybara::Selenium::Driver.new(app, {browser: :chrome, http_client: client})
end
Solved similar problem by using my own version of visit method:
def safe_visit(url)
max_retries = 3
times_retried = 0
begin
visit url
rescue Net::ReadTimeout => error
if times_retried < max_retries
times_retried += 1
puts "Failed to visit #{url}, retry #{times_retried}/#{max_retries}"
retry
else
puts error.message
puts error.backtrace.inspect
exit(1)
end
end
end
Here is what you need to do if you need to configure it for headless chrome
Capybara.register_driver :headless_chrome do |app|
client = Selenium::WebDriver::Remote::Http::Default.new
client.timeout = 120 # instead of the default 60
options = Selenium::WebDriver::Chrome::Options.new
options.headless!
Capybara::Selenium::Driver.new(app, {
browser: :chrome,
http_client: client,
options: options
})
end
Capybara.default_driver = :headless_chrome
Capybara.javascript_driver = :headless_chrome
Passing headless argument in capabilities was not working for me.
capabilities = Selenium::WebDriver::Remote::Capabilities.chrome(
chromeOptions: { args: %w[headless disable-gpu] }
)
Here is more details about why headless in capabilities was not working.
Related
Here is the issue that has been nagging for weeks and all solutions found online do not seem to work... ie. wait for ajax, etc...
here is versions of gems:
capybara (2.10.1, 2.7.1)
selenium-webdriver (3.0.1, 3.0.0)
rspec (3.5.0)
running ruby 2.2.5
ruby 2.2.5p319 (2016-04-26 revision 54774) [x64-mingw32]
in the env.rb
Capybara.register_driver :selenium do | app |
browser = (ENV['browser'] || 'firefox').to_sym
Capybara::Driver::Selenium.new(app, :browser => browser.to_sym, :resynchronize => true)
Capybara.default_max_wait_time = 5
end
Here is my dynamicpage.feature
Given I visit page X
Then placeholder text appears
And the placeholder text is replaced by the content provided by the json service
and the step.rb
When(/^I visit page X$/) do
visit('mysite.com/productx/')
end
When(/^placeholder text appears$/) do
expect(page).to have_css(".text-replacer-pending")
end
Then(/^the placeholder text is replaced by the content provided by the json service$/) do
expect(page).to have_css(".text-replacer-done")
end
the webpage in question, which I cannot add it here as it is not publicly accessible, contains the following on page load:
1- <span class="text-replacer-pending">Placeholder Text</span>
after a call to an external service (which provides the Json data), the same span class gets refreshed/updated to the following;
2- <span class="text-replacer-done">Correct Data</span>
The problem I have with the "visit" method in capybara + selenium is that as soon as it visits the page, it thinks everything loaded and freezes the browser, and it never lets the service be called to dynamically update the content.
I tried the following solutions but without success:
Capybara.default_max_wait_time = 5
Capybara::Driver::Selenium.new(app, :browser => browser.to_sym, :resynchronize => true)
add sleep 5 after the visit method
wait for ajax solution from several websites, etc...
adding after hooks
etc...
I am at a complete loss why "visit" can't wait or at least provide a simple solution to an issue i am sure is very common.
I am aware of the capybara methods that wait and those that don't wait such as 'visit' but the issue is;
there is no content that goes from hidden to displayed
there is there is no user interaction either, just the content is getting updated.
also unsure if this is a capybara issue or a selenium or both.
Anyhow have insight on any solutions? i am fairly new to ruby and cucumber so specifically what code goes in what file/folder would be much appreciated.
Mel
Restore wait_until method (add it to your spec_helpers.rb)
def wait_until(timeout = DEFAULT_WAIT_TIME)
Timeout.timeout(timeout) do
sleep(0.1) until value = yield
value
end
end
And then:
# time in seconds
wait_until(20) { has_no_css?('.text-replacer-pending') }
expect(page).to have_css(".text-replacer-done")
#maxple and #nattf0dd
Just to close the loop on our issue here...
After looking at this problem from a different angle,
we finally found out Cucumber/Capybara/ is not a problem at all :-)
The issue we are having lies with the browser Firefox driver (SSL related), since we have no issues when running the same test with the Chrome driver.
I do appreciate the replies and suggestions and will keep those in mind for future.
thanks again!
I am working on a program for downloading bills that will automatically navigate to a webpage using the web browser object, log into it, navigate to the Bills page and download the most recent bill.
It all works well up until where I actually download it, as using the invokemember("click") code fires up IE and asks me to log in again which is not what I want to do. The DownloadFile method also does not work, and downloads a file that says the object has been moved to he login URL.
Right clicking and selecting "save target as" from the web browser works, but I'm not sure how to go about automating it.
Edit: Here's the code for the downloading part, parts of it were borrowed from another question on here but I can't remember who or where.
Dim IsRightElement As Boolean = False
For Each curElement As HtmlElement In Browser.Document.Links()
If curElement.GetAttribute("InnerText") = "Download Call Charges" Then
IsRightElement = True
End If
If IsRightElement Then
Dim Link As String = curElement.DomElement.href.ToString()
'This is where I'm stuck
'My.Computer.Network.DownloadFile(Link, "C:\Users\user\Desktop\PhoneBill.csv", "<username>", "<password>")
'The above does not work
'curElement.InvokeMember("contextmenu")
'Not sure what to do here
IsRightElement = False
Exit For
End If
Next
Me.Close()
I've got got a page with some AJAX code that I'm trying to build a spec for using Capybara. This test works:
context 'from the JobSpec#index page' do
scenario 'clicks the Run now link' do
visit job_specs_path
within( find('tr', text: #job_spec.name) ) { click_link 'run_now' }
visit current_path
expect(find('tr', text: #job_spec.name)).to have_content(/(running|success)/)
end
end
After clicking the link 'run_now', a job is launched and the user is redirected to the launched_jobs_path, which has a Datatable that uses some AJAX to pull currently running jobs. This test passes. However, I wanted to add to the test to check to make sure that the job wasn't already running before clicking the 'run_now' button (which would give me a false positive). I started playing around with it, but then I noticed that even simply putting visit launched_jobs_path before visit job_specs_path would cause the test to fail with the error
Failure/Error: expect(find('tr', text: #job_spec.name)).to have_content(/(running|success)/)
Capybara::ElementNotFound:
Unable to find css "tr" with text "job_spec_collection-1"
I'm fairly certain that this is an issue with the Javascript not running at the right time, but I don't quite know how to resolve it. Any ideas? This is Rails 4.1 with Capybara 2.4, and Capybara-webkit 1.3.
Thanks!
Update
In the parent context of the spec, I do have :js => true
feature 'User runs a JobSpec now', :js => true do
before do
Capybara.current_driver = :webkit
#job_spec = FactoryGirl.create(:job_spec, :with_client_spaces)
end
after { Capybara.use_default_driver }
context 'from the JobSpec#index page' do
...
end
end
I think it might be as simple as adding :js => true to your scenario header so that it knows to launch webkit / selenium to test the js / ajax.
scenario 'clicks the Run now link', :js => true do
Hope that helps!
UPDATE
Well here's one more idea. You could try adding this to your 'rails_helper' or 'spec_helper' file to set the :js driver.
Capybara.javascript_driver = :webkit
Do you have other :js tests that are working properly with :webkit? Have you tried using save_and_open_page to see if the element you are looking for is actually on the page and Capybara just isn't finding it?
Ok, pretty sure I've got something that will at least work for now. If anyone has more elegant solutions I'd love to hear them.
I think the issue is in the complexity of what happens in the 'run_now' controller. It adds a job to a Rufus scheduler process, which then has to start the job (supposed to start immediately). Only after the launched job is running via Rufus does an entry in the launched_jobs table get created.
While developing, I would see the job just launched as soon as I clicked 'Run now'. However, I'm guessing that WebKit is so fast that the job hasn't been launched by the time it gets to the launched_jobs#index page. The extra visit current_path there seem to help at first because it would refresh the page, but it only worked occasionally. Wrapping these last two steps in a synchronize block seems to work reliably (I've run it about 20 times now without any failures):
context 'from the JobSpec#index page' do
scenario 'clicks the Run now link' do
# First make sure that the job isn't already running
visit launched_jobs_path
expect(page).to have_no_selector('tr', text: #job_spec.name)
# Now launch the job
visit job_specs_path
within( find('tr', text: #job_spec.name) ) { click_link 'run_now' }
page.document.synchronize do
visit current_path
expect(find('tr', text: #job_spec.name)).to have_content(/(running|success)/)
end
end
end
I'm very new to Capybara and have also never used Selenium before. I'm doing a ruby on rails project on MacOSX and for whatever reason, a browser window never opens when I run my test.
My stack is: Capybara, Selenium, Rspec, Ruby on Rails.
My test is as follows:
describe 'Downloads', js: true do
context ' compress zip and download file' do
before do
Capybara.current_driver = :selenium
session = Capybara::Session.new(:selenium)
session.visit '/users/sign_in'
find('#tab_signin').click
within("#new_user") do
fill_in 'user_login', :with => 'admin#local.host'
fill_in 'user_password', :with => 'password'
end
click_button 'Sign in'
end
it 'downloads the project and navigates to downloads page' do
visit 'some/path'
within '.create-download' do
find(:select).find("option[value='zip']").select_option
end
sleep 3
page.should have_css('#download-modal.in')
end
end
end
I've also tried to change stuff in my features/support/env.rb to be this:
Capybara.javascript_driver = :selenium
Capybara.register_driver :selenium do |app|
profile = Selenium::WebDriver::Firefox::Profile.new
Capybara::Selenium::Driver.new(app, :browser => :firefox, :profile => profile)
end
Update
Not only is the browser not opening, but the test is failing with the following output:
Failure/Error: visit '/users/sign_in'
ArgumentError:
unknown option: {:resynchronize=>true}
So after a lot of work I finally figured it out. Thanks to #RAJ for the suggestion of where to put that config info. The feature/support/env.rb is for cucumber and I'm using rspec.
Most of the articles I read about selenium and capybara told me to use the js: true option at the start of the block, but that didn't work. Once I changed that to feature: true it worked. My final solution looks like this:
describe 'Downloads', feature: true do
context 'compress zip and download file' do
before do
visit '/users/sign_in'
find("a[href$='signin']").click
within("#new_user") do
fill_in 'user_login', :with => 'admin#local.host'
fill_in 'user_password', :with => 'password'
end
click_button 'Sign in'
end
it 'downloads the project and navigates to downloads page' do
visit 'some/path'
within '.create-download' do
find(:select).find("option[value='zip']").select_option
end
sleep 3
page.should have_css('#download-modal.in')
end
end
end
Then my spec_helper.rb looks like:
Capybara.register_driver :selenium do |app|
profile = Selenium::WebDriver::Firefox::Profile.new
Capybara::Selenium::Driver.new( app, :profile => profile)
end
Capybara.default_wait_time = 10
Capybara.current_driver = :selenium
Capybara.app_host = 'http://localhost:3000'
Another thing that I did and I wasn't aware before was install the Selenium IDE on Firefox. Since I'd never used Selenium before, I thought all I needed was the gem.
I'm trying to scrape a site that's jQuery based and I'm having trouble with getting the page to load completely before extracting the elements with Selenium. The page has multiple modules, each of which is a different query. I tried using the wait commands I found in the documentation, but it would usually hang the browser after one of the multiple queries load.
For reference, my OS is Windows 7, Firefox 30.0, Python 2.7 and Selenium 2.42.1
The commands and results I tried are as follows:
Explicit Wait: Browser hangs after loading the first query (Firefox Not Responding)
try:
element = WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.XPATH, path)))
finally:
browser.quit()
Expected Conditions: Browser hangs after loading the first query (Firefox Not Responding)
wait = WebDriverWait(browser, 10)
element = wait.until(EC.element_to_be_clickable((By.XPATH,path)))
Implicit Wait: Firefox hangs after loading the first query (Firefox Not Responding)
browser.implicitly_wait(10) # seconds
myDynamicElement = browser.find_element_by_xpath(path)
Custom Function: Page loads, but selenium starts scraping before the second query is loaded resulting in an error
def wait_for_condition(browser,c):
for x in range(1,10):
print "Waiting for jquery: " + c
x = browser.execute_script("return " + c)
if(x):
return
time.sleep(1)
def main():
wait_for_condition(browser,"jQuery.active == 0")
#First element to be clicked on to scrape:
path="//a[starts-with(#class, 'export db')]"
browser.find_element_by_xpath(path).click()
The error is:
selenium.common.exceptions.NoSuchElementException: Message: u'Unable to locate element: {"method":"xpath","selector":"//a[starts-with(#class, \'export db\')]"}' ;
Catching this exception and running wait_for_condition again in the except block causes the browser to stop loading the rest of the queries and hang:
wait_for_condition(browser,"jQuery.active == 0")
try:
path="//a[starts-with(#class, 'export db')]"
browser.find_element_by_xpath(path).click()
except NoSuchElementException:
wait_for_condition(browser,"jQuery.active == 0")
path="//a[starts-with(#class, 'export db')]"
browser.find_element_by_xpath(path).click()
Please let me know if you have any suggestions to solving the problems.
Thanks in advance,
Teresa