PhantomJS does not see dropdown options populated by AJAX
I've been trying to scrape a site using a PhantomJS/Selenium setup in Python.
There's a dropdown in the page that, when selected, populates the options of another dropdown. When I'm automating this with Firefox, when I select_by_visible_text on the first dropdown, the second one gets populated.
However, the same code doesn't work with PhantomJS.
browser = webdriver.PhantomJS(executable_path=PHANTOMJS_PATH) ## browser = webdriver.Firefox() wait = WebDriverWait(browser, WAIT_TIME) browser.get(URL) Select(browser.find_element_by_id('DropDown0')).select_by_visible_text('XXX') def condition_wait_for_dropdown(driver): ret = False if driver.find_elements_by_xpath('//*[@id="DropDown1"]/option'): ret = True return ret wait.until(condition_wait_for_dropdown)
I have tried using a hard-coded sleep instead of the custom waiting condition, and checked the DOM - the second dropdown never gets populated at all.
Can this be a problem with the webpage itself? If so, how do I work around this?
EDIT: Just to clarify, this works with Selenium+Firefox, so the possibility exists that this is a bug in the PJS driver itself.
I found a fix for this, so best to leave it here for posterity.
I had an input field in the page form, so after selecting the first dropdown I moved the focus to the input field, and sent an ENTER to it. The second dropdown got loaded immediately.
Still don't know wth is wrong with this web page.