Question: Help with scraping <tBody> information rendered dynamically

Hey folks,

Looking for a point in the right direction....

Main Questions:

• How scrape table information that appears to be rendered dynamically via JS?
• How to modify selenium so that html elements visible via chrome inspection are also visible to selenium?

Tech Stack:

• I'm using Scrapy & Selenium
• Chrome Driver

Context:

• Very much a novice at web scraping. Trying to pull information for another project.
• Trying to scrape the doctors information located in this table: https://ishrs.org/find-a-doctor/
• When I inspect the html in chrome tools I see the elements I'm looking for
• When I capture the html from driver.page_source I do not see the table elements which makes me think the table is rendered via js
• I've tried:

​

EC.presence_of_element_located((By.CSS_SELECTOR, "tfoot select.nt_pager_selection"))
EC.visibility_of_element_located((By.CSS_SELECTOR, "tfoot select.nt_pager_selection"))  

• I've increased the delay WebDriverWait(driver, 20)

Thoughts?