Source
r/learnpython•
FunCicada

Requests-HTML Confusion

In the tutorial, Kenneth has code along the lines of:

from requests_html import HTMLSession

session = HTMLSession()
r = session.get('websiteurl')
all_links = r.html.links     # gets links on page

Now here's where I'm confused - the get() method in HTMLSession returns a Response object, but where is html being called from? The Response object doesn't seem to list html as an attribute or as a method. See here for what I'm looking at: https://html.python-requests.org/

Posted Aug 11, 2018, 5:04 PM UTC · edited

65
rinyre•

It's definitely listed as one of the main classes. See here: https://html.python-requests.org/#requests_html.HTML.html

3
jeffrey_f•

r.text will contain the text of the html

1
FunCicadaOP•

But what is the html - is it an object?

1
TangibleLight•

HTMLResponse.html is an HTML object. The reason you're not seeing it as an attribute is that it's a property. You'll be able to see it in dir(HTMLResponse). HTML inherits from BaseParser, which contains all the stuff like links, xpath, and pq. The HTML class itself mainly just adds the render method and adds support for iterating through webpages.

5
jeffrey_f•

To you, me and the browser, it is the underlying source code to every webpage.

To python, EVERYTHING is an object.

1