Why my curl doesn't curl
Hi folks.
I am writing a simple scraping script. I am using scalpel, and it depends on curl.
The starting webpage of my scraper is https://mmhaskell.com/blog/. Unfortunately, when I make a request to that URL using curlGetString, I get back a value of CurlHttpReturnedError with no further data. If I use curlGetString on http://www.google.com, I get the expected value.
Oh - and using get from wreq works just fine. Am I missing a setting or something? I'd like to use scalpel if this problem is fixable.
Posted Mar 12, 2018, 9:18 PM UTC
14