Does Python Scrapy Work Properly On Localhost?

I have written a scrapy spider to scrape out some html tags. Now the problem is that this spider works perfectly for a url that is running on internet but not for a url that is on

Solution 1:

The error

exceptions.IndexError: list index outofrange

on this line

title = hxs.select("//h3")[0].extract() 

indicates that the list hxs.select("//h3") is empty ([]) since attempting to access the first item (index 0) with hxs.select("//h3")[0] uses an index which Python tells us is out of range.

The html you are parsing apparently has no <h3> tags.

---

Also, after you fix the above error, you'll need to put a comma after the a in (a,):

cur.execute("""Insert into heads(h2) Values(%s )""",(a,))

(a) is evaluated to a, whereas (a,) represents a tuple with 1 element inside.