Python - Resume Web Session From Urllib2 After Manual Browser Login

Say, I browse to a website (on intranet too) that require a login to access the contents. I will fill in the required fields... e.g. username, password and any captcha, etc. that i

Solution 1:

Did you consider Selenium? It's about browser automation instead of http requests (urllib2), and you can manipulate the browser in between steps.

Solution 2:

You want to use the cookielib module.

http://docs.python.org/library/cookielib.html

You can log on using your browser, then export the cookies into a Netscape-style cookie.txt file. Then from python you'll be able to load this and fetch the resource you require. The cookie will be good until the website expires your session (often around 30 days).

import cookielib, urllib2cj= cookielib.MozillaCookieJar()
cj.load('cookie.txt')
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
r = opener.open("http://example.com/resource")

There are add-ons for Chrome and Firefox that will export the cookies in this format. For example:

https://chrome.google.com/webstore/detail/lopabhfecdfhgogdbojmaicoicjekelh

https://addons.mozilla.org/en-US/firefox/addon/export-cookies/