Reply to thread

Message

<blockquote data-quote="Brackson" data-source="post: 242816" data-attributes="member: 34747">Current version: v1.1[code]#!/usr/bin/env pythonimport urllib2, os, timedef store_into_file():&nbsp;&nbsp; url = 'http://google.com/' # URL that you want to mine.&nbsp;&nbsp; data = urllib2.urlopen(url).read() # Get the HTML source of URL.&nbsp;&nbsp; current_time = time.strftime('%H:%M:%S', time.localtime()) # Get the current time so we can use if for the txt filename.&nbsp;&nbsp; r = open('%s.txt' % (current_time), 'w') # Create the file.&nbsp;&nbsp; r.write(data) # Put the source in the file.&nbsp;&nbsp; r.close() # Close the file.def main():&nbsp;&nbsp; store_into_file()while True:&nbsp;&nbsp; main()[/code](<a href="http://pastebin.com/vgXh761a" target="_blank">with syntax formatting</a>)PREVIOUS VERSIONS<ul>
<li data-xf-list-type="ul"><a href="http://pastebin.com/Bi9fhYQU" target="_blank">v1.0</a></li>
</ul>This script allows you to take the HTML contents of a webpage, and store it in a text file. You do not to install any modules for this (AFAIK). If you want to parse the content before it's stored into the file, you can use <a href="http://www.crummy.com/software/BeautifulSoup/" target="_blank">BeautifulSoup</a>.This would come in handy if you were building an archive site, if you want to log a website's content just because, etc.I made this for educational purposes, and I thought I'd just release it because I don't really need it. Thanks for viewing!</blockquote>

[QUOTE="Brackson, post: 242816, member: 34747"] Current version: v1.1 [code] #!/usr/bin/env python import urllib2, os, time def store_into_file(): url = 'http://google.com/' # URL that you want to mine. data = urllib2.urlopen(url).read() # Get the HTML source of URL. current_time = time.strftime('%H:%M:%S', time.localtime()) # Get the current time so we can use if for the txt filename. r = open('%s.txt' % (current_time), 'w') # Create the file. r.write(data) # Put the source in the file. r.close() # Close the file. def main(): store_into_file() while True: main() [/code] ([URL='http://pastebin.com/vgXh761a']with syntax formatting[/URL]) [U]PREVIOUS VERSIONS[/U] [LIST] [*][URL='http://pastebin.com/Bi9fhYQU']v1.0[/URL] [/LIST] This script allows you to take the HTML contents of a webpage, and store it in a text file. You do not to install any modules for this (AFAIK). If you want to parse the content before it's stored into the file, you can use [URL='http://www.crummy.com/software/BeautifulSoup/']BeautifulSoup[/URL]. This would come in handy if you were building an archive site, if you want to log a website's content just because, etc. I made this for educational purposes, and I thought I'd just release it because I don't really need it. Thanks for viewing! [/QUOTE]

Verification

Reply to thread

Connect with us

Newest members