{"id":749,"date":"2015-01-19T00:33:38","date_gmt":"2015-01-18T22:33:38","guid":{"rendered":"http:\/\/sven-seeberg.de\/wp\/?p=749"},"modified":"2015-12-05T14:59:04","modified_gmt":"2015-12-05T12:59:04","slug":"website-mapper","status":"publish","type":"post","link":"https:\/\/sven-seeberg.de\/wp\/?p=749","title":{"rendered":"Website Mapper"},"content":{"rendered":"<p>Here is a small python program a friend and I wrote to map and visualize the structure of websites. It is licensed with the <a title=\"http:\/\/opensource.org\/licenses\/bsd-license.php\" href=\"http:\/\/opensource.org\/licenses\/bsd-license.php\">BSD license<\/a>. The programs opens all links in the start address and recursivly searches through all following links. The crawling process may take a long time for more than 1 or 2 recursions. This may cause a lot of traffic for a website, therefore please contact the site&#8217;s admin before you run the program.<\/p>\n<p>Download: <a title=\"https:\/\/sven-seeberg.de\/download\/iac.py\" href=\"https:\/\/sven-seeberg.de\/download\/iac.py\">https:\/\/sven-seeberg.de\/download\/iac.py<\/a><\/p>\n<p>It creates a text file which can be visualized by <a title=\"http:\/\/graphviz.org\" href=\"http:\/\/graphviz.org\">graphviz<\/a>. The result may look like <a title=\"https:\/\/sven-seeberg.de\/download\/sitemap2.svg\" href=\"https:\/\/sven-seeberg.de\/download\/sitemap2.svg\">this<\/a> or with 2 recursions like <a title=\"https:\/\/sven-seeberg.de\/download\/sitemap.svg\" href=\"https:\/\/sven-seeberg.de\/download\/sitemap.svg\">this<\/a>.<\/p>\n<p>The program requires Python 2.7, urllib2 and BeautifulSoup.<\/p>\n<p>Usage:<\/p>\n<blockquote><p>$ python(.exe) iac.py [parameters]<\/p><\/blockquote>\n<p>The required parameters are:<br \/>\n-f [path to result file]<br \/>\n-r [number of recursions]<br \/>\n-n [&#8220;title&#8221;|&#8221;url&#8221;] (map site titles or URls)<br \/>\n-u [url to map]<\/p>\n<p>Create a visualization with graphviz:<\/p>\n<blockquote><p>$ dot -Tsvg result.txt -o sitemap.svg<\/p><\/blockquote>\n","protected":false},"excerpt":{"rendered":"<p>Here is a small python program a friend and I wrote to map and visualize the structure of websites. It is licensed with the BSD license. The programs opens all links in the start address and recursivly searches through all &hellip; <a href=\"https:\/\/sven-seeberg.de\/wp\/?p=749\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[],"class_list":["post-749","post","type-post","status-publish","format-standard","hentry","category-software"],"_links":{"self":[{"href":"https:\/\/sven-seeberg.de\/wp\/index.php?rest_route=\/wp\/v2\/posts\/749","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sven-seeberg.de\/wp\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sven-seeberg.de\/wp\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sven-seeberg.de\/wp\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sven-seeberg.de\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=749"}],"version-history":[{"count":16,"href":"https:\/\/sven-seeberg.de\/wp\/index.php?rest_route=\/wp\/v2\/posts\/749\/revisions"}],"predecessor-version":[{"id":806,"href":"https:\/\/sven-seeberg.de\/wp\/index.php?rest_route=\/wp\/v2\/posts\/749\/revisions\/806"}],"wp:attachment":[{"href":"https:\/\/sven-seeberg.de\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=749"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sven-seeberg.de\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=749"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sven-seeberg.de\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=749"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}