/**************************************************
PasteBin Scraper
by Andrew MacPherson
andrew@andrewmohawk.com
@AndrewMohawk
------------------------
Readme
**************************************************/

INDEX
------
1 // What is PasteBinScraper?
2 // How does it work?
3 // How do i use it?
4 // How do i install?
5 // Extending
6 // TODO
7 // Thanks

1. What is PasteBinScraper
---------------------------
Originally concieved as a method to enumerate various pastebins (slexy,pastie,pastebin,gisthub) as a response to corelans 
pastenum - http://www.corelan.be/index.php/2011/03/22/pastenum-pastebinpastie-enumeration-tool/, its changed slightly to a more general enumerator. 
Pastenum is a ruby application that runs client side to enumerate various bins, this just took it a bit further by putting it in a web application
as mine kept crashing trying to load the previous results.

Think of it as a means of searching various pastebins for information.

Pro Tip:
Try "facebook password","gmail password","password enable","scada",etc

2. How does it work?
---------------------
PasteBinScraper works in one of two ways:

Using Yahoo!BOSS Search API:
Basically does <term> site:<site> and returns all the results with highlighting of the phrases.
eg. go to yahoo, and search for "DB_PASSWORD site:pastebin.com"

Using Cusom Scraping Scripts:
At the moment i've just included a basic (read ugly, badly coded) script to enumerate pastie.org (i did speak to the owner to ask if it was alright!),
essentially the frontend simply calls the script and returns the results under one of the tabs. It calls the script with a 'q' GET parameter so something like
http://<server>/libs/myscript.php?q=Search+Term

These scripts are all located in the libs/ directory under the root.

3. How do i use it?
-------------------
Err, type in a phrase top right and click search :D The tabs will show which have loaded and once you see the 'show' button you can simply click on it
to view the results :)

4. How do i install?
--------------------
Pretty simple, pull the archive from wherever (not sure where it will be hosted at time of writing this) and extract it to somewhere within your 
webservers (apache/iis/etc) webroot... something like /var/www/PasteBin/

You can then simply browse to it and go :)

For it to work you will need a webserver that has php, php5-curl and you may want to change the 'memory_limit =' field in your php.ini to something like
memory_limit = 128MB. This is because the multi-threaded requests take a bit of mem :)

If you are on ubuntu basically its:
sudo apt-get install apache2 php5 libapache2-mod-php5 php5-curl
(but most hosts should have this)


5. Extending
---------------
Extending is a relatively simple process whereby you either add a site, or you add a custom script.

Within libs/config.php you will see two arrays under the pastebin section:

/* PasteBin Section
--------------------
Please note there needs to be the same number of identifiers as types... *duh*
*/
$PasteBin_Identifiers = array(
	"Pastie.org",
	"Pastebin.com",
	"Codepad.org",
	"Slexy.org"
);

//Types can either be 'yahoo' (for doing <term> site:<identifier>) or a custom script, like 'mypastie.php' 
$PasteBin_Types = array(
	"pastie.php",
	"yahoo",
	"yahoo",
	"yahoo",
);

Basically you want to add to both of these to extend it. $PasteBin_Identifiers is an array of the sites to enumerate and $PasteBin_Types defines the method. 
For the types you can set it as a specific script (which you need to put in libs/) such as 'pastie.php' in the default application. Alternatively you can use
the keyword "yahoo" to tell the app to use a search engine.

So with the above definitions if i wanted to add gist.github.com to search aswell and i didnt want to code anything i would change them as follows:

Before:
-------
$PasteBin_Identifiers = array(
	"Pastie.org",
	"Pastebin.com",
	"Codepad.org",
	"Slexy.org"
);

After:
------
$PasteBin_Identifiers = array(
	"Pastie.org",
	"Pastebin.com",
	"Codepad.org",
	"Slexy.org",
	"gist.github.com"
);

BAM! now refresh the interface and you will notice it updated and now has a new tab :)

Alternatively if you wanted to code something up you would place your script in the libs/ directory and then change it to the following:
Before:
-------
$PasteBin_Identifiers = array(
	"Pastie.org",
	"Pastebin.com",
	"Codepad.org",
	"Slexy.org"
);

After:
------
$PasteBin_Identifiers = array(
	"Pastie.org",
	"Pastebin.com",
	"Codepad.org",
	"Slexy.org",
	"my_gist_script.php"
);

again, BAM! etc etc 


6. TODO
--------
6.1 The app definitely needs to be checked for anything dodgy (xss etc). (if you find it, PLEASE dont own my box, i will cry.)
6.2 'API' functionality -- something to just change the output to be xml so that other applications can use it (although they have the code anyway -shrug-)
6.3 Rework the code into a real framework (but im lazy and just like coding PoC stuff)
6.4 Suggestions?

7. Thanks
-----------
Just wanna say thanks to the corelan guys for getting me off my ass and doing something else (hey 2 days is better than 0).
Thanks for the motivation from nullthreat,corelanc0d3r,ekse,singe,etc. Josh from pastie.org for giving me the A-OK etc.

If theres any changes/stuff/hatemail feel free to mail me (andrew@andrewmohawk.com) or @andrewmohawk. 
Payments in alchoholic beverage form, attractive females and other will be considered.

