I’ve probably rewritten/re-factored creep about 6/7 times now, it’s an absolute shambles. I created a new branch in the repo yesterday so I could finally start developing with unit tests, but I’ve run into a slight problem. How the fuck do you unit test the OptParse library? This is before I even start looking at unit testing the async network stuff in Tornado (which has its own unit testing support classes) which is partly the reason I’ve started rewriting creep again. I want creep to be async network capable because it has to crawl the site before it can start sifting through the source code to find potentially ‘interesting’ things. My definition of ‘interesting’ appears to be broadening as well. Here’s what creep flags so far:
- PHP messages (fatals, errors, warns etc)
- MySQL messages
- Comments in source code
- Linux file paths (the regex for this is quite difficult to get right)
- PHP code (code that hasn’t been parsed by the PHP interpreter)
- Header banners
I want to extend the functionality of creep to crawl pages mentioned in actions (in forms), as well as basic XSS and SQL injection. Firstly though, I need to make the application work again (the repo branch is essentially a rewrite containing lots of test code, and not much application code).
I also want to start messing around with GET / POST parameters. I figured one way to do this would be to generate a list of potentially interesting GET / POST parameters and values, and that’s when I ran into my next problem. How do I represent this data? I could have a list with a delimiter, but then what about parameters which may warrant more than one input (debug=1, debug=true, debug=yes, debug=full)? I could have two delimiters, but then I need to start thinking about a class to dissect the list. That’s when I started thinking about a database.. SQLite is bound to have an easy to use interface available in some Python library, but I digress, creep has a long way to go yet.