Personal Web Spiders

I know some very large corporate companies use this technique to identify keywords that their competitors are using, I think this would be a very useful and profitable too. I know their are services out there that can do this for you, but why not do it yourself and keep the data for yourself? Then you are always one step ahead of the game?

I will be thinking very hard about if this is something to look in to, and information on them or questions are welcome.