"I estimate close to 50% of our server CPU resources go to serving crawler robots." http://goo.gl/QUIAI