Be Aware of Bad Analytics

This FICO post warns: “The word ‘analytics’ now means anything from advanced optimization to recording the number of cars in the car park. And because it is now so general, it has been divorced from the underlying technical considerations that were once associated with the word.” More quotes:

“The algorithm has no knowledge of selection bias, sample truncation and so on, and so the result can be widely wrong no matter how clever the technique is.”

“More data doesn’t solve this problem if the data all has the same limitation. For example, a model may observe that a price fall is associated with increased production, but completely miss the fact that there are no substitute products that the producer can switch to, so output increases to maintain earnings.”

“We hope that bad analytics get exposed and rejected by the market. Unfortunately, given the current demand, this is likely to happen after some businesses bet their future on the performance of these models. Oops.”

This entry was posted in Business Analytics, Trends. Bookmark the permalink.

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s