Power Law


P2P relevance http://backspaces.net/sun/PLaw/

Zipf, Power-law, Pareto - a ranking tutorial - http://ginger.hpl.hp.com/shl/papers/ranking/ranking.html





the curve becomes steeper due to Network Effect-s.

The 2 ends of the curve: the A-List (Big Head, Fat Head) and the Long Tail.

Cosma Shalizi notes that distributions that look like a Power Law often aren't (quite). Ask yourself whether you really care. Maybe you don't. A lot of the time, we think, all that's genuine important is that the tail is heavy, and it doesn't really matter whether it decays linearly in the log of the variable (power law) or quadratically (Log Normal) or something else. If that's all that matters, then you should really consider doing some kind of non-parametric density estimation (e.g. Markovitch and Krieger's (preprint)). Sometimes, though, you do care. Maybe you want to make a claim which depends heavily on just how common hugely large observations are. Or maybe you have a particular model in mind for the data-generating process, and that model predicts some particular distribution for the tail. Then knowing whether it really is a power law, or closer to a power law than (say) a stretched exponential, actually matters to you. In that case, you owe it to yourself to do the data analysis right. You also owe it to yourself to think carefully about whether there are other ways of checking your model. If the only testable prediction it makes is about the shape of the tail, it doesn't sound like a very good model, and it will be intrinsically hard to check it.

  • update based on Robin Hanson critique...I bring up the OMG DOOM because some people, Hanson very much included, like to extrapolate from supposed power laws for various Bad Things to scenarios where THE BIG ONE kills off most of humanity. But, at least with the data we found, the magnitudes of forest fires, solar flares, earthquakes and wars were all better fit by log-normals, by stretched exponentials and by cut-off power laws than by power laws.

Edited:    |       |    Search Twitter for discussion