Rudrajit Das

Rudrajit Das

Just another guy working towards a PhD in Machine Learning

Extreme Value Theory (EVT) for Limiting Distributions of Extreme Events

2 minute read

Published:

This short post contains a few references for the relatively unknown (in my opinion) EVT among people who aren’t statisticians but use it heavily (such as in machine learning).

The EVT basically states that the limiting distribution of the maximum or minimum of i.i.d random variables can be modelled by just 3 types of distributions, which are the Gumbel, Frechet and Weibull types, as long as their individual distributions obey certain conditions (which are generally obeyed for standard distributions that we usually deal with). It is in some ways the analogue of the central limit theorem (which is regarding the distribution of the sums of i.i.d random variables).

I’m writing about EVT here because until about a month ago, I had never heard about it but while working on my previous paper, I had to do some analysis on the maximum of a significantly large number of i.i.d Chi-squared random variables and the EVT result came in really handy here. And this was after wasting a lot of time trying with the usual CDF of the max. of $n$ random variables, say each with CDF $F(.)$, which would be simply $F(.)^{n}$. The power of the EVT really captivated me and I thought it’s worthy of being mentioned.

Coming to the references, the book “Modelling Extremal Events for Insurance and Finance” (yes finance!) by Paul Embrechts, Claudia Klüppelberg, Thomas Mikosch contains several useful results related to EVT for several classes of distributions. Unfortunately, obtaining a free and complete online version of this book might be difficult - it’s available as Google Books but with several pages missing! There are also quite a few useful stack exchange discussions which one might find on the EVT such as this one here. Also if one is interested in a more detailed discussion on EVT, there are some online pdfs available, one of which you can find here.

Please note that the EVT discussed here works only for i.i.d random variables, so if you are dealing with non i.i.d random variables (as is often the case, for example with non-stationary stochastic processes), you have to suitably modify (which is non-trivial) the EVT. The discussion of that is beyond the scope of this post.

Hope this post somewhat enlightens you about the existence of the EVT which can be very useful in certain situations.

You May Also Enjoy

Good Initialization for Alternating Minimization

2 minute read

Published:

This short post is on the importance of having proper initialization while using alternating minimization for two variables, such that the objective function under consideration is convex in each variable individually but not jointly convex with respect to both the variables.

Recent Advances in Non-Convex Optimization for Deep Learning

8 minute read

Published:

This post contains a summary of recent advances in non-convex optimization in deep learning, discussing about the optimality of local minima for several models, the issue of saddle points and modifications to stochastic gradient descent which are robust to saddle points.

Theoretical Research in Deep Learning

6 minute read

Published:

This post contains some guidelines (gathered from self experience and also from some highly experienced people) for doing theoretical research in deep learning (and machine learning in general), strictly for newbies!