One of the projects we are working on is predicting the future positions of\nagents based on past observations. As we all konw such prediction cannot be\naccurate when it goes far to the future. The question is then, how inaccurate?
\nTechniques such as VAE can be used to regress the target value as well as the\ndistribution, which gives you some idea about how uncertain the value is.\nHowever, VAEs are hard to train and therefore less practical.
\nOne of my colleagues introduces us the quantile regression as an alternative.\nQuantile regression does not directly estimate the distribution. Instead, it\nanswers questions such as:
\nSuch estimation is not as precise as the distribution since it only estimates\nsome statistics about the distribution. However, in practice it is usually good\nenough and even more useful to do so since that is how you are going to use the\ndistribution anyway.
\n\nThe key of quantile regression as I understand it is about a series of slightly\ntweaked loss functions. I found the inutition behind them very interesting, and\ndecieded to write about them to help others better understand them.
\nThe most widely used loss function used in supervised learning (e.g. the\nregression we are talking about) is mean square. Mean square loss can be written\nas the sum (actually mean, but the quantity is a constant anyway) of fi(y)=(y−yi)2.
\nLet’s first try to reason about the mean square loss, i.e. how does it drive the\noptimization (training). Imagine you choose some parametrized function to\nestimate y, for example a neural network, and use mean square loss as the last\nlayer. If we look at how the backward pass of the last layer alone guide the\noptimization, it is striaght forward to see that it is trying to get y to its\noptimal value, which is the mean of all the yi.
\nQuantile losses are not so different from the ordinary mean square loss. In\nfact, they are just replacing the fi(y) with L1 norm and its friends (in mean\nsquare loss, it is L2 norm).
\n