Inference for unreliable grading: The case of recommendation letters

In this work, a well defined procedure to assign a probability distribution to a score is presented. By considering a score 0 ≤ t ≤ 1 and using Bayesian inference together with Jaynes' Maximum Entropy Principle, we are able to assign an estimation <t> to the score based on the available information. In order to correctly define a score t, we assume a resolution Δt that enables us to assign a a score t∗ so that t∗ -Δt/2 ≤ t ≤ t∗ + Δt/2 with a confidence p, and infer the parameters of the maximum entropy distribution as a function of p and t∗. This framework may provide insights on how to state problems with uncertain evaluation of performance in learning in several contexts.