Supported predictions

This page explains three supported types of predictions.

Probabilities

If the predictions are provided as a vector of probabilities, they correspond to a set of Bernoulli distributions with these parameters. Accordingly, the synthetic targets will be boolean values (true if the predicted outcome occurs, false if not).

We define a vector of 100 randomly sampled probabilities.

using ConsistencyResampling

probabilities = rand(100)

Consistency resampling yields a tuple consisting of a randomly sampled probability and a sample from the corresponding Bernoulli distribution.

probability, target = rand(Consistent(probabilities))

(0.869142175000603, true)

probability in probabilities

true

Multiple samples are returned as a vector of tuples.

rand(Consistent(probabilities), 10)

10-element Vector{Tuple{Float64, Bool}}:
 (0.8290760489325268, 1)
 (0.19573696453541478, 0)
 (0.1809976599934553, 0)
 (0.895855015412153, 1)
 (0.33385905430793017, 0)
 (0.8113243037087731, 1)
 (0.4213882946411107, 0)
 (0.6524645128338589, 0)
 (0.38136299883493363, 0)
 (0.9464532262313834, 1)

Vectors of probabilities

If the predictions are vectors of real-valued vectors, they correspond to the class probabilities of categorical distributions. In this case, the targets will be 1,…,m where the number of classes m is given by the length of the real-valued vectors.

We generate 100 randomly sampled vectors of probabilities for m = 5 classes.

using ConsistencyResampling

predictions = map(1:100) do _
    x = rand(5)
    x ./= sum(x)
    return x
end

Consistency resampling returns a tuple of a randomly sampled prediction and a sample from the corresponding categorical distribution.

prediction, target = rand(Consistent(predictions))

([0.31169343646775927, 0.22387144211532295, 0.011051607488009393, 0.40619240567555503, 0.047191108253353396], 5)

prediction in predictions

true

Vectors of distributions

Additionally, predictions can be provided as a vector of distributions. These distributions have to support the Random API. More precisely,

Random.Sampler(::Type{<:Random.AbstractRNG}, ::MyDistributionType, ::Random.Repetition)

must be defined and it must be possible to sample from the resulting samplers with rand. More information about the interface can be found in the documentation of Random.

We define vector of 100 normal distributions with randomly sampled mean and standard deviation.

using ConsistencyResampling
using Distributions

predictions = map(Normal, randn(100), rand(100))

Consistency resampling yields a randomly sampled prediction and a sample from it.

prediction, target = rand(Consistent(predictions))

(Distributions.Normal{Float64}(μ=-1.252895809245693, σ=0.07251510685257079), -1.366933438972655)

prediction in predictions

true

using Distributions
using Random

Random.Sampler(::Type{<:AbstractRNG}, s::Sampleable, ::Val{1}) = s
Random.Sampler(::Type{<:AbstractRNG}, s::Sampleable, ::Val{Inf}) = sampler(s)