Model Transformations

Occasionally, one might wish to perform coordinate transformations on the parameter space of a model without having to redefine the entire model as this can be a cumbersome process for complex models. For example, this might be useful in the fitting process when the allowable parameter range spans several orders of magnitude or when trying to enforce the positivity of parameters.

A few methods are provided to make this process more convenient. These include: LogTransform, ExpTransform, Log10Transform, Exp10Transform, ReflectionTransform and ScaleTransform. These methods accept a vector of booleans as an optional second argument to restrict the application of the transformation to specific parameter components if desired. For first argument, one can either provide just a model function to obtain its transformed counterpart or alternatively supply an entire DataModel to be transformed.

DM = DataModel(DataSet([1,2,3,4], [4,5,6.5,9], [0.5,0.45,0.6,1]), LinearModel)
logDM = LogTransform(DM)
ExpDM = ExpTransform(Exp10Transform(DM, [false, true]), [true, false])
SymbolicModel(logDM), SymbolicModel(ExpDM)

("y(x,θ) = log(θ[2]) + log(θ[1])*x", "y(x,θ) = exp10(θ[2]) + exp(θ[1])*x")

It is also possible to provide other differentiable functions for parameter transformations by hand using the following method:

InformationGeometry.ComponentwiseModelTransform — Function

ComponentwiseModelTransform(DM::AbstractDataModel, F::Function, idxs=trues(pdim(DM))) -> DataModel
ComponentwiseModelTransform(model::Function, idxs, F::Function) -> Function

Transforms the parameters of the model by the given scalar function F such that newmodel(x, θ) = oldmodel(x, F.(θ)). By providing idxs, one may restrict the application of the function F to broadcast only to specific parameter components.

For vector-valued transformations, see ModelEmbedding.

source

The provided scalar function F should be strictly monotonic to avoid problems when differentiating the model.

In addition to componentwise application of scalar functions to the parameters, there are also higher-dimensional transformations such as TranslationTransform, LinearTransform and their combination AffineTransform which allow for mixing between the components.

Lastly, the method LinearDecorrelation is a special case of AffineTransform which subtracts the MLE from the parameters and applies the cholesky decomposition (i.e. "square root") of the inverse Fisher metric at the best fit. This centers the confidence regions on the origin and will result in confidence boundaries which constitute concentric circles / spheres for linearly parametrized models. For models which are non-linear with respect to their parameters, the confidence boundaries of the "linearly decorrelated" model showcase the deviations of the confidence boundaries of the original model from ellipsoidal shape, therefore nicely illustrating the magnitude of the coordinate distortion present on the parameter space.

For general (differentiable) multivariable transformations on the parameter space, one can use:

InformationGeometry.ModelEmbedding — Function

ModelEmbedding(DM::AbstractDataModel, Emb::Function, invEmb::Function; MLE::AbstractVector=MLE(DM), SkipOptim::Bool=true, SkipTests::Bool=true, 
            ADmode::Val=Val(:ForwardDiff), levels::Int=1, GenerateNewScore::Bool=true, ScoreFn::Union{Nothing,Function}=nothing, 
            GenerateNewFisher::Bool=false, FisherInfoFn::Union{Nothing,Function}=nothing, 
            GenerateNewCostHessian::Bool=false, CostHessianFn::Union{Nothing,Function}=nothing, kwargs...)
ModelEmbedding(DM::AbstractDataModel, Emb::Function, start::AbstractVector; Domain::HyperCube=FullDomain(length(start),Inf), kwargs...)

Transforms a model function via newmodel(x, θ) = oldmodel(x, Emb(θ)) and returns the associated DataModel. Need to pass new MLE or starting configuration of correct length for Emb via MLE kwarg.

For GenerateNewScore=true, a new score function which includes the embedding is regenerated via ForwardDiff from the log-likelihood. Otherwise, the new score is implemented via explicit multiplication of the old score with the embedding Jacobian, which is typically slower. For the CostHessian, the added recompilation time when using ForwardDiff is often significant, wherefore neglecting second derivatives of the embedding and implementing the new CostHessian via multiplication with the embedding Jacobian is often preferable in practice, despite being approximate rather than exact.

For component-wise transformations see ComponentwiseModelTransform.

source