MOI.modify version for multiple changes at once

[guilhermebodin]

Many solvers support changing many coefficients at once.

Some examples are functions in Gurobi (https://www.gurobi.com/documentation/9.5/refman/c_xchgcoeffs.html) and Xpress (https://www.fico.com/fico-xpress-optimization/docs/latest/solver/optimizer/HTML/XPRSchgmcoef.html).

This is a draft about making an MOI version of such methods and a proper fallback. Honestly, I don't know what would be the best approach but I am happy to make a PR with the correct implementation.

[odow]

Can we make a benchmark that demonstrates the need for this before adding? Is there a material difference, or is this just a nice to have? What would be the use-case?

[guilhermebodin]

I can try to do some benchmarks in Xpress.

For now, I have the benchmarks of ParametricOptInterface jump-dev/ParametricOptInterface.jl#71 The last ones that allocate more than a few Gb are trying to make 5000 modifications 10 times. The main bottleneck there is the MOI.modify function.

[odow]

Is the benchmark just in MOI? No Xpress? Why is it allocating? If so, we should try and fix that in MOI first.

[guilhermebodin]

This benchmark is only in MOI. Here there is a MWE, it is spending quite a lot time in copy in the _modifycoefficient function.

using MathOptInterface
using ParametricOptInterface
const MOI = MathOptInterface
const POI = ParametricOptInterface
using Profile
using PProf

N = 10_000
second(x) = x[2]
model = POI.Optimizer(MOI.Utilities.Model{Float64}())
x = MOI.add_variables(model, N/2)
ycy = MOI.add_constrained_variable.(model, POI.Parameter.(ones(Int(N/2))))
y = first.(ycy)
cy = map(second, ycy)
MOI.add_constraint(
        model,
        MOI.ScalarQuadraticFunction(
            MOI.ScalarQuadraticTerm.(1.0, y, x),
            MOI.ScalarAffineTerm{Float64}[],
            0.0,
        ),
        MOI.GreaterThan(1.0),
    )
MOI.set.(model, MOI.ConstraintSet(), cy, POI.Parameter.(0.5))

@profile POI.update_parameter_in_quadratic_constraints_pv!(model)

pprof()

[blegat]

So MOI.Utilities.Model would have a specialized method for modified several constraints at once that would be more efficient ?
I would understand for the objective since it is several modifications for the same function but for constraints, your PR seems to suggest it's one modification per constraint

[guilhermebodin]

Hi @blegat and @odow,

To be the clearest possible, my intentions with this draft were simply to add a fallback in the interface that allows solvers to implement their version of MOI.modify that does multiple modifications at once if this is available in the low-level interface (written in C in most cases).

I am going to illustrate the reason with a use case of ParametricOptInterface:

I have a model with 5000 variables and 5000 parameters in a certain constraint. I first solve the model with the constraint function being

p1 * v1 + p2 * v2 + ... + p5000 * v5000

After that, I would like to update the values of the parameters and solve the problem with the updated parameters. The new constraint function is now something like

p1' * v1 + p2' * v2 + ... + p5000' * v5000

To perform this update I have to use the MOI.modify function 5000 times to make some 5000 ScalarCoefficientChange . I would like to have an interface that allowed me to call MOI.modify only once and pass 5000 ConstraintIndexes and 5000 ScalarCoefficientChange as an argument and use the low level interface of the solvers to do this operation.

To be more concrete, in Xpress we have two functions, one is called XPRSchgcoef that allows users to make a single modification to matrix coefficients and another function called XPRSchgmcoef that allows users to make multiple modifications to matrix coefficients at once. Currently MOI does not have a fallback method to have a version of MOI.modify that would call XPRSchgmcoef in the MOI_wrapper.jl

I have implemented a dummy version of this function in Xpress MOI_wrapper.jl

function MOI.modify(
    model::Optimizer,
    cis::Vector{MOI.ConstraintIndex{MOI.ScalarAffineFunction{Float64}, MOI.GreaterThan{Float64}}},
    chgs::Vector{MOI.ScalarCoefficientChange{Float64}}
)
    nels = length(cis)
    @assert nels == length(chgs)

    rows = Vector{Int}(undef, nels)
    cols = Vector{Int}(undef, nels)
    coefs = Vector{Float64}(undef, nels)

    for i in 1:nels
        rows[i] = _info(model, cis[i]).row
        cols[i] = _info(model, chgs[i].variable).column
        coefs[i] = chgs[i].new_coefficient
    end

    Xpress.chgmcoef(
        model.inner,
        nels,
        rows,
        cols,
        coefs
    )
    return
end

This implementation makes a reasonable difference in the performance of updates. In the first function, I call the current version of MOI.modify that I call 5000 times and in the second function I call my version of MOI.modify that I call only once with 5000 modifications

julia> @btime update_parameters_multiple_times_with_single_change(num_variables, num_solves)
  27.503 ms (241327 allocations: 15.01 MiB)

julia> @btime update_parameters_multiple_times_with_multiple_changes(num_variables, num_solves)
  14.975 ms (191487 allocations: 16.54 MiB)

[blegat]

Thanks for the detailed explanation. One thing to consider is whether we want the user to provide a list of constraint indices or a single constraint. For instance, for MOI.Utilities.VectorOfConstraints, you can only gain something if you modify several times the same constraints. So if all constraint indices in the vector of constraints are unique, you won't gain anything. I guess it's not the same for CPLEX as it stores all constraints in the same matrix and that would also apply for MOI.Utilities.MatrixOfConstraints.

[joaquimg]

I think we could have both:

function modify(
    model::ModelLike,
    cis::AbstractVector{ConstraintIndex},
    changes::AbstractVector{<:AbstractFunctionModification},
)

and

function modify(
    model::ModelLike,
    ci::ConstraintIndex,
    changes::AbstractVector{<:AbstractFunctionModification},
)

The first one that @guilhermebodin proposed is key to having fast in-place updates that we are currently working on ParametricOptInterface. This will also be important for DiffOpt which will require in-place updating to be efficient. The latter will require follow-up work on MOI.Utilities.MatrixOfConstraints indeed.

Therefore I think the multi-constraint version is especially important for high-performance applications. CachingOptimizer is not super important for such applications that usually end up relying on direct_mode.

Still, if we want the single-constraint version, it is possible to add a default fallback that orders things properly and adds batch by batch.

[blegat]

Ok, I see, then it's best to start with the version of the paper and add

function modify(
    model::ModelLike,
    ci::ConstraintIndex,
    changes::AbstractVector{<:AbstractFunctionModification},
)

later if there is a use case for it.

[guilhermebodin]

@blegat I also added docs and tests

[odow]

Looks good, modulo the doc fixes.