OSQP for Supportive Vector Regression

Hallo,

i have considered your OSQP solver to speed up my QP-work. I think the solution is very smart! I am working on supportive vector regressions in R and not sure how to transform the A matrix for QSQP accordingly.

Maybe you could please give me a hint how to solve this issue?

I have added an example with the ipop solver from kernlab, i want to get the same solutions from OSQP. Any help is highly appreciated!

Thank you & Kind regards

library(rosqp)
library(Matrix)
library(kernlab)
library(RMKL)

x=as.matrix(10runif(200),ncol=1)
y=x
sin(x)+rnorm(200)

K=kernels.gen(data=x, train.samples = 1:150, kernels = c(‘radial’,‘radial’),
sigma = c(0.25,4))

outcome=y[1:150]
epsilon=0.1
penalty=1e8
k=K$K.train

#solve with kernlabs ipop

n=length(outcome)
A=c(rep(1,n),rep(-1,n))
b=0
r=0
l=rep(0,2n)
u=rep(penalty,2
n)
c=c(epsilon-outcome,epsilon+outcome)
u=rep(penalty,2n)
iters=0
#delta=rep(1,length(k))
m=length(k)
gamma=rep(1/m,m)
#gamma_all=list()
iters=iters+1
#gamma_all[[iters]]=gamma
kk=Reduce(’+’,mapply("
", k, gamma,SIMPLIFY = FALSE))
H=as.matrix(cbind(rbind(kk,-kk),rbind(-kk,kk)))
#eps = 1e-11
model=kernlab::ipop(c,H+eps*diag(H),A,b,l,u,r, margin=1e-2)

#solve with rosqp

Dmat<- H
M <- Matrix(Dmat, sparse = TRUE)
Dmat<-M
dvec<- c
Amat<- Matrix(A, sparse = TRUE) #thinking this is incorrect and shoudl be somehow transformed
res = solve_osqp(Dmat, dvec,Amat) #osqp(P=NULL, q=NULL, A=NULL, l=NULL, u=NULL, pars=osqpSettings())

#Error in osqp(P, q, A, l, u, pars) : dim(A) == c(m, n) are not all TRUE

Maybe a simple example for SVR in R would be excellent to add, too.

Thanks!

You appear not to be providing any lower or upper bounds for your constraints. Don’t you want

res = solve_osqp(Dmat, dvec,Amat, l, u)

to get \ell \le Ax \le u there?

Thanks for the reply, yes it is true i forgot to add here l & u. So res = solve_osqp(Dmat, dvec,Amat,l=l,u=u). But the problem is the A matrix. Any help is appreciated! Thank you!

I can’t get your example to run (something is wrong with the mapply call maybe) so can’t advise directly. However, it seems clear that the problem is that the dimension of your A matrix is incompatible with the upper / lower bounds.

The row dimension of A should agree with the number of elements in both l and u. The column dimension of P should agree with both the row dimension of A and the number of elements in q. The matrix P should also be square.

Hallo pgoulart,

thanks for your answer. I have updated the example to make it independent of mapply.
Still the matrix A is incorrect of dimensions. Still wondering how to transform A to have the same output as in ipop.

Kind regards

library(rosqp)
library(Matrix)
library(kernlab)

x=as.matrix(10runif(2),ncol=1)
y=x
sin(x)+rnorm(2)

outcome=y[1:2]
epsilon=0.1
penalty=1e8
k=K$K.train

#solve with kernlabs ipop
n=length(outcome)
A=c(rep(1,n),rep(-1,n))
b=0
r=0
l=rep(0,2n)
u=rep(penalty,2
n)
c=c(epsilon-outcome,epsilon+outcome)
u=rep(penalty,2*n)
H=matrix(c(0.5, 0.283345, 0.5, 0.283345, 0.283345, 0.5, 0.283345, 0.5, 0.5, 0.283345, 0.5, 0.283345, 0.283345, 0.5, 0.283345, 0.5),4,4)
sol=kernlab::ipop(c,H,A,b,l,u,r, margin=1e-9)

#solve with rosqp
Dmat<- H
M <- Matrix(Dmat, sparse = TRUE)
Dmat<-M
dvec<- c
n=length(outcome)
Amat <- A
Amat<- Matrix(Amat, sparse = TRUE) #thinking this is incorrect and shoudl be somehow transformed
res = solve_osqp(Dmat, dvec,Amat,l=l,u=u) #with penalty osqp(P=NULL, q=NULL, A=NULL, l=NULL, u=NULL, pars=osqpSettings())

#compare results
res$x
primal(sol)

Hallo,

i have updated my problem:

eps=0.03
cost=45
f<-c(13.7, 30.9, 22.3, 21.1)
matrix(c(2.8, 4.3, 2.6, 1.3, 1.0, 4.5, 3.7, 3.3, 2.3,1.8,1.3,4.3,3.2,3.2,4.6),4,4)

fvec <- c(13.7, 30.9, 22.3, 21.1) # target function vector
amat <- matrix(c(2.8, 4.3, 2.6, 1.3, 1.0, 4.5, 3.7, 3.3, 2.3,1.8,1.3,4.3,3.2,3.2,4.6,4),4,4) # attribute value matrix
dpmat <- amat%*%t(amat) # dot product matrix

#solve with quadprog
Dmat=nearPD(rbind(cbind(dpmat, -dpmat), cbind(-dpmat, dpmat)))$mat
dvec=c(fvec-eps, -fvec-eps)
Amat=matrix(c(rep(1, nrow(data)), rep(-1, nrow(data)),diag(1, 2nrow(data)), diag(-1, 2nrow(data))),nrow=2nrow(data))
bvec=c(0, rep(0, 2
nrow(data)), rep(-cost, 2*nrow(data)))
meq=1
sol=solve.QP(Dmat, dvec, Amat, bvec, meq=1, factorized=FALSE) #fisrt colunm is a equal constraint meq=1

#solve with ipop
c=c(-fvec+eps, fvec+eps)
H=rbind(cbind(dpmat, -dpmat), cbind(-dpmat, dpmat))
A=c(rep(1, nrow(data)), rep(-1, nrow(data)))
b=0
l=rep(0, 2nrow(data))
u=rep(cost, 2
nrow(data))
r=0
sol2=ipop(c, H, A, b, l, u, r,margin = 0.000000000000001)

#solve with osqp
Dmat<-Matrix(Dmat, sparse=TRUE)
Amat<-Matrix(Amat, sparse=TRUE)
bvec<-Matrix(bvec, sparse=TRUE)
settings <- osqpSettings(verbose = FALSE, eps_abs=1e-8, eps_rel = 1e-8)
solve_osqp(Dmat, -dvec,t(Amat), bvec,pars=settings)

[1] -7.033377e-14 1.623610e+00 -1.210545e-12 3.751486e-01 1.106638e+00 6.712196e-19 8.921211e-01 -2.109370e-14 #quadprog
[1] 4.141427e-14 1.623611e+00 -2.879912e-18 3.751489e-01 1.106638e+00 1.050124e-15 8.921218e-01 1.050474e-15 #ipop
[1] -1.479521e-10 1.509620e+00 -1.573976e-10 2.914813e-01 7.152557e-01 1.523699e-10 8.418707e-01 1.499065e-10 #osqp

Is the OSQP problem stated accordingly? The solutions differ by the way. Why could this be?

Kind regards