% Section 8.6.2, Boyd & Vandenberghe "Convex Optimization"
% Original by Lieven Vandenberghe
% Adapted for CVX by Joelle Skaf - 10/23/05
% (a figure is generated)
%
% The goal is to find the polynomial of degree 4 on R^n that separates
% two sets of points {x_1,...,x_N} and {y_1,...,y_N}. We are trying to find
% the coefficients of an order-4-polynomial P(x) that would satisfy:
%           minimize    t
%               s.t.    P(x_i) <= t  for i = 1,...,N
%                       P(y_i) >= t   for i = 1,...,M

% Data generation
rand('state',0);
N = 100;
M = 120;

% The points X lie within a circle of radius 0.9, with a wedge of points
% near [1.1,0] removed. The points Y lie outside a circle of radius 1.1,
% with a wedge of points near [1.1,0] added. The wedges are precisely what
% makes the separation difficult and interesting.
X = 2 * rand(2,N) - 1;
X = X * diag(0.9*rand(1,N)./sqrt(sum(X.^2)));
Y = 2 * rand(2,M) - 1;
Y = Y * diag((1.1+rand(1,M))./sqrt(sum(Y.^2)));
d = sqrt(sum((X-[1.1;0]*ones(1,N)).^2));
Y = [ Y, X(:,d<0.9) ];
X = X(:,d>1);
N = size(X,2);
M = size(Y,2);

% Construct Vandermonde-style monomial matrices
p1   = [0,0,1,0,1,2,0,1,2,3,0,1,2,3,4]';
p2   = [0,1,1,2,2,2,3,3,3,3,4,4,4,4,4]'-p1;
np   = length(p1);
op   = ones(np,1);
monX = X(op,:) .^ p1(:,ones(1,N)) .* X(2*op,:) .^ p2(:,ones(1,N));
monY = Y(op,:) .^ p1(:,ones(1,M)) .* Y(2*op,:) .^ p2(:,ones(1,M));

% Solution via CVX
fprintf(1,'Finding the optimal polynomial of order 4 that separates the 2 classes...');

cvx_begin
    variables a(np) t(1)
    minimize ( t )
    a'*monX <= t;
    a'*monY >= -t;
    % For normalization purposes only
    norm(a) <= 1;
cvx_end

fprintf(1,'Done! \n');

% Displaying results
nopts = 2000;
angles = linspace(0,2*pi,nopts);
cont = zeros(2,nopts);
for i=1:nopts
   v = [cos(angles(i)); sin(angles(i))];
   l = 0;  u = 1;
   while ( u - l > 1e-3 )
      s = (u+l)/2;
      x = s * v;
      if a' * ( x(op,:) .^ p1 .* x(2*op) .^ p2 ) > 0,
          u = s;
      else
          l = s;
      end
   end;
   s = (u+l)/2;
   cont(:,i) = s*v;
end;

graph = plot(X(1,:),X(2,:),'o', Y(1,:), Y(2,:),'o', cont(1,:), cont(2,:), '-');
set(graph(2),'MarkerFaceColor',[0 0.5 0]);
title('Optimal order-4 polynomial that separates the 2 classes')
% print -deps min-deg-discr.eps

%%%% Dual infeasible ?????

Finding the optimal polynomial of order 4 that separates the 2 classes... 
Calling Mosek 9.1.9: 228 variables, 17 equality constraints
   For improved efficiency, Mosek is solving the dual problem.
------------------------------------------------------------

MOSEK Version 9.1.9 (Build date: 2019-11-21 11:32:15)
Copyright (c) MOSEK ApS, Denmark. WWW: mosek.com
Platform: MACOSX/64-X86

MOSEK warning 710: #1 (nearly) zero elements are specified in sparse col '' (31) of matrix 'A'.
Problem
  Name                   :                 
  Objective sense        : min             
  Type                   : CONIC (conic optimization problem)
  Constraints            : 17              
  Cones                  : 1               
  Scalar variables       : 228             
  Matrix variables       : 0               
  Integer variables      : 0               

Optimizer started.
Presolve started.
Linear dependency checker started.
Linear dependency checker terminated.
Eliminator started.
Freed constraints in eliminator : 0
Eliminator terminated.
Eliminator - tries                  : 1                 time                   : 0.00            
Lin. dep.  - tries                  : 1                 time                   : 0.00            
Lin. dep.  - number                 : 0               
Presolve terminated. Time: 0.00    
Problem
  Name                   :                 
  Objective sense        : min             
  Type                   : CONIC (conic optimization problem)
  Constraints            : 17              
  Cones                  : 1               
  Scalar variables       : 228             
  Matrix variables       : 0               
  Integer variables      : 0               

Optimizer  - threads                : 8               
Optimizer  - solved problem         : the primal      
Optimizer  - Constraints            : 16
Optimizer  - Cones                  : 1
Optimizer  - Scalar variables       : 227               conic                  : 16              
Optimizer  - Semi-definite variables: 0                 scalarized             : 0               
Factor     - setup time             : 0.00              dense det. time        : 0.00            
Factor     - ML order time          : 0.00              GP order time          : 0.00            
Factor     - nonzeros before factor : 136               after factor           : 136             
Factor     - dense dim.             : 0                 flops                  : 5.91e+04        
ITE PFEAS    DFEAS    GFEAS    PRSTATUS   POBJ              DOBJ              MU       TIME  
0   1.0e+00  0.0e+00  2.0e+00  0.00e+00   1.000000000e+00   0.000000000e+00   1.0e+00  0.00  
1   3.6e-01  8.9e-16  1.2e-01  1.98e+00   2.844057249e-02   -2.692825782e-01  3.6e-01  0.01  
2   2.5e-01  1.8e-15  6.2e-02  8.98e+00   1.579292632e-02   -6.770937750e-03  2.5e-01  0.01  
3   1.1e-01  3.6e-15  1.4e-02  1.49e+00   4.177907458e-03   -5.443218172e-03  1.1e-01  0.01  
4   7.2e-02  8.9e-15  7.6e-03  1.10e+00   2.922507480e-03   -3.777454964e-03  7.2e-02  0.01  
5   1.7e-02  2.7e-15  1.5e-03  7.15e-01   2.243340673e-03   5.706381852e-04   1.7e-02  0.01  
6   4.9e-03  6.2e-15  7.4e-04  -5.81e-01  7.210723047e-03   9.231164533e-03   4.9e-03  0.01  
7   3.4e-03  3.6e-15  3.2e-04  -9.33e-02  2.492456091e-02   2.493770965e-02   3.4e-03  0.01  
8   2.0e-03  5.3e-15  1.4e-04  3.33e-01   2.508769607e-02   2.483220243e-02   2.0e-03  0.01  
9   4.2e-04  3.6e-15  1.7e-05  4.40e-01   3.627400999e-02   3.627382372e-02   4.2e-04  0.01  
10  2.0e-04  2.2e-15  6.2e-06  5.30e-01   3.853167902e-02   3.854114677e-02   2.0e-04  0.01  
11  6.1e-05  1.1e-14  1.2e-06  6.12e-01   4.028611633e-02   4.028286811e-02   6.1e-05  0.01  
12  8.3e-06  6.2e-15  5.9e-08  7.87e-01   4.088928864e-02   4.088780570e-02   8.3e-06  0.01  
13  6.1e-07  4.4e-15  1.1e-09  9.12e-01   4.098714045e-02   4.098686333e-02   6.1e-07  0.02  
14  2.3e-09  6.3e-13  1.8e-13  9.95e-01   4.099320912e-02   4.099320672e-02   2.3e-09  0.02  
Optimizer terminated. Time: 0.02    


Interior-point solution summary
  Problem status  : PRIMAL_AND_DUAL_FEASIBLE
  Solution status : OPTIMAL
  Primal.  obj: 4.0993209116e-02    nrm: 1e+00    Viol.  con: 2e-09    var: 2e-09    cones: 0e+00  
  Dual.    obj: 4.0993206718e-02    nrm: 5e+00    Viol.  con: 0e+00    var: 2e-13    cones: 0e+00  
Optimizer summary
  Optimizer                 -                        time: 0.02    
    Interior-point          - iterations : 14        time: 0.02    
      Basis identification  -                        time: 0.00    
        Primal              - iterations : 0         time: 0.00    
        Dual                - iterations : 0         time: 0.00    
        Clean primal        - iterations : 0         time: 0.00    
        Clean dual          - iterations : 0         time: 0.00    
    Simplex                 -                        time: 0.00    
      Primal simplex        - iterations : 0         time: 0.00    
      Dual simplex          - iterations : 0         time: 0.00    
    Mixed integer           - relaxations: 0         time: 0.00    

------------------------------------------------------------
Status: Solved
Optimal value (cvx_optval): -0.0409932
 
Done!