Accelerating the pace of engineering and science

# Global Optimization Toolbox

## Simulated Annealing Options

This example shows how to create and manage options for the simulated annealing function SIMULANNEALBND using SAOPTIMSET in the Global Optimization Toolbox.

Optimization Problem Setup

SIMULANNEALBND searches for a minimum of a function using simulated annealing. For this example we use SIMULANNEALBND to minimize the objective function DEJONG5FCN. This function is a real valued function of two variables and has many local minima making it difficult to optimize. There is only one global minimum at x=(-32,-32), where f(x) = 0.998. To define our problem, we must define the objective function, start point, and bounds specified by the range -64 <= x(i) <= 64 for each x(i).

```ObjectiveFunction = @dejong5fcn;
startingPoint = [-30 0];
lb = [-64 -64];
ub = [64 64];
```

The function PLOTOBJECTIVE in the toolbox plots the objective function over the range -64 <= x1 <= 64, -64 <= x2 <= 64.

```plotobjective(ObjectiveFunction,[-64 64; -64 64]);
view(-15,150);
```

Now, we can run the SIMULANNEALBND solver to minimize our objective function.

```[x,fval,exitFlag,output] = simulannealbnd(ObjectiveFunction,startingPoint,lb,ub);

fprintf('The number of iterations was : %d\n', output.iterations);
fprintf('The number of function evaluations was : %d\n', output.funccount);
fprintf('The best function value found was : %g\n', fval);
```
```Optimization terminated: change in best function value less than options.TolFun.
The number of iterations was : 1095
The number of function evaluations was : 1104
The best function value found was : 2.98211
```

Note that when you run this example, your results may be different from the results shown above because simulated annealing algorithm uses random numbers to generate points.

SIMULANNEALBND can accept one or more plot functions through an 'options' argument. This feature is useful for visualizing the performance of the solver at run time. Plot functions are selected using SAOPTIMSET. The help for SAOPTIMSET contains a list of plot functions to choose from, or you can provide your own custom plot functions.

To select multiple plot functions, SAOPTIMSET is used to create an options structure. For this example, we select SAPLOTBESTF, which plots the best function value every iteration, SAPLOTTEMPERATURE, which shows the current temperature in each dimension at every iteration, SAPLOTF, which shows the current function value (remember that the current value is not necessarily the best one), and SAPLOTSTOPPING, which plots the percentage of stopping criteria satisfied every iteration.

```options = saoptimset('PlotFcns',{@saplotbestf,@saplottemperature,@saplotf,@saplotstopping});
```

Run the solver.

```simulannealbnd(ObjectiveFunction,startingPoint,lb,ub,options);
```
```Optimization terminated: change in best function value less than options.TolFun.
```

Specifying Temperature Options

The temperature parameter used in simulated annealing controls the overall search results. The temperature for each dimension is used to limit the extent of search in that dimension. The toolbox lets you specify initial temperature as well as ways to update temperature during the solution process. The two temperature-related options are the InitialTemperature and the TemperatureFcn.

Specifying initial temperature

The default initial temperature is set to 100 for each dimension. If you want the initial temperature to be different in different dimensions then you must specify a vector of temperatures. This may be necessary in cases when problem is scaled differently in each dimensions. For example,

```options = saoptimset('InitialTemperature',[300 50]);
```

InitialTemperature can be set to a vector of length less than the number of variables (dimension); the solver expands the vector to the remaining dimensions by taking the last element of the initial temperature vector. Here we want the initial temperature to be the same in all dimensions so we need only specify the single temperature.

```options = saoptimset('InitialTemperature',100);
```

Specifying a temperature function

The default temperature function used by SIMULANNEALBND is called TEMPERATUREEXP. In the temperatureexp schedule, the temperature at any given step is .95 times the temperature at the previous step. This causes the temperature to go down slowly at first but ultimately get cooler faster than other schemes. If another scheme is desired, e.g. Boltzmann schedule or "Fast" schedule annealing, then TEMPERATUREBOLTZ or TEMPERATUREFAST can be used respectively. To select the fast temperature schedule, we can update our previously created options structure 'options' by passing 'options' to SAOPTIMSET.

```options = saoptimset(options,'TemperatureFcn',@temperaturefast);
```

Specifying reannealing

Reannaling is a part of annealing process. After a certain number of new points are accepted, the temperature is raised to a higher value in hope to restart the search and move out of a local minima. Performing reannealing too soon may not help the solver identify a minimum, so a relatively high interval is a good choice. The interval at which reannealing happens can be set using the ReannealInterval option. Here, we reduce the default reannealing interval to 50 because the function seems to be flat in many regions and solver might get stuck rapidly.

```options = saoptimset(options,'ReannealInterval',50);
```

We can also set the Display option to control the level of display on the command prompt. Here we set the display option to 'iter' and change the display interval to 400.

```options = saoptimset(options,'Display','iter','DisplayInterval',400);
```

Now that we have setup the new temperature options we run the solver again

```[x,fval,exitFlag,output] = simulannealbnd(ObjectiveFunction,startingPoint,lb,ub,options);
```
```                           Best        Current           Mean
Iteration   f-count         f(x)         f(x)         temperature
0          1        65.2159        65.2159            100
*  166        169        21.0727        21.0729        17.7907
*  358        363        16.4409        16.7827        12.2086
400        405        16.4409        20.3586        1.88714
*  545        552        16.4409        16.4417        14.1675
*  718        727        16.4409        16.6691        12.3336
800        809        16.4409        17.2669        1.07018
*  920        931        16.4409        16.4409        15.8571
* 1116       1129        16.4409        16.4474        15.5745
1200       1213        16.4409        16.4417        1.10255
* 1291       1306        16.4409        16.4452        16.1383
Optimization terminated: change in best function value less than options.TolFun.
```

Notice that every iteration at which reannealing happens is denoted by a '*' and is always displayed.

```fprintf('The number of iterations was : %d\n', output.iterations);
fprintf('The number of function evaluations was : %d\n', output.funccount);
fprintf('The best function value found was : %g\n', fval);
```
```The number of iterations was : 1306
The number of function evaluations was : 1321
The best function value found was : 16.4409
```

Reproducing Results

SIMULANNEALBND is a nondeterministic algorithm. This means that running the solver more than once without changing any settings may give different results. This is because SIMULANNEALBND utilizes MATLAB random number generators when it generates subsequent points and also when it determines whether or not to accept new points. Every time a random number is generated the state of the random number generators change.

To see this, two runs of SIMULANNEALBND solver yields:

```[x,fval] = simulannealbnd(ObjectiveFunction,startingPoint,lb,ub,options);
fprintf('The best function value found was : %g\n', fval);
```
```                           Best        Current           Mean
Iteration   f-count         f(x)         f(x)         temperature
0          1        65.2159        65.2159            100
*  178        181        10.7632        10.8556         306.84
*  337        342        1.99203        2.23278        43.5661
400        405        1.99203        3.80458        1.50999
*  529        536        1.99203          1.993        120.264
*  711        720        1.99203        1.99252        19.8934
800        809        1.99203        1.99279        1.06345
*  905        916        1.99203        1.99203         5032.9
* 1086       1099        1.99203        1.99221        12.0733
1200       1213        1.99203        2.17076       0.792482
* 1268       1283        1.99203        1.99208        13.7625
Optimization terminated: change in best function value less than options.TolFun.
The best function value found was : 1.99203
```

And,

```[x,fval] = simulannealbnd(ObjectiveFunction,startingPoint,lb,ub,options);
fprintf('The best function value found was : %g\n', fval);
```
```                           Best        Current           Mean
Iteration   f-count         f(x)         f(x)         temperature
0          1        65.2159        65.2159            100
*  194        197        10.7632         10.772        32.3858
*  395        400        10.7632        11.2931        12.9278
400        405        10.7632        11.2931        7.44822
*  578        585        10.7632        10.7644        16.0489
*  768        777        10.7632        12.0341        13.0872
800        809        10.7632        12.0341         2.4326
*  931        942        10.7632        10.7634        16.1797
* 1105       1118        10.7632        11.7832        116.913
Optimization terminated: change in best function value less than options.TolFun.
The best function value found was : 10.7632
```

In the previous two runs SIMULANNEALBND gives different results.

We can reproduce our results if we reset the states of the random number generators between runs of the solver by using information returned by SIMULANNEALBND. SIMULANNEALBND returns the states of the random number generators at the time SIMULANNEALBND is called in the output argument. This information can be used to reset the states. Here we reset the states between runs using this output information so the results of the next two runs are the same.

```[x,fval,exitFlag,output] = simulannealbnd(ObjectiveFunction,startingPoint,lb,ub,options);
fprintf('The best function value found was : %g\n', fval);
```
```                           Best        Current           Mean
Iteration   f-count         f(x)         f(x)         temperature
0          1        65.2159        65.2159            100
*  193        196        20.1535        20.1537        17.8501
*  381        386        20.1535        20.5812        20.9127
400        405        20.1535        20.5812        4.19923
*  574        581        20.1535        20.1543        16.6134
*  751        760        20.1535         20.304        13.2824
800        809        20.1535         20.304        1.72145
*  939        950        20.1535        20.1589        16.3603
* 1117       1130        20.1535         20.155        15.8317
1200       1213        20.1535        20.3304        1.11621
* 1299       1314        20.1535        20.1686        12.9936
* 1487       1504        20.1535        20.1535        14.4676
Optimization terminated: change in best function value less than options.TolFun.
The best function value found was : 20.1535
```

We reset the state of the random number generator.

```set(RandStream.getGlobalStream,'State',output.rngstate.state);
```

Now, let's run SIMULANNEALBND again.

```[x,fval] = simulannealbnd(ObjectiveFunction,startingPoint,lb,ub,options);
fprintf('The best function value found was : %g\n', fval);
```
```                           Best        Current           Mean
Iteration   f-count         f(x)         f(x)         temperature
0          1        65.2159        65.2159            100
*  193        196        20.1535        20.1537        17.8501
*  381        386        20.1535        20.5812        20.9127
400        405        20.1535        20.5812        4.19923
*  574        581        20.1535        20.1543        16.6134
*  751        760        20.1535         20.304        13.2824
800        809        20.1535         20.304        1.72145
*  939        950        20.1535        20.1589        16.3603
* 1117       1130        20.1535         20.155        15.8317
1200       1213        20.1535        20.3304        1.11621
* 1299       1314        20.1535        20.1686        12.9936
* 1487       1504        20.1535        20.1535        14.4676
Optimization terminated: change in best function value less than options.TolFun.
The best function value found was : 20.1535
```

Modifying the Stopping Criteria

SIMULANNEALBND uses six different criteria to determine when to stop the solver. SIMULANNEALBND stops when the maximum number of iterations or function evaluation is exceeded; by default the maximum number of iterations is set to Inf and the maximum number of function evaluations is 3000*numberOfVariables. SIMULANNEALBND keeps track of the average change in the function value for StallIterLimit iterations. If the average change is smaller than the function tolerance, TolFun, then the algorithm will stop. The solver will also stop when the objective function value reaches ObjectiveLimit. Finally the solver will stop after running for TimeLimit seconds. Here we set the TolFun to 1e-5.

```options = saoptimset(options,'TolFun',1e-5);
```

Run the SIMULANNEALBND solver.

```[x,fval,exitFlag,output] = simulannealbnd(ObjectiveFunction,startingPoint,lb,ub,options);
fprintf('The number of iterations was : %d\n', output.iterations);
fprintf('The number of function evaluations was : %d\n', output.funccount);
fprintf('The best function value found was : %g\n', fval);
```
```                           Best        Current           Mean
Iteration   f-count         f(x)         f(x)         temperature
0          1        65.2159        65.2159            100
*  179        182        11.7187        11.7211        33.0831
*  369        374        11.7187        11.7596        180.265
400        405        11.7187        11.7596        2.97641
*  551        558        11.7187        12.5042        21.3737
*  729        738        11.7187        12.0319         53.538
800        809        6.91013        8.26212        1.34928
*  897        908        6.90334        6.91869        55.3227
* 1099       1112        6.90334        6.90842        13.1488
1200       1213        6.90334        6.92373       0.907765
* 1266       1281        6.90334        6.90334        31.4322
* 1462       1479        6.90334        7.11941        13.1345
1600       1617        6.90334        7.00217       0.678746
* 1636       1655        6.90334        6.90523        21.6782
* 1807       1828        6.90334        7.86653        17.3648
Optimization terminated: change in best function value less than options.TolFun.
The number of iterations was : 1843
The number of function evaluations was : 1864
The best function value found was : 6.90334
```