#------------------------------------------------------------------------------
# irace: An implementation in R of (Elitist) Iterated Racing
# Version: 3.2.2215
# Copyright (C) 2010-2019
# Manuel Lopez-Ibanez     <manuel.lopez-ibanez@manchester.ac.uk>
# Jeremie Dubois-Lacoste  
# Leslie Perez Caceres    <leslie.perez.caceres@ulb.ac.be>
#
# This is free software, and you are welcome to redistribute it under certain
# conditions.  See the GNU General Public License for details. There is NO
# WARRANTY; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#
# irace builds upon previous code from the race package:
#     race: Racing methods for the selection of the best
#     Copyright (C) 2003 Mauro Birattari
#------------------------------------------------------------------------------
# installed at: /home/manu/R/x86_64-pc-linux-gnu-library/3.4/irace
# called with: --parallel 2
Warning: A default scenario file './scenario.txt' has been found and will be read
# Adding 1 initial configuration(s) from file '/home/manu/work/irace/trunk/examples/vignette-example/default.txt' 
# 2019-04-04 11:52:51 BST: Initialization
# Elitist race
# Elitist new instances: 1
# Elitist limit: 2
# nbIterations: 5
# minNbSurvival: 5
# nbParameters: 11
# seed: 340101640
# confidence level: 0.95
# budget: 1000
# mu: 5
# deterministic: FALSE

# 2019-04-04 11:52:51 BST: Iteration 1 of 5
# experimentsUsedSoFar: 0
# remainingBudget: 1000
# currentBudget: 200
# nbConfigurations: 33
  Markers:
     x No test is performed.
     - The test is performed and some configurations are discarded.
     = The test is performed but no configuration is discarded.
     ! The test is performed and configurations could be discarded but elite configurations are preserved.
     . All alive configurations are elite and nothing is discarded

+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
| |   Instance|      Alive|       Best|      Mean best| Exp so far|  W time|  rho|KenW|  Qvar|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
|x|          1|         33|          6|    23254140.00|         33|00:02:53|   NA|  NA|    NA|
|x|          2|         33|         21|    23206384.50|         66|00:02:52|+0.96|0.98|0.0069|
|x|          3|         33|         21|    23237162.67|         99|00:02:52|+0.95|0.97|0.0069|
|x|          4|         33|         32|    23266553.50|        132|00:02:52|+0.97|0.97|0.0069|
|-|          5|          3|         32|    23246622.20|        165|00:02:52|-0.05|0.16|0.6111|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
Best-so-far configuration:          32    mean value:     23246622.20
Description of the best-so-far configuration:
   .ID. algorithm localsearch  alpha   beta    rho ants nnls     q0 dlb rasrank elitistants .PARENT.
32   32       acs           3 3.4899 9.9275 0.2071   42   10 0.0868   0      NA          NA       NA

# 2019-04-04 12:07:15 BST: Elite configurations (first number is the configuration ID; listed from best to worst according to the sum of ranks):
   algorithm localsearch  alpha   beta    rho ants nnls     q0 dlb rasrank elitistants
32       acs           3 3.4899 9.9275 0.2071   42   10 0.0868   0      NA          NA
21       acs           3 4.2535 2.5375 0.6954    6   15 0.7730   1      NA          NA
6        acs           3 2.0533 2.1353 0.7127   25   20 0.7251   0      NA          NA
# 2019-04-04 12:07:15 BST: Iteration 2 of 5
# experimentsUsedSoFar: 165
# remainingBudget: 835
# currentBudget: 208
# nbConfigurations: 31
  Markers:
     x No test is performed.
     - The test is performed and some configurations are discarded.
     = The test is performed but no configuration is discarded.
     ! The test is performed and configurations could be discarded but elite configurations are preserved.
     . All alive configurations are elite and nothing is discarded

+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
| |   Instance|      Alive|       Best|      Mean best| Exp so far|  W time|  rho|KenW|  Qvar|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
|x|          6|         31|         21|    23215678.00|         31|00:02:42|   NA|  NA|    NA|
|x|          1|         31|         21|    23237188.00|         59|00:02:23|+0.98|0.99|0.0132|
|x|          4|         31|         32|    23298775.67|         87|00:02:23|+0.96|0.97|0.0545|
|x|          2|         31|         32|    23255125.25|        115|00:02:22|+0.96|0.97|0.0463|
|-|          5|          6|         32|    23237479.60|        143|00:02:22|-0.08|0.14|0.8802|
|=|          3|          6|         32|    23247690.50|        146|00:00:20|+0.03|0.19|0.8182|
|=|          7|          6|         32|    23192329.71|        152|00:00:30|+0.09|0.22|0.7569|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
Best-so-far configuration:          32    mean value:     23192329.71
Description of the best-so-far configuration:
   .ID. algorithm localsearch  alpha   beta    rho ants nnls     q0 dlb rasrank elitistants .PARENT.
32   32       acs           3 3.4899 9.9275 0.2071   42   10 0.0868   0      NA          NA       NA

# 2019-04-04 12:20:19 BST: Elite configurations (first number is the configuration ID; listed from best to worst according to the sum of ranks):
   algorithm localsearch  alpha   beta    rho ants nnls     q0 dlb rasrank elitistants
32       acs           3 3.4899 9.9275 0.2071   42   10 0.0868   0      NA          NA
21       acs           3 4.2535 2.5375 0.6954    6   15 0.7730   1      NA          NA
6        acs           3 2.0533 2.1353 0.7127   25   20 0.7251   0      NA          NA
35       acs           3 0.9232 3.5994 0.5889    8   35 0.6366   1      NA          NA
43      mmas           3 1.4244 8.7845 0.5345   40    8     NA   0      NA          NA
# 2019-04-04 12:20:19 BST: Iteration 3 of 5
# experimentsUsedSoFar: 317
# remainingBudget: 683
# currentBudget: 227
# nbConfigurations: 32
  Markers:
     x No test is performed.
     - The test is performed and some configurations are discarded.
     = The test is performed but no configuration is discarded.
     ! The test is performed and configurations could be discarded but elite configurations are preserved.
     . All alive configurations are elite and nothing is discarded

+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
| |   Instance|      Alive|       Best|      Mean best| Exp so far|  W time|  rho|KenW|  Qvar|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
|x|          8|         32|         64|    23066039.00|         32|00:02:42|   NA|  NA|    NA|
|x|          5|         32|         35|    23122735.50|         59|00:02:22|+0.86|0.93|0.0243|
|x|          7|         32|         64|    23028243.33|         86|00:02:22|+0.85|0.90|0.0312|
|x|          3|         32|         32|    23114761.50|        113|00:02:22|+0.79|0.84|0.0590|
|-|          1|          9|         32|    23144841.60|        140|00:02:22|+0.14|0.31|0.6624|
|=|          6|          9|         78|    23159552.33|        144|00:00:20|+0.13|0.27|0.7384|
|-|          4|          7|         78|    23193750.00|        148|00:00:20|+0.16|0.28|0.6113|
|=|          2|          7|         32|    23184943.38|        150|00:00:10|+0.08|0.20|0.7184|
|=|          9|          7|         32|    23159695.56|        157|00:00:40|+0.11|0.21|0.7219|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
Best-so-far configuration:          32    mean value:     23159695.56
Description of the best-so-far configuration:
   .ID. algorithm localsearch  alpha   beta    rho ants nnls     q0 dlb rasrank elitistants .PARENT.
32   32       acs           3 3.4899 9.9275 0.2071   42   10 0.0868   0      NA          NA       NA

# 2019-04-04 12:34:01 BST: Elite configurations (first number is the configuration ID; listed from best to worst according to the sum of ranks):
   algorithm localsearch  alpha   beta    rho ants nnls     q0 dlb rasrank elitistants
32       acs           3 3.4899 9.9275 0.2071   42   10 0.0868   0      NA          NA
78       acs           3 3.3244 5.2122 0.2630    6   28 0.3217   1      NA          NA
64       acs           3 1.7469 0.3687 0.3568   15   30 0.6474   1      NA          NA
21       acs           3 4.2535 2.5375 0.6954    6   15 0.7730   1      NA          NA
35       acs           3 0.9232 3.5994 0.5889    8   35 0.6366   1      NA          NA
# 2019-04-04 12:34:01 BST: Iteration 4 of 5
# experimentsUsedSoFar: 474
# remainingBudget: 526
# currentBudget: 263
# nbConfigurations: 30
  Markers:
     x No test is performed.
     - The test is performed and some configurations are discarded.
     = The test is performed but no configuration is discarded.
     ! The test is performed and configurations could be discarded but elite configurations are preserved.
     . All alive configurations are elite and nothing is discarded

+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
| |   Instance|      Alive|       Best|      Mean best| Exp so far|  W time|  rho|KenW|  Qvar|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
|x|         10|         30|         64|    23309564.00|         30|00:02:32|   NA|  NA|    NA|
|x|          8|         30|         64|    23187801.50|         55|00:02:11|+0.68|0.84|0.0100|
|x|          9|         30|         64|    23113651.00|         80|00:02:11|+0.67|0.78|0.0136|
|x|          4|         30|         64|    23189241.50|        105|00:02:11|+0.62|0.71|0.0131|
|-|          3|         16|         64|    23225830.60|        130|00:02:11|-0.09|0.13|1.0154|
|=|          7|         16|         64|    23165755.67|        141|00:01:00|-0.06|0.11|0.9731|
|=|          1|         16|         64|    23180781.71|        152|00:01:00|-0.02|0.12|0.9233|
|=|          5|         16|         64|    23177347.75|        163|00:01:00|-0.00|0.12|0.9138|
|=|          6|         16|         95|    23188578.78|        174|00:01:00|-0.01|0.10|0.9255|
|=|          2|         16|        110|    23178781.00|        185|00:01:00|+0.01|0.11|0.9073|
|=|         11|         16|        110|    23149932.55|        201|00:01:20|+0.01|0.10|0.9034|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
Best-so-far configuration:         110    mean value:     23149932.55
Description of the best-so-far configuration:
    .ID. algorithm localsearch  alpha   beta    rho ants nnls     q0 dlb rasrank elitistants .PARENT.
110  110       acs           3 1.6928 0.2335 0.4496   11   10 0.5141   1      NA          NA       64

# 2019-04-04 12:51:43 BST: Elite configurations (first number is the configuration ID; listed from best to worst according to the sum of ranks):
    algorithm localsearch  alpha   beta    rho ants nnls     q0 dlb rasrank elitistants
110       acs           3 1.6928 0.2335 0.4496   11   10 0.5141   1      NA          NA
95        acs           3 1.5151 1.6723 0.3781   10   34 0.7918   1      NA          NA
64        acs           3 1.7469 0.3687 0.3568   15   30 0.6474   1      NA          NA
112       acs           3 2.0947 7.8084 0.1220   23   11 0.2665   0      NA          NA
32        acs           3 3.4899 9.9275 0.2071   42   10 0.0868   0      NA          NA
# 2019-04-04 12:51:43 BST: Iteration 5 of 5
# experimentsUsedSoFar: 675
# remainingBudget: 325
# currentBudget: 325
# nbConfigurations: 31
  Markers:
     x No test is performed.
     - The test is performed and some configurations are discarded.
     = The test is performed but no configuration is discarded.
     ! The test is performed and configurations could be discarded but elite configurations are preserved.
     . All alive configurations are elite and nothing is discarded

+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
| |   Instance|      Alive|       Best|      Mean best| Exp so far|  W time|  rho|KenW|  Qvar|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
|x|         12|         31|        131|    23210995.00|         31|00:02:41|   NA|  NA|    NA|
|x|          4|         31|        131|    23300342.50|         57|00:02:11|+0.53|0.77|0.1089|
|x|         11|         31|        131|    23156010.00|         83|00:02:11|+0.42|0.61|0.1699|
|x|          1|         31|        136|    23173809.25|        109|00:02:11|+0.45|0.58|0.1633|
|-|          5|         20|        110|    23184165.60|        135|00:02:11|-0.11|0.11|1.0253|
|=|          3|         20|        136|    23190209.00|        150|00:01:20|-0.06|0.12|0.9785|
|=|          6|         20|        136|    23197214.14|        165|00:01:20|-0.04|0.11|0.9602|
|=|          2|         20|        129|    23193127.38|        180|00:01:20|-0.01|0.12|0.9389|
|=|          7|         20|        110|    23159859.11|        195|00:01:20|+0.01|0.12|0.9219|
|=|         10|         20|        133|    23182665.30|        210|00:01:20|+0.02|0.12|0.9113|
|=|          8|         20|        133|    23172694.45|        225|00:01:20|+0.04|0.12|0.9117|
|=|          9|         20|        133|    23157506.42|        240|00:01:20|+0.04|0.12|0.9056|
|=|         13|         20|        129|    23160461.38|        260|00:01:41|+0.04|0.11|0.9081|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
Best-so-far configuration:         129    mean value:     23160461.38
Description of the best-so-far configuration:
    .ID. algorithm localsearch  alpha   beta    rho ants nnls     q0 dlb rasrank elitistants .PARENT.
129  129       acs           3 1.4332 0.7319 0.3364   16   16 0.5681   1      NA          NA      110

# 2019-04-04 13:14:18 BST: Elite configurations (first number is the configuration ID; listed from best to worst according to the sum of ranks):
    algorithm localsearch  alpha   beta    rho ants nnls     q0 dlb rasrank elitistants
129       acs           3 1.4332 0.7319 0.3364   16   16 0.5681   1      NA          NA
133       acs           3 1.6209 0.9782 0.7217    9   13 0.4398   1      NA          NA
110       acs           3 1.6928 0.2335 0.4496   11   10 0.5141   1      NA          NA
95        acs           3 1.5151 1.6723 0.3781   10   34 0.7918   1      NA          NA
135       acs           3 1.6560 0.2396 0.3828    9   13 0.7661   1      NA          NA
# 2019-04-04 13:14:18 BST: Iteration 6 of 6
# experimentsUsedSoFar: 935
# remainingBudget: 65
# currentBudget: 65
# nbConfigurations: 9
  Markers:
     x No test is performed.
     - The test is performed and some configurations are discarded.
     = The test is performed but no configuration is discarded.
     ! The test is performed and configurations could be discarded but elite configurations are preserved.
     . All alive configurations are elite and nothing is discarded

+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
| |   Instance|      Alive|       Best|      Mean best| Exp so far|  W time|  rho|KenW|  Qvar|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
|x|         14|          9|        140|    23104760.00|          9|00:00:50|   NA|  NA|    NA|
|x|          1|          9|        140|    23168308.50|         13|00:00:20|+0.73|0.87|0.0509|
|x|          6|          9|        140|    23184764.67|         17|00:00:20|+0.54|0.70|0.0911|
|x|          2|          9|        133|    23172023.00|         21|00:00:20|+0.43|0.57|0.0925|
|-|          8|          7|        133|    23152215.60|         25|00:00:20|-0.11|0.11|0.8788|
|=|          9|          7|        133|    23125252.67|         27|00:00:10|-0.02|0.15|0.7786|
|=|          4|          7|        133|    23167226.71|         29|00:00:10|-0.05|0.10|0.8504|
|=|         12|          7|        133|    23176624.50|         31|00:00:10|-0.01|0.11|0.8244|
|=|          5|          7|         95|    23172760.22|         33|00:00:10|-0.04|0.08|0.8635|
|=|         13|          7|        140|    23182954.00|         35|00:00:10|-0.02|0.08|0.8532|
|=|         11|          7|        129|    23144833.64|         37|00:00:10|-0.03|0.06|0.8452|
|=|         10|          7|        129|    23169650.25|         39|00:00:10|-0.06|0.03|0.8916|
|=|          7|          7|        140|    23148480.00|         41|00:00:10|-0.05|0.03|0.8875|
|=|          3|          7|        140|    23157234.64|         43|00:00:10|-0.03|0.04|0.8789|
|=|         15|          7|        140|    23148936.33|         50|00:00:40|-0.04|0.03|0.8823|
+-+-----------+-----------+-----------+---------------+-----------+--------+-----+----+------+
Best-so-far configuration:         140    mean value:     23148936.33
Description of the best-so-far configuration:
    .ID. algorithm localsearch  alpha   beta    rho ants nnls     q0 dlb rasrank elitistants .PARENT.
140  140       acs           3 1.2462 2.1628 0.2982   16   20 0.4988   1      NA          NA      129

# 2019-04-04 13:18:41 BST: Elite configurations (first number is the configuration ID; listed from best to worst according to the sum of ranks):
    algorithm localsearch  alpha   beta    rho ants nnls     q0 dlb rasrank elitistants
140       acs           3 1.2462 2.1628 0.2982   16   20 0.4988   1      NA          NA
133       acs           3 1.6209 0.9782 0.7217    9   13 0.4398   1      NA          NA
129       acs           3 1.4332 0.7319 0.3364   16   16 0.5681   1      NA          NA
110       acs           3 1.6928 0.2335 0.4496   11   10 0.5141   1      NA          NA
142       acs           3 1.3394 0.8031 0.5894    7   15 0.4967   1      NA          NA
# 2019-04-04 13:18:41 BST: Stopped because there is not enough budget left to race more than the minimum (5)
# You may either increase the budget or set 'minNbSurvival' to a lower value
# Iteration: 7
# nbIterations: 7
# experimentsUsedSoFar: 985
# timeUsed: 0
# remainingBudget: 15
# currentBudget: 15
# number of elites: 5
# nbConfigurations: 5
# Best configurations (first number is the configuration ID; listed from best to worst according to the sum of ranks):
    algorithm localsearch  alpha   beta    rho ants nnls     q0 dlb rasrank elitistants
140       acs           3 1.2462 2.1628 0.2982   16   20 0.4988   1      NA          NA
133       acs           3 1.6209 0.9782 0.7217    9   13 0.4398   1      NA          NA
129       acs           3 1.4332 0.7319 0.3364   16   16 0.5681   1      NA          NA
110       acs           3 1.6928 0.2335 0.4496   11   10 0.5141   1      NA          NA
142       acs           3 1.3394 0.8031 0.5894    7   15 0.4967   1      NA          NA
# Best configurations as commandlines (first number is the configuration ID; same order as above):
140  --acs --localsearch 3 --alpha 1.2462 --beta 2.1628 --rho  0.2982 --ants 16 --nnls 20 --q0 0.4988 --dlb 1
133  --acs --localsearch 3 --alpha 1.6209 --beta 0.9782 --rho  0.7217 --ants 9 --nnls 13 --q0 0.4398 --dlb 1
129  --acs --localsearch 3 --alpha 1.4332 --beta 0.7319 --rho  0.3364 --ants 16 --nnls 16 --q0 0.5681 --dlb 1
110  --acs --localsearch 3 --alpha 1.6928 --beta 0.2335 --rho  0.4496 --ants 11 --nnls 10 --q0 0.5141 --dlb 1
142  --acs --localsearch 3 --alpha 1.3394 --beta 0.8031 --rho  0.5894 --ants 7 --nnls 15 --q0 0.4967 --dlb 1
 

# 2019-04-04 13:18:41 BST: Testing configurations (in no particular order): 32 110 129 140 133 142
    algorithm localsearch  alpha   beta    rho ants nnls     q0 dlb rasrank elitistants
32        acs           3 3.4899 9.9275 0.2071   42   10 0.0868   0      NA          NA
110       acs           3 1.6928 0.2335 0.4496   11   10 0.5141   1      NA          NA
129       acs           3 1.4332 0.7319 0.3364   16   16 0.5681   1      NA          NA
140       acs           3 1.2462 2.1628 0.2982   16   20 0.4988   1      NA          NA
133       acs           3 1.6209 0.9782 0.7217    9   13 0.4398   1      NA          NA
142       acs           3 1.3394 0.8031 0.5894    7   15 0.4967   1      NA          NA
# Testing of elite configurations: 5 
# Testing iteration configurations: TRUE 
# 2019-04-04 13:23:44 BST: Testing results (column number is configuration ID in no particular order):
          32      110      129      140      133      142
1t  23365389 23353730 23374999 23354991 23362055 23379817
2t  23158681 23176360 23146736 23168217 23143084 23103167
3t  23064240 23053820 23023056 23026067 22986371 23066480
4t  23031740 23044139 23008672 23051943 23093371 23043474
5t  23209025 23188063 23215787 23241732 23249566 23186894
6t  23405205 23442994 23374269 23446188 23375515 23380036
7t  23335011 23324526 23351169 23359563 23351186 23375381
8t  23262829 23263361 23228639 23234191 23281635 23213823
9t  23339334 23300092 23295216 23278792 23303729 23293629
10t 23061345 23032704 23094304 23095169 23024431 23032993
# 2019-04-04 13:23:44 BST: Finished testing
