Merge pull request #9 from kstrauch94/clingo-dl

Clingo dl

Author: wanko

README.md

@@ -47,6 +47,9 @@ python scripts/acclingo --instance_dir <directory> --fn_suffix ".gz" --binary <s
 - `--tae_class acclingo/tae/clasp_opt_tae.py`: TAE that expects `<solver>` to be a `clingo` binary and grounded instances to include `#minimize` statements. If no solution was found or the resulting cost is bigger then the ParN score, the ParN score is returned as costs. Per default 10*cutoff. Otherwise the cost is runtime times percental difference of achieved quality to best known bound.
 - `--tae_args "{\"best_known\": \"<csv file>\"}"`: String in json format that passes auxiliary arguments to the TAE. The TAE expects the csv file to have instance names without extension in the first column and integers representing the best known bound in the second.
 
+# Additional Options
 
-
-
+The --tae_args can also be used to change the mode of clingo. By default it is set to "clasp". If the instances require an additional file(encoding) to run, you can also provide it with the --tae_args option using "encoding":
+```
+--tae_args "{\"mode\": \"clingo\", \"encoding\": \"path/to/encoding.lp\"}"
+```

acclingo/tae/README.md

@@ -1,6 +1,6 @@
 # Asprin TAE
 
-This TAE is very similar to the basic clasp TAE. The only differences are that it only counts a solution as a succesful solution if the optimum was found and it needs an asprin "binary" or a way to call asprin. The penalty for no solution or no optimal solution is the maximum available time multiplied by some penalty. This penalty can be given as a parameter:
+This TAE is very similar to the basic clasp TAE. The only differences are that it only counts a solution as a succesful solution if the optimum was found and that it needs an asprin "binary" or a way to call asprin. The penalty for no solution or no optimal solution is the maximum available time multiplied by some penalty value . This penalty value can be given as a parameter:
 
 ⋅⋅* Parameter to penalize no optimal solution = "penalty": float
 
@@ -14,13 +14,13 @@ example:
 The optimization weights TAE is a variant of "clasp\_opt\_tae" where the cost of not finding the optimal solution is calculated based on the time it takes to find some solution (or no solution) and the solution quality.
 
 ## Solution quality
-There are two ways to calculate the cost of the solution, normalized cost and not normalized cost. For both calculations we assume that a higher value is always worse.
+There are two ways to calculate the cost of the solution: normalized cost and not normalized cost. For both calculations we assume that a higher value is always worse.
 
 Normalized cost uses the following formula:
 ```
     solution_quality = 1 - (best_known_solution / found_solution_quality)
 ```
-this formula will be between 0 and 1 for all values *worse* than the best solution. This also assumes that worse values are higher than better values(E.g 10 is worse than 5). 
+this formula will be between 0 and 1 for all values *worse* than the best solution. A solution is worse than another if its value is higher(E.g 10 is worse than 5). If the solution found is better, the value will become negative.
 
 Not normalized quality cost uses the following formula:
 ```
@@ -29,7 +29,7 @@ Not normalized quality cost uses the following formula:
 This formula basically gives a ratio of the quality of the found solution to the best known solution. So, a value higher than 1 means that the found solution is worse and a value below 1 means that the found solution is better.
 
 ## Runtime quality
-The Above formulas are used to gage the solution quality. To calculate the actual cost we also want to take a loot at the time quality. The following formula is used regardless of the which quality formula is used:
+The Above formulas are used to gage the solution quality. To calculate the actual cost we also want to take a look at the time quality. The following formula is used regardless of the which quality formula is used:
 ```
     runtime_quality = (found_solution_runtime / cutoff)
 ```
@@ -41,7 +41,7 @@ If a solution was not found for the current instance, the following formula is u
 ```
     cost = par_factor * unsolved_penalty
 ```
-Par factor is usually set to 10 and unsolved penalty is a predefined value that modifies par factor. Users need to be careful when choosing a low unsolved_penalty when using the not normalized formula for solution quality as the value for solution quality can potentially be very big.
+Par factor is usually set to 10 and unsolved penalty is a predefined value that modifies par factor.
 
 The actual cost is then calculated with the following formula:
 ```

acclingo/tae/asprin_tae.py

@@ -67,11 +67,22 @@ def __init__(self, ta_bin:str, runsolver_bin:str,
             self.penalty = misc["penalty"]
         else:
             self.penalty = 2
+
+        self.encoding = ""
+
+        self.mode = "clasp"
+
+        if "encoding" in misc:
+            self.encoding = misc["encoding"]
+
+        if "mode" in misc:
+            self.mode = misc["mode"]
 
 
     def run(self, config, instance,
             cutoff,
             seed=12345,
+            budget=None,
             instance_specific="0"
             ):
         """
@@ -105,9 +116,9 @@ def run(self, config, instance,
 
 
         if not instance.endswith(".gz"):
-            cmd = "%s %s --seed=%d " %(self.ta_bin, instance, seed)       
+            cmd = "{bin} {encoding} {instance} --seed {seed} ".format(bin=self.ta_bin, encoding=self.encoding, instance=instance, seed=seed)       
         else:
-            cmd = "bash -c 'zcat %s | %s --seed=%d " %(instance, self.ta_bin, seed )       
+            cmd = "bash -c 'zcat {instance} | {bin} {encoding} --seed {seed} ".format(instance=instance, bin=self.ta_bin, encoding=self.encoding, seed=seed )       
 
         params = []
         for name in config:
@@ -118,9 +129,28 @@ def run(self, config, instance,
             params.append(value)
         thread_to_params, thread_to_solver, params_to_tags = self.parse_parameters(params)
 
-        for t, p_list in thread_to_params.items():
-             for p in p_list:
-                 cmd += " "+p
+        thread_count = int(max(thread_to_params.keys()))
+        config_file = "config_file.tmp"
+        with open(config_file, "w") as cfile:
+            for t, p_list in thread_to_params.items():
+                if t == 0: # global params
+                    for p in p_list:
+                            cmd += " " + p
+                else:
+                    new_p_list = []
+                    thread_config = "auto"
+                    for p in p_list:
+                        if "--configuration" in p:
+                            thread_config = p.split("=")[1]
+                        else:
+                            new_p_list.append(p)
+
+                    cfile.write("{}({}): {}\n".format(t, thread_config, " ".join(new_p_list)))
+        
+        cmd += " --mode={}".format(self.mode)
+        cmd += " --configuration={}".format(config_file)
+        cmd += " --quiet=2 --stats"
+        cmd += " --parallel-mode={}".format(thread_count)
 
         if instance.endswith(".gz"):
            cmd += "'"
@@ -146,8 +176,11 @@ def run(self, config, instance,
 
         ta_status_rs, ta_runtime, ta_exit_code = self.read_runsolver_output(watcher_file)
 
-        ta_status, ta_quality = self.parse_output(fn=solver_file, exit_code=ta_exit_code)
-        
+        ta_status, ta_quality, clingo_runtime = self.parse_output(fn=solver_file, exit_code=ta_exit_code)
+                
+        if ta_runtime is None:
+            ta_runtime = clingo_runtime
+         
         if not ta_status:
             ta_status = ta_status_rs
 
@@ -378,8 +411,10 @@ def parse_output(self, fn, exit_code):
             ta_status = StatusType.TIMEOUT
         elif re.search('INDETERMINATE', data):
             ta_status = StatusType.TIMEOUT
-        
-        return ta_status, ta_quality 
+            
+        clingo_runtime = re.search(r"Time[ ]*:[ ]*(\d+\.\d+)s", data).group(1)
+
+        return ta_status, ta_quality, float(clingo_runtime) 
 
     def read_runsolver_output(self, watcher_fn: str):
         '''

acclingo/tae/clasp_opt_tae.py

@@ -68,9 +68,20 @@ def __init__(self, ta_bin:str, runsolver_bin:str,
                 for line in f.readlines():
                    self.best_known[line.split(',')[0].strip()]=int(line.split(',')[1].strip())
 
+        self.encoding = ""
+
+        self.mode = "clasp"
+
+        if "encoding" in misc:
+            self.encoding = misc["encoding"]
+
+        if "mode" in misc:
+            self.mode = misc["mode"]
+
     def run(self, config, instance,
             cutoff,
             seed=12345,
+            budget=None,
             instance_specific="0"
             ):
         """
@@ -104,9 +115,9 @@ def run(self, config, instance,
 
 
         if not instance.endswith(".gz"):
-            cmd = "%s %s --seed %d " %(self.ta_bin, instance, seed)       
+            cmd = "{bin} {encoding} {instance} --seed {seed} ".format(bin=self.ta_bin, encoding=self.encoding, instance=instance, seed=seed)       
         else:
-            cmd = "bash -c 'zcat %s | %s --seed %d " %(instance, self.ta_bin, seed )       
+            cmd = "bash -c 'zcat {instance} | {bin} {encoding} --seed {seed} ".format(instance=instance, bin=self.ta_bin, encoding=self.encoding, seed=seed )       
 
         params = []
         for name in config:
@@ -117,11 +128,28 @@ def run(self, config, instance,
             params.append(value)
         thread_to_params, thread_to_solver, params_to_tags = self.parse_parameters(params)
 
-        for t, p_list in thread_to_params.items():
-             for p in p_list:
-                 cmd += " "+p
+        thread_count = int(max(thread_to_params.keys()))
+        config_file = "config_file.tmp"
+        with open(config_file, "w") as cfile:
+            for t, p_list in thread_to_params.items():
+                if t == 0: # global params
+                    for p in p_list:
+                            cmd += " " + p
+                else:
+                    new_p_list = []
+                    thread_config = "auto"
+                    for p in p_list:
+                        if "--configuration" in p:
+                            thread_config = p.split("=")[1]
+                        else:
+                            new_p_list.append(p)
+
+                    cfile.write("{}({}): {}\n".format(t, thread_config, " ".join(new_p_list)))
 
-        cmd += " --mode=clasp"
+        cmd += " --mode={}".format(self.mode)
+        cmd += " --configuration={}".format(config_file)
+        cmd += " --quiet=2 --stats"
+        cmd += " --parallel-mode={}".format(thread_count)
 
         if instance.endswith(".gz"):
            cmd += "'"
@@ -147,8 +175,11 @@ def run(self, config, instance,
 
         ta_status_rs, ta_runtime, ta_exit_code = self.read_runsolver_output(watcher_file)
 
-        ta_status, ta_quality = self.parse_output(fn=solver_file, exit_code=ta_exit_code)
+        ta_status, ta_quality, clingo_runtime = self.parse_output(fn=solver_file, exit_code=ta_exit_code)
 
+        if ta_runtime is None:
+            ta_runtime = clingo_runtime
+            
         if not ta_status:
             ta_status = ta_status_rs
 
@@ -168,7 +199,7 @@ def run(self, config, instance,
             diff = float(ta_quality) - float(best)
             prct = float(best)/100.0 if best!=0 else 1.0
             cost = min(float(cutoff * self.par_factor),ta_runtime*(1+diff/prct))
-            
+        
         return ta_status, cost, ta_runtime, {}
 
     def parse_parameters(self, params,prefix="--",separator="="):
@@ -383,12 +414,14 @@ def parse_output(self, fn, exit_code):
             ta_status = StatusType.SUCCESS
         elif re.search('OPTIMUM FOUND', data):
             ta_status = StatusType.SUCCESS
-        elif re.search('s UNKNOWN', data):
+        elif re.search('UNKNOWN', data):
             ta_status = StatusType.TIMEOUT
         elif re.search('INDETERMINATE', data):
             ta_status = StatusType.TIMEOUT
+
+        clingo_runtime = re.search(r"Time[ ]*:[ ]*(\d+\.\d+)s", data).group(1)
 
-        return ta_status, ta_quality 
+        return ta_status, ta_quality, float(clingo_runtime)
 
     def read_runsolver_output(self, watcher_fn: str):
         '''

acclingo/tae/clasp_opt_weights_tae.py

@@ -90,10 +90,21 @@ def __init__(self, ta_bin:str, runsolver_bin:str,
 
         if "normalized" in misc:
             self.normalized = misc["normalized"]
+
+        self.encoding = ""
+
+        self.mode = "clasp"
+
+        if "encoding" in misc:
+            self.encoding = misc["encoding"]
+
+        if "mode" in misc:
+            self.mode = misc["mode"]
 
     def run(self, config, instance,
             cutoff,
             seed=12345,
+            budget=None,
             instance_specific="0"
             ):
         """
@@ -127,9 +138,9 @@ def run(self, config, instance,
 
 
         if not instance.endswith(".gz"):
-            cmd = "%s %s --seed %d " %(self.ta_bin, instance, seed)       
+            cmd = "{bin} {encoding} {instance} --seed {seed} ".format(bin=self.ta_bin, encoding=self.encoding, instance=instance, seed=seed)       
         else:
-            cmd = "bash -c 'zcat %s | %s --seed %d " %(instance, self.ta_bin, seed )       
+            cmd = "bash -c 'zcat {instance} | {bin} {encoding} --seed {seed} ".format(instance=instance, bin=self.ta_bin, encoding=self.encoding, seed=seed )       
 
         params = []
         for name in config:
@@ -140,11 +151,28 @@ def run(self, config, instance,
             params.append(value)
         thread_to_params, thread_to_solver, params_to_tags = self.parse_parameters(params)
 
-        for t, p_list in thread_to_params.items():
-             for p in p_list:
-                 cmd += " "+p
+        thread_count = int(max(thread_to_params.keys()))
+        config_file = "config_file.tmp"
+        with open(config_file, "w") as cfile:
+            for t, p_list in thread_to_params.items():
+                if t == 0: # global params
+                    for p in p_list:
+                            cmd += " " + p
+                else:
+                    new_p_list = []
+                    thread_config = "auto"
+                    for p in p_list:
+                        if "--configuration" in p:
+                            thread_config = p.split("=")[1]
+                        else:
+                            new_p_list.append(p)
+
+                    cfile.write("{}({}): {}\n".format(t, thread_config, " ".join(new_p_list)))
 
-        cmd += " --mode=clasp"
+        cmd += " --mode={}".format(self.mode)
+        cmd += " --configuration={}".format(config_file)
+        cmd += " --quiet=2 --stats"
+        cmd += " --parallel-mode={}".format(thread_count)
 
         if instance.endswith(".gz"):
            cmd += "'"
@@ -170,7 +198,10 @@ def run(self, config, instance,
 
         ta_status_rs, ta_runtime, ta_exit_code = self.read_runsolver_output(watcher_file)
 
-        ta_status, ta_quality = self.parse_output(fn=solver_file, exit_code=ta_exit_code)
+        ta_status, ta_quality, clingo_runtime = self.parse_output(fn=solver_file, exit_code=ta_exit_code)
+                
+        if ta_runtime is None:
+            ta_runtime = clingo_runtime
 
         if not ta_status:
             ta_status = ta_status_rs
@@ -423,7 +454,9 @@ def parse_output(self, fn, exit_code):
         elif re.search('INDETERMINATE', data):
             ta_status = StatusType.TIMEOUT
 
-        return ta_status, ta_quality 
+        clingo_runtime = re.search(r"Time[ ]*:[ ]*(\d+\.\d+)s", data).group(1)
+        
+        return ta_status, ta_quality, float(clingo_runtime) 
 
     def read_runsolver_output(self, watcher_fn: str):
         '''