-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The current state of clpsmt-miniKanren
#9
Comments
Thanks for your thoughtful issue.
did you find it educational to read the canonical implementation before diving into faster-miniKanren? if that's the case, maybe we should keep it even though we would always use faster-miniKanren? I think it's a bit premature to pick one implementation, so it's good to have them in separate branches in a separate repo? Eventually for benchmarks, we might want to try a few implementations. I guess we could all have them in one repo under different names, but that seems unwieldy too.
|
What do you think? Any ideas on your side? Thanks! |
It's worth also looking at https://github.com/michaelballantyne/faster-miniKanren/commits/smt-numsonly by @michaelballantyne |
Hi! Glad you find this interesting. :) Let me start with the things I was thinking about when we left off work, and then I'll get to your specific questions. Interacting with the solverWe were exploring the best way of interacting with the solver. smt-assumptions was Nada's version of using check-sat-assuming, but its code was different enough than the basic implementation that I was worried we weren't comparing correctly. To attempt an apples-to-apples comparison of different approaches I created the smt-numsoly branch (https://github.com/michaelballantyne/faster-miniKanren/tree/smt-numsonly), which supports parameters that configure several modes:
I don't have recorded benchmark runs, but the notes I wrote suggest that reset-as-push-pop provided a significant gain vs naive, but push-pop and assumptions didn't provide much further benefit. From our work on the smt-assumptions branch we had previously thought that using check-sat-assuming was useful, but I think we had fooled ourselves. The real effects were:
However, our collaborator Aaron Bembenek found that the check-sat-assuming strategy was useful in his datalog system, as documented here: https://people.seas.harvard.edu/~bembenek/papers/datalog-incr-smt-iclp2020.pdf It would be useful to benchmark these options more thoroughly as we explore extensions. Integrating equalities, disequalities, and typesAs you point out, giving the solver information about equalities and disequalities improves its ability to refute certain queries. The integration is a little tricky because miniKanren treats variables as dynamically typed, whereas SMT-lib is statically typed. Types for SMT variables must be provided at variable declaration time. If at
we want to provide the equality to the solver, we need to declare the variables. But if we support multiple theories (say, bitvectors in addition to numbers), we haven't yet seen a constraint that determines the type of the variable, so we don't know the right type to declare the variable with! I believe this is why smt-assumptions and smt-numsonly only support integers. I imagine solving this problem by accumulating equality and disequality constraint information and only sending it to the solver after a constraint that determines a particular type has been applied. Things like I think it would work to rely on the solver to handle the interaction between disequality and finite domains. A query asserting that three variables are pairwise disequal would succeed; adding additional constraints indicating that all three variables are of an algebraic data type with only two constructors would lead to failure when the constraints get checked by the solver. As an alternative we could create a statically or gradually typed miniKanren so that we have type information to correctly declare types to the solver even before reaching theory-specific constraints. My angle on your questions
We stalled because we hadn't yet found applications that we thought would be compelling to people beyond the miniKanren community: where might this approach be more effective than a classic constraint logic programming system, or a constrained horn clause solver? As one such application, we had hoped that the combination of miniKanren's unique search and SMT constraints would allow us to solve example-based program synthesis queries for programs that manipulate things like numbers and arrays. The Barliman repository has branches corresponding to the faster-minikanren ones where we ported over the various implementations to Barliman's version of miniKanren. From my notes it looks like we weren't able to get Barliman with SMT to synthesize the full implementation of factorial. The tests we were exploring are here: https://github.com/webyrd/Barliman/blob/smt-numsonly/cocoa/Barliman/mk-and-rel-interp/test-interp.scm Barliman is a bit finicky though, so I'm not sure that the problem was the performance of SMT solving. This could use some more digging. Or, you might have other applications in mind? Of course, a clean and complete implementation would definitely be of value to folks already interested in miniKanren regardless of whether we find uses of interest to a broader audience.
I think this is an artifact of the development process; Nada started in clpsmt-miniKanren and then wanted to try a faster-minikanren version. The clpsmt-miniKanren repository has all of the examples that we were trying across the original version based on canonical minikanren, the faster-minikanren branches, and the barliman branches.
Starting further work from a copy of smt-numsonly would let us continue to explore the different modes of interacting with the solver as we use new theories. It would be great to extend that implementation to support all the SMT-LIB datatypes and constraints we can. Then it could subsume the other implementations. It's nice to have a separate repository for this project, but it is a bit of a pain to keep everything in sync. @namin, would it make sense to set up a branch of faster-minikanren as a submodule of this repository? I haven't looked into it too deeply, but it might be possible to have a single smt.scm used by both the faster-minikanren and the canonical miniKanren versions, if we want to have both. |
Thanks for your great answers ! @namin @michaelballantyne . I have a lot of things like to write, but they are not well organized yet. I will add it later, sorry. |
About "Integrating equalities, disequalities, and types" This section gave me a lot of inspiration, thanks. I was thinking that it seems not a big problem if minikanren with SMT does not support constraints promotion? Take diseq-3 as an example, suppose the users know how the
Then no problem to refute this query. So users seemly need to learn some technique for working around this restriction... In the example (diseq-3),
Since the user has used So he must declare Then since The result is the same as the above. Note that, for For example, suppose the original program is
The only way to rewrite is
They look a bit like type inference. For a simple program, user can manually rewrite it, but for a larger program, it will be more complicated (Imagine that here Therefore, I think, although the In contrast, although As @michaelballantyne said,
So is the "constraints promotion" really necessary? What do you think about it? PS. The "constraint promotion" is the term that I invented myself. I don’t know whether there is a specific term for that purpose (i.e. converting miniKanren's constraints to SMT assertions). Perhaps using "constraint conversion" or "constraint reification" is more precise? |
Hmm. Maybe it would be useful to work through how you would add arithmetic operations to a relational interpreter without constraint promotion, but while ensuring refutation? I think you'd at least be forced to tag the numbers, but I'm not certain that would be enough. Because the relational interpreter supports many kinds of values, we don't know statically whether a given unification or disequality relates things that correspond to solver variables. For example, in the interpreter |
(I'm also not sure on the best terminology; "constraint promotion" is a good shorthand for our communication, but I think I would avoid it in formal writing.) |
@michaelballantyne I haven't tried the relational interpreter, but for a simple naive version
revised version
However, I tried another example which can represent refutation:
naive version
revised version
I will try the relational interpreter later. Note that these programs crashes in the current clpsmt-miniKanren, because there is a bug I have not fixed. If you'd like to try, you can temporarily uncomment this line. |
In the I think it's good to experiment with different approaches. The tag approach seems a bit annoying -- think how you'd define |
Yes.
Yes, it is annoying. For example,
However, if we integrate the I have hacked the
Currently I only tagged int and bool in PS. I found @webyrd used a similar method for abstract-interp-tagged.
I don't know how to do. As you see that the Another reason is that I don’t know how to construct negation in miniKanren, which make the code very ugly. For example, how to represent
But using tags, we just write:
PS. We can use |
If it's just for hooking SMT, it doesn't seem no much difference (I now even think
Feeling that the Perhaps we need record and replay those SMT commands to check whether the incremental query is really effective? (Ignore the impact of miniKanren)
I will look at it. Thanks. BTW is it possible that the process communication is a bottleneck?
The current implementation of I'm thinking about how to improve it. For Racket, we can modify
I am reading all the tests. I will report if I find some illegitimate case. In fact, I have found some. I will create issues later. |
I am reading the test cases these days. IMO, all the examples are interesting :) Here are some examples that I think are most interesting:
|
There are two main repos of miniKanren's SMT hook: clpsmt-miniKanren and smt-assumptions
These two repos seemly have their own advantages and restrictions.
smt-assumptions
usescheck-sat-assuming
to improve performance and supports constraints promotion.However, its the variable' domain seems only supports number? currently?
BTW, I noticed that the variables' domain (FD) in
faster-miniKanren
is really tricky.As @michaelballantyne said in his comment:
I hope the SMT hook can work around this restriction.
clpsmt-miniKanren
is slower thansmt-assumptions
and does not support constraints promotion, but it can support more variable's domains (e.g. Boolean, bit-vector, Set, even ADT).PS.
smt
is basically the same asclpsmt-miniKanren
, except that usingfaster-miniKanren
to improve performance.My question is that
Does this repo still under development or has done?
As you said that
So ideally any patches/examples of
clpsmt-miniKanren
now should be committed tofaster-miniKanren
repo (e.g.smt
orsmt-assumptions
) instead ofclpsmt-miniKanren
. Is that right?Another way is replacing the
canonical miniKanren implementation
ofclpsmt-miniKanren
withfaster-miniKanren
.Why
smt-assumptions
/smt
use a remote link toclpsmt-miniKanren
?They must be hardcoded by path name explicitly, e.g.
(load "../clpsmt-miniKanren/z3-server.scm")
or(require "../faster-miniKanren/mk.rkt")
.Is it intentional?
IMO it's very inconvenient. Shall we move all the SMT-server code to faster-mk version?
What do you think about the above restrictions of
clpsmt-miniKanren
andsmt-assumption
?Is it necessary to fix these restrictions?
For example, merging constraints promotion to
smt
or make thesmt-assumption
support more variable's domain?Are the
clpsmt-miniKanren
/smt
andsmt-assumption
both useful?I do not quite understand
Is there any next plan about this project?
I can help.
Thanks.
The text was updated successfully, but these errors were encountered: