unify prototypes/defs with compatible but different itypes

[mwhicks1]

If we have this code:

int *foo(int *a) {
 a = (int *)5;
 return a;
}

Then converting it (using liberalized itypes) leads to:

int *foo(int *a : itype(_Ptr<int>)) : itype(_Ptr<int>) {
 a = (int *)5;
 return a;
}

However, if we add the liberal prototype at the outset, as follows, 3c makes no changes to the file, i.e., definition of foo is not changed.

int *foo(int *a : itype(_Ptr<int>)) : itype(_Ptr<int>);
int *foo(int *a) {
 a = (int *)5;
 return a;
}

This is OK as far as Checked C's clang is concerned, but I think we should at the least add the itype to it, with the goal of making the prototypes uniform.

[mwhicks1]

But, we might aim to be even better than that, giving foo the best possible type to both foo as a definition, and the prototype (which might necessitate the change of other prototypes too). For example, if we had

int *foo(int *a : itype(_Ptr<int>)) : itype(_Ptr<int>);
int *foo(int *a) {
 return a;
}

we might like this to convert to

_Ptr<int> foo(_Ptr<int>);
_Ptr<int> foo(_Ptr<int> *a) {
 return a;
}

The problem with this conversion is that it could break some clients that call foo assuming the old itype, e.g., those clients that are passing an unsafe pointer. It might make sense to transfer the itype to the definition, unchanged, and then have a follow-up pass that removes the itype if it's not needed by the callers anymore (e.g., because those callers have no unsafe pointers they are passing in).

[mattmccutchen-cci]

However, if we add the liberal prototype at the outset, as follows, 3c makes no changes to the file, i.e., definition of foo is not changed.

FTR, as also noted here, I believe this is because 3C rewrites a function signature only if the external type changed on the current 3C run. This condition is met in the first example (where 3C inserted the liberal itype) but not in the second (where after declaration merging, 3C sees the external type as already being _Ptr<int> and not changing). Maybe we should change that behavior.

and then have a follow-up pass that removes the itype if it's not needed by the callers anymore (e.g., because those callers have no unsafe pointers they are passing in).

Private comment

[mwhicks1]

So one solution might be that when there are multiple prototypes for a function, and one of them is part-checked, we should consider rewriting those we can (i.e., that are in files in the base-dir) so that the types match?

This would change the current behavior, which simply confirms whether types are compatible when they are different, but does no rewriting of any of the types when one of them is part-checked (I think you are saying).

[mattmccutchen-cci]

So one solution might be that when there are multiple prototypes for a function, and one of them is part-checked, we should consider rewriting those we can (i.e., that are in files in the base-dir) so that the types match?

Yes, that's what I would have assumed 3C should do, except for one reservation: if 3C has a problem rewriting one of the prototypes (e.g., due to a macro), should 3C handle it in the same way as a necessary rewrite (e.g., by inlining the macro or generating a warning), or is it not worth the user's trouble and 3C should silently skip the rewrite?

Of course, figuring out how to make 3C do that without breaking anything else may be a challenge.