Allow complex promotion in functions, not just operators

[WardBrian]

This address the first portion of #998. We still need to do static casting in the code generation so the C++ will compile.

Currently, things like

    complex foo_two(complex z, complex z2){
      return add(z,z2);
    }
   
    foo_two(1, 3.5);

throw signature errors because we're not properly allowing promotion for functions.

Partially this was caused by the fact that we have different systems for checking functions and checking operators, but this does not fix that issue, just repair this symptom for now.

However, this now fails at C++ time.

So does something like proj(1.0).

Submission Checklist

• Run unit tests
• Documentation
  • OR, no user-facing changes were made

Release notes

Fixed an issue with calling functions that expect complex arguments with reals or integers.

Copyright and Licensing

By submitting this pull request, the copyright holder is agreeing to
license the submitted work under the BSD 3-clause license (https://opensource.org/licenses/BSD-3-Clause)

[WardBrian]

@SteveBronder this is now getting failures on C++ compile:

--- Compiling, linking C++ code ---
clang++-6.0 -std=c++1y -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes      -I stan/lib/stan_math/lib/tbb_2020.3/include    -O0 -I src -I stan/src -I lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.3.9 -I stan/lib/stan_math/lib/boost_1.75.0 -I stan/lib/stan_math/lib/sundials_5.7.0/include    -DBOOST_DISABLE_ASSERTS          -c -include-pch stan/src/stan/model/model_header.hpp.gch -x c++ -o ../../test/integration/good/lang/complex_scalar.o ../../test/integration/good/lang/complex_scalar.hpp
../../test/integration/good/lang/complex_scalar.hpp:869:26: error: no matching function for call to 'foo_two'
      assign(gq_complex, foo_two(1, 3.0, pstream__),
                         ^~~~~~~
../../test/integration/good/lang/complex_scalar.hpp:199:1: note: candidate template ignored: could not match 'complex<type-parameter-0-0>' against 'int'
foo_two(const std::complex<T0__>& z, const std::complex<T1__>& z2,
^

Do we need to do something on the c++ side to allow complex promotion in UDFs?

[WardBrian]

Marked as a draft until C++ compilation is resolved. @SteveBronder - feel free to work on this branch as well if you get the opportunity

[SteveBronder]

I'll look at it today!

[WardBrian]

Thanks!
Feel free to work in the complex-promotion branch I'm using in #999. I've added a couple tests to explicitly expose the issue, and I think we need to solve both simultaneously anyway.

[SteveBronder]

Will do! Trying to finish up some of the adjoint ODE fix today then I will look at this

[SteveBronder]

Oh, this is a little tricky. So int -> double works in signatures since int is implicitly convertible to double but the same is not true for complex<>

https://godbolt.org/z/ov184n6do

So, I think what we need to do here is handle promotion ourselves and that means before printing the C++ we need to look at the type_ for each function input and check it against the type that the function expects. If the functions expected type is complex but the input type is int or real then we need to promote them to complex. i.e. something like the following

functions {
// For this signature
vector blah(complex x, array[] complex y);
}

transformed data {
// called as 
real yy[10] yy = fill_in_y(10, 10);
complex whatever = blah(1, yy);
}

The cpp is pseudo like

// function decl
template <typename T1, typename T2>
auto blah(std::complex<T1> x, std::vector<std::complex<T2>> y) {
  /// stuff
}

// l8r in the model

std::vector<double> yy = fill_in_y(10, 10);
// call site
std::complex<double> whatever = blah(std::complex<double>(1), 
 stan::math::promote_scalar_t<std::complex<double>, std::vector<double>>(yy));

But stan::math::promote_scalar_t already exists in math and in the case above should return a std::vector<std::complex<double>> and I'm pretty sure works with complex. If not then we can use some other function for promotion.

Do we allow for array int to be promoted to array real? I can't remember. But if so we should allow for array real to array complex

The Q I'm thinking about is where should that happen? Like looking at pp_expr for printing UDFs we have no knowledge of what the functions signature is. (link here.

  | FunApp (UserDefined (f, suffix), es) ->
      pp_user_defined_fun ppf (f, suffix, es)

So I think we would have to do this in transformed mir at the least right? Because then I have access to the functions block to lookup the signatures (or can I actually get the signature from UserDefined somehow).

What if we add a Promotion Expr type? Kind of a heavy handed answer, but I think the most likely to not fail and probably useful elsewhere. Another alt would be to keep the signature information around with UserDefined, which does seem like a good idea. Do you have any preferences or other ideas?

Also I just realized idt we can generate functions that take in an array of array of complex types?

[WardBrian]

Also I just realized idt we can generate functions that take in an array of array of complex types?

Why not?

Do we allow for array int to be promoted to array real? I can't remember. But if so we should allow for array real to array complex

We don't, but we should. It's semantically valid, and we allow it during assignment, just not during function calls. Long been on @bob-carpenter's wishlist

I think that generating a CompilerInternal Promotion type isn't a terrible idea. We will need to do it with any backend that doesn't have the same covariance/contravariances as ours, and in others like python it would just be a noop. I wouldn't target array int -> array real immediately, but we should keep it in mind with the way we do this IMO. I'm not 100% sure, but I would also guess this may be helpful for some parts of tuples (@rybern ?)

[SteveBronder]

Also I just realized idt we can generate functions that take in an array of array of complex types?

Why not?

How do you write the signature? I think the C++ always gens const std::vector<std::complex<T1__>>& no? Maybe I just don't know the syntax

I'll take a stab at the compiler internal promotion type tmrw. That feels like the cleanest way imo

[WardBrian]

Does void foo(array[,,] complex zs){...} not codegen at the moment? That sounds like a whole other kind of issue if so

Edit: Compiles to

template <typename T0__>
void
foo(const std::vector<std::vector<std::vector<std::complex<T0__>>>>& zs,
    std::ostream* pstream__) {
...

seems fine?

[SteveBronder]

Oh whoops nvm!

[SteveBronder]

Working on this rn, but just a sanity check, should we allow for promotions from double to complex for UDF functions in the language? tmk everything in the math library has explicit function signatures for combinations of double and complex. For UDFs, it feels kind of inefficient to promote a double to a complex. Though of course that requires folks to write two different functions. I think its sort of a question on whether we should force users to write things efficiently or be nice and allow implicit promotions. For instance a promotion from std::vector<double> to std::vector<complex> is pretty costly!

It almost feels like when we do this we should also throw a warning in line of something like,

Warning on line ____ for function ___: Stan will perform implicit promotion for input X 
  to type Y in this function, but this is expensive and it is recommended to make an
  explicit function signature

[WardBrian]

I think users will just call to_complex on their arguments if we don’t do this automatically. It’s also a bit of a consistency thing with int/real, since we’re just treating complex as another scalar type

[WardBrian]

Reading your message again: I don’t think we need to do conversation from real[] to complex[] in this PR unless you’d really like to - we don’t do it for int [] to real[] during function calls at the moment anyway (but we do let you assign up, which should probably be the same here)

[SteveBronder]

The backend for this is pretty much the same whether I do the big or small thing so I'm just going to the full promotion thing

[bob-carpenter]

Yes! We should be promoting int to real and real to complex (and by transitivity, int to complex). We then want to make sure all of our containers are covariant in their types. That means that array[...] int should promote to array[...] real and that to array[...] complex.

When we do vectors and matrices, we'll want those to have covariance, too, so that vector can be assigned to complex_vector and so on.

We also need to promote during brace construction so that this works:

array[2] complex a = {1 + 2i, 3.0};

I'm not sure we ever extended the array construction logic to complex numbers.

[bob-carpenter]

P.S. We should not be throwing a warning in any of these situations any more than we should warn people about promoting integers to reals or reals to complex. There's probably not going to be a whole lot of places where people are going to want to promote real containers to complex containers if we get all the heterogeneous operators working.

[WardBrian]

Array literal promotion for complex works in Stan, I’ll need to check that the C++ compiles and that we have it in the test

[WardBrian]

Superseded by #1004