Consider restoring _PyLong_New() function as public

[skirpichev]

This was used to streamline conversion between ints and gmp-based integers. E.g. in the gmpy2: https://github.com/aleaxit/gmpy/blob/e0944795008e6929c3d6cdfabb8344ce8a9e99d6/src/gmpy2_convert_gmp.c#L148-L181 or in the Sage: https://github.com/sagemath/sage/blob/2f1a76dc24af6c8ca9a804b10913c33a936b4fb8/src/sage/libs/gmp/pylong.pyx#L57-L69

Clearly, we want a better C-API e.g. for import/export ints, like mpz_import/export (probably, #102471 is the right metaissue here), but I don't see good reasons to hide this function now, without any alternatives.

See #106320 and #108604.

[skirpichev]

CC @vstinner

[vstinner]

GMP import and export functions: https://gmplib.org/manual/Integer-Import-and-Export

[vstinner]

cc @serhiy-storchaka

[vstinner]

I don't think that accessing directly PyLongObject members and having a public "PyLong_New" API to create an uninitialized integer is a good pattern: see capi-workgroup/problems#56

I would prefer to have a generic API to create a Python int object from any format, with a convenient, but generic enough, "import" API.

https://github.com/aleaxit/gmpy/blob/e0944795008e6929c3d6cdfabb8344ce8a9e99d6/src/gmpy2_convert_gmp.c#L148-L181

This function calls also _PyLong_SetSignAndDigitCount(), not only _PyLong_New().

https://github.com/sagemath/sage/blob/2f1a76dc24af6c8ca9a804b10913c33a936b4fb8/src/sage/libs/gmp/pylong.pyx#L57-L69

This function calls also Py_SET_SIZE(L, -pylong_size) to sets the integer sign (make it negative).

The problem is to have a generic but efficient API. A generic API would be to:

• (1) Convert a GMP integer into an "array of digits" allocated on the stack or on the heap (depending on the size)
• (2) Create a Python integer from this array
• (3) Delete the temporary array

The problem is to avoid (1) and (3) steps.

[skirpichev]

I don't think that accessing directly PyLongObject members and having a public "PyLong_New" API to create an uninitialized integer is a good pattern

No, that's not what was proposed. Sure, I would prefer a better alternative: mpz_export/import-like pair + functions to get/set the sign of int and to get the length of an "array of digits". Something like #102471. (see also @casevh comment)

The point is: we shouldn't hide _PyLong_New() without introducing an alternative in 3.13.

This function calls also _PyLong_SetSignAndDigitCount(), not only _PyLong_New().

Yes. GMPy_MPZ_From_PyLong() and GMPy_PyLong_From_MPZ() are typical consumers of this "_API".

PS: In principle, I think some parts like _PyLong_IsNegative() and _PyLong_DigitCount() could be exposed as public (or as an unstable API at least) right now, but this doesn't make too much sense without PyInt_Import/Export() functions...

[vstinner]

The _PyLong_New() function was restored by #112115

Should others functions such as _PyLong_SetSignAndDigitCount(), _PyLong_IsNegative() and _PyLong_DigitCount() be restored as well?

[skirpichev]

They are copied in the gmpy2, yes. (See https://github.com/aleaxit/gmpy/blob/1edb0369690cce69ec5d90964eaead31a57e7a9b/src/gmpy2_convert.h#L135-L155) And to sage: sagemath/sage@dc71bd0

[skirpichev]

I don't think that accessing directly PyLongObject members and having a public "PyLong_New" API to create an uninitialized integer is a good pattern: see capi-workgroup/problems#56

@vstinner, do you think that mpz_limbs_read/write-like API to access unsigned value of "big enough" PyLongObject as an array of "digits" is an example of this bad pattern or it does make sense and could be discussed somewhere?

With this kind of API (probably much less complex than PyInt_Import/Export) on the CPython side - Sage/gmpy2/python-flint/etc could do fast conversion to/from PyLong's using mpz_import/export functions.

[vstinner]

@vstinner, do you think that mpz_limbs_read/write-like API to access unsigned value of "big enough" PyLongObject as an array of "digits" is an example of this bad pattern or it does make sense and could be discussed somewhere?

Does the new PyLong_FromNativeBytes() API fit your needs? https://docs.python.org/dev/c-api/long.html#c.PyLong_FromNativeBytes