Statically initialize PyArg_Parser variables in clinic.py.

[ericsnowcurrently]

https://bugs.python.org/issue46772

Changes will be in Tools/clinic/clinic.py.

[ericsnowcurrently]

blocked by #285 and #287

[kumaraditya303]

I have a rough plan of how this can be implemented, and to remove code duplication between deepfreeze and clinic, it can be factored into a separate module.

Clinic can generate the static tuple object and since the keywords are identifiers, it can use _Py_ID macro to declare the keywords in the ob_item array. Since there won't be any changes to the generated tuple object, it can be even be made static const if possible to move to the read only section of the executable which will be faster, parser_init could be made to check the generated tuple in debug mode and in release builds it will be simply removed which should make parsing overhead less as it would avoid a branch, and finally the parsers would not have to be deallocated simplifying runtime shutdown and this would be subinterpreter friendly.

[kumaraditya303]

@ericsnowcurrently Have you started working on this or you have a draft branch with using internal APIs?

[ericsnowcurrently]

Yeah, I've been working on this. However, feel free to take over! 🙂 It sounds like you have a similar understanding to mine on what needs to be done. I have plenty of context at this point and would be glad to talk through any of the details with you. I can also show you my branch if you like.

[kumaraditya303]

I can also show you my branch if you like.

Yes, I would like to take a look. thanks

[ericsnowcurrently]

to remove code duplication between deepfreeze and clinic, it can be factored into a separate module.

FWIW, I agree that there is significant overlap between clinic and deepfreeze (as well as other tools like generate_frozen_modules.py). Factoring our common parts may be a good idea. Just remember that the tools aren't as carefully tested as the CPython runtime and that it's easy to introduce bugs when refactoring (especially large-scale refactoring). That doesn't mean it's a bad idea. We just need to be consider the possible downsides seriously.

I'll also note that the approach clinic takes a substantially different approach for code generation compared to deepfreeze. Deepfreeze generates the output as it goes, with some structural helpers to simplify the process (@gvanrossum's design). In contrast, clinic does the following (more or less):

1. parse input
2. generate a template
3. apply the parsed data to the template

The clinic approach makes it much harder to generate concise code for things like statically initialized _PyArg_Parser structs. The deepfreeze approach is much better suited, so from that standpoint I like the idea of refactoring clinic to share the deepfreeze approach. I'd even considered it briefly. However, it looked like way more work than I have time to spare. 🙂

If you want to give it a shot then I say go for it. Just don't be afraid to fight through the challenge. And don't be afraid to abandon the change if it starts looking like it's too much to handle or if the level of churn to too high relative to the benefit. (You can sometimes get away with a little more churn in tools than in runtime code, but don't push it.)

[ericsnowcurrently]

I can also show you my branch if you like.

Yes, I would like to take a look. thanks

python/cpython@main...ericsnowcurrently:global-objects-pyargs-parser-keywords-init

I've temporarily dropped all generated code from the branch to make it easier to see the actual changes.

[ericsnowcurrently]

Also, let's move any further discussion on this to https://bugs.python.org/issue46772.