Soft-deprecating outdated helper macros

[encukou]

In a follow-up to #86,

I propose formally soft-deprecating helper macros which have equivalents in C99.
We won't be updating them, and we won't be using them as precedent for adding similar macros.
I propose undefining them internally (with Py_BUILD_CORE), and switch away if we still use them.

• Py_LL & Py_ULL: Use LL/ULL suffixes.
• Py_MEMCPY (already soft-deprecated)
• Py_VA_COPY: Use va_copy directly.
• PY_LONG_LONG, PY_LLONG_MIN, PY_LLONG_MAX, PY_ULLONG_MAX, PY_INT32_T, PY_UINT32_T, PY_INT64_T, PY_UINT64_T, PY_SIZE_MAX: Use C99 types/limits
• Py_UNICODE_SIZE: Use sizeof(wchar_t) directly
• PY_FORMAT_SIZE_T: Use "z" directly
• Py_BUILD_ASSERT: Use _Static_assert (C11) or static_assert (C++ & C23) instead
• Py_ALIGNED: Use _Alignas (C11) or alignas (C++ & C23) instead.
  • This one should be undefined internally (with Py_BUILD_CORE); we should switch to the more robust _Py_ALIGNED_DEF instead.

Additionally I propose soft-deprecating the following:

• Py_BUILD_ASSERT_EXPR: In virtually all cases, Py_BUILD_ASSERT is enough.
  In CPython it's only used to define Py_BUILD_ASSERT (for old compilers) and Py_ARRAY_LENGTH (which can do without it).
• PY_DWORD_MAX: Not a good precedent for public API. CPython internals should continue using it.

For Py_UNICODE_WIDE, which is scheduled for removal, I propose soft-deprecating instead. There's been no visible warning outside docs, and removing an #ifdef macro is indistinguishable from making it “false”.

---

I consider this clerical clean-up so I haven't started a wider discussion. Let me know if you think any of this is controversial.

---

• @encukou
• @erlend-aasland
• @serhiy-storchaka
• @vstinner
• @zooba

[encukou]

ping @serhiy-storchaka @vstinner @zooba

[zooba]

I disagree with deprecating all the ones you proposed that are followed by "use ... or ... instead". If there's a reasonable choice like that - particularly the C vs C++ ones - I think we should keep papering over those differences if we're using them in public headers.

I think the precedent here is "used in our public headers and potentially different across platforms/tools". We don't have to start adding full sets of max values just because we did it for the one we needed, but equally we shouldn't have to litter our headers with inline preprocessor checks (and users who clone from our headers shouldn't either) rather than having an alignment macro.

"There are equivalents in C99" isn't a sufficient justification, IMHO.

[encukou]

I disagree with deprecating all the ones you proposed that are followed by "use ... or ... instead"

There are two of those.

Py_ALIGNED is bad on its own; it silently does nothing if you're not on GCC(-like) compiler.
We now use _Py_ALIGNED_DEF instead: it's more relaible, but needs a dufferent syntax. Also, we added it as private API -- was that a mistake?
I don't think it was. New code should use the standard spelling, unless it has specific, increasingly rare, requirement to support an older compiler (without extensions).

Py_BUILD_ASSERT is not used in public headers.

In both cases, you can include a standard header to paper over the C11 vs C++ difference.

used in our public headers and potentially different across platforms/tools

Should we re-evaluate making pyatomic.h public? That's similar to alignments or long long in 2000.

[zooba]

Should we re-evaluate making pyatomic.h public?

I think I lost the argument on that one already, so perhaps I just deserve to lose the argument here as well? My concern there was the depth and breadth - they aren't just renaming compile-time keywords, whereas here we're really just taking a range of non-standard compiler names and giving them one of our own.

Py_ALIGNED is bad on its own; it silently does nothing if you're not on GCC(-like) compiler.

If it's bad, then let's hard deprecate it. I remember the different syntax discussion now, and maybe the new one should've been public as we use it in public structs? (My idea of "what embedders may need to do" has shifted over the last few months, which is why I'm considering it more likely that they'll have to extract/copy struct definitions out of our headers, and e.g. may be recompiling CPython without necessarily rebuilding all the 3rd party extensions).

Py_BUILD_ASSERT is not used in public headers.

It's also not documented, which means we would only update it for our own needs, but would accept PRs to fill in gaps without prioritising bugs that don't affect us. So, it seems like a one-liner (okay, five-ish) that we can just leave alone? Much like the others? We can set a policy with a comment - we don't have to make behavioural changes to avoid treating old precedent as current.

[vstinner]

Please leave Py_BUILD_ASSERT() and Py_BUILD_ASSERT_EXPR() unchanged (no deprecation). It's more convenient that _Static_assert() which requires an error message.

Is it really an issue to provide PY_DWORD_MAX? I would prefer to not deprecate it.

I propose undefining them internally (with Py_BUILD_CORE), and switch away if we still use them.

I don't think that it's worth it to make header files more complicated just to ensure that we don't use these macros/functions internally.

[encukou]

OK, leaving controversial stuff out of this proposal:

• undefining for internal use
• keep Py_BUILD_ASSERT, Py_BUILD_ASSERT_EXPR, PY_DWORD_MAX

If [Py_ALIGNED]'s bad, then let's hard deprecate it.

Yeah, but... it's a macro; raising a deprecation warning for it is problematic. You can view soft-deprecation as an easy first step -- with the caveat that I won't be taking the next one.

---

just taking a range of non-standard compiler names and giving them one of our own.

That's exactly what pyatomic does, though. (Well, it doesn't exclude standard names, but neither do the macros like _Py_ALIGNED_DEF/Py_ALIGNED.)

(My idea of "what embedders may need to do" has shifted over the last few months, which is why I'm considering it more likely that they'll have to extract/copy struct definitions out of our headers, and e.g. may be recompiling CPython without necessarily rebuilding all the 3rd party extensions).

But if they do that, they need the macro definitions as well -- and I'd say that if they're copying struct definitions, then extracting private macros (without re-exposing them to users) is fair game.

FWIW, this is an argument to get macros (and other complications) out of our headers as soon as we can -- I expect a generic extraction tool would happier with uint32_t than PY_UINT32_T.

We can set a policy with a comment - we don't have to make behavioural changes to avoid treating old precedent as current.

I'm not proposing behavioural changes. Soft-deprecation is mostly about documentation -- which is how I ended up here: when writing the docs, I found myself recommending something else instead, and I want to make that official.

[vstinner]

I'm fine with soft deprecating the list (current) list proposed by @encukou.

[zooba]

Yeah, the reduced list is okay by me.

just taking a range of non-standard compiler names and giving them one of our own.

That's exactly what pyatomic does, though.

Only for platforms/tools where they exist, and on others I don't recall whether it would fail or a no-op, but I don't remember us reaching enough agreement to make them part of our public API.

But if they do that, they need the macro definitions as well -- and I'd say that if they're copying struct definitions, then extracting private macros (without re-exposing them to users) is fair game.

It's more that we can change private macros whenever we like, and that's painful for people that have had to copy code out for some reason. If the entire struct is private API, sure, but public API structs should use public API macros.

[encukou]

ping @erlend-aasland @serhiy-storchaka

[encukou]

Only for platforms/tools where they exist, and on others I don't recall whether it would fail or a no-op

#  error "no available pyatomic implementation for this platform/compiler"

---

Anyway, for embedders: I think I get the use case; I don't necessarily agree on your solution. But, we don't need to agree on that here & now.

[encukou]

ping @erlend-aasland @serhiy-storchaka

[encukou]

Thanks for the vote!
@erlend-aasland @serhiy-storchaka are inactive & count as abstaining.

I've opened this as python/cpython#146175.