PHP 8.1 compatibility

[emteknetnz]

Split off from #10219

Upgrade all the support modules to support php 8.1

Agreed way forward to resolve common errors #10237 (comment)

Reference to some breaking changes in php 8.1 - https://stitcher.io/blog/new-in-php-81#breaking-changes

Note

Steve created some science experiment to add a _a method that can note docblock/parameter type mismatched and a _c method that can recast variable to their expected type. He also has a script to apply it to the entire code base.

Update: this approach won't work for php 8.1. Reason being that the _a() method profiling will certainly detect the say all the string and null arguments being passed in for a param, we cannot determine whether or not null is valid in this instance or not, so we don't know if _c('string', $myvar, 1) should be included or not

Acceptance criteria

• PHP8.1 build are green.
• We don't break semver.
• The magical process for auto updating our code base is documented somewhere. (it's in this issue)
• CI fails on php deprecation warnings
• graphql4 is also updated
• Ensure an installable version of php 8.1 compatible tractorcow deps is available - see ENH PHP 8.1 compatibility tractorcow/classproxy#5 (comment)

Explanation of the conversion process in the PRs below

• Automatic code modification using https://github.com/emteknetnz/php81-fixer
• Static code analysis that ensures any arguments passed to native php functions are not null
  - If the native param only accepts a single type of (string|bool|int|float) then it is cast i.e (string)
  - If the native param only accepts an array then it uses the ternary operator i.e. ?: []
  - If the native param accepts multiple types then it uses the ternary operator with a bias to using string i.e. ?: ''
• Use the null coalescing operator for variables passed to native functions that do not accept null values
• There are some conditions where arguments are not converted such as the argument is a native constant, is an inner native function call with a single return type, is using the class static property e.g. SiteTree::class

Because this is an automated process this certainly goes on the side of over converting code. If we move to full-type hinting param and return type in the future, we could run another code sweep to remove a bunch of the code conversions. Currently we cannot rely on the docblock types as they are very unreliable.

There are also #[\ReturnTypeWIllChange] method attributes added to class methods where relevant, as well as some other methods that are updated, such as adding __serialized()/__unserialized()

Current PHP 8.1 unit test status

• https://github.com/emteknetnz/silverstripe-installer/actions
• https://github.com/emteknetnz/recipe-kitchen-sink/actions

Follow up PRs

• MNT Fail tests on warnings silverstripe-travis-shared#38

PRs

GraphQL 3 PRs

• DEP Upgrade webonyx/graphql-php to v14 silverstripe-graphql#454
• MNT Compatibility with webonyx/graphql-php v14 silverstripe-elemental#973
• MNT Compatibility with webonyx/graphql-php v14 silverstripe-asset-admin#1266
• MNT Compatibility with webonyx/graphql-php v14 silverstripe-versioned#359

"Manual fix" PRs

• ENH Fix deprecation issues for PHP 8.1 compatibility #10273
• ENH Fix deprecation issues for PHP 8.1 compatibility silverstripe-versioned#357
• ENH Fix deprecation issues for PHP 8.1 compatibility silverstripe-assets#484
• ENH Various fixes for PHP 8.1 compatibility silverstripe-assets#485
• ENH Various fixes for PHP 8.1 compatibility #10275
• ENH Various fixes for PHP 8.1 compatibility silverstripe-cms#2731
• ENH Various fixes for PHP 8.1 compatibility silverstripe-versioned#358
• ENH Various fixes for PHP 8.1 compatibility silverstripe-mfa#449
• ENH Various fixes for PHP 8.1 compatibility silverstripe-webauthn-authenticator#106
• ENH Various fixes for PHP 8.1 compatibility silverstripe-queuedjobs#370
• FIX Use composer update for prefer-lowest silverstripe-travis-shared#36
• ENH Various fixes for PHP 8.1 compatibility #10281
• ENH Various fixes for PHP 8.1 compatibility silverstripe-html5#36
• ENH Various fixes for PHP 8.1 compatibility silverstripe-blog#676
• ENH Various fixes for PHP 8.1 compatibility silverstripe-elemental#976
• NEW Use composer 2 bringyourownideas/silverstripe-composer-update-checker#62
• DEP Use v3 of silverstripe-composer-update-checker recipe-reporting-tools#26

solr-php-client fork

• DEP Use silverstripe account solr-php-client#1
• DEP Use silverstripe fork silverstripe-fulltextsearch#321

Installer (all prs use null coalesce)

• ENH PHP 8.1 compatibility silverstripe-admin#1294
• ENH PHP 8.1 compatibility silverstripe-asset-admin#1264
• ENH PHP 8.1 compatibility silverstripe-assets#483
• ENH PHP 8.1 compatibility silverstripe-behat-extension#212
• ENH PHP 8.1 compatibility silverstripe-campaign-admin#230
• ENH PHP 8.1 compatibility silverstripe-config#56
• ENH PHP 8.1 compatibility silverstripe-cms#2730
• ENH PHP 8.1 compatibility #10222
• ENH PHP 8.1 compatibility silverstripe-graphql#450 (graphql 3)
• ENH PHP 8.1 compatibility silverstripe-graphql#456 (graphql 4)
• ENH PHP 8.1 compatibility silverstripe-login-forms#112
• ENH PHP 8.1 compatibility silverstripe-mimevalidator#53
• ENH PHP 8.1 compatibility silverstripe-reports#148
• ENH PHP 8.1 compatibility silverstripe-serve#29
• ENH PHP 8.1 compatibility silverstripe-session-manager#104
• ENH PHP 8.1 compatibility silverstripe-testsession#73
• ENH PHP 8.1 compatibility silverstripe-versioned#356
• ENH PHP 8.1 compatibility silverstripe-versioned-admin#238
• ENH PHP 8.1 compatibility MinkFacebookWebDriver#18
• ENH PHP 8.1 compatibility recipe-plugin#20
• ENH PHP 8.1 compatibility vendor-plugin#52
• ENH PHP 8.1 compatibility silverstripe-gridfieldextensions#338

No changes

• silverstripe/silverstripe-errorpage
• silverstripe/silverstripe-siteconfig

Kitchen-sink (all prs use null coalesce)

• ENH PHP 8.1 compatibility silverstripe-akismet#40
• ENH PHP 8.1 compatibility silverstripe-auditor#42
• ENH PHP 8.1 compatibility silverstripe-blog#675
• ENH PHP 8.1 compatibility silverstripe-ckan-registry#266
• ENH PHP 8.1 compatibility silverstripe-comment-notifications#44
• ENH PHP 8.1 compatibility silverstripe-comments#338
• ENH PHP 8.1 compatibility silverstripe-contentreview#163
• ENH PHP 8.1 compatibility silverstripe-documentconverter#29
• ENH PHP 8.1 compatibility silverstripe-elemental-bannerblock#85
• ENH PHP 8.1 compatibility silverstripe-environmentcheck#76
• ENH PHP 8.1 compatibility silverstripe-externallinks#77
• ENH PHP 8.1 compatibility silverstripe-frameworktest#102
• ENH PHP 8.1 compatibility silverstripe-fulltextsearch#319
• ENH PHP 8.1 compatibility silverstripe-graphql-devtools#41
• ENH PHP 8.1 compatibility silverstripe-gridfieldqueuedexport#66
• ENH PHP 8.1 compatibility silverstripe-html5#35
• ENH PHP 8.1 compatibility silverstripe-hybridsessions#70
• ENH PHP 8.1 compatibility silverstripe-iframe#58
• ENH PHP 8.1 compatibility silverstripe-ldap#39
• ENH PHP 8.1 compatibility silverstripe-lumberjack#111
• ENH PHP 8.1 compatibility silverstripe-mfa#451
• ENH PHP 8.1 compatibility silverstripe-realme#83
• ENH PHP 8.1 compatibility silverstripe-registry#67
• ENH PHP 8.1 compatibility silverstripe-restfulserver#98
• ENH PHP 8.1 compatibility silverstripe-securityreport#54
• ENH PHP 8.1 compatibility silverstripe-segment-field#64
• ENH PHP 8.1 compatibility silverstripe-sharedraftcontent#164
• ENH PHP 8.1 compatibility silverstripe-sitewidecontent-report#51
• ENH PHP 8.1 compatibility silverstripe-spamprotection#82
• ENH PHP 8.1 compatibility silverstripe-spellcheck#50
• ENH PHP 8.1 compatibility silverstripe-subsites#478
• ENH PHP 8.1 compatibility silverstripe-tagfield#209
• ENH PHP 8.1 compatibility silverstripe-textextraction#72
• ENH PHP 8.1 compatibility silverstripe-totp-authenticator#89
• ENH PHP 8.1 compatibility silverstripe-userforms#1141
• ENH PHP 8.1 compatibility silverstripe-versionfeed#70
• ENH PHP 8.1 compatibility silverstripe-webauthn-authenticator#107
• ENH PHP 8.1 compatibility silverstripe-widgets#189
• ENH PHP 8.1 compatibility silverstripe-advancedworkflow#460
• ENH PHP 8.1 compatibility symbiote/silverstripe-multivaluefield#80
• ENH PHP 8.1 compatibility silverstripe-queuedjobs#371
• ENH PHP 8.1 compatibility bringyourownideas/silverstripe-maintenance#177
• ENH PHP 8.1 compatibility cwp-agencyextensions#72
• ENH PHP 8.1 compatibility cwp#309
• ENH PHP 8.1 compatibility cwp-core#106
• ENH PHP 8.1 compatibility cwp-pdfexport#23
• ENH PHP 8.1 compatibility cwp-search#42
• ENH PHP 8.1 compatibility silverstripe-elemental#974
• ENH PHP 8.1 compatibility silverstripe-gridfield-bulk-editing-tools#222
• ENH PHP 8.1 compatibility tractorcow/classproxy#5
• ENH PHP 8.1 compatibility tractorcow-farm/silverstripe-fluent#763

No changes

• content-widget
• crontask
• elemental-fileblock
• security-extensions
• taxonomy
• dnadesign/silverstripe-elemental-userforms
• tractorcow/silverstripe-proxy-db

PRs for dependencies

• Add method attribute for PHP 8.1 compatibility PTCInc/solr-php-client#8
  ^ if this isn't merged, then we should look to copy into thirdparty folder << update have moved to thirdparty folder in fulltextsearch module

[emteknetnz]

Honestly this is starting to look like a nightmare due to the sheer number of call to native php functions like str_replace, where suddently null values are no longer allowed

The core issue of this is that Silverstripe was originally written for PHP 5 before param types were a thing, so everything is mixed value

We basically need to cast params in a stunningly large number of places, e.g.:

silverstripe/framework Convert.php

     * @param string $path
     * @param string $separator
     * @param bool $multiple Collapses multiple slashes or not
     * @return string
     */
    public static function slashes($path, $separator = DIRECTORY_SEPARATOR, $multiple = true)
    {
        $path = (string) $path; // << we need to do this

Like wise we have the same issue with a lack of return type enforcement:

silverstripe/versioned Versioned.php

     * @return string
     */
    public static function get_reading_mode()
    {
        return self::$reading_mode;
    }

Which, despite what the docblock says, it can return null.

So any calls to this method can now fail, such as str_replace('abc', 'def', Versioned::get_reading_mode());

I think we might actually be better off being more ambitious here and converting ALL the docblock to actual param and return types. If not, were essentially stuck with the $path = (string) $path; casting of params instead.

If we're going to do all the work, I think we should do it properly, rather than this somewhat improper casting method

This would be a breaking change, though so is php 8.1

[emteknetnz]

For those not familiar with the issue here, when upgrading PHP 8.1 deprecation warnings are now thrown when passing null as a argument instead of the typed value. Since we cannot realistically turn of deprecation warnings on local environments, this leads to hundreds of deprecation warnings when doing local dev.

So in practice, PHP 8.1 is effectively a breaking API change itself:

PHP 8.0:
strpos(?string $haystack, ?string $needle, ?int $offset = 0): int|false

PHP 8.1:
strpos(string $haystack, string $needle, int $offset = 0): int|false

Options here are:
a) Use param casting method $myvar = (string) $myvar; and release as a minor of 4
b) Use param types which is a big API change, and release as 5.0 (new major)
c) Use param types which is a big API change, and release as a new minor of 4 (semver violation) - with a very large note in the changelog saying there's a high chance you'll get regressions, so really do test this upgrade

@silverstripe/contributing-committers
@maxime-rainville

[michalkleiner]

I'm for B) for the sole reason of (possibly) starting to release major versions faster so that we don't have to put major changes on the back burner for months or years until 5 finally comes.
C) feels risky as it may have impacts you wouldn't expect from a minor version change even with a huge note in the changelog.
A) is just obscuring the problem and letting it come back at us later, or creating a technical debt that will be hard to repay.

[lozcalver]

I think we’d need to carefully consider the implications of jumping into a 5.x release for this. If we want to get away from monolithic major releases then that’s great, but we need to consider the implications for things like modules, support timelines and community expectations about what a major release is. I don’t think any of those challenges are insurmountable of course, but if a 5.x release is suddenly on the table then it should probably be part of a wider discussion around versioning/releases. If we do decide to take that path then we’d need to decide which breaking changes to include - just the PHP compatibility? Or include embed v4 in that release instead of 4.x? Any other major pain points we can address at the same time? I honestly have no idea what the state of our master branch currently is, but forking a 5 branch from 4 instead of master is an option in the short-term.

Given the description of the issue in the comments above though, I don’t have any problem with option A. For core PHP functions we can cast arguments to their expected types, I can’t foresee any future problems or technical debt from that? For casting issues caused by core code, I think we just have to use our judgement. For the Versioned::get_reading_mode() example, I see two approaches:

1. We can cast the value returned anywhere we use it in core code, and users who upgrade to PHP 8.1 will have to do the same in their code (and module authors will have to do the same too)
2. We treat docblocks as the source of truth, and implement casting in core code. Whether this is a BC-break is a bit ambiguous given the docblocks, but we just have to use our judgement to figure out how likely it is to actually cause issues for people. For Versioned::get_reading_mode() my instinct is that this approach would be fine, but there may be other methods where it wouldn’t be

Also cc-ing @silverstripe/core-team because the contributing committers group is smaller 😉

[emteknetnz]

A) is feasible, and short term it's easier. My issues with it is that it's that it's still a significant amount of effort to implement, it's a frankly weird solution and it's kicking the can down the road

At some point we'll modernize the code and put types on everything anyway, so it seems like we should just do this step to begin with i.e. do it properly the first time around

If we do decide to take that path then we’d need to decide which breaking changes to include - just the PHP compatibility?

If we do a major I think it's important to try and limit the amount of changes we try to put in for a few reasons:

• The more we breaking changes we put in, the more upgrade pain experienced by end users, who will have the major upgrade forced on them rather than by choice
• 3 to 4 was a painful experience for many due to the many breaking changes, and many site owners had a very negative perception of the value of the major upgrade. With 4 to 5 it would be good if we could basically do the opposite
• People are leaning towards doing more frequent, smaller major upgrades, so if something misses the boat now, it's not lost forever.

[maxime-rainville]

I would like us to get to a point where we can release new majors again. But that would require us to have a wider discussion around what we put in major and what kind of changes we ship in there. My preference would be to take the Symfony approach where new majors are just removing deprecated API and paying down some technical debt ... so basically, transitioning from CMS4 to CMS5 should be a much smaller step than CMS3 to CMS4.

But realistically, there's no way we could be in a position to do this before the end of this year.

So the question is do we want to have official PHP8.1 support in the 4 release line? If the answer is yes, then we have to decide between A and C. Option C seems like a bridge too far ... basically it's like releasing CMS 5, but pretending we are not.

I could live with Option A ... and we have taken this general approach in the past, but maybe we shouldn't have.

I guess there's an Option D here which would be tell people to suppress deprecation warning if they want to use PHP8.1. Maybe, that can be a transitory step to Silverstripe CMS 5.