Merge pull request #526 from asgrim/435-prompt-to-install-missing-system-deps

435: prompt to install missing system deps

Author: asgrim

.noai

@@ -2,11 +2,12 @@
 
 ## What is PIE?
 
-PIE is a new installer for PHP extensions, intended to eventually replace PECL.
-It is distributed as a [PHAR](https://www.php.net/manual/en/intro.phar.php),
-just like Composer, and works in a similar way to Composer, but it installs PHP
-extensions (PHP Modules or Zend Extensions) to your PHP installation, rather
-than pulling PHP packages into your project or library.
+PIE is the official installer for PHP extensions, which replaces
+[PECL](https://pecl.php.net/) (which is now deprecated). PIE is distributed as a
+[PHAR](https://www.php.net/manual/en/intro.phar.php), just like Composer, and
+works in a similar way to Composer, but it installs PHP extensions (PHP Modules
+or Zend Extensions) to your PHP installation, rather than pulling PHP packages
+into your project or library.
 
 # Using PIE - what do I need to get started?
 
@@ -15,12 +16,8 @@ than pulling PHP packages into your project or library.
 You will need PHP 8.1 or newer to run PIE, but PIE can install an extension to
 any other installed PHP version.
 
-On Linux, you will need a build toolchain installed. On Debian/Ubuntu type
-systems, you could run something like:
-
-```shell
-sudo apt install gcc make autoconf libtool bison re2c pkg-config php-dev
-```
+On Linux/OSX, if any build tools needed are missing, PIE will ask if you would
+like to automatically install them first (this is a new feature in 1.4.0).
 
 On Windows, you do not need any build toolchain installed, since PHP extensions
 for Windows are distributed as pre-compiled packages containing the extension
@@ -38,7 +35,9 @@ Further installation details can be found in the [usage](./docs/usage.md) docs.
 This documentation assumes you have moved `pie.phar` into your `$PATH`, e.g.
 `/usr/local/bin/pie` on non-Windows systems or created an alias in your shell RC file.
 
-## Installing a single extension using PIE
+## Using PIE
+
+### Installing a single extension using PIE
 
 You can install an extension using the `install` command. For example, to
 install the `example_pie_extension` extension, you would run:
@@ -57,7 +56,7 @@ You must now add "extension=example_pie_extension" to your php.ini
 $
 ```
 
-## Installing all extensions for a PHP project
+### Installing all extensions for a PHP project
 
 When in your PHP project, you can install any missing top-level extensions:
 
@@ -87,6 +86,12 @@ The following packages may be suitable, which would you like to install:
 Finished checking extensions.
 ```
 
+> [!TIP]
+> If you are running PIE in a non-interactive shell (for example, CI, a
+> container), pass the `--allow-non-interactive-project-install` flag to run
+> this command. It may still fail if more than one PIE package provides a
+> particular extension.
+
 ## Extensions that support PIE
 
 A list of extensions that support PIE can be found on
@@ -105,6 +110,6 @@ A list of extensions that support PIE can be found on
 If you are an extension maintainer wanting to add PIE support to your extension,
 please read [extension-maintainers](./docs/extension-maintainers.md).
 
-## More documentation...
+# More documentation...
 
 The full documentation for PIE can be found in [usage](./docs/usage.md) docs.

README.md

@@ -46,6 +46,7 @@
         "bnf/phpstan-psr-container": "^1.1",
         "doctrine/coding-standard": "^14.0.0",
         "phpstan/phpstan": "^2.1.38",
+        "phpstan/phpstan-phpunit": "^2.0",
         "phpstan/phpstan-webmozart-assert": "^2.0",
         "phpunit/phpunit": "^10.5.63"
     },

composer.json

@@ -345,11 +345,16 @@ The list of accepted OS families: "windows", "bsd", "darwin", "solaris", "linux"
 
 #### Extension dependencies
 
-Extension authors may define some dependencies in `require`, but practically,
+Extension authors may define some dependencies in `require`, but typically,
 most extensions would not need to define dependencies, except for the PHP
-versions supported by the extension. Dependencies on other extensions may be
-defined, for example `ext-json`. However, dependencies on a regular PHP package
-(such as `monolog/monolog`) SHOULD NOT be specified in your `require` section.
+versions supported by the extension, and system libraries.
+
+Dependencies on a regular PHP package (such as `monolog/monolog`) SHOULD NOT be
+specified in your extension's `require` section.
+
+##### Dependencies on other extensions
+
+Dependencies on other extensions may be defined, for example `ext-json`.
 
 It is worth noting that if your extension does define a dependency on another
 dependency, and this is not available, someone installing your extension would
@@ -360,6 +365,47 @@ Cannot use myvendor/myextension's latest version 1.2.3 as it requires
 ext-something * which is missing from your platform.
 ```
 
+##### System Library Dependencies
+
+In PIE 1.4.0, the ability for extension authors to define system library
+dependencies was added, and in some cases, automatically install them.
+
+The following libraries are supported at the moment. **If you would like to add
+a library, please [open a discussion](https://github.com/php/pie/discussions)
+in the first instance.** Don't just open a PR without discussing first please!
+
+We are adding libraries and improving this feature over time. If the automatic
+install of a system dependency that is supported below in your package manager
+is NOT working, then please [report a bug](https://github.com/php/pie/issues).
+
+| Library       | Checked by PIE | Auto-installs in   |
+|---------------|----------------|--------------------|
+| lib-curl      | ✅              | apt, apk, dnf, yum |
+| lib-enchant   | ✅              | ❌                  |
+| lib-enchant-2 | ✅              | ❌                  |
+| lib-sodium    | ✅              | apt, apk, dnf, yum |
+| lib-ffi       | ✅              | apt, apk, dnf, yum |
+| lib-xslt      | ✅              | apt, apk, dnf, yum |
+| lib-zip       | ✅              | apt, apk, dnf, yum |
+| lib-png       | ✅              | ❌                  |
+| lib-avif      | ✅              | ❌                  |
+| lib-webp      | ✅              | ❌                  |
+| lib-jpeg      | ✅              | apt, apk, dnf, yum |
+| lib-xpm       | ✅              | ❌                  |
+| lib-freetype2 | ✅              | ❌                  |
+| lib-gdlib     | ✅              | ❌                  |
+| lib-gmp       | ✅              | ❌                  |
+| lib-sasl      | ✅              | ❌                  |
+| lib-onig      | ✅              | ❌                  |
+| lib-odbc      | ✅              | ❌                  |
+| lib-capstone  | ✅              | ❌                  |
+| lib-pcre      | ✅              | ❌                  |
+| lib-edit      | ✅              | ❌                  |
+| lib-snmp      | ✅              | ❌                  |
+| lib-argon2    | ✅              | ❌                  |
+| lib-uriparser | ✅              | ❌                  |
+| lib-exslt     | ✅              | ❌                  |
+
 #### Checking the extension will work
 
 First up, you can use `composer validate` to check your `composer.json` is

composer.lock

@@ -281,6 +281,31 @@ pie install example/some-extension --with-some-library-name=/path/to/the/lib
 pie install example/some-extension --with-some-library-name=/path/to/the/lib --enable-some-functionality
 ```
 
+### Build tools check
+
+PIE will attempt to check the presence of build tools (such as gcc, make, etc.)
+before running. If any are missing, an interactive prompt will ask if you would
+like to install the missing tools. If you are running in non-interactive mode
+(for example, in a CI pipeline, container build, etc), PIE will **not**
+install these tools automatically. If you would like to install the build tools
+in a non-interactive terminal, pass the `--auto-install-build-tools` and the
+prompt will be skipped.
+
+To skip the build tools check entirely, pass the `--no-build-tools-check` flag.
+
+### System library dependencies check
+
+PIE will attempt to check the presence of system library dependencies before
+installing an extension. If any are missing, an interactive prompt will ask if
+you would like to install the missing tools. If you are running in
+non-interactive mode (for example, in a CI pipeline, container build, etc), PIE
+will **not** install these dependencies automatically. If you would like to
+install the system dependencies in a non-interactive terminal, pass the
+`--auto-install-system-dependencies` and the prompt will be skipped.
+
+To skip the dependencies check entirely, pass the
+`--no-system-dependencies-check` flag.
+
 ### Configuring the INI file
 
 PIE will automatically try to enable the extension by adding `extension=...` or

docs/extension-maintainers.md

@@ -288,6 +288,12 @@ parameters:
 			count: 1
 			path: src/Platform.php
 
+		-
+			message: '#^Parameter \#1 \$callback of function array_map expects \(callable\(Composer\\Package\\BasePackage\)\: mixed\)\|null, Closure\(Composer\\Package\\CompletePackageInterface\)\: Php\\Pie\\DependencyResolver\\Package given\.$#'
+			identifier: argument.type
+			count: 1
+			path: src/Platform/InstalledPiePackages.php
+
 		-
 			message: '#^Call to function array_key_exists\(\) with 1 and array\{non\-falsy\-string, non\-empty\-string, non\-empty\-string\} will always evaluate to true\.$#'
 			identifier: function.alreadyNarrowedType
@@ -354,18 +360,6 @@ parameters:
 			count: 1
 			path: test/integration/Command/InstallCommandTest.php
 
-		-
-			message: '#^Parameter \#1 \$originalClassName of method PHPUnit\\Framework\\TestCase\:\:createMock\(\) expects class\-string\<object\>, string given\.$#'
-			identifier: argument.type
-			count: 1
-			path: test/integration/Command/InstallExtensionsForProjectCommandTest.php
-
-		-
-			message: '#^Unable to resolve the template type RealInstanceType in call to method PHPUnit\\Framework\\TestCase\:\:createMock\(\)$#'
-			identifier: argument.templateType
-			count: 1
-			path: test/integration/Command/InstallExtensionsForProjectCommandTest.php
-
 		-
 			message: '#^Call to function assert\(\) with true will always evaluate to true\.$#'
 			identifier: function.alreadyNarrowedType

docs/usage.md

@@ -2,6 +2,7 @@ includes:
     - phpstan-baseline.neon
     - vendor/bnf/phpstan-psr-container/extension.neon
     - vendor/phpstan/phpstan-webmozart-assert/extension.neon
+    - vendor/phpstan/phpstan-phpunit/extension.neon
 
 parameters:
     level: 10

phpstan-baseline.neon

@@ -11,17 +11,19 @@
 use Php\Pie\ComposerIntegration\PieComposerRequest;
 use Php\Pie\ComposerIntegration\PieOperation;
 use Php\Pie\DependencyResolver\BundledPhpExtensionRefusal;
+use Php\Pie\DependencyResolver\DependencyInstaller\PrescanSystemDependencies;
 use Php\Pie\DependencyResolver\DependencyResolver;
 use Php\Pie\DependencyResolver\InvalidPackageName;
 use Php\Pie\DependencyResolver\UnableToResolveRequirement;
 use Php\Pie\Installing\InstallForPhpProject\FindMatchingPackages;
+use Php\Pie\Platform\PackageManager;
 use Php\Pie\SelfManage\BuildTools\CheckAllBuildTools;
-use Php\Pie\SelfManage\BuildTools\PackageManager;
 use Psr\Container\ContainerInterface;
 use Symfony\Component\Console\Attribute\AsCommand;
 use Symfony\Component\Console\Command\Command;
 use Symfony\Component\Console\Input\InputInterface;
 use Symfony\Component\Console\Output\OutputInterface;
+use Throwable;
 
 use function sprintf;
 
@@ -34,6 +36,7 @@ final class BuildCommand extends Command
     public function __construct(
         private readonly ContainerInterface $container,
         private readonly DependencyResolver $dependencyResolver,
+        private readonly PrescanSystemDependencies $prescanSystemDependencies,
         private readonly ComposerIntegrationHandler $composerIntegrationHandler,
         private readonly FindMatchingPackages $findMatchingPackages,
         private readonly IOInterface $io,
@@ -88,6 +91,22 @@ public function execute(InputInterface $input, OutputInterface $output): int
             ),
         );
 
+        if (CommandHelper::shouldCheckSystemDependencies($input)) {
+            try {
+                ($this->prescanSystemDependencies)(
+                    $composer,
+                    $targetPlatform,
+                    $requestedNameAndVersion,
+                    CommandHelper::autoInstallSystemDependencies($input),
+                );
+            } catch (Throwable $anything) {
+                $this->io->writeError(
+                    '<comment>Skipping system dependency pre-scan due to exception:</comment> ' . $anything->getMessage(),
+                    verbosity: IOInterface::VERBOSE,
+                );
+            }
+        }
+
         try {
             $package = ($this->dependencyResolver)(
                 $composer,

phpstan.neon

@@ -63,6 +63,8 @@ final class CommandHelper
     private const OPTION_NO_CACHE                             = 'no-cache';
     private const OPTION_AUTO_INSTALL_BUILD_TOOLS             = 'auto-install-build-tools';
     private const OPTION_SUPPRESS_BUILD_TOOLS_CHECK           = 'no-build-tools-check';
+    private const OPTION_AUTO_INSTALL_SYSTEM_DEPENDENCIES     = 'auto-install-system-dependencies';
+    private const OPTION_SUPPRESS_SYSTEM_DEPENDENCIES_CHECK   = 'no-system-dependencies-check';
 
     private function __construct()
     {
@@ -154,6 +156,19 @@ public static function configureDownloadBuildInstallOptions(Command $command, bo
             'Do not perform the check to see if build tools are present on the system.',
         );
 
+        $command->addOption(
+            self::OPTION_AUTO_INSTALL_SYSTEM_DEPENDENCIES,
+            null,
+            InputOption::VALUE_NONE,
+            'If system dependencies missing, automatically install them, instead of prompting.',
+        );
+        $command->addOption(
+            self::OPTION_SUPPRESS_SYSTEM_DEPENDENCIES_CHECK,
+            null,
+            InputOption::VALUE_NONE,
+            'Do not perform the check to see if system dependencies are present on the system.',
+        );
+
         /**
          * Allows additional options for the `./configure` command to be passed here.
          * Note, this means you probably need to call {@see self::validateInput()} to validate the input manually...
@@ -267,6 +282,22 @@ public static function shouldCheckForBuildTools(InputInterface $input): bool
             || ! $input->getOption(self::OPTION_SUPPRESS_BUILD_TOOLS_CHECK);
     }
 
+    public static function autoInstallSystemDependencies(InputInterface $input): bool
+    {
+        return $input->hasOption(self::OPTION_AUTO_INSTALL_SYSTEM_DEPENDENCIES)
+            && $input->getOption(self::OPTION_AUTO_INSTALL_SYSTEM_DEPENDENCIES);
+    }
+
+    public static function shouldCheckSystemDependencies(InputInterface $input): bool
+    {
+        if (Platform::isWindows()) {
+            return false;
+        }
+
+        return ! $input->hasOption(self::OPTION_SUPPRESS_SYSTEM_DEPENDENCIES_CHECK)
+            || ! $input->getOption(self::OPTION_SUPPRESS_SYSTEM_DEPENDENCIES_CHECK);
+    }
+
     public static function requestedNameAndVersionPair(InputInterface $input): RequestedPackageAndVersion
     {
         $requestedPackageString = $input->getArgument(self::ARG_REQUESTED_PACKAGE_AND_VERSION);