[Performance] ORM and DI performance issues due to lack of cache

[guvra]

Tempest version

2.14

PHP version

8.4

Operating system

Linux

Description

TLDR

Tempest has huge performance issues related to the DI container and the database mapper/inspector.

Currently, when interacting with the container or the ORM, the data they resolve isn't kept in memory.

For example, if the method get of the container is called 1000 times with the same arguments, it will do the same calculations 1000 times instead of returning a cached value.

Recommended solution:

• ORM: caching the database metadata in memory (php) by default in dev mode, and in filesystem by default in production mode.
• Container: caching resolve dependencies in memory (php) at the very least, ideally in filesystem if it's technically possible.
• Being able to customize the cache adapter to use depending on the environment.

For example, Symfony uses the following caching system:

• Symfony uses a cache for Doctrine metadata and queries (documented here). The configuration allows to use any cache provider. I think it's none by default in dev mode, and filesystem by default in production mode.
• Symfony compiles a cached version of the DI container (documented here).

DI Component Performance

I used xdebug to profile the request:

The slowest part of the code is the get method of the GenericContainer class.
This method is called hundreds of thousands of time in this reproduction scenario.

This wouldn't be an issue if the container kept resolved dependencies in memory, but it doesn't.

Conclusion: the DI container must keep in memory (or store in cache) anything that it resolves.
The constant recalculations have a huge impact on performance.

To confirm this, I added some cache around the get method (by building a hash from the function arguments), the HTTP request is now served in ~200 ms instead of 650 ms (70% improvement) with 677 entities loaded.

ORM/Mapper Performance

The SelectModelMapper object uses the ModelInspector object to fetch information from the database.

These two objects don't keep anything in memory, so for example when mapping a collection of 677 entities, everything is recalculated 677 times.

Conclusion: these objects (and more generally all database components) need to keep in memory (or store in cache) anything that they resolve.

For example, I added some cache around the methods getRelations, getBelongsTo, getHasOne, getAsMany and getTableDefinition. The HTTP response is now served in 60ms instead of 200ms (and 650ms before the DI container optimization). This is now nearly as fast as when I do not use the ORM! (only 10 ms slower)

Unfortunately I can't provide a PR because I only did some quick and dirty hacks in order to confirm that these components were indeed the reason behind the slow performance.

Steps to reproduce

Create two tables that depend on each other.
For example:

• product: id name, description, price, category_id
• category: id name, description

Populate the product table with hundreds of items, the category table doesn't need to have many rows.

Then use the ORM to fetch all products:

public function __invoke(): Json
{
    $products = $this->productRepository->findByCategoryId(1);

    return new Json(['products' => $products]);
}

And the function in the repository:

public function findByCategoryId(int $categoryId): array
{
    /** @var Product[] */
    return query(Product::class)
        ->select()
        ->with('categories')
        ->where('category_id = ?', $categoryId)
        ->all();
}

On my dev environnement, the HTTP response time is ~650 ms with 677 rows.

If i don't use the ORM (e.g. by building the SQL query manually and not using the mapper), the response time is 50 ms instead (-92%).

[guvra]

Another possible optimization is to replicate Doctrine behavior for initializing relations.

Example of an object returned by the find method of Doctrine:

object(App\Entity\Record)#333 {
    ["id"]=>
    int(1)
    ["description"]=>
    string(30) "xxxxxxxxxxx"
    ["category"]=>
    lazy ghost object(App\Entity\Category)#350 {            <---- relation is lazy-loaded
      ["id"]=>
      int(6)
      ["name"]=>
      uninitialized(string)
      ["description"]=>
      uninitialized(string)
    }
}

Doctrine lazy loads relations, so the object isn't actually instanciated until it is used.

[innocenzi]

Would happily accept performance PRs (on 3.x)! Even if incomplete. Thanks for the detailed issue, we definitely should fix this before next release.

[innocenzi]

Also related: #562