diff --git a/.github/workflows/blackbox.yml b/.github/workflows/blackbox.yml
index fa37e54f..e4ffd29d 100644
--- a/.github/workflows/blackbox.yml
+++ b/.github/workflows/blackbox.yml
@@ -32,6 +32,8 @@ jobs:
run: docker-compose up -d
- name: Run Blackbox with APC
run: docker-compose run phpunit env ADAPTER=apc vendor/bin/phpunit tests/Test/
+ - name: Run Blackbox with APCng
+ run: docker-compose run phpunit env ADAPTER=apcng vendor/bin/phpunit tests/Test/
- name: Run Blackbox with Redis
run: docker-compose run phpunit env ADAPTER=redis vendor/bin/phpunit tests/Test/
diff --git a/README.APCng.md b/README.APCng.md
new file mode 100644
index 00000000..dd536873
--- /dev/null
+++ b/README.APCng.md
@@ -0,0 +1,66 @@
+
+## Using either the APC or APCng storage engine:
+```php
+$registry = new CollectorRegistry(new APCng());
+ // or...
+$registry = new CollectorRegistry(new APC());
+
+// then...
+$counter = $registry->registerCounter('test', 'some_counter', 'it increases', ['type']);
+$counter->incBy(3, ['blue']);
+
+$renderer = new RenderTextFormat();
+$result = $renderer->render($registry->getMetricFamilySamples());
+```
+
+## Performance comparions vs the original APC engine
+The difference between `APC` and `APCng` is that `APCng` is re-designed for servers which have millions of entries in their APCu cache and/or receive hundreds to thousands of requests per second. Several key data structures in the original `APC` engine require repeated scans of the entire keyspace, which is far too slow and CPU-intensive for a busy server when APCu contains more than a few thousand keys. `APCng` avoids these scans for the most part, the trade-off being creation of new metrics is slightly slower than it is with the `APC` engine, while other operations are approximately the same speed, and collecting metrics to report is 1-2 orders of magnitude faster when APCu contains 10,000+ keys.
+In general, if your APCu cache contains over 1000 keys, consider using the `APCng` engine.
+In my testing, on a system with 100,000 keys in APCu and 500 Prometheus metrics being tracked, rendering all metrics took 35.7 seconds with the `APC` engine, but only 0.6 seconds with the `APCng` engine. Even with a tiny cache (50 metrics / 1000 APC keys), `APCng` is over 2.5x faster generating reports. As the number of APCu keys and/or number of tracked metrics increases, `APCng`'s speed advantage grows.
+The following table compares `APC` and `APCng` processing time for a series of operations, including creating each metric, incrementing each metric, the wipeStorage() call, and the collect() call, which is used to render the page that Prometheus scrapes. Lower numbers are better! Increment is the most frequently used operation, followed by collect, which happens every time Prometheus scrapes the server. Create and wipe are relatively infrequent operations.
+
+| Configuration | Create (ms) | Increment (ms) | WipeStorage (ms) | Collect (ms) | Collect speedup over APC |
+|--------------------------------|------------:|---------------:|-----------------:|-------------:|-------------------------:|
+| APC 1k keys / 50 metrics | n/t | n/t | n/t | 29.0 | - |
+| APC 10k keys / 50 metrics | 9.2 | 0.7 | 1.1 | 131.9 | - |
+| APC 100k keys / 50 metrics | 9.3 | 1.3 | 11.9 | 3474.1 | - |
+| APC 1M keys / 50 metrics | 12.7 | 1.4 | 19.2 | 4805.8 | - |
+| APC 1k keys / 500 metrics | n/t | n/t | n/t | 806.5 | - |
+| APC 10k keys / 500 metrics | 26.7 | 9.3 | 4.2 | 1770.9 | - |
+| APC 100k keys / 500 metrics | 44.8 | 13.1 | 16.6 | 35758.3 | - |
+| APC 1M keys / 500 metrics | 39.9 | 25.9 | 22.9 | 46489.1 | - |
+| APC 1k keys / 2500 metrics | n/t | n/t | n/t | n/t | n/t |
+| APC 10k keys / 2500 metrics | 196.7 | 95.1 | 17.6 | 24689.5 | - |
+| APC 100k keys / 2500 metrics | 182.6 | 82.0 | 34.4 | 216526.5 | - |
+| APC 1M keys / 2500 metrics | 172.7 | 93.3 | 38.3 | 270596.3 | - |
+| | | | | | |
+| APCng 1k keys / 50 metrics | n/t | n/t | n/t | 11.1 | 2.6x |
+| APCng 10k keys / 50 metrics | 8.6 | 0.6 | 1.3 | 15.2 | 8.6x |
+| APCng 100k keys / 50 metrics | 10.1 | 1.0 | 11.7 | 69.7 | 49.8x |
+| APCng 1M keys / 50 metrics | 10.4 | 1.3 | 17.3 | 100.4 | 47.9x |
+| APCng 1k keys / 500 metrics | n/t | n/t | n/t | 108.3 | 7.4x |
+| APCng 10k keys / 500 metrics | 25.2 | 7.2 | 5.9 | 118.6 | 14.9x |
+| APCng 100k keys / 500 metrics | 55.0 | 12.3 | 18.6 | 603.9 | 59.2x |
+| APCng 1M keys / 500 metrics | 39.9 | 14.1 | 22.9 | 904.2 | 51.4x |
+| APCng 1k keys / 2500 metrics | n/t | n/t | n/t | n/t | n/t |
+| APCng 10k keys / 2500 metrics | 181.3 | 80.3 | 17.9 | 978.8 | 25.2x |
+| APCng 100k keys / 2500 metrics | 274.7 | 84.0 | 34.6 | 4092.4 | 52.9x |
+| APCng 1M keys / 2500 metrics | 187.8 | 87.7 | 40.7 | 5396.4 | 50.1x |
+
+The suite of engine-performance tests can be automatically executed by running `docker-compose run phpunit vendor/bin/phpunit tests/Test --group Performance`. This set of tests in not part of the default unit tests which get run, since they take quite a while to complete. Any significant change to the APC or APCng code should be followed by a performance-test run to quantify the before/after impact of the change. Currently this is triggered manually, but it could be automated as part of a Github workflow.
+
+## Known limitations
+One thing to note, the current implementation of the `Summary` observer should be avoided on busy servers. This is true for both the `APC` and `APCng` storage engines. The reason is simple: each observation (call to increment, set, etc) results in a new item being written to APCu. The default TTL for these items is 600 seconds. On a busy server that might be getting 1000 requests/second, that results in 600,000 APC cache items continually churning in and out of existence. This can put some interesting pressure on APCu, which could lead to rapid fragmentation of APCu memory. Definitely test before deploying in production.
+
+For a future project, the existing algorithm that stores one new key per observation could be replaced with a sampling-style algorithm (`t-digest`) that only stores a handful of keys, and updates their weights for each request. This is considerably less likely to fragment APCu memory over time.
+
+Neither the `APC` or `APCng` engine performs particularly well once more than ~1000 Prometheus metrics are being tracked. Of course, "good performance" is subjective, and partially based on how often you scrape for data. If you only scrape every five minutes, then spending 4 seconds waiting for collect() might be perfectly acceptable. On the other hand, if you scrape every 2 seconds, you'll want collect() to be as fast as possible.
+
+## How it works under the covers
+Without going into excruciating detail (you can read the source for that!), the general idea is to remove calls to APCUIterator() whenever possible. In particular, nested calls to APCUIterator are horrible, since APCUIterator scales O(n) where n is the number of keys in APCu. This means the busier your server is, the slower these calls will run. Summary is the worst: it has APCUIterator calls nested three deep, leading to O(n^3) running-time.
+
+The approach `APCng` takes is to keep a "metadata cache" which stores an array of all the metadata keys, so instead of doing a scan of APCu looking for all matching keys, we just need to retrieve one key, deserialize it (which turns out to be slow), and retrieve all the metadata keys listed in the array. Once we've done that, there is some fancy handwaving which is used to deterministically generate possible sub-keys for each metadata item, based on LabelNames, etc. Not all of these keys exist, but it's quicker to attempt to fetch them and fail, then it is to run another APCUIterator looking for a specific pattern.
+
+Summaries, as mentioned before, have a third nested APCUIterator in them, looking for all readings w/o expired TTLs that match a pattern. Again, slow. Instead, we store a "map", similar to the metadata cache, but this one is temporally-keyed: one key per second, which lists how many samples were collected in that second. Once this is done, an expensive APCUIterator match is no longer needed, as all possible keys can be deterministically generated and checked, by retrieving each key for the past 600 seconds (if it exists), extracting the sample-count from the key, and then generating all the APCu keys which would refer to each observed sample.
+
+There is the concept of a metadata cache TTL (default: 1 second) which offers a trade-off of performance vs responsiveness. If a collect() call is made and then a new metric is subsequently tracked, the new metric won't show up in subsequent collect() calls until the metadata cache TTL is expired. By keeping this TTL short, we avoid hammering APCu too heavily (remember, deserializing that metainfo cache array is nearly as slow as calling APCUIterator -- it just doesn't slow down as you add more keys to APCu). However we want to cap how long a new metric remains "hidden" from the Prometheus scraper. For best performance, adjust the TTL as high as you can based on your specific use-case. For instance if you're scraping every 10 seconds, then a reasonable TTL could be anywhere from 5 to 10 seconds, meaning a 50 to 100% chance that the metric won't appear in the next full scrape, but it will be there for the following one. Note that the data is tracked just fine during this period - it's just not visible yet, but it will be! You can set the TTL to zero to disable the cache. This will return to `APC` engine behavior, with no delay between creating a metric and being able to collect() it. However, performance will suffer, as the metainfo cache array will need to be deserialized from APCu each time collect() is called -- which might be okay if collect() is called infrequently and you simply must have zero delay in reporting newly-created metrics.
diff --git a/README.md b/README.md
index fa1fefde..415f2acc 100644
--- a/README.md
+++ b/README.md
@@ -8,9 +8,9 @@ If using Redis, we recommend running a local Redis instance next to your PHP wor
## How does it work?
Usually PHP worker processes don't share any state.
-You can pick from three adapters.
-Redis, APC or an in memory adapter.
-While the first needs a separate binary running, the second just needs the [APC](https://pecl.php.net/package/APCU) extension to be installed. If you don't need persistent metrics between requests (e.g. a long running cron job or script) the in memory adapter might be suitable to use.
+You can pick from four adapters.
+Redis, APC, APCng, or an in-memory adapter.
+While the first needs a separate binary running, the second and third just need the [APC](https://pecl.php.net/package/APCU) extension to be installed. If you don't need persistent metrics between requests (e.g. a long running cron job or script) the in-memory adapter might be suitable to use.
## Installation
@@ -94,6 +94,15 @@ $renderer = new RenderTextFormat();
$result = $renderer->render($registry->getMetricFamilySamples());
```
+Using the APC or APCng storage:
+```php
+$registry = new CollectorRegistry(new APCng());
+ or
+$registry = new CollectorRegistry(new APC());
+```
+(see the `README.APCng.md` file for more details)
+
+
### Advanced Usage
#### Advanced Histogram Usage
@@ -150,5 +159,19 @@ Pick the adapter you want to test.
```
docker-compose run phpunit env ADAPTER=apc vendor/bin/phpunit tests/Test/
+docker-compose run phpunit env ADAPTER=apcng vendor/bin/phpunit tests/Test/
docker-compose run phpunit env ADAPTER=redis vendor/bin/phpunit tests/Test/
```
+
+## Performance testing
+
+This currently tests the APC and APCng adapters head-to-head and reports if the APCng adapter is slower for any actions.
+```
+phpunit vendor/bin/phpunit tests/Test/ --group Performance
+```
+
+The test can also be run inside a container.
+```
+docker-compose up
+docker-compose run phpunit vendor/bin/phpunit tests/Test/ --group Performance
+```
diff --git a/examples/flush_adapter.php b/examples/flush_adapter.php
index 544dcbdd..1c00eab7 100644
--- a/examples/flush_adapter.php
+++ b/examples/flush_adapter.php
@@ -12,6 +12,8 @@
$adapter = new Prometheus\Storage\Redis(['host' => REDIS_HOST]);
} elseif ($adapterName === 'apc') {
$adapter = new Prometheus\Storage\APC();
+} elseif ($adapterName === 'apcng') {
+ $adapter = new Prometheus\Storage\APCng();
} elseif ($adapterName === 'in-memory') {
$adapter = new Prometheus\Storage\InMemory();
}
diff --git a/examples/metrics.php b/examples/metrics.php
index e523f795..9c0fdb80 100644
--- a/examples/metrics.php
+++ b/examples/metrics.php
@@ -13,6 +13,8 @@
$adapter = new Prometheus\Storage\Redis();
} elseif ($adapter === 'apc') {
$adapter = new Prometheus\Storage\APC();
+} elseif ($adapter === 'apcng') {
+ $adapter = new Prometheus\Storage\APCng();
} elseif ($adapter === 'in-memory') {
$adapter = new Prometheus\Storage\InMemory();
}
diff --git a/examples/some_counter.php b/examples/some_counter.php
index 25d3736c..c7426ce8 100644
--- a/examples/some_counter.php
+++ b/examples/some_counter.php
@@ -12,6 +12,8 @@
$adapter = new Prometheus\Storage\Redis();
} elseif ($adapter === 'apc') {
$adapter = new Prometheus\Storage\APC();
+} elseif ($adapter === 'apcng') {
+ $adapter = new Prometheus\Storage\APCng();
} elseif ($adapter === 'in-memory') {
$adapter = new Prometheus\Storage\InMemory();
}
diff --git a/examples/some_gauge.php b/examples/some_gauge.php
index f8202d28..9e8b3da2 100644
--- a/examples/some_gauge.php
+++ b/examples/some_gauge.php
@@ -15,6 +15,8 @@
$adapter = new Prometheus\Storage\Redis();
} elseif ($adapter === 'apc') {
$adapter = new Prometheus\Storage\APC();
+} elseif ($adapter === 'apcng') {
+ $adapter = new Prometheus\Storage\APCng();
} elseif ($adapter === 'in-memory') {
$adapter = new Prometheus\Storage\InMemory();
}
diff --git a/examples/some_histogram.php b/examples/some_histogram.php
index aeb5c1d2..2f1a5f98 100644
--- a/examples/some_histogram.php
+++ b/examples/some_histogram.php
@@ -14,6 +14,8 @@
$adapter = new Prometheus\Storage\Redis();
} elseif ($adapter === 'apc') {
$adapter = new Prometheus\Storage\APC();
+} elseif ($adapter === 'apcng') {
+ $adapter = new Prometheus\Storage\APCng();
} elseif ($adapter === 'in-memory') {
$adapter = new Prometheus\Storage\InMemory();
}
diff --git a/examples/some_summary.php b/examples/some_summary.php
index adea1796..363f9190 100644
--- a/examples/some_summary.php
+++ b/examples/some_summary.php
@@ -14,6 +14,8 @@
$adapter = new Prometheus\Storage\Redis();
} elseif ($adapter === 'apc') {
$adapter = new Prometheus\Storage\APC();
+} elseif ($adapter === 'apcng') {
+ $adapter = new Prometheus\Storage\APCng();
} elseif ($adapter === 'in-memory') {
$adapter = new Prometheus\Storage\InMemory();
}
diff --git a/phpunit.xml.dist b/phpunit.xml.dist
index 9fc8a727..88981644 100644
--- a/phpunit.xml.dist
+++ b/phpunit.xml.dist
@@ -21,4 +21,10 @@
./src/Prometheus
+
+
+
+ Performance
+
+
diff --git a/src/Prometheus/Storage/APCng.php b/src/Prometheus/Storage/APCng.php
new file mode 100644
index 00000000..bdb2bb29
--- /dev/null
+++ b/src/Prometheus/Storage/APCng.php
@@ -0,0 +1,803 @@
+prometheusPrefix = $prometheusPrefix;
+ $this->metainfoCacheKey = implode(':', [ $this->prometheusPrefix, 'metainfocache' ]);
+ }
+
+ /**
+ * @return MetricFamilySamples[]
+ */
+ public function collect(): array
+ {
+ $metrics = $this->collectHistograms();
+ $metrics = array_merge($metrics, $this->collectGauges());
+ $metrics = array_merge($metrics, $this->collectCounters());
+ $metrics = array_merge($metrics, $this->collectSummaries());
+ return $metrics;
+ }
+
+ /**
+ * @param mixed[] $data
+ * @throws RuntimeException
+ */
+ public function updateHistogram(array $data): void
+ {
+ // Initialize or atomically increment the sum
+ // Taken from https://github.com/prometheus/client_golang/blob/66058aac3a83021948e5fb12f1f408ff556b9037/prometheus/value.go#L91
+ $sumKey = $this->histogramBucketValueKey($data, 'sum');
+ $done = false;
+ $loopCatcher = self::CAS_LOOP_RETRIES;
+ while (!$done && $loopCatcher-- > 0) {
+ $old = apcu_fetch($sumKey);
+ if ($old !== false) {
+ $done = apcu_cas($sumKey, $old, $this->toBinaryRepresentationAsInteger($this->fromBinaryRepresentationAsInteger($old) + $data['value']));
+ } else {
+ // If sum does not exist, initialize it, store the metadata for the new histogram
+ apcu_add($sumKey, $this->toBinaryRepresentationAsInteger(0));
+ apcu_store($this->metaKey($data), json_encode($this->metaData($data)));
+ $this->storeLabelKeys($data);
+ }
+ }
+ if ($loopCatcher <= 0) {
+ throw new RuntimeException('Caught possible infinite loop in ' . __METHOD__ . '()');
+ }
+
+ // Figure out in which bucket the observation belongs
+ $bucketToIncrease = '+Inf';
+ foreach ($data['buckets'] as $bucket) {
+ if ($data['value'] <= $bucket) {
+ $bucketToIncrease = $bucket;
+ break;
+ }
+ }
+
+ // Initialize and increment the bucket
+ apcu_add($this->histogramBucketValueKey($data, $bucketToIncrease), 0);
+ apcu_inc($this->histogramBucketValueKey($data, $bucketToIncrease));
+ }
+
+ /**
+ * For each second, store an incrementing counter which points to each individual observation, like this:
+ * prom:bla..blabla:value:16781560:observations = 199
+ * Then we know that for the 1-second period at unix timestamp 16781560, 199 observations are stored, and they can
+ * be retrieved using APC keynames "prom:...:16781560.0" thorough "prom:...:16781560.198"
+ * We can deterministically calculate the intervening timestamps by subtracting maxAge, to get a range of seconds
+ * when generating a summary, e.g. 16781560 back to 16780960 for a 600sec maxAge. Then collect observation counts
+ * for each second, programmatically generate the APC keys for each individual observation, and we're able to avoid
+ * performing a full APC key scan, which can block for several seconds if APCu contains a few million keys.
+ *
+ * @param mixed[] $data
+ * @throws RuntimeException
+ */
+ public function updateSummary(array $data): void
+ {
+ // store value key; store metadata & labels if new
+ $valueKey = $this->valueKey($data);
+ $new = apcu_add($valueKey, $this->encodeLabelValues($data['labelValues']));
+ if ($new) {
+ apcu_add($this->metaKey($data), $this->metaData($data));
+ $this->storeLabelKeys($data);
+ }
+ $sampleKeyPrefix = $valueKey . ':' . time();
+ $sampleCountKey = $sampleKeyPrefix . ':observations';
+
+ // Check if sample counter for this timestamp already exists, so we can deterministically store observations+counts, one key per second
+ // Atomic increment of the observation counter, or initialize if new
+ $done = false;
+ $loopCatcher = self::CAS_LOOP_RETRIES;
+ while (!$done && $loopCatcher-- > 0) {
+ $sampleCount = apcu_fetch($sampleCountKey);
+ if ($sampleCount !== false) {
+ $done = apcu_cas($sampleCountKey, $sampleCount, $sampleCount + 1);
+ } else {
+ apcu_add($sampleCountKey, 0, $data['maxAgeSeconds']);
+ }
+ }
+ if ($loopCatcher <= 0) {
+ throw new RuntimeException('Caught possible infinite loop in ' . __METHOD__ . '()');
+ }
+
+ // We now have a deterministic keyname for this observation; let's save the observed value
+ $sampleKey = $sampleKeyPrefix . '.' . $sampleCount;
+ apcu_add($sampleKey, $data['value'], $data['maxAgeSeconds']);
+ }
+
+ /**
+ * @param mixed[] $data
+ * @throws RuntimeException
+ */
+ public function updateGauge(array $data): void
+ {
+ $valueKey = $this->valueKey($data);
+ if ($data['command'] === Adapter::COMMAND_SET) {
+ apcu_store($valueKey, $this->toBinaryRepresentationAsInteger($data['value']));
+ apcu_store($this->metaKey($data), json_encode($this->metaData($data)));
+ $this->storeLabelKeys($data);
+ } else {
+ // Taken from https://github.com/prometheus/client_golang/blob/66058aac3a83021948e5fb12f1f408ff556b9037/prometheus/value.go#L91
+ $done = false;
+ $loopCatcher = self::CAS_LOOP_RETRIES;
+ while (!$done && $loopCatcher-- > 0) {
+ $old = apcu_fetch($valueKey);
+ if ($old !== false) {
+ $done = apcu_cas($valueKey, $old, $this->toBinaryRepresentationAsInteger($this->fromBinaryRepresentationAsInteger($old) + $data['value']));
+ } else {
+ apcu_add($valueKey, $this->toBinaryRepresentationAsInteger(0));
+ apcu_store($this->metaKey($data), json_encode($this->metaData($data)));
+ $this->storeLabelKeys($data);
+ }
+ }
+ if ($loopCatcher <= 0) {
+ throw new RuntimeException('Caught possible infinite loop in ' . __METHOD__ . '()');
+ }
+ }
+ }
+
+ /**
+ * @param mixed[] $data
+ * @throws RuntimeException
+ */
+ public function updateCounter(array $data): void
+ {
+ // Taken from https://github.com/prometheus/client_golang/blob/66058aac3a83021948e5fb12f1f408ff556b9037/prometheus/value.go#L91
+ $valueKey = $this->valueKey($data);
+ $done = false;
+ $loopCatcher = self::CAS_LOOP_RETRIES;
+ while (!$done && $loopCatcher-- > 0) {
+ $old = apcu_fetch($valueKey);
+ if ($old !== false) {
+ $done = apcu_cas($valueKey, $old, $this->toBinaryRepresentationAsInteger($this->fromBinaryRepresentationAsInteger($old) + $data['value']));
+ } else {
+ apcu_add($valueKey, 0);
+ apcu_store($this->metaKey($data), json_encode($this->metaData($data)));
+ $this->storeLabelKeys($data);
+ }
+ }
+ if ($loopCatcher <= 0) {
+ throw new RuntimeException('Caught possible infinite loop in ' . __METHOD__ . '()');
+ }
+ }
+
+ /**
+ * @param array $metaData
+ * @param string $labels
+ * @return string
+ */
+ private function assembleLabelKey(array $metaData, string $labels): string
+ {
+ return implode(':', [ $this->prometheusPrefix, $metaData['type'], $metaData['name'], $labels, 'label' ]);
+ }
+
+ /**
+ * Store ':label' keys for each metric's labelName in APCu.
+ *
+ * @param array $data
+ * @return void
+ */
+ private function storeLabelKeys(array $data): void
+ {
+ // Store labelValues in each labelName key
+ foreach ($data['labelNames'] as $seq => $label) {
+ $this->addItemToKey(implode(':', [
+ $this->prometheusPrefix,
+ $data['type'],
+ $data['name'],
+ $label,
+ 'label'
+ ]), isset($data['labelValues']) ? $data['labelValues'][$seq] : ''); // may not need the isset check
+ }
+ }
+
+ /**
+ * Ensures an array serialized into APCu contains exactly one copy of a given string
+ *
+ * @return void
+ * @throws RuntimeException
+ */
+ private function addItemToKey(string $key, string $item): void
+ {
+ // Modify serialized array stored in $key
+ $arr = apcu_fetch($key);
+ if (false === $arr) {
+ $arr = [];
+ }
+ if (!array_key_exists($item, $arr)) {
+ $arr[$this->encodeLabelKey($item)] = 1;
+ apcu_store($key, $arr);
+ }
+ }
+
+ /**
+ * Removes all previously stored data from apcu
+ *
+ * NOTE: This is non-atomic: while it's iterating APCu, another thread could write a new Prometheus key that doesn't get erased.
+ * In case this happens, getMetas() calls scanAndBuildMetainfoCache before reading metainfo back: this will ensure "orphaned"
+ * metainfo gets enumerated.
+ *
+ * @return void
+ */
+ public function wipeStorage(): void
+ {
+ // / / | PCRE expresion boundary
+ // ^ | match from first character only
+ // %s: | common prefix substitute with colon suffix
+ // .+ | at least one additional character
+ $matchAll = sprintf('/^%s:.+/', $this->prometheusPrefix);
+
+ foreach (new APCUIterator($matchAll) as $key => $value) {
+ apcu_delete($key);
+ }
+ }
+
+ /**
+ * Sets the metainfo cache TTL; how long to retain metainfo before scanning APCu keyspace again (default 1 second)
+ *
+ * @param int $ttl
+ * @return void
+ */
+ public function setMetainfoTTL(int $ttl): void
+ {
+ $this->metainfoCacheTTL = $ttl;
+ }
+
+ /**
+ * Scans the APCu keyspace for all metainfo keys. A new metainfo cache array is built,
+ * which references all metadata keys in APCu at that moment. This prevents a corner-case
+ * where an orphaned key, while remaining writable, is rendered permanently invisible when reading
+ * or enumerating metrics.
+ *
+ * Writing the cache to APCu allows it to be shared by other threads and by subsequent calls to getMetas(). This
+ * reduces contention on APCu from repeated scans, and provides about a 2.5x speed-up when calling $this->collect().
+ * The cache TTL is very short (default: 1sec), so if new metrics are tracked after the cache is built, they will
+ * be readable at most 1 second after being written.
+ *
+ * Setting $apc_ttl less than 1 will disable the cache.
+ *
+ * @param int $apc_ttl
+ * @return array
+ */
+ private function scanAndBuildMetainfoCache(int $apc_ttl = 1): array
+ {
+ $arr = [];
+ $matchAllMeta = sprintf('/^%s:.*:meta/', $this->prometheusPrefix);
+ foreach (new APCUIterator($matchAllMeta) as $apc_record) {
+ $arr[] = $apc_record['key'];
+ }
+ if ($apc_ttl >= 1) {
+ apcu_store($this->metainfoCacheKey, $arr, $apc_ttl);
+ }
+ return $arr;
+ }
+
+ /**
+ * @param mixed[] $data
+ * @return string
+ */
+ private function metaKey(array $data): string
+ {
+ return implode(':', [$this->prometheusPrefix, $data['type'], $data['name'], 'meta']);
+ }
+
+ /**
+ * @param mixed[] $data
+ * @return string
+ */
+ private function valueKey(array $data): string
+ {
+ return implode(':', [
+ $this->prometheusPrefix,
+ $data['type'],
+ $data['name'],
+ $this->encodeLabelValues($data['labelValues']),
+ 'value',
+ ]);
+ }
+
+ /**
+ * @param mixed[] $data
+ * @param string|int $bucket
+ * @return string
+ */
+ private function histogramBucketValueKey(array $data, $bucket): string
+ {
+ return implode(':', [
+ $this->prometheusPrefix,
+ $data['type'],
+ $data['name'],
+ $this->encodeLabelValues($data['labelValues']),
+ $bucket,
+ 'value',
+ ]);
+ }
+
+ /**
+ * @param mixed[] $data
+ * @return mixed[]
+ */
+ private function metaData(array $data): array
+ {
+ $metricsMetaData = $data;
+ unset($metricsMetaData['value'], $metricsMetaData['command'], $metricsMetaData['labelValues']);
+ return $metricsMetaData;
+ }
+
+ /**
+ * When given a ragged 2D array $labelValues of arbitrary size, and a 1D array $labelNames containing one
+ * string labeling each row of $labelValues, return an array-of-arrays containing all possible permutations
+ * of labelValues, with the sub-array elements in order of labelName.
+ *
+ * Example input:
+ * $labelNames: ['endpoint', 'method', 'result']
+ * $labelValues: [0] => ['/', '/private', '/metrics'], // "endpoint"
+ * [1] => ['put', 'get', 'post'], // "method"
+ * [2] => ['success', 'fail'] // "result"
+ * Returned array:
+ * [0] => ['/', 'put', 'success'], [1] => ['/', 'put', 'fail'], [2] => ['/', 'get', 'success'],
+ * [3] => ['/', 'get', 'fail'], [4] => ['/', 'post', 'success'], [5] => ['/', 'post', 'fail'],
+ * [6] => ['/private', 'put', 'success'], [7] => ['/private', 'put', 'fail'], [8] => ['/private', 'get', 'success'],
+ * [9] => ['/private', 'get', 'fail'], [10] => ['/private', 'post', 'success'], [11] => ['/private', 'post', 'fail'],
+ * [12] => ['/metrics', 'put', 'success'], [13] => ['/metrics', 'put', 'fail'], [14] => ['/metrics', 'get', 'success'],
+ * [15] => ['/metrics', 'get', 'fail'], [16] => ['/metrics', 'post', 'success'], [17] => ['/metrics', 'post', 'fail']
+ * @param array $labelNames
+ * @param array $labelValues
+ * @return array
+ */
+ private function buildPermutationTree(array $labelNames, array $labelValues): array
+ {
+ $treeRowCount = count(array_keys($labelNames));
+ $numElements = 1;
+ $treeInfo = [];
+ for ($i = $treeRowCount - 1; $i >= 0; $i--) {
+ $treeInfo[$i]['numInRow'] = count($labelValues[$i]);
+ $numElements *= $treeInfo[$i]['numInRow'];
+ $treeInfo[$i]['numInTree'] = $numElements;
+ }
+
+ $map = array_fill(0, $numElements, []);
+ for ($row = 0; $row < $treeRowCount; $row++) {
+ $col = $i = 0;
+ while ($i < $numElements) {
+ $val = $labelValues[$row][$col];
+ $map[$i] = array_merge($map[$i], array($val));
+ if (++$i % ($treeInfo[$row]['numInTree'] / $treeInfo[$row]['numInRow']) == 0) {
+ $col = ++$col % $treeInfo[$row]['numInRow'];
+ }
+ }
+ }
+ return $map;
+ }
+
+ /**
+ * @return MetricFamilySamples[]
+ */
+ private function collectCounters(): array
+ {
+ $counters = [];
+ foreach ($this->getMetas('counter') as $counter) {
+ $metaData = json_decode($counter['value'], true);
+ $data = [
+ 'name' => $metaData['name'],
+ 'help' => $metaData['help'],
+ 'type' => $metaData['type'],
+ 'labelNames' => $metaData['labelNames'],
+ 'samples' => [],
+ ];
+ foreach ($this->getValues('counter', $metaData) as $value) {
+ $parts = explode(':', $value['key']);
+ $labelValues = $parts[3];
+ $data['samples'][] = [
+ 'name' => $metaData['name'],
+ 'labelNames' => [],
+ 'labelValues' => $this->decodeLabelValues($labelValues),
+ 'value' => $this->fromBinaryRepresentationAsInteger($value['value']),
+ ];
+ }
+ $this->sortSamples($data['samples']);
+ $counters[] = new MetricFamilySamples($data);
+ }
+ return $counters;
+ }
+
+ /**
+ * When given a type ('histogram', 'gauge', or 'counter'), return an iterable array of matching records retrieved from APCu
+ *
+ * @param string $type
+ * @return array
+ */
+ private function getMetas(string $type): array
+ {
+ $arr = [];
+ $metaCache = apcu_fetch($this->metainfoCacheKey);
+ if (!is_array($metaCache)) {
+ $metaCache = $this->scanAndBuildMetainfoCache($this->metainfoCacheTTL);
+ }
+ foreach ($metaCache as $metaKey) {
+ if ((1 === preg_match('/' . $this->prometheusPrefix . ':' . $type . ':.*:meta/', $metaKey)) && false !== ($gauge = apcu_fetch($metaKey))) {
+ $arr[] = [ 'key' => $metaKey, 'value' => $gauge ];
+ }
+ }
+ return $arr;
+ }
+
+ /**
+ * When given a type ('histogram', 'gauge', or 'counter') and metaData array, return an iterable array of matching records retrieved from APCu
+ *
+ * @param string $type
+ * @param array $metaData
+ * @return array
+ */
+ private function getValues(string $type, array $metaData): array
+ {
+ $labels = $arr = [];
+ foreach (array_values($metaData['labelNames']) as $label) {
+ $labelKey = $this->assembleLabelKey($metaData, $label);
+ if (is_array($tmp = apcu_fetch($labelKey))) {
+ $labels[] = array_map([$this, 'decodeLabelKey'], array_keys($tmp));
+ }
+ }
+ // Append the histogram bucket-list and the histogram-specific label 'sum' to labels[] then generate the permutations
+ if (isset($metaData['buckets'])) {
+ $metaData['buckets'][] = 'sum';
+ $labels[] = $metaData['buckets'];
+ $metaData['labelNames'][] = '__histogram_buckets';
+ }
+ $labelValuesList = $this->buildPermutationTree($metaData['labelNames'], $labels);
+ unset($labels);
+ $histogramBucket = '';
+ foreach ($labelValuesList as $labelValues) {
+ // Extract bucket value from permuted element, if present, then construct the key and retrieve
+ if (isset($metaData['buckets'])) {
+ $histogramBucket = ':' . array_pop($labelValues);
+ }
+ $key = $this->prometheusPrefix . ":{$type}:{$metaData['name']}:" . $this->encodeLabelValues($labelValues) . $histogramBucket . ':value';
+ if (false !== ($value = apcu_fetch($key))) {
+ $arr[] = [ 'key' => $key, 'value' => $value ];
+ }
+ }
+ return $arr;
+ }
+
+ /**
+ * @return MetricFamilySamples[]
+ */
+ private function collectGauges(): array
+ {
+ $gauges = [];
+ foreach ($this->getMetas('gauge') as $gauge) {
+ $metaData = json_decode($gauge['value'], true);
+ $data = [
+ 'name' => $metaData['name'],
+ 'help' => $metaData['help'],
+ 'type' => $metaData['type'],
+ 'labelNames' => $metaData['labelNames'],
+ 'samples' => [],
+ ];
+ foreach ($this->getValues('gauge', $metaData) as $value) {
+ $parts = explode(':', $value['key']);
+ $labelValues = $parts[3];
+ $data['samples'][] = [
+ 'name' => $metaData['name'],
+ 'labelNames' => [],
+ 'labelValues' => $this->decodeLabelValues($labelValues),
+ 'value' => $this->fromBinaryRepresentationAsInteger($value['value']),
+ ];
+ }
+ $this->sortSamples($data['samples']);
+ $gauges[] = new MetricFamilySamples($data);
+ }
+ return $gauges;
+ }
+
+ /**
+ * @return MetricFamilySamples[]
+ */
+ private function collectHistograms(): array
+ {
+ $histograms = [];
+ foreach ($this->getMetas('histogram') as $histogram) {
+ $metaData = json_decode($histogram['value'], true);
+
+ // Add the Inf bucket so we can compute it later on
+ $metaData['buckets'][] = '+Inf';
+
+ $data = [
+ 'name' => $metaData['name'],
+ 'help' => $metaData['help'],
+ 'type' => $metaData['type'],
+ 'labelNames' => $metaData['labelNames'],
+ 'buckets' => $metaData['buckets'],
+ ];
+
+ $histogramBuckets = [];
+ foreach ($this->getValues('histogram', $metaData) as $value) {
+ $parts = explode(':', $value['key']);
+ $labelValues = $parts[3];
+ $bucket = $parts[4];
+ // Key by labelValues
+ $histogramBuckets[$labelValues][$bucket] = $value['value'];
+ }
+
+ // Compute all buckets
+ $labels = array_keys($histogramBuckets);
+ sort($labels);
+ foreach ($labels as $labelValues) {
+ $acc = 0;
+ $decodedLabelValues = $this->decodeLabelValues($labelValues);
+ foreach ($data['buckets'] as $bucket) {
+ $bucket = (string)$bucket;
+ if (!isset($histogramBuckets[$labelValues][$bucket])) {
+ $data['samples'][] = [
+ 'name' => $metaData['name'] . '_bucket',
+ 'labelNames' => ['le'],
+ 'labelValues' => array_merge($decodedLabelValues, [$bucket]),
+ 'value' => $acc,
+ ];
+ } else {
+ $acc += $histogramBuckets[$labelValues][$bucket];
+ $data['samples'][] = [
+ 'name' => $metaData['name'] . '_' . 'bucket',
+ 'labelNames' => ['le'],
+ 'labelValues' => array_merge($decodedLabelValues, [$bucket]),
+ 'value' => $acc,
+ ];
+ }
+ }
+
+ // Add the count
+ $data['samples'][] = [
+ 'name' => $metaData['name'] . '_count',
+ 'labelNames' => [],
+ 'labelValues' => $decodedLabelValues,
+ 'value' => $acc,
+ ];
+
+ // Add the sum
+ $data['samples'][] = [
+ 'name' => $metaData['name'] . '_sum',
+ 'labelNames' => [],
+ 'labelValues' => $decodedLabelValues,
+ 'value' => $this->fromBinaryRepresentationAsInteger($histogramBuckets[$labelValues]['sum']),
+ ];
+ }
+ $histograms[] = new MetricFamilySamples($data);
+ }
+ return $histograms;
+ }
+
+ /**
+ * @return MetricFamilySamples[]
+ */
+ private function collectSummaries(): array
+ {
+ $math = new Math();
+ $summaries = [];
+ foreach ($this->getMetas('summary') as $summary) {
+ $metaData = $summary['value'];
+ $data = [
+ 'name' => $metaData['name'],
+ 'help' => $metaData['help'],
+ 'type' => $metaData['type'],
+ 'labelNames' => $metaData['labelNames'],
+ 'maxAgeSeconds' => $metaData['maxAgeSeconds'],
+ 'quantiles' => $metaData['quantiles'],
+ 'samples' => [],
+ ];
+
+ foreach ($this->getValues('summary', $metaData) as $value) {
+ $encodedLabelValues = $value['value'];
+ $decodedLabelValues = $this->decodeLabelValues($encodedLabelValues);
+ $samples = [];
+
+ // Deterministically generate keys for all the sample observations, and retrieve them. Pass arrays to apcu_fetch to reduce calls to APCu.
+ $end = time();
+ $begin = $end - $metaData['maxAgeSeconds'];
+ $valueKeyPrefix = $this->valueKey(array_merge($metaData, ['labelValues' => $decodedLabelValues]));
+
+ $sampleCountKeysToRetrieve = [];
+ for ($ts = $begin; $ts <= $end; $ts++) {
+ $sampleCountKeysToRetrieve[] = $valueKeyPrefix . ':' . $ts . ':observations';
+ }
+ $sampleCounts = apcu_fetch($sampleCountKeysToRetrieve);
+ unset($sampleCountKeysToRetrieve);
+ if (is_array($sampleCounts)) {
+ foreach ($sampleCounts as $k => $sampleCountThisSecond) {
+ $tstamp = explode(':', $k)[5];
+ $sampleKeysToRetrieve = [];
+ for ($i = 0; $i < $sampleCountThisSecond; $i++) {
+ $sampleKeysToRetrieve[] = $valueKeyPrefix . ':' . $tstamp . '.' . $i;
+ }
+ $newSamples = apcu_fetch($sampleKeysToRetrieve);
+ unset($sampleKeysToRetrieve);
+ if (is_array($newSamples)) {
+ $samples = array_merge($samples, $newSamples);
+ }
+ }
+ }
+ unset($sampleCounts);
+
+ if (count($samples) === 0) {
+ apcu_delete($value['key']);
+ continue;
+ }
+
+ // Compute quantiles
+ sort($samples);
+ foreach ($data['quantiles'] as $quantile) {
+ $data['samples'][] = [
+ 'name' => $metaData['name'],
+ 'labelNames' => ['quantile'],
+ 'labelValues' => array_merge($decodedLabelValues, [$quantile]),
+ 'value' => $math->quantile($samples, $quantile),
+ ];
+ }
+
+ // Add the count
+ $data['samples'][] = [
+ 'name' => $metaData['name'] . '_count',
+ 'labelNames' => [],
+ 'labelValues' => $decodedLabelValues,
+ 'value' => count($samples),
+ ];
+
+ // Add the sum
+ $data['samples'][] = [
+ 'name' => $metaData['name'] . '_sum',
+ 'labelNames' => [],
+ 'labelValues' => $decodedLabelValues,
+ 'value' => array_sum($samples),
+ ];
+ }
+
+ if (count($data['samples']) > 0) {
+ $summaries[] = new MetricFamilySamples($data);
+ } else {
+ apcu_delete($summary['key']);
+ }
+ }
+ return $summaries;
+ }
+
+ /**
+ * @param mixed $val
+ * @return int
+ * @throws RuntimeException
+ */
+ private function toBinaryRepresentationAsInteger($val): int
+ {
+ $packedDouble = pack('d', $val);
+ if ((bool)$packedDouble !== false) {
+ $unpackedData = unpack("Q", $packedDouble);
+ if (is_array($unpackedData)) {
+ return $unpackedData[1];
+ }
+ }
+ throw new RuntimeException("Formatting from binary representation to integer did not work");
+ }
+
+ /**
+ * @param mixed $val
+ * @return float
+ * @throws RuntimeException
+ */
+ private function fromBinaryRepresentationAsInteger($val): float
+ {
+ $packedBinary = pack('Q', $val);
+ if ((bool)$packedBinary !== false) {
+ $unpackedData = unpack("d", $packedBinary);
+ if (is_array($unpackedData)) {
+ return $unpackedData[1];
+ }
+ }
+ throw new RuntimeException("Formatting from integer to binary representation did not work");
+ }
+
+ /**
+ * @param mixed[] $samples
+ */
+ private function sortSamples(array &$samples): void
+ {
+ usort($samples, function ($a, $b): int {
+ return strcmp(implode("", $a['labelValues']), implode("", $b['labelValues']));
+ });
+ }
+
+ /**
+ * @param mixed[] $values
+ * @return string
+ * @throws RuntimeException
+ */
+ private function encodeLabelValues(array $values): string
+ {
+ $json = json_encode($values);
+ if (false === $json) {
+ throw new RuntimeException(json_last_error_msg());
+ }
+ return base64_encode($json);
+ }
+
+ /**
+ * @param string $values
+ * @return mixed[]
+ * @throws RuntimeException
+ */
+ private function decodeLabelValues(string $values): array
+ {
+ $json = base64_decode($values, true);
+ if (false === $json) {
+ throw new RuntimeException('Cannot base64 decode label values');
+ }
+ $decodedValues = json_decode($json, true);
+ if (false === $decodedValues) {
+ throw new RuntimeException(json_last_error_msg());
+ }
+ return $decodedValues;
+ }
+
+ /**
+ * @param string $keyString
+ * @return string
+ */
+ private function encodeLabelKey(string $keyString): string
+ {
+ return base64_encode($keyString);
+ }
+
+ /**
+ * @param string $str
+ * @return string
+ * @throws RuntimeException
+ */
+ private function decodeLabelKey(string $str): string
+ {
+ $decodedKey = base64_decode($str, true);
+ if (false === $decodedKey) {
+ throw new RuntimeException('Cannot base64 decode label key');
+ }
+ return $decodedKey;
+ }
+}
diff --git a/tests/Test/Performance/PerformanceTest.php b/tests/Test/Performance/PerformanceTest.php
new file mode 100644
index 00000000..ff4ac4f3
--- /dev/null
+++ b/tests/Test/Performance/PerformanceTest.php
@@ -0,0 +1,183 @@
+initializeAPC($num_apc_keys);
+ foreach (['Prometheus\Storage\APC', 'Prometheus\Storage\APCng'] as $driverClass) {
+ $begin = microtime(true);
+ $x = new TestEngineSpeed($driverClass, $num_metrics);
+ $x->doCreates();
+ $elapsedTime = microtime(true) - $begin;
+ fprintf(STDOUT, "%7d APC keys and %4d metrics; Creating metrics took %7.4f seconds. (%s)\n", $num_apc_keys, $num_metrics, $elapsedTime, $driverClass);
+ $results["{$num_apc_keys}keys_{$num_metrics}metrics"][$driverClass] = $elapsedTime;
+ }
+ }
+ }
+
+ // should be around the same speed or slightly faster; minimum of 70% the APC speed to account for variations between test runs
+ foreach ($results as $resultName => $elapsedTimeByDrivername) {
+ self::assertThat(
+ $elapsedTimeByDrivername['Prometheus\Storage\APCng'] * 0.7,
+ self::lessThan($elapsedTimeByDrivername['Prometheus\Storage\APC']),
+ "70% of APCng time of {$elapsedTimeByDrivername['Prometheus\Storage\APCng']} is not less than APC time of {$elapsedTimeByDrivername['Prometheus\Storage\APC']}"
+ );
+ }
+ }
+
+ /**
+ * @test
+ * @group Performance
+ * Compare the speed of incrementing existing metrics beteween both engines.
+ * Incrementing items should be O(1) regardless of NUM_APC_ITEMS or NUM_PROM_ITEMS
+ */
+ public function testIncrements(): void
+ {
+ $results = [];
+ foreach ($this::NUM_APC_KEYS as $num_apc_keys) {
+ foreach ($this::NUM_PROM_METRICS as $num_metrics) {
+ $this->initializeAPC($num_apc_keys);
+ foreach (['Prometheus\Storage\APC', 'Prometheus\Storage\APCng'] as $driverClass) {
+ // create the prometheus items on the first pass, then increment them (and time it) on the second pass
+ $x = new TestEngineSpeed($driverClass, $num_metrics);
+ $x->doCreates();
+ $begin = microtime(true);
+ $x->doCreates();
+ $elapsedTime = microtime(true) - $begin;
+ fprintf(STDOUT, "%7d APC keys and %4d metrics; Incrementing metrics took %7.4f seconds. (%s)\n", $num_apc_keys, $num_metrics, $elapsedTime, $driverClass);
+ $results["{$num_apc_keys}keys_{$num_metrics}metrics"][$driverClass] = $elapsedTime;
+ }
+ }
+ }
+
+ // should be around the same speed or slightly faster; minimum of 70% the APC speed to account for variations between test runs
+ foreach ($results as $resultName => $elapsedTimeByDrivername) {
+ self::assertThat(
+ $elapsedTimeByDrivername['Prometheus\Storage\APCng'] * 0.7,
+ self::lessThan($elapsedTimeByDrivername['Prometheus\Storage\APC']),
+ "70% of APCng time of {$elapsedTimeByDrivername['Prometheus\Storage\APCng']} is not less than APC time of {$elapsedTimeByDrivername['Prometheus\Storage\APC']}"
+ );
+ }
+ }
+
+ /**
+ * @test
+ * @group Performance
+ * Compare the speed of calling wipeStorage() between both engines.
+ * Clearing cache should be unaffected by the number of objects stored in APC
+ */
+ public function testWipeStorage(): void
+ {
+ $results = [];
+ foreach ($this::NUM_APC_KEYS as $num_apc_keys) {
+ foreach ($this::NUM_PROM_METRICS as $num_metrics) {
+ $this->initializeAPC($num_apc_keys);
+ foreach (['Prometheus\Storage\APC', 'Prometheus\Storage\APCng'] as $driverClass) {
+ $apc = new $driverClass();
+ $registry = new CollectorRegistry($apc);
+ for ($i = 0; $i < $num_metrics; $i++) {
+ $registry->getOrRegisterCounter("namespace", "counter{$i}", "counter help")->inc();
+ $registry->getOrRegisterGauge("namespace", "gauge{$i}", "gauge help")->inc();
+ $registry->getOrRegisterHistogram("namespace", "histogram{$i}", "histogram help")->observe(1);
+ $registry->getOrRegisterSummary("namespace", "summary{$i}", "summary help")->observe(1);
+ }
+ $begin = microtime(true);
+ $x = new TestEngineSpeed($driverClass, $num_metrics);
+ $x->doWipeStorage();
+ $elapsedTime = microtime(true) - $begin;
+ fprintf(STDOUT, "%7d APC keys and %4d metrics; Wiping stored metrics took %7.4f seconds. (%s)\n", $num_apc_keys, $num_metrics, $elapsedTime, $driverClass);
+ $results["{$num_apc_keys}keys_{$num_metrics}metrics"][$driverClass] = $elapsedTime;
+ }
+ }
+ }
+
+ // should be around the same speed or slightly faster; minimum of 70% the APC speed to account for variations between test runs
+ foreach ($results as $resultName => $elapsedTimeByDrivername) {
+ self::assertThat(
+ $elapsedTimeByDrivername['Prometheus\Storage\APCng'] * 0.7,
+ self::lessThan($elapsedTimeByDrivername['Prometheus\Storage\APC']),
+ "70% of APCng time of {$elapsedTimeByDrivername['Prometheus\Storage\APCng']} is not less than APC time of {$elapsedTimeByDrivername['Prometheus\Storage\APC']}"
+ );
+ }
+ }
+
+ /**
+ * @test
+ * @group Performance
+ * Compare the speed of collecting the metrics into a report between both engines.
+ * Enumerating all values in cache should be unaffected by the number of objects stored in APC
+ */
+ public function testCollect(): void
+ {
+ $results = [];
+ foreach ($this::NUM_APC_KEYS as $num_apc_keys) {
+ foreach ($this::NUM_PROM_METRICS as $num_metrics) {
+ $this->initializeAPC($num_apc_keys);
+ foreach (['Prometheus\Storage\APC', 'Prometheus\Storage\APCng'] as $driverClass) {
+ $apc = new $driverClass();
+ $registry = new CollectorRegistry($apc);
+ for ($i = 0; $i < $num_metrics; $i++) {
+ $registry->getOrRegisterCounter("namespace", "counter{$i}", "counter help")->inc();
+ $registry->getOrRegisterGauge("namespace", "gauge{$i}", "gauge help")->inc();
+ $registry->getOrRegisterHistogram("namespace", "histogram{$i}", "histogram help")->observe(1);
+ $registry->getOrRegisterSummary("namespace", "summary{$i}", "summary help")->observe(1);
+ }
+ $begin = microtime(true);
+ $x = new TestEngineSpeed($driverClass, $num_metrics);
+ $x->doCollect();
+ $elapsedTime = microtime(true) - $begin;
+ fprintf(STDOUT, "%7d APC keys and %4d metrics; Collecting/Reporting metrics took %8.4f seconds. (%s)\n", $num_apc_keys, $num_metrics, $elapsedTime, $driverClass);
+ $results["{$num_apc_keys}keys_{$num_metrics}metrics"][$driverClass] = $elapsedTime;
+ $results["{$num_apc_keys}keys_{$num_metrics}metrics"]['n_metrics'] = $num_metrics;
+ }
+ }
+ }
+
+ // should be strictly faster than APC across all runs, by an O(lg2(num_metrics)) factor
+ foreach ($results as $resultName => $elapsedTimeByDrivername) {
+ self::assertThat(
+ $elapsedTimeByDrivername['Prometheus\Storage\APCng'] * (log($elapsedTimeByDrivername['n_metrics'], 2) / 2),
+ self::lessThan($elapsedTimeByDrivername['Prometheus\Storage\APC']),
+ "70% of APCng time of {$elapsedTimeByDrivername['Prometheus\Storage\APCng']} is not less than APC time of {$elapsedTimeByDrivername['Prometheus\Storage\APC']}"
+ );
+ }
+ }
+}
diff --git a/tests/Test/Performance/TestEngineSpeed.php b/tests/Test/Performance/TestEngineSpeed.php
new file mode 100644
index 00000000..160899b2
--- /dev/null
+++ b/tests/Test/Performance/TestEngineSpeed.php
@@ -0,0 +1,61 @@
+num_metrics = $num_metrics;
+ $this->driver = new $driver();
+ }
+
+ /**
+ * Create new (or increment existing) metrics
+ */
+ public function doCreates(): void
+ {
+ $registry = new CollectorRegistry($this->driver);
+ for ($i = 0; $i < $this->num_metrics; $i++) {
+ $registry->getOrRegisterCounter("namespace", "counter{$i}", "counter help")->inc();
+ $registry->getOrRegisterGauge("namespace", "gauge{$i}", "gauge help")->inc();
+ $registry->getOrRegisterHistogram("namespace", "histogram{$i}", "histogram help")->observe(1);
+ $registry->getOrRegisterSummary("namespace", "summary{$i}", "summary help")->observe(1);
+ }
+ }
+
+ /**
+ * Remove all metrics
+ */
+ public function doWipeStorage(): void
+ {
+ $this->driver->wipeStorage();
+ }
+
+ /**
+ * Collect a report of all recorded metrics
+ */
+ public function doCollect(): void
+ {
+ $registry = new CollectorRegistry($this->driver);
+ $renderer = new RenderTextFormat();
+ $result = $renderer->render($registry->getMetricFamilySamples());
+ }
+}
diff --git a/tests/Test/Prometheus/APCng/CollectorRegistryTest.php b/tests/Test/Prometheus/APCng/CollectorRegistryTest.php
new file mode 100644
index 00000000..a1904128
--- /dev/null
+++ b/tests/Test/Prometheus/APCng/CollectorRegistryTest.php
@@ -0,0 +1,21 @@
+adapter = new APCng();
+ $this->adapter->wipeStorage();
+ }
+}
diff --git a/tests/Test/Prometheus/APCng/CounterTest.php b/tests/Test/Prometheus/APCng/CounterTest.php
new file mode 100644
index 00000000..a190a16a
--- /dev/null
+++ b/tests/Test/Prometheus/APCng/CounterTest.php
@@ -0,0 +1,21 @@
+adapter = new APCng();
+ $this->adapter->wipeStorage();
+ }
+}
diff --git a/tests/Test/Prometheus/APCng/GaugeTest.php b/tests/Test/Prometheus/APCng/GaugeTest.php
new file mode 100644
index 00000000..43c312d9
--- /dev/null
+++ b/tests/Test/Prometheus/APCng/GaugeTest.php
@@ -0,0 +1,21 @@
+adapter = new APCng();
+ $this->adapter->wipeStorage();
+ }
+}
diff --git a/tests/Test/Prometheus/APCng/HistogramTest.php b/tests/Test/Prometheus/APCng/HistogramTest.php
new file mode 100644
index 00000000..7102130c
--- /dev/null
+++ b/tests/Test/Prometheus/APCng/HistogramTest.php
@@ -0,0 +1,21 @@
+adapter = new APCng();
+ $this->adapter->wipeStorage();
+ }
+}
diff --git a/tests/Test/Prometheus/APCng/SummaryTest.php b/tests/Test/Prometheus/APCng/SummaryTest.php
new file mode 100644
index 00000000..0a92a5f2
--- /dev/null
+++ b/tests/Test/Prometheus/APCng/SummaryTest.php
@@ -0,0 +1,21 @@
+adapter = new APCng();
+ $this->adapter->wipeStorage();
+ }
+}
diff --git a/tests/Test/Prometheus/Storage/APCngTest.php b/tests/Test/Prometheus/Storage/APCngTest.php
new file mode 100644
index 00000000..d3288d4c
--- /dev/null
+++ b/tests/Test/Prometheus/Storage/APCngTest.php
@@ -0,0 +1,168 @@
+getOrRegisterCounter("namespace", "counter", "counter help")->inc();
+ $registry->getOrRegisterGauge("namespace", "gauge", "gauge help")->inc();
+ $registry->getOrRegisterHistogram("namespace", "histogram", "histogram help")->observe(1);
+ $registry->getOrRegisterSummary("namespace", "summary", "summary help")->observe(1);
+ $apc->wipeStorage();
+
+ $cacheEntries = iterator_to_array(new APCuIterator(null), true);
+ $cacheMap = array_map(function ($item) {
+ return $item['value'];
+ }, $cacheEntries);
+
+ self::assertThat(
+ $cacheMap,
+ self::equalTo([
+ 'not a prometheus metric key' => 'data',
+ ])
+ );
+ }
+
+ /**
+ * @test
+ */
+ public function itShouldUseConfiguredPrefix(): void
+ {
+ $apc = new APCng('custom_prefix');
+ $apc->wipeStorage();
+
+ $registry = new CollectorRegistry($apc);
+
+ $registry->getOrRegisterCounter('namespace', 'counter', 'counter help')->inc();
+ $registry->getOrRegisterGauge('namespace', 'gauge', 'gauge help')->inc();
+ $registry->getOrRegisterHistogram('namespace', 'histogram', 'histogram help')->observe(1);
+ $registry->getOrRegisterSummary("namespace", "summary", "summary help")->observe(1);
+
+ $entries = iterator_to_array(new APCUIterator('/^custom_prefix:.*:meta$/'), true);
+
+ $cacheKeys = array_map(function ($item) {
+ return $item['key'];
+ }, $entries);
+
+ self::assertArrayHasKey('custom_prefix:counter:namespace_counter:meta', $cacheKeys);
+ self::assertArrayHasKey('custom_prefix:gauge:namespace_gauge:meta', $cacheKeys);
+ self::assertArrayHasKey('custom_prefix:histogram:namespace_histogram:meta', $cacheKeys);
+ self::assertArrayHasKey('custom_prefix:summary:namespace_summary:meta', $cacheKeys);
+ }
+
+ /**
+ * @test
+ * Ensure orphaned items added after the metainfo cache has been created get "picked up" and stored
+ * in the metainfo cache. Ensure the TTL is honored (off-by-one errors in APCu TTL handling notwithstanding).
+ */
+ public function itShouldHonorMetainfoCacheTTL(): void
+ {
+ $ttl = 1; // 1-second TTL
+
+ $apc = new APCng();
+ $apc->setMetainfoTTL($ttl);
+ $registry = $this->metainfoCacheTestPartOne($apc);
+
+ $metrics = $apc->collect();
+ $metricHelpStrings = array_map(function ($item): string {
+ return $item->getHelp();
+ }, $metrics);
+ self::assertContains('gauge help', $metricHelpStrings);
+ self::assertNotContains('counter help', $metricHelpStrings);
+ self::assertContains('histogram help', $metricHelpStrings);
+ self::assertContains('summary help', $metricHelpStrings);
+
+ // Let the TTL expire, the hidden metric will appear. Increment counter before & after cache expiry to prove all inc() calls were processed
+ $registry->getOrRegisterCounter("namespace", "counter", "counter help")->incBy(3);
+ sleep($ttl + 1); // APCu needs one extra second. Off-by-one error somewhere?
+ $registry->getOrRegisterCounter("namespace", "counter", "counter help")->incBy(5);
+ $metrics = $apc->collect();
+ foreach ($metrics as $metric) {
+ if ('counter' === $metric->getType()) {
+ self::assertEquals(9, $metric->getSamples()[0]->getValue());
+ }
+ }
+ $metricHelpStrings = array_map(function ($item): string {
+ return $item->getHelp();
+ }, $metrics);
+ self::assertContains('gauge help', $metricHelpStrings);
+ self::assertContains('counter help', $metricHelpStrings);
+ self::assertContains('histogram help', $metricHelpStrings);
+ self::assertContains('summary help', $metricHelpStrings);
+ }
+
+ /**
+ * @test
+ */
+ public function itShouldHonorZeroMetainfoCacheTTL(): void
+ {
+ $this->metainfoCacheDisabledTest(0); // cache disabled
+ }
+
+ /**
+ * @test
+ */
+ public function itShouldHandleNegativeMetainfoCacheTTLAsZero(): void
+ {
+ $this->metainfoCacheDisabledTest(-1);
+ }
+
+ /* Helper function for metainfo cache testing, reduces copypaste */
+ private function metainfoCacheTestPartOne(APCng $apc): CollectorRegistry
+ {
+ apcu_clear_cache();
+
+ $registry = new CollectorRegistry($apc);
+ $registry->getOrRegisterGauge("namespace", "gauge", "gauge help")->inc();
+ $registry->getOrRegisterHistogram("namespace", "histogram", "histogram help")->observe(1);
+ $registry->getOrRegisterSummary("namespace", "summary", "summary help")->observe(1);
+
+ $metrics = $apc->collect();
+ $metricHelpStrings = array_map(function ($item): string {
+ return $item->getHelp();
+ }, $metrics);
+ self::assertContains('gauge help', $metricHelpStrings);
+ self::assertNotContains('counter help', $metricHelpStrings);
+ self::assertContains('histogram help', $metricHelpStrings);
+ self::assertContains('summary help', $metricHelpStrings);
+
+ $registry->getOrRegisterCounter("namespace", "counter", "counter help")->inc();
+ return $registry;
+ }
+
+ /* Helper function for metainfo cache-disabled results testing, reduces more copypaste when only $ttl is changing */
+ private function metainfoCacheDisabledTest(int $ttl): void
+ {
+ $apc = new APCng();
+ $apc->setMetainfoTTL($ttl);
+ $this->metainfoCacheTestPartOne($apc);
+ $metrics = $apc->collect();
+ $metricHelpStrings = array_map(function ($item): string {
+ return $item->getHelp();
+ }, $metrics);
+ self::assertContains('gauge help', $metricHelpStrings);
+ self::assertContains('counter help', $metricHelpStrings);
+ self::assertContains('histogram help', $metricHelpStrings);
+ self::assertContains('summary help', $metricHelpStrings);
+ }
+}
diff --git a/tests/bootstrap.php b/tests/bootstrap.php
index f44f7215..12456d29 100644
--- a/tests/bootstrap.php
+++ b/tests/bootstrap.php
@@ -14,5 +14,6 @@
}
$loader = require $autoload;
$loader->add('Test\\Prometheus', __DIR__);
+$loader->add('Test\\Performance', __DIR__);
define('REDIS_HOST', isset($_ENV['REDIS_HOST']) ? $_ENV['REDIS_HOST'] : '127.0.0.1');