DPS calculator is inaccurate in many situations due to prayer assumptions

[oolong-osrs]

Related plugin component(s)
Deserved Damage

Describe the bug
Plugin assumes that the appropriate offensive prayer (from Piety, Rigour, and Augury) is always used when calculating deserved damage. This results in overestimates of deserved damage, as well as overestimates of your expected freeze count. This is especially noticeable in Pure/Zerker LMS, where only the 15% boost prayers are accessible.

To reproduce
Steps to reproduce the behavior:

1. Start LMS game.
2. Play your first fight without using offensive prayers at all. Eat to ensure a long fight.
3. Observe that your expected damage is much higher than your actual damage.

Or just play a lot of LMS and observe the statistics over time. The expected damage does not converge to the actual damage.

Expected behavior
Ideally the plugin actually reads the player's prayers to calculate damage.

I imagine this isn't feasible, so I think that there should be a configurable setting for prayer assumptions.

Current assumptions around prayer usage
Right now, the prayer assumptions are highly opinionated:

pvp-performance-tracker/src/main/java/matsyir/pvpperformancetracker/controllers/PvpDamageCalc.java

Line 67 in 5672af3

// Offensive pray: assume you have valid. Piety for melee, Rigour for range, Augury for mage

// Offensive pray: assume you have valid. Piety for melee, Rigour for range, Augury for mage
...
// Defensive pray: Assume you have one of the defensive prays active, but don't assume you have augury
// while getting maged, since you would likely be planning to range or melee & using rigour/piety instead.

While these settings are certainly a reasonable use-case, it's not at all the behavior that I would expect.
The assumptions are:

1. Player has access to Piety/Rigour/Augury
2. Player always uses correct offensive prayer from Piety/Rigour/Augury
3. When attacking with any style, the opponent is using a +25% def prayer (Piety/Rigour/Augury).
4. When attacking with magic, opponent is not praying Rigour (optimal tank prayer)

The consequences of these assumptions:

1. Attack roll is consistently overestimated if a player doesn't have access to Piety/Rigour/Augury (for example, in Pure/Zerker LMS)
2. Attack roll is consistently overestimated when a player misses an offensive prayer.
3. Defense roll is consistently overestimated if the target player doesn't have access to Piety/Rigour/Augury. (I think this effect is overshadowed by the attack roll overestimation.)
4. Mage defense roll is consistently underestimated if target player uses Augury or Mystic Might to tank mage hits.

Thus if you play a lot of LMS and always use the correct offensive prayer, your long term stats will show that deserved damage is higher than actual damage ("unlucky"), because it's assuming that you have access to Piety whenever you're playing Pure/Zerker. If your opponents are using Augury or Mystic Might as defensive prayers against mage, the tracker will also show you as being unlucky on freezes.

The worse you are at using offensive prayers, the more visible this discrepancy will be.

Suggested config options
For offensive prayer assumptions, the following settings are all reasonable:
O1. No offensive prayer.
O2. Always using 15% accuracy prayers.
O3. Always using Piety/Rigour/Augury prayers. (this is current behavior)

For defensive prayer assumptions:
D1. No defensive prayer.
D2. Always praying +15% Def, +15% Mage Def. (optimal tanking for pure/zerk)
D3. Always praying Augury. (optimal tanking for mains)
D4. Always praying Piety. (this is current behavior)
(Note: Piety/Augury/Rigour all have the same def bonus, but Rigour has additional mage def.)

To avoid configuration bloat, we can package these together into sensible presets. For example, we can ignore the use-case where the attacker is using 15% accuracy prayers and the defender is using Augury.
A. Perfect Offense + Defense (Mains): O3 + D2
B. Perfect Offense + Defense (Pure/Zerk): O2 + D2
C. Optimistic Main: O3 + D1
D. Pessimistic Main: O1 + D3
E. Optimistic Zerk/Pure: O2 + D1
F. Pessimistic Zerk/Pure: O1 + D2
G. Prayer Off: O1 + D1
H. Legacy behavior: O3 + D4

Each "optimistic" setting results in expected damage/freezes being overestimated. If you miss a lot of offensive prayers, then you will appear to be particularly unlucky.
Each "pessimistic" setting results in expected damage/freezes being underestimated. Using your offensive prayers well will make your stats appear to be lucky.

[oolong-osrs]

The pessimistic settings are not really useful in modeling real combat situations. I wouldn't use pessimistic settings personally, so I'm not that attached to the idea.

It's just annoying that every LMS lobby is full of people complaining about their RNG on this plugin, because the default settings make everyone appear unlucky on Pure/Zerk mode. And you appear more unlucky if you are missing a significant number of offensive prayers. You also appear to be unlucky on freezes whenever your opponent is good at tanking them with Mystic Might or Rigour.

So pessimistic mode is proposed to allow people to feel lucky instead of unlucky: assume a 0% success rate on prayers instead of a 100% rate. Perhaps a more realistic setting would be to assume something like 70-80% success, but I think that's pretty arbitrary and it's probably not worth adding another configuration parameter for "assumed on-prayer success rate".

[Matsyir]

Great writeup. Very understandable concerns regarding the existing assumptions, however I have mixed feelings about this idea.

From a technical and theoretical perspective, I think the idea is great. But I fear it would also cause more confusion surrounding the tracker and exacerbate the problem of people complaining about their RNG, or some would just rag on you for not using their preferred "assumption setting" if you discussed further and disagreed on which setting makes more sense for that fight. A large amount of users (maybe even the majority) already don't touch the config at all or even realize any assumptions are being made in the calculations, they often just assume it works perfectly. I can't really blame them for that, I can't say I dive into every hub plugin's wiki/readme to fully understand the details, and without knowing anything about how RL plugins are built, you wouldn't have much reason to assume there are inaccuracies until you experience them for yourself.

While the current behavior is nowhere near perfect, I like how it keeps everyone's tracker a bit more consistent and easier to compare between different clients - especially in LMS, where most settings are automatically forced/balanced and thus both players' trackers should be nearly the same. With a new setting like this, you'd never really know how to respond to someone complaining about either their rng or the tracker's behavior, because you wouldn't know what setting they're on, and their tracker may look completely different from yours (though I'd imagine most would be using the default option).

Me & Mazhar definitely did consider some of these options when deserved damage was initially added, and I felt like this statement somewhat summarized our thoughts on adding any options like this, along with the idea to just keep it as neutral and consistent as possible for both players:

Perhaps a more realistic setting would be to assume something like 70-80% success, but I think that's pretty arbitrary and it's probably not worth adding another configuration parameter for "assumed on-prayer success rate".

Then again, the existing behavior is also pretty arbitrary so I definitely see where you're coming from. But at least it's currently consistent for everyone. I dunno, I like it but I feel like most users are not ready to use and discuss this option properly.

---

One part of this issue could definitely be addressed without any confusion or drama - it should be possible to, when the player's defence level is found to be <60 when the fight starts, apply O2 (Always using 15% accuracy prayers - pure/zerk best) & D2 (15% def/magedef instead of 25%) or maybe D2 without the magedef, if we are otherwise keeping the current behavior... This would at least provide more realistic expectations for pure/zerk fights without adding any heavy options like this.

[oolong-osrs]

I like how it keeps everyone's tracker a bit more consistent and easier to compare between different clients

Yeah, that's reasonable. I admittedly don't pay much attention to plugin configurations when a plugin appears to be working as intended.

What do you think about a single checkbox configuration?

☑ Optimal mage tanking
Alt-text: "If checked, assume your opponent is always praying Augury or Mystic Might for mage defense."

This would at least provide more realistic expectations for pure/zerk fights without adding any heavy options like this.

This would be great.

Ultimately, what I'm hoping for is a more advanced feedback mechanism for the player's offensive prayers. I'd like to be able to look back at my fight log and observe the difference between attacking with the correct/incorrect offensive prayer. At present, the feedback I get is "I got X/Y offensive prayers correct," but the value of having the right offensive prayer is situational. It depends on the particular combat interaction.

Example 1a: If you're bolting someone in robes as they protect range, rigour is relatively low value, since you don't need the accuracy and the extra damage is muffled by the overhead.

Example 1b: If you're bolting someone in tank as they pray the wrong overhead, rigour is much higher value. (theoretically, at least. I didn't actually calculate any numbers for these examples.)

Example 2: If you have a high enough mage bonus and your opponent has low enough mage def, using augury offers only a marginal increase in accuracy.

Example 3: If you use your melee spec without your best melee prayers, it's a blunder that can lose you the fight.

I think the tracker could be a useful tool for asking questions like "Do I really need to use the 15% attack prayer in pure mode?", but this can't be done at present.

I imagine there's some technical obstruction to reading the offensive prayer when damage is calculated during a fight. But the prayer is available in the log, so it seems like there should be enough information to get accurate calculations. Maybe there could be a post-fight analysis that looks at correct/incorrect offensive prayers and recalculates the expected damage?

[Matsyir]

I discussed this feature (the initial detailed one from the OP) with Mazhar and he also thinks it's a good idea. We are willing to implement it in some form, maybe with some small adjustments, but as long as the default option is a sensible configuration it shouldn't affect most users too significantly while also giving them more realistic expectations than the existing behavior, plus of course the option to configure it how you'd prefer or expect your fights to go.

---

What do you think about a single checkbox configuration?

I do think this is a reasonable compromise, but it kind of introduces the same issue of changing up the tracker's behavior between users, without being too customizable or accurate anyways - so kinda meh but I do appreciate the simplicity of it. If we are adding something like this, I'd rather just go with the initially suggested options to provide further customization.

Yeah, you give a good description and examples of the issues with the existing assumptions. I definitely hear you on that front, we've observed and tried to explain a lot of these situations which can be very hard to understand if you don't already know all the existing assumptions being made.

I imagine there's some technical obstruction to reading the offensive prayer when damage is calculated during a fight. But the prayer is available in the log, so it seems like there should be enough information to get accurate calculations. Maybe there could be a post-fight analysis that looks at correct/incorrect offensive prayers and recalculates the expected damage?

Correct, we can accurately detect the offensive prayer, however that's only available for the local/client player, so to make the deserved damage fair for both players it uses these assumptions. The correct prayer is still saved, for the "fight merge / fight analysis" feature, which takes data from one fight from two clients and merges them to provide more accurate stats (this feature has some occasional issues that cause it to fail completely but it usually works). This might be exactly the feature you had in mind, but it's a bit tedious to use (and requires you to be in communication with your opponent outside of the game).

It is in the realm of possibility to have users opt into using a server and automatically perform some type of merge like that at the end of the fight, though that would be a very big task to undertake properly and for now we'll leave that for another issue/another time.

[oolong-osrs]

This might be exactly the feature you had in mind, but it's a bit tedious to use (and requires you to be in communication with your opponent outside of the game).

One approach is to extend this "fight merge" feature to allow you to merge a fight against certain "dummy players". That is, rather than actually asking your opponent for their data, you could use some preset options like "assume your opponent has perfect prayers" and "assume your opponent has no prayers".

[Matsyir]

I still don't see myself personally implementing this anytime soon, but while looking at the plugin due to the recent issues I had some good ideas surrounding this feature, although it's more optional & nice-to-haves:

Instead of using the config for this feature, we could bring in a lot of the work from the ui-rework branch (which introduced tabs to the side-panel for multiple "sections"); I'm pretty sure the tabs were working nicely in this branch, but the intended additional content inside of the sections was incomplete. However, it's been a while since I touched that branch so I have no idea to what extent it's possible to just bring over those changes. I believe the initial intent of tabs in that branch was just to provide detailed information about how each statistic works, and links to the various wiki sections. I considered an entire custom config as well, so I could do things like hide the overlay options if overlay is disabled, but that was kind of too big of a scope for the custom config.

Regardless of the original intent of those tabs, one of those new tabs could be dedicated to your "fight configuration", i.e:

• The prayer-assumption-style described in this issue
• Stats/levels (with a dropdown to common builds, like max main, max pure, max zerk, etc)
• Possibly, a dropdown for which prayer book, if we get around to implementing that (no idea if that thing is used). This might make more sense to include/combine with the prayer-assumption-style thing.

This should make the feature much more accessible, and easy to use for those who commonly switch between different builds or expected playstyle/performance. But as I said, it's really just nice-to-haves, and if anyone wants to try implement this feature without bothering with all that UI stuff, go for it.