Damage Meters Benchmarking: Recount Maintainer Feedback

Elsia, the current maintainer of the benchmarked Recount, took the time to post extensive feedback both on general accuracy considerations and on the pet damage discrepancies noted with the now outdated version I used during the first part of the series:

1) It’s a very widely held believe that WWS is most accurate and establishes a correct base line.

This believe is not correct for healing. WWS only has as information what the combat log provides, whereas online damage meters have extra information such as the actual current health level and max health of a player. This information is not present in the combat log. It’s important to note that the combat log does not report overhealing.

In game overhealing is calculated from the difference of current max health minus health before the heal landed compared to the incoming heal. If the incoming heal is larger than that difference the remainder is overhealing.

WWS does not have this information, so instead it estimates overhealing by taking the incoming damage number and comparing it to the incoming healing number. If there is more heals than damage that’ll be calculated to be overhealing. This algorithm will lead WWS to underestimate overhealing and overestimate actual healing.

In general for healing volume and overhealing stats in-game damage meters (if not subject to bugs) are more accurate than WWS.

In general one can expect WWS to underestimate overhealing and overestimate healing. This is what you saw in your test.

As a side-remark, the same holds for damage. If a mob has 20 health and is hit by 2000 damage all damage meters online and offline that I know will count the full 2000 damage, whereas actually this should be 20 damage and 2000 over-damage. We don’t really do this because mana-preservation isn’t such a longevity issues as it is for healers. Hence in this department WWS and online damage meters tend to agree very much, because they just calculate total damage done as displayed in the combat log.

In retrospect, I realize that I quite simply stated that WWS would be used as a baseline without positioning its own limitations. Guilty as charged, and Elsia clarified the limitations of both types of measuring (online and combat log parsing) very eloquently. Indeed I should have cautioned somewhere that in this particular case, WWS does not serve as a baseline for absolute accuracy due to its own limitations, but more simply as a trendline which has two advantages, the first one not being actually benchmarked and the second one the trust it enjoys from the player community.

What is certainly remarkable nonetheless is that in each of the exercises so far we have seen a clear and firm majority result which was consistent through different tools as well as the offline measuring offered by WWS. Which speaks quite highly for the huge amount of work produced for each of the addons by their respective authors, small discrepancies notwithstanding. Something I haven’t acknowledged so far.

2) Recount: I have since made various changes and improvements to recount, specifically to pet handling. The current version of recount will properly track pet healing, overhealing on pets, damage done to pets. With 2.4 Recount will also get the pet handling rewritten and I hope that the discrepancy you reported here will be removed by then.

Finally with 2.4 I’d expect the accuracy between meters to normalize. “Parsing” is straight forward, and certain issues that each meter may have handled differently will be unified.

Which is of course excellent news for each addon’s userbase, and provided that every of the surveyed meters gets transitioned over to the new method, users will be able to pick the addon based on the feature set he wants without compromising with accuracy. As the mini-review in the first part of the benchmarking has hopefully hinted at, there is a vast choice available today and each of the tools bring their own unique design and data display to the table.