A Call to Arms - Cataclysm Mechanics testing

[theckhd]

Klaud's data sets:
#1 (1010 AP / 497 SP)

Code: Select all

CS       274   273     -1    -0
JoT      604   604      0     0
HW      1301  1301     -0    -0
SoT       29    29     -0    -1
Cens     888   885     -4    -0
HotR      30    31      1     4
HaNova   831   838      7     1
Melee     99    99     -0    -0

#2 (2020 AP / 981 SP) & HoW on Grol from #3

Code: Select all

ShoR    2424     0  -2424  -100
CS       477   478      1     0
JoT     1199  1192     -7    -1
HW      1518  1519      1     0
HoW     1633  1640      7     0
SoT       47    47     -0    -0
Cens    1771  1767     -4    -0
HotR      48    48     -0    -0
HaNova  1004  1032     28     3
Melee    160   161      1     0

HW code seems to be working properly

#4 (Grol HoW):

Code: Select all

HoW     1398  1413     15     1
SoT       29    31      2     8
Cens     860   954     94    11
Melee     99   143     44    44

#5 (Grol HoW "naked"):

Code: Select all

HoW     1287  1326     39     3
Melee     68   117     49    72

Out of curiosity, what did you mean by "20/10" and "10/5"? 20%-10% and 10%-5%? Did you change weapons at that point to get a stat difference? If so, that would explain the SoT/Cens/Melee changes.

Conclusion: HoW formula is correct.

[Klaudandus]

I called 'em 20/10 and 10/5 since I was going for the 2000AP/1000SP and 1000AP/500SP combos, so I named 'em like that.

Did not change weapons, still Dalaran sword and same white shield with only armor.

The only thing I believe could have made a difference was the accidental haste that I got while seeking for the SP mark. I think the problem with #2 is that you might have combined my three logs. I used the same stats for 2 different tests. First one was CS on 2 dummies with HW, second was HotR on 2 Dummies with HW, in that case, HoW was never used (since you cant use it against the dummies), third parse was test was Grol and HoW

[theckhd]

Well, the problem isn't in the log I merged, since I just used the Hammer of Wrath data from the Grol log. I didn't merge any of the other skills.

The problem is in #4 and #5, where the other abilities aren't lining up properly. Looking at the code, I treated him as an 80 boss, so that explains the melee being high. Not sure why Censure is high though. Either way, I'm not too concerned.

I think I've managed to get something figured out for Hammer. My first attempts at solving the linear system gave me nonsense for the SP coefficient. On a wild guess that they removed the SP scaling entirely, here's what I get when fitting just the AP and damage values with a linear model:

Code: Select all

Linear model Poly1:
       f(x) = p1*x + p2
Coefficients (with 95% confidence bounds):
       p1 =      0.1865  (0.1242, 0.2488)
       p2 =       653.9  (570.6, 737.3)

Goodness of fit:
  SSE: 28.28
  R-square: 0.9993
  Adjusted R-square: 0.9986
  RMSE: 5.318

It's notable that the old base damage was 653, so this seems pretty likely. Fixing p2, we get:

Code: Select all

Linear model Poly1:
       f(x) = p1*x + p2
Coefficients (with 95% confidence bounds):
       p1 =      0.1871  (0.18, 0.1941)
       p2 =         653  (fixed at bound)

Goodness of fit:
  SSE: 28.83
  R-square: 0.9993

If I had to guess, the formula is (653 + 0.187*AP). We can do a few sanity checks on this though.

#1: Parse with any gear setup, dalaran sword, get at least 30 HotR's in. Leave one spot empty on your gear for an item that has only intellect on it, no STR or AP.
#2: Put that intellect item in the empty gear slot, and repeat. 30 HotR's or more.
#3: Buff yourself with Might, parse again.

Make sure you write down AP and SP for each trial.

This should give us definitive proof that the SP scaling is 0 and should confirm the AP coefficient.

[Klaudandus]

#1: Parse with any gear setup, dalaran sword, get at least 30 HotR's in. Leave one spot empty on your gear for an item that has only intellect on it, no STR or AP.
#2: Put that intellect item in the empty gear slot, and repeat. 30 HotR's or more.
#3: Buff yourself with Might, parse again.

Make sure you write down AP and SP for each trial.

This should give us definitive proof that the SP scaling is 0 and should confirm the AP coefficient.

Ok, I'll have these tomorrow morning.

[Staryx]

The blue line is Staryx's data set, the red and green lines are Kierly and Arianne respectively. These are the single-target damage values, but the same effect happens in the 3-target tests. From this we have to conclude that something about those two gear sets is affecting the HW damage, and ignore those two points.

It was indeed my Ret gear that messed with those final 2 points. They changed the 2pT10 bonus for Ret to increase all damage by 5%. I failed to notice this prior to testing. Accounting for this should bring my numbers back in line.

[Arianne]

#1: Parse with any gear setup, dalaran sword, get at least 30 HotR's in. Leave one spot empty on your gear for an item that has only intellect on it, no STR or AP.
#2: Put that intellect item in the empty gear slot, and repeat. 30 HotR's or more.
#3: Buff yourself with Might, parse again.

Rhia - 80 Dwarf paladin 0/36/0 with Dalaran Sword hitting 3 dummies (60, 70, 80) with each cast
1: 3706 AP, 1132 spellpower : http://www.worldoflogs.com/reports/oeer5llhxd6w9yrn/
2: 3706 AP, 1285 spellpower : http://www.worldoflogs.com/reports/x0ueyltqmmc4f1lw/
3: 4077 AP, 1285 spellpower : http://www.worldoflogs.com/reports/ez2nzdi16nqmuejy/

[theckhd]

Looks good. Increasing SP by 153 only gives a damage increase of 4.8, well below the noise level (if it were an actual increase, the coefficient would be 0.0314). Increasing AP by 371 gives an increase of 67.9, or 0.183. Within rounding and statistical error, that agrees with the 0.187 coefficient we calculated earlier (371*0.187=69.4).

One final test would be to try comparing naked damage to damage in a set of Holy gear - again, no AP change, but a very significant SP change (several thousand at least). That would cement the lower bound on SP scaling.

[Klaudandus]

standard 0/34/0 lvl 80 BE
Naked
516 AP / 179 SP -- http://www.worldoflogs.com/reports/qcyzcmmb3gddgzdy/

516 AP / 511 SP -- http://www.worldoflogs.com/reports/ymbucvjuoir69a05/

516 AP / 1017 SP -- http://www.worldoflogs.com/reports/fcbmejhfb9n6pmq0/

516 AP / 1613 SP -- http://www.worldoflogs.com/reports/w7h5fjzw4k2oncnx/

Might
568 AP (Might) / 179 SP -- http://www.worldoflogs.com/reports/4zspwowyh1a5z1in/

568 AP (Might) / 1017 SP -- http://www.worldoflogs.com/reports/vlnqsreo579ejab8/

568 AP (Might) / 1613 SP -- http://www.worldoflogs.com/reports/4swcvbakpt8mpsnx/

[theckhd]

Hm... those sample sizes are still giving us a lot of statistical noise. Hammer has a pretty huge damage range, so it's hard to be more accurate than +/- 10 damage with these parses.

That said, here's the data:

Code: Select all

AP   SP   DMG
516   179  746.1
516   511  765.1
516  1017  754.0
516  1613  751.6
568  1017  755.7
568   179  765.9
568  1613  758.8


If we assume that the damage difference is around the noise level (let's say, 10 damage), then seeing that variation over 838 spellpower gives a coefficient of around 0.01, low enough that it's unlikely they'd bother implementing it.

You've already done a lot, so I don't want to ask you to repeat this. But to get a serious determination, I think we'd need to see a parse of the first and third configurations with 1000+ samples. However, since it's spell damage, if you can find a spot where your Hammer hits multiple dummies, you can collect up to 5 or 6 samples per cast, which would significantly reduce the time required.

[twilytgardnfaery]

Looks good. Increasing SP by 153 only gives a damage increase of 4.8, well below the noise level (if it were an actual increase, the coefficient would be 0.0314). Increasing AP by 371 gives an increase of 67.9, or 0.183. Within rounding and statistical error, that agrees with the 0.187 coefficient we calculated earlier (371*0.187=69.4).

One final test would be to try comparing naked damage to damage in a set of Holy gear - again, no AP change, but a very significant SP change (several thousand at least). That would cement the lower bound on SP scaling.

Would a prot/holy parse comparison at 85 be acceptable? My pally hit 82 yesterday, so I don't have base 80 stats anymore, and I haven't got a holy set on the beta because I trashed it about a month before I got my invite in favor of ret

[theckhd]

We're actually looking at lvl 80 PTR data right now, the beta is using an entirely different build.

Eventually I'll need this sort of information for 85 on the beta, but there's no sense doing tests right now since as far as I'm aware, they still haven't pushed the 3-second CS patch to beta yet. We have a few months before tanking at 85 is a serious concern, so I'm focusing my efforts on 4.0.1 right now.

Plus, they're likely to do more numbers passes at 85 before Cataclysm's release, so nailing down exact numbers right now could just be a waste of time.

[Klaudandus]

If we assume that the damage difference is around the noise level (let's say, 10 damage), then seeing that variation over 838 spellpower gives a coefficient of around 0.01, low enough that it's unlikely they'd bother implementing it.

You've already done a lot, so I don't want to ask you to repeat this. But to get a serious determination, I think we'd need to see a parse of the first and third configurations with 1000+ samples. However, since it's spell damage, if you can find a spot where your Hammer hits multiple dummies, you can collect up to 5 or 6 samples per cast, which would significantly reduce the time required.

Can do, problem is getting people to leave the dummies alone for long enough. That's probably the reason why some of my parses seem out of whack, I had this rogue that would sneak from behind and attack the 2nd dummy, at which point I would stop the combat log. That's why some of the logs are rather short. Also, what about the 1613 SP parses? You were probably working already in my first parses when I posted those last two.

I need to work on some documents, as soon as I finish, I'll hop onto the PTR and pray no one bothers me for 30 mins per parse.

[theckhd]

Edited that data into the previous table, that pushes it down to between 0.005 and 0.0091.

Like I said, the damage range on Hammer is just huge, which means that you either need many samples to get accurate statistics or a huge variation in spellpower. I think the 1.6k SP data is probably close enough to conclude it's 0.

That said, if you can find a spot where you can hit 5 dummies, 1000 swings would only take 10 minutes. I haven't been on the PTR at all, so I don't have a feel for how crowded it is. Are Darnassus and Exodar even swamped (or whatever the horde equivalents would be - Silvermoon and TB I guess)?

[Klaudandus]

Even TB is swamped - and the only spot that is secluded only has two dummies, which are those lvl 80 ones. Which sets would you like me to repeat then?

[theckhd]

Probably 516/179 and 516/1613, though if you had a way of getting 2-3k that would be better. You may just want to wait and see if someone with an exceptional Holy gear set is willing to run the experiment. Especially since with only 2 dummies, it could take 30 minutes.