A Call to Arms - Cataclysm Mechanics testing

[theckhd]

I'll try to start working on parses again, just been very out of it and sick -- problem is that I dont have any SP items, but I know of 3 I can get easily. Expect me to hit Grol later today.

It might be easier to use the Argent Tournament dummies. I've heard that they count as low-HP and you can chain HoW them on cooldown. The level of the target should be completely irrelevant for this test.

[Rabs]

Ideally I'd like to have two sets with the same AP but vastly different SP to nail down the SP scaling, and a couple more at whatever AP/SP values I can get my hand on. If anyone has time for this, please parse around 50 HoW's in as many different gear configs as you have time for.

HoW:

• 50 HoW's.
• Stats: 820 AP, 3177 SP, level 80 Melee Target at the Argent Tournament Grounds.

http://www.worldoflogs.com/reports/j3dryvg2ze037l9q/

Editting this post in a few minutes with a parse with lower SP but identical AP

Edit:

• ~50 HoW's.
• Stats: 820 AP, 400 SP, level 80 Melee Target at the Argent Tournament Grounds.

http://www.worldoflogs.com/reports/ktfg5wrb60ltgw32/

Edit2:

• ~50 HoW's.
• Stats: 3822 AP, 1168 SP, level 80 Melee Target at the Argent Tournament Grounds.

http://www.worldoflogs.com/reports/3bu0x3dd73t1lubx/

[theckhd]

That's.... very interesting.

Code: Select all

  AP    SP    D1    D2
820  3177  5442  5464
820   400  5046  4981
3822  1168  6714  6717

D1 and D2 are the average damage calculated two ways - D1 just uses the average hit value, D2 is a weighted average of hits, crits, and blocks (i.e. Nhit*Dhit+Ncrit*Dcrit/2+Nblock*Dblock/0.7)/(Nhit+Ncrit+Nblock)).

The first two traces unambiguously show that the SP scale factor is not 0.4. Using D1 and D2 I get:
(5442-5046)/(3177-400) = 0.1426
(5464-4981)/(3177-400) = 0.1739

Both of these are way off of 0.4, but straddle the old coefficient of 0.15. It's also trivial to show that if you take D1 or D2 and subtract off 0.4*(AP+SP), the result (which should be base damage) isn't consistent, indicating that something's wrong with the model.

If we look at the third parse, it gets more interesting. Taking the extra damage from 2->3, subtracting off the expected contribution from SP with a coefficient of 0.15, and dividing by the AP difference gives us the following AP coefficient:
((6717-4981)-0.15*(1168-400))/(3822-820) = 0.5399

Repeating this for the other three of the four possibilities (D1 and D2 for traces 1 and 2) gives us 0.5183, 0.5178, 0.5251, or a mean of 0.5253.

I can't use the nonlinear fitting algorithm without a few more data sets, but a model of:

4500+x*AP+y*SP

gives me less than one percent error for a continuum of [x,y] pairs in the 0.5-0.55 and 0.15-0.19 range.

For the moment, I'm going to guess at 4500+0.53*AP+0.17*SP, since that puts me in about the middle of the range.

[theckhd]

Bumping because
A) I need more data sets (and Rabs has worked hard enough already), and
B) I told Redcape I'd request some beta parses to see if the scaling is the same on beta.

[Klaudandus]

Dalaran Sword on Melee dummy from Argent Tournament Grounds.

Hammer of Wrath Test

3058 AP/968 SP -- http://www.worldoflogs.com/reports/060h0bb7hfkg5yp5/

Code: Select all

STR 1419
AGI 161
STA 3958
INT 177
SPI 123

3058 AP/1172 SP -- http://www.worldoflogs.com/reports/a6bjia6ln0k5ve6a/

Code: Select all

STR - 1419
AGI - 161
STA - 4180
INT - 331
SPI - 221

530 AP/393 SP -- http://www.worldoflogs.com/reports/3kd29kjrai8np268/

Code: Select all

STR 155
AGI 96
STA 580
INT 310
SPI 201

530 AP/189 SP -- http://www.worldoflogs.com/reports/pwr24b9eveh1w8yg/

Code: Select all

STR - 155
AGI - 96
STA - 245
INT - 106
SPI - 103

DONE

[theckhd]

Thanks, I'll take a look at those tomorrow.

Glyph of HotR testing:
Glyphed: http://www.worldoflogs.com/reports/rt-5 ... details/3/
No Glyph: http://www.worldoflogs.com/reports/rt-l ... details/0/

Using Last Word (277 ilvl, 1.8 speed and 315-586 damage), 3810 AP, 1165 SP, on two lvl 80 dummies. (edit: these aren't including the BoL buff).

Difference of 11 damage in the Nova portion (11/1780=0.0062, or 0.62% difference, not statistically significant)
Difference of 21.8 damage in the physical portion, (21.8/273.4=0.0797, or 7.97% difference).

This means that:
1) HotR glyph is still bugged
2) It's probably additive with Crusade rather than multiplicative:

(1.3+0.1)/1.3=1.077, or a 7.7% increase.

[theckhd]

T10 2-piece testing:
2pc: http://www.worldoflogs.com/reports/rt-1 ... details/5/
no 2pc: http://www.worldoflogs.com/reports/rt-4qv1yvph5o4vjcez/

Last Word, 3772 AP, 1173 SP (with BoLight buff)

Again, 10 damage difference in the Nova, (10/1713.5=0.0058 or 0.58% difference, not statistically significant)

Melee damage was poisoned thanks to annoying warrior, retook the data on a 70 dummy:
no 2pc: http://www.worldoflogs.com/reports/rt-n ... details/6/
2pc: http://www.worldoflogs.com/reports/rt-s ... details/6/

Again, no significant difference in Nova damage.
Melee: (289.6-258.7)/258.7=0.119, or 11.9%. A little lower than the expected 15.38% if it's additive with Crusade. But it's closer to that than 20%, so chances are it's additive.

[theckhd]

Dalaran Sword on Melee dummy from Argent Tournament Grounds.

Hammer of Wrath Test
....

DONE

Was that data from live or beta Klaud? I can get it to fit reasonably well with 4585 base and 0.42*(AP+SP), though the nonlinear fitting algorithm seems to want it to be 0.466*AP+0.285*SP. Either way, it doesn't agree at all with Rabs' data - his SP scaling is much lower than yours.

If it's from Beta, it's a good chance that it's still 0.42*(AP+SP). If it's from live... then something weird is going on.

[Klaudandus]

That was live. If I don't have RS25 in the next 10 mins, I'll parse 'em again if you need me to do so -- otherwise, I'll do everything again tomorrow.

Forgot to mention that I had kings at that time -- but should be reflected on the initial stats -- since I wrote down the stat breakdown per gear set for each different parse.

[theckhd]

OK, that's definitely weird. I'm going to need more data points. If you (or others) could repeat those tests with different gear sets when you have some free time, I'd appreciate it. The more different AP/SP combinations the better the fit.

Also, AP-only changes would be good if you can get them. Things like with/without Might, for example.

It also might help to increase the parses to 100 casts. The damage range on HoW is high enough that smaller parses can be misleading.

[Malthrax]

Dunno how much help these are, my gear options are a tad limited

Each run ~50 HoW on the Argent Tournament Dummy, wielding a Dalaran Sword. No buffs other than Seal of Insight for mana recovery.


AP:516, SP:179 - http://www.worldoflogs.com/reports/w43a ... amageDone/

AP:516, SP:520 - http://www.worldoflogs.com/reports/m2mk ... amageDone/

AP:1210, SP:179 - http://www.worldoflogs.com/reports/j2x7 ... amageDone/

AP:3410, SP: 1058 - http://www.worldoflogs.com/reports/6p2x ... amageDone/

AP:1529, SP: 179 - http://www.worldoflogs.com/reports/awrj ... amageDone/

AP:1903, SP: 179 - http://www.worldoflogs.com/reports/yq19 ... amageDone/

[theckhd]

Been doing some fooling around with your data. There are a few clear conclusions to be had.

Code: Select all

          AP          SP          D1          D2
         516         179        4899        4902
         516         520        4918        4917
        1210         179        5204        5230
        3410        1058        6488        6490
        1529         179        5431        5388
        1903         179        5657        5628

D1 and D2 are the same as last time - D1 is just hits, D2 is a weighted average of hits and crits. To see what the AP scaling is, we can look at sets 1, 3, 5, and 6, which all had the same SP. For SP scaling, we could only directly compare 1 & 2, which gives an abnormally low coefficient (for reasons which will be obvious in a second). However, we can infer some data by calculating the AP coefficient from 1/3/5/6 and subtract estimated damage due to AP to get residual damage. That's what I've plotted below; The green dots are sets 1/3/5/6, the blue line is a fit to that data, the black dots and red line are the residuals plotted against SP:



Fit details - AP:

Code: Select all

Linear model Poly1:
       f(x) = p1*x + p2
Coefficients (with 95% confidence bounds):
       p1 =      0.5159  (0.3987, 0.6332)
       p2 =        4622  (4459, 4784)

Goodness of fit:
  SSE: 1542
  R-square: 0.9945
  Adjusted R-square: 0.9917
  RMSE: 27.77

Fit details - SP:

Code: Select all

Linear model Poly1:
       f(x) = p1*x + p2
Coefficients (with 95% confidence bounds):
       p1 =      0.1211  (0.05005, 0.1921)
       p2 =        4598  (4563, 4634)

Goodness of fit:
  SSE: 1677
  R-square: 0.8485
  Adjusted R-square: 0.8106
  RMSE: 20.48

First of all, the AP fit is good, but not terribly convincing. The fit predicts a coefficient 0.52, but the 95% confidence interval includes everything from 0.399 and 0.633.

The SP data is trickier. The only two "pure" data points in this set are 1 & 2, which gives something between 15/341=0.044 and 19/341=0.0557. Both of those are ridiculously low, and probably due to the large base damage range of HoW. We're looking at a small increase in a measurement that has a large range, which makes it hard to get good fits without larger data sets.

If we subtract off 0.5159*AP from the damage data to get residuals, and plot that against AP, we get the black dots and the red line as a fit. This certainly suggests that SP scales differently from AP, though it's not a completely fair test, as we've imposed an AP scaling on the data.

If we plot D2 vs (AP+SP) and try and fit it, we get the following:

Code: Select all

Linear model Poly1:
       f(x) = p1*x + p2
Coefficients (with 95% confidence bounds):
       p1 =      0.4326  (0.3427, 0.5225)
       p2 =        4605  (4402, 4809)

Goodness of fit:
  SSE: 3.827e+004
  R-square: 0.9781
  Adjusted R-square: 0.9726
  RMSE: 97.82


Which certainly doesn't look that bad - the fit isn't as good, but it's not terrible either. The average error is a bit higher, but would agree with Redcape's 0.42*(AP+SP) model. That said, I don't think this is correct - more likely than not, this is what you'd stumble across with a limited data set, and only after scrutinizing more data would you notice that AP and SP seem to scale very differently.

In particular, this is what happens if we include the data from Klaud and Rabs:

Code: Select all

Linear model Poly1:
       f(x) = p1*x + p2
Coefficients (with 95% confidence bounds):
       p1 =      0.3867  (0.2935, 0.4799)
       p2 =        4613  (4344, 4883)

Goodness of fit:
  SSE: 6.369e+005
  R-square: 0.8835
  Adjusted R-square: 0.8729
  RMSE: 240.6


The data point from Rabs with very high SP really demonstrates the problem. If the scaling was equal (and thus depended only on SP+AP), then we wouldn't expect that data point to be almost 800 damage lower than the curve. In general, the fits would all be a lot better as well. So Rabs' data pretty much seals the deal - there's no question that SP is scaling at a much lower rate than AP.

If we plot the residuals again (but for all of the data this time), we get:

Code: Select all

Linear model Poly1:
       f(x) = p1*x + p2
Coefficients (with 95% confidence bounds):
       p1 =      0.1416  (0.1166, 0.1665)
       p2 =        4581  (4554, 4609)

Goodness of fit:
  SSE: 1.173e+004
  R-square: 0.9342
  Adjusted R-square: 0.9282
  RMSE: 32.65

Again, not perfect, but pretty good, and far better than the equal-scaling model.

Finally, let's do something a little more rigorous than simple linear fits to plots. We'll go back to our nonlinear regression function (nlinfit in MATLAB) and calculate the coefficients and confidence intervals using all of the data:

Code: Select all

X=[ones(size(a)) a s];Y=d2;
[BETA,R,J,COVB,MSE]=nlinfit(X,Y,@myfun,[4600;0.52;0.14]);CI=NLPARCI(BETA,R,'covar',COVB);

gives us (myfun is a simple X*B, where B is the column vector of coefficients passed to nlinfit):

Code: Select all

           Fit      95% Conf. Int.
Base      4584      4546      4623
AP      0.5138    0.4957    0.5320
SP      0.1423    0.1152    0.1693

The first thing to note is that 0.42 is completely outside of both confidence intervals. The second is that we're definitely narrowing down on the correct values, though there's still quite a bit of wiggle room in the SP coefficient.

Finally, here's all of the data used along with the residuals R (i.e. the error from the fit):

Code: Select all

  AP    SP    D2    R
516   179  4902   26.8343
516   520  4917   -6.30559
1210   179  5230   -1.28829
3410  1058  6490    2.7306
1529   179  5388   -7.56617
1903   179  5628   40.1567
3058   968  6256  -37.6434
3058  1172  6344   21.4639
530   393  4934   21.4025
530   189  4897   12.9587
820  3177  5464    6.31203
820   400  4981  -81.6269
3822  1168  6717    2.57167

Note that almost all of the data points fall within about 20 damage from the expected value, and the one that's fairly far off is at low AP/SP where we expect the base damage variance to be a large source of error.

Conclusion: Hammer of Wrath's damage formula at 80 is approximately 4585+0.515*AP+0.14*SP. Some clarification on the SP coefficient would be nice if someone doesn't mind suiting up in Holy Gear/Spec and taking some data covering the range from about 200 SP to 4k SP.

[Arianne]

Would Holy gear and Ret spec be ok? If not, do you want Holy gear and no spec in addition to Holy gear and Holy spec? Do you have a particular spec in mind?

[theckhd]

Would Holy gear and Ret spec be ok? If not, do you want Holy gear and no spec in addition to Holy gear and Holy spec? Do you have a particular spec in mind?

I guess spec should be irrelevant if you're just using Holy gear. The main thing we want to test is the SP scaling, so we want a bunch of data sets with the same AP but different values of SP. Holy Gear should do that, since it will have Int but no STR.

The advantage of Holy spec is that you don't have the STR->SP conversion, so you can test AP and SP essentially independently by wearing prot gear and holy gear. But I think we have the AP coefficient mostly nailed down, so I'd rather focus effort on the SP scaling right now.

[Xenix]

I've got some spellpower scaling data on HoW for you - since I didn't have my combat log text parsing code handy, I just did this in Excel to be lazy (text-to-columns ftw). I had a constant 544 ap and varied sp from 179 to 3534 with around 70 HoW's per data point. Standard deviation on the HoW damage was 2.7% on average.

First, here is the world of logs report. I didn't realize WoL wouldn't separate out my different combats, but I split them up by just looking for any time interval > 20 seconds between HoW's. Next, here is an excel file of all the data with ap/sp/average/stdev calculated (For any crits, I divided the damage from WoL by 2). Plotting that data and having Excel calculate a trendline versus spellpower gives the following (X-axis is SP, Y-axis is damage):



The 6 significant digit scaling coefficient I get for these six data points is 0.151276 with R Squared of 0.999873.

Edit: One other note - I did this test in pure holy gear and prot spec so I wouldn't screw up the numbers with the Conviction talent. Also, I just threw out the few blocked hits and blocked crits.