Theck's MATLAB thread - MoP/5.x

Warning: Theorycraft inside.

Moderators: Fridmarr, Worldie, Aergis, theckhd

Re: Theck's MATLAB thread - MoP/5.x

Postby Thels » Tue Jan 28, 2014 3:39 am

CSw>J>AS+GC>HW>AS>ES+ex>HoW>ES>Cons - Prioritizing regular HW over AS is a clear loss in DPS.

CSw>J>AS+GC>HW+FW>AS>HW>ES+ex>HoW>ES>Cons - This rotation seems to max it out, but the difference is marginal in comparison to a much easier rotation, such as CSw>J>AS>HW>ES>HoW>Cons

Especially considering the differences in results, I don't think there's that much to say for strange Execute rotations. For example, in your previous post, CSw>J>AS>HW>ES+ex>HoW>ES>Cons showed as better than CSw>J>AS>HW>ES>HoW>Cons, but now the reverse is true.

Seems to me that CSw>J>AS>HW>T90>HoW>Cons is the way to go, and any further detailing in the rotation demands attention that is best spent elsewhere.
ImageImage
User avatar
Thels
 
Posts: 1314
Joined: Tue May 15, 2007 7:30 am
Location: The Netherlands

Re: Theck's MATLAB thread - MoP/5.x

Postby theckhd » Tue Jan 28, 2014 7:18 am

Below are updated results, this time with 250k iterations. "DPS Error" drops to about 40 in this set.

I looked into what we're reporting as "DPS Error." It's related to Standard Error of the Mean. In particular, by default it's 1.96*SE, which is the single-sided bound for a 95% confidence interval. In other words, you expect that if you re-run the simulation, 95% of the time your new mean will fall between (mean-1.96*SE) and (mean+1.96*SE).

In that light, the earlier results make a lot of sense. We expect about a +/-90 DPS variation from the mean, or a window of about 200 DPS in which our results will fall. I did some empirical testing of this and found an experimental 95% CI of about 228 DPS with 100 trials, suggesting that this is at least mostly accurate. There are some underlying issues that may or may not be affecting the accuracy of this estimate - in particular we might be weakly violating some of the assumptions inherent in the Central Limit Theorem on which we're relying. Hope to run a bunch more investigation sims and write up a blog post about statistics, because stats are fun.

From a practical point of view, however, it means that we should treat the sims as only accurate to about 2*DPS_Error at a minimum, because for all we know one sim could be on the low end of the CI while the sim we're comparing it to is on the high side. I'd even argue that for true statistical certainty in this case, we might want to use a wider range (like 3* or 4*) just to be safe (essentially using a larger CI, like 99%).

Code: Select all

basic Rotations
Max DPS Error: 37
Talents: 312232
Glyphs: focused_shield/word_of_glory

|               Rotation |    DPS |    HPS |   DTPS |    TMI |   Var |  SotR |  Wait |
|        CS>J>AS>Cons>HW | 373620 | 160001 | 160342 |  10983 |  6156 | 71.0% | 14.5% |
|        CS>J>AS>HW>Cons | 379608 | 159521 | 159854 |   4287 |   971 | 71.4% | 13.2% |
|   CS+W0.3>J>AS>HW>Cons | 373814 | 157738 | 158054 |    533 |   117 | 73.2% | 12.7% |
|       CSw>J>AS>Cons>HW | 368204 | 157862 | 158182 |    460 |    47 | 73.1% | 13.8% |
|       CSw>J>AS>HW>Cons | 373591 | 157666 | 157983 |    427 |    55 | 73.2% | 12.7% |
|       CSw>AS>J>HW>Cons | 373952 | 158483 | 158804 |    451 |    38 | 72.5% | 12.8% |
|       J>CSw>AS>HW>Cons | 368396 | 162575 | 162942 |  90576 | 83031 | 68.5% | 17.4% |
|       J>AS>CSw>HW>Cons | 372886 | 163157 | 163529 |  61525 | 35811 | 67.9% | 17.0% |
|       AS>J>CSw>HW>Cons | 372490 | 163965 | 164342 | 174459 | 57861 | 67.2% | 17.4% |
|       AS>CSw>J>HW>Cons | 378485 | 159759 | 160092 |   1971 |   365 | 71.2% | 13.0% |
|      HotR>J>AS>HW>Cons | 371873 | 159521 | 159857 |   5113 |  1894 | 71.4% | 13.2% |
| AS+GC>CSw>J>AS>HW>Cons | 374633 | 157958 | 158289 |    727 |   145 | 72.9% | 12.7% |
| CSw>AS+GC>J>AS>HW>Cons | 373734 | 157767 | 158086 |    409 |    52 | 73.1% | 12.7% |
| CSw>AS+GC>J>HW>AS>Cons | 373243 | 157700 | 158021 |    391 |    43 | 73.2% | 12.5% |
| CSw>AS+GC>J>HW>Cons>AS | 372838 | 158084 | 158405 |    429 |    79 | 72.8% | 12.3% |


execute Rotations
Max DPS Error: 41
Talents: 312232
Glyphs: focused_shield/word_of_glory/final_wrath
Code: Select all
|                   Rotation |    DPS |    HPS |   DTPS |  TMI | Var |  SotR |  Wait |
|       CSw>J>AS>HW>Cons>HoW | 383714 | 157727 | 158045 |  379 |  30 | 73.2% | 11.2% |
|       CSw>J>AS>HW>HoW>Cons | 384536 | 157678 | 157999 |  436 | 111 | 73.2% | 11.0% |
|       CSw>J>AS>HoW>HW>Cons | 383834 | 157566 | 157879 |  431 | 121 | 73.3% | 10.9% |
|       CSw>J>HoW>AS>HW>Cons | 383380 | 157868 | 158196 |  529 | 135 | 73.0% | 10.9% |
|       CSw>HoW>J>AS>HW>Cons | 383612 | 157968 | 158297 |  519 | 123 | 72.9% | 10.9% |
|       HoW>CSw>J>AS>HW>Cons | 383963 | 158370 | 158738 | 2348 | 761 | 72.5% | 11.0% |
| CSw>J>HW+FW>AS>HW>HoW>Cons | 384673 | 157751 | 158072 |  397 |  41 | 73.2% | 11.0% |
| CSw>HW+FW>J>AS>HW>HoW>Cons | 384846 | 158089 | 158426 |  458 |  65 | 72.8% | 11.0% |
| HW+FW>CSw>J>AS>HW>HoW>Cons | 385184 | 158096 | 158435 |  632 | 229 | 72.8% | 11.0% |


defensive Rotations
Max DPS Error: 40
Talents: 313232
Glyphs: focused_shield/word_of_glory/final_wrath
Code: Select all
|                            Rotation |    DPS |    HPS |   DTPS |    TMI |   Var |  SotR | Wait |
|             CSw>J>AS>HW>HoW>Cons>SS | 367667 | 111879 | 120831 | 230953 | 15250 | 68.8% | 3.3% |
|          CSw>J>AS>HW>HoW>SS+R1>Cons | 370778 | 113784 | 122153 | 151427 |  5737 | 69.9% | 8.9% |
|       CSw>J>AS>HW>HoW>SS+R1>Cons>SS | 367419 | 111758 | 118767 | 116705 |  3947 | 68.8% | 3.3% |
|       CSw>J>AS>HW>SS+R1>HoW>Cons>SS | 367205 | 111699 | 118253 |  99323 |  3247 | 68.8% | 3.3% |
|          CSw>J>AS>SS+R1>HW>HoW>Cons | 368861 | 113136 | 119949 | 100877 |  3655 | 70.0% | 9.1% |
|       CSw>J>AS>SS+R1>HW>HoW>Cons>SS | 366914 | 111684 | 118101 |  98289 |  3051 | 68.8% | 3.3% |
| CSw>J>AS+GC>SS+R1>AS>HW>HoW>Cons>SS | 366800 | 111667 | 118048 |  98425 |  4289 | 68.8% | 3.3% |
|       CSw>J>AS>SS+R2>HW>HoW>Cons>SS | 367064 | 111661 | 118041 |  95695 |  2790 | 68.8% | 3.3% |
|       CSw>J>AS>SS+R3>HW>HoW>Cons>SS | 366863 | 111692 | 118097 |  98717 |  2810 | 68.8% | 3.3% |
|       CSw>J>AS>SS+R4>HW>HoW>Cons>SS | 366868 | 111676 | 118065 |  96511 |  2942 | 68.8% | 3.3% |
|       CSw>J>AS>SS+R5>HW>HoW>Cons>SS | 366886 | 111665 | 118051 |  96597 |  3131 | 68.8% | 3.3% |
|          CSw>J>SS+R1>AS>HW>HoW>Cons | 368547 | 112834 | 119190 |  87898 |  2947 | 70.0% | 9.1% |
|          CSw>SS+R1>J>AS>HW>HoW>Cons | 368234 | 112800 | 119149 |  96260 |  5332 | 69.7% | 9.2% |
|          SS+R1>CSw>J>AS>HW>HoW>Cons | 368466 | 112689 | 119040 |  95784 |  7286 | 69.7% | 9.1% |


talents Rotations
Max DPS Error: 41
Talents: 312232+custom
Glyphs: focused_shield/word_of_glory/final_wrath
Code: Select all
|                                    Rotation |    DPS |    HPS |   DTPS | TMI | Var |  SotR |  Wait |
|                     CSw>J>AS>HW>HoW>Cons>ES | 404748 | 157849 | 158167 | 591 | 332 | 73.1% |  9.6% |
|                     CSw>J>AS>HW>HoW>ES>Cons | 406988 | 157828 | 158147 | 454 | 110 | 73.1% |  9.7% |
|                     CSw>J>AS>HW>ES>HoW>Cons | 407136 | 157867 | 158187 | 406 |  46 | 73.1% |  9.7% |
|                     CSw>J>AS>ES>HW>HoW>Cons | 406419 | 157757 | 158077 | 517 | 235 | 73.2% |  9.9% |
|                     CSw>J>ES>AS>HW>HoW>Cons | 406392 | 157809 | 158126 | 433 |  48 | 73.1% |  9.9% |
|                     CSw>ES>J>AS>HW>HoW>Cons | 405022 | 157857 | 158175 | 539 | 117 | 73.0% | 10.0% |
|                     ES>CSw>J>AS>HW>HoW>Cons | 405441 | 158113 | 158437 | 665 | 134 | 72.7% | 10.0% |
|               CSw>J>AS>HW>ES+ex>HoW>ES>Cons | 406953 | 157844 | 158164 | 437 |  56 | 73.1% |  9.8% |
|               CSw>J>AS>ES+ex>HW>HoW>ES>Cons | 406957 | 157850 | 158170 | 431 |  43 | 73.0% |  9.8% |
|         CSw>J>AS>HW+FW>ES+ex>HW>HoW>ES>Cons | 407279 | 157836 | 158156 | 504 | 137 | 73.1% |  9.7% |
|         CSw>J>AS+GC>HW>AS>ES+ex>HoW>ES>Cons | 406188 | 157770 | 158090 | 380 |  30 | 73.1% |  9.6% |
|   CSw>J>AS+GC>HW+FW>AS>HW>ES+ex>HoW>ES>Cons | 407030 | 157819 | 158139 | 492 | 120 | 73.1% |  9.8% |
|                     CSw>J>AS>HW>HoW>Cons>LH | 393512 | 157962 | 158280 | 308 |  22 | 73.1% |  9.6% |
|                     CSw>J>AS>HW>HoW>LH>Cons | 393944 | 158002 | 158310 | 329 |  51 | 73.1% |  9.8% |
|                     CSw>J>AS>HW>LH>HoW>Cons | 394156 | 158013 | 158320 | 327 |  80 | 73.1% |  9.8% |
|                     CSw>J>AS>LH>HW>HoW>Cons | 393962 | 157918 | 158221 | 316 |  38 | 73.2% | 10.0% |
|                     CSw>J>LH>AS>HW>HoW>Cons | 394201 | 158002 | 158305 | 324 |  59 | 73.1% | 10.0% |
|                     CSw>LH>J>AS>HW>HoW>Cons | 394363 | 158040 | 158344 | 326 |  50 | 73.0% | 10.0% |
|                     LH>CSw>J>AS>HW>HoW>Cons | 394678 | 158286 | 158595 | 421 |  89 | 72.8% | 10.0% |
|               CSw>J>AS>HW>LH+ex>HoW>LH>Cons | 393933 | 158033 | 158339 | 301 |  28 | 73.1% |  9.8% |
|               CSw>J>AS>LH+ex>HW>HoW>LH>Cons | 393755 | 157986 | 158291 | 421 | 182 | 73.1% |  9.8% |
|         CSw>J>AS>HW+FW>LH+ex>HW>HoW>LH>Cons | 394034 | 158034 | 158340 | 315 |  36 | 73.1% |  9.8% |
|         CSw>J>AS+GC>HW>AS>LH+ex>HoW>LH>Cons | 392845 | 157978 | 158284 | 606 | 631 | 73.1% |  9.7% |
|   CSw>J>AS+GC>HW+FW>AS>HW>LH+ex>HoW>LH>Cons | 394090 | 157995 | 158302 | 338 |  54 | 73.1% |  9.8% |
|                    CSw>J>AS>HW>HoW>Cons>HPr | 396882 | 158186 | 158505 | 339 |  27 | 72.9% |  7.6% |
|                    CSw>J>AS>HW>HoW>HPr>Cons | 396093 | 158345 | 158655 | 336 |  37 | 72.8% |  7.8% |
|                    CSw>J>AS>HW>HPr>HoW>Cons | 395777 | 158348 | 158657 | 363 |  63 | 72.7% |  7.9% |
|                    CSw>J>AS>HPr>HW>HoW>Cons | 394603 | 158170 | 158476 | 300 |  23 | 72.9% |  8.0% |
|                    CSw>J>HPr>AS>HW>HoW>Cons | 394422 | 158173 | 158479 | 383 |  65 | 72.9% |  8.0% |
|                    CSw>HPr>J>AS>HW>HoW>Cons | 393947 | 158426 | 158732 | 343 |  35 | 72.7% |  8.1% |
|                    HPr>CSw>J>AS>HW>HoW>Cons | 395775 | 159554 | 159877 | 521 |  74 | 71.6% |  8.2% |
|             CSw>J>AS>HW>HPr+ex>HoW>HPr>Cons | 395840 | 158414 | 158723 | 371 |  45 | 72.7% |  7.9% |
|             CSw>J>AS>HPr+ex>HW>HoW>HPr>Cons | 395620 | 158381 | 158687 | 357 |  48 | 72.7% |  7.9% |
|       CSw>J>AS>HW+FW>HPr+ex>HW>HoW>HPr>Cons | 395698 | 158384 | 158691 | 326 |  27 | 72.7% |  7.9% |
|       CSw>J>AS+GC>HW>AS>HPr+ex>HoW>HPr>Cons | 395200 | 158229 | 158536 | 395 |  94 | 72.9% |  7.7% |
| CSw>J>AS+GC>HW+FW>AS>HW>HPr+ex>HoW>HPr>Cons | 395787 | 158381 | 158690 | 353 |  30 | 72.7% |  7.9% |
"Theck, Bringer of Numbers and Pounding Headaches," courtesy of Grehn|Skipjack.
Simcraft 6.x, Call to Arms 6.0, Talent Spec & Glyph Guide 6.x, Blog: Sacred Duty
User avatar
theckhd
Moderator
 
Posts: 8031
Joined: Thu Jul 31, 2008 3:06 pm
Location: Harrisburg, PA

Re: Theck's MATLAB thread - MoP/5.x

Postby theckhd » Tue Jan 28, 2014 8:19 pm

Thels wrote:The thing about T90 is that we earlier determined that outside of Execute range, it was best to save T90 for last, so it would fill an empty GCD. However, during Execute range, we have no empty GCDs, so that was a good reason to bump T90 up the priority list. Not sure if that's still the case.

It would therefor make sense to me to first establish the optimal rotation without using the Final Wrath glyph or Hammer of Wrath, much like you're already doing in the first post, but with the T90 abilities, and once we got the optimal rotation, use variants of that rotation with HW+FW, HoW and T90+EX placed in various locations.


Revisiting this: Can you see any reason that CSw>J>AS>HW>ES>HoW>Cons would differ from CSw>J>AS>HW+FW>ES+ex>HW>HoW>ES>Cons? If you're not in execute range, that's CSw>J>AS>HW>ES>Cons. If you're in execute range, it's CSw>J>AS>HW>ES>HoW>Cons.

I think I'm going back to my earlier stance that HoW>ES and ES>HoW span the space of possibilities here, at least until you start moving things farther afield.

Also, I noticed the LH data has some oddities - it's still showing up as highest DPS to put it at the front of the queue.
"Theck, Bringer of Numbers and Pounding Headaches," courtesy of Grehn|Skipjack.
Simcraft 6.x, Call to Arms 6.0, Talent Spec & Glyph Guide 6.x, Blog: Sacred Duty
User avatar
theckhd
Moderator
 
Posts: 8031
Joined: Thu Jul 31, 2008 3:06 pm
Location: Harrisburg, PA

Re: Theck's MATLAB thread - MoP/5.x

Postby Thels » Wed Jan 29, 2014 4:41 am

Interesting stuff there. It's quite surprising that CS>J>AS>HW>Cons is quite a bit of a DPS upgrade over CSw>J>AS>HW>Cons, but a clear loss in TMI. Is there a clear reason why you picked 0.3 seconds? Also, does it benefit to do the same for Judgment? Are these reports available somewhere, so we can compare the number of CS with the number of J?

CSw>J>AS>HW>ES/LH>HoW>Cons still seems very close to the optimal HoPo+DPS, and is a simple priority list that doesn't suddenly change at execute range or grand crusader, therefor probably recommended to all but the most skilled of players. If you take Holy Prism, instead use it at the end of the list: CSw>J>AS>HW>HoW>Cons>HPr.

For the more detailed rotations, there's some entries that indeed do not make sense now that you made me think about it:

Code: Select all
|               CSw>J>AS>HW>ES+ex>HoW>ES>Cons | 406953 | 157844 | 158164 | 437 |  56 | 73.1% |  9.8% |
|               CSw>J>AS>ES+ex>HW>HoW>ES>Cons | 406957 | 157850 | 158170 | 431 |  43 | 73.0% |  9.8% |
|         CSw>J>AS>HW+FW>ES+ex>HW>HoW>ES>Cons | 407279 | 157836 | 158156 | 504 | 137 | 73.1% |  9.7% |
|         CSw>J>AS+GC>HW>AS>ES+ex>HoW>ES>Cons | 406188 | 157770 | 158090 | 380 |  30 | 73.1% |  9.6% |
|   CSw>J>AS+GC>HW+FW>AS>HW>ES+ex>HoW>ES>Cons | 407030 | 157819 | 158139 | 492 | 120 | 73.1% |  9.8% |


For starters, all the lines have "ES+ex" in them, which means that "ES" is always outside of execute range. Since "HoW" is always during execute range, "ES" and "HoW" can never clash, so we might as well switch them in the priority list, which brings us to:

CSw>J>AS>HW>ES+ex>ES>HoW>Cons
CSw>J>AS>ES+ex>HW>ES>HoW>Cons
CSw>J>AS>HW+FW>ES+ex>HW>ES>HoW>Cons
CSw>J>AS+GC>HW>AS>ES+ex>ES>HoW>Cons
CSw>J>AS+GC>HW+FW>AS>HW>ES+ex>ES>HoW>Cons

Likewise, the third line has "HW+FW" in it, which means that "HW" is always outside of execute range. Since "ES+ex" is always during execute range, "HW" and "ES+ex" can never clash (for the lines that contain HW+FW), so we might as well switch them in the priority list, which brings us to:

CSw>J>AS>HW>ES+ex>ES>HoW>Cons
CSw>J>AS>ES+ex>HW>ES>HoW>Cons
CSw>J>AS>HW+FW>HW>ES+ex>ES>HoW>Cons
CSw>J>AS+GC>HW>AS>ES+ex>ES>HoW>Cons
CSw>J>AS+GC>HW+FW>AS>HW>ES+ex>ES>HoW>Cons

Naturally, we can simplify "ES+ex>ES" into "ES", and "HW+FW>HW" into "HW", which brings us to:

Code: Select all
|                     CSw>J>AS>HW>ES>HoW>Cons | 406953 | 157844 | 158164 | 437 |  56 | 73.1% |  9.8% |
|               CSw>J>AS>ES+ex>HW>ES>HoW>Cons | 406957 | 157850 | 158170 | 431 |  43 | 73.0% |  9.8% |
|                     CSw>J>AS>HW>ES>HoW>Cons | 407279 | 157836 | 158156 | 504 | 137 | 73.1% |  9.7% |
|               CSw>J>AS+GC>HW>AS>ES>HoW>Cons | 406188 | 157770 | 158090 | 380 |  30 | 73.1% |  9.6% |
|         CSw>J>AS+GC>HW+FW>AS>HW>ES>HoW>Cons | 407030 | 157819 | 158139 | 492 | 120 | 73.1% |  9.8% |


This makes the first and third entry identical, but they still have quite a difference in DPS (over 300). This difference in DPS makes me wonder if trying out any other rotations would have any meaning whatsoever, as the difference in DPS seems to be too small to measure properly.

I'm only curious about Holy Prism, to see if bumping it is worth it during Execute range if we leave it at the very bottom of the priority list.
ImageImage
User avatar
Thels
 
Posts: 1314
Joined: Tue May 15, 2007 7:30 am
Location: The Netherlands

Re: Theck's MATLAB thread - MoP/5.x

Postby theckhd » Wed Jan 29, 2014 5:28 pm

OK, so I can dumpster that one duplicate rotation and simplify the expressions for several others.

Regarding the DPS variance, this plot might interest you:
Image

For this, I've run the same profile through SimC, changing only the number of iterations. At each iteration level I run 100 sims and measure the 95% confidence interval ("Observed") and compare that to the predicted one based on SimC's DPS_Error output ("Reported"). The curious part is that the actual 95% CI seems to roughly match 1/N^(1/3) rather than 1/N^(1/2), which the prediction obviously represents since it's hard-coded that way. I don't have any idea why.

More pertinently to us at the moment is the fact that for low numbers of iterations, the actual error is usually less than what we predict. However, for 50k iterations or more, we're under-estimating the error. I'm not yet sure if the green line bottoms out at a ~200 DPS CI or if it continues to shrink. Going to try and test that tonight.

Moral of the story is that we may not actually be able to get accuracy of the mean better than +/- 100 DPS out of these sims, and as you've noted, even the same rotation can vary by up to 300 from trial to trial.
"Theck, Bringer of Numbers and Pounding Headaches," courtesy of Grehn|Skipjack.
Simcraft 6.x, Call to Arms 6.0, Talent Spec & Glyph Guide 6.x, Blog: Sacred Duty
User avatar
theckhd
Moderator
 
Posts: 8031
Joined: Thu Jul 31, 2008 3:06 pm
Location: Harrisburg, PA

Re: Theck's MATLAB thread - MoP/5.x

Postby theckhd » Wed Jan 29, 2014 5:39 pm

Thels wrote:Interesting stuff there. It's quite surprising that CS>J>AS>HW>Cons is quite a bit of a DPS upgrade over CSw>J>AS>HW>Cons, but a clear loss in TMI. Is there a clear reason why you picked 0.3 seconds? Also, does it benefit to do the same for Judgment? Are these reports available somewhere, so we can compare the number of CS with the number of J?


Oh, forgot about this part. There's no obvious reason for 0.3 vs. 0.35. I tried both, 0.35 just happens to be what we're using in the default profiles in SimC. I wanted to play with that a bit, but for the most part that value shouldn't matter much. It has to be larger than the difference between two hasted spell GCDs and two hasted melee GCDs, but anything over that point (within reason) works roughly equivalently.

Ex: 30% melee haste, but 50.2% spell haste (*1.1*1.05 for SoI & raid buff). Your hasted spell GCD is capped at 1.0 seconds, but your melee GCD is still 1.1538 seconds. If you do CS-HW-Cons-, your GCD is up in 3.1538 seconds. But CS's cooldown is 4.5/1.3=3.4615 seconds (which is exactly three hasted melee GCDs, of course).

So in this example there's a 0.3077-second gap where you could be using a spell (and delaying CS). You generally don't want to do that for HPG purposes, so you'd want a wait that's a little larger than 0.3077 seconds, hence 0.35. Any higher haste and the gap shrinks thanks to the 1.0-second GCD cap; the worst case scenario is at zero haste, where you're looking at a 0.4026-second gap. This probably suggests we should amp that delay up to 0.45 seconds in the default profiles for next expansion provided nothing else changes.

Note that the haste level of the gear set in question should be in the 35% range, so even 0.3 should be sufficient, and the CS+W0.3 results should be identical to CS+W0.35=CSw.

Originally I had intended to try a bunch of wait values to see what happened, but never got around to it. Now that the framework is in place it would be easy to do though, to show that there's a binary cutoff.
"Theck, Bringer of Numbers and Pounding Headaches," courtesy of Grehn|Skipjack.
Simcraft 6.x, Call to Arms 6.0, Talent Spec & Glyph Guide 6.x, Blog: Sacred Duty
User avatar
theckhd
Moderator
 
Posts: 8031
Joined: Thu Jul 31, 2008 3:06 pm
Location: Harrisburg, PA

Re: Theck's MATLAB thread - MoP/5.x

Postby Thels » Thu Jan 30, 2014 3:04 am

theckhd wrote:OK, so I can dumpster that one duplicate rotation and simplify the expressions for several others.


Actually, you can toss 2 of them in the trash. The 1st and 3rd special rotation simplify into CSw>J>AS>HW>ES>HoW>Cons, which we already had listed.

Would there be some use in adding a Jw? This would of course require an additional check to make sure that waiting for J would result in us not pressing J but waiting for CS by the time J's up. It should be easy to check if this is required by checking the number of Judgments used. If it's 2/3th of the number of Crusader Strikes, then we're making optimal use of J.



EDIT: Come to think of it, CSw>J>AS>ES+ex>HW>ES>HoW>Cons can go sleep with the fishes as well. There is only one reason to bump T90 during Execute phase, and that is that the addition of HoW might push T90 too far back. However, since we already prioritize ES over HoW, this will not happen. The only thing this list does is pushing HW down during execute phase, which makes no sense as HW deals more damage during execute phase.

That leaves 2 special rotations on the list:

CSw>J>AS+GC>HW>AS>ES>HoW>Cons
CSw>J>AS+GC>HW+FW>AS>HW>ES>HoW>Cons

The first one prioritizes HW over AS without Grand Crusader.
The second one does the same, but only during Execute phase.

Perhaps the following option is still worth including for completion's sake:

CSw>J>AS>HW+FW>ES>HW>HoW>Cons



Considering the margin of error, would it make sense to find the best rotation outside of execute range, and the best rotation during execute range separately?

Currently, to test the best execute priority, we basically only use 20% of the data, but include the margin error from 100% of the data. IE, if we compare say CSw>J>AS>HW>ES>HoW>Cons to CSw>J>AS>HW>HoW>ES>Cons, we only do something different in the last 20% of the fight, but we still include the variations in RNG for the first 80% of the fight, basically including 5 times as much noise as is needed.

If it's possibly to artificially keep the boss in execute phase 0% or 100% of the fight, the entire sim would matter, instead of only the last 20%.
ImageImage
User avatar
Thels
 
Posts: 1314
Joined: Tue May 15, 2007 7:30 am
Location: The Netherlands

Re: Theck's MATLAB thread - MoP/5.x

Postby theckhd » Thu Jan 30, 2014 8:06 am

Have to run, but the blog post will be going up at 11:30 EST today. Most of the changes you mentioned are alreadymade, with the exception of the last HW+FW>ES>HW rotation (which isn't likely to be a gain anyway, since ES>HW is inferior to HW>ES).

I'll have to think about Jw, that might be worth considering for the future. I'm not sure if we have cases where we're doing CS-X-Y-CS-J- instead of CS-X-J-CS-X-X- because of the GCD coming up early enough that we cast Y even though J will be available <0.2 seconds later.

It's possible to make a boss that's in execute range 100% of the fight (or, conversely, for 0% of the fight) in simc, but it's tricky/annoying. In the end, I'm not sure it's much more accurate than what we're doing either. I'll look into it, but realistically I think we should be satisfied with knowing that two rotations differ by less than e.g. 0.5% overall DPS.
"Theck, Bringer of Numbers and Pounding Headaches," courtesy of Grehn|Skipjack.
Simcraft 6.x, Call to Arms 6.0, Talent Spec & Glyph Guide 6.x, Blog: Sacred Duty
User avatar
theckhd
Moderator
 
Posts: 8031
Joined: Thu Jul 31, 2008 3:06 pm
Location: Harrisburg, PA

Re: Theck's MATLAB thread - MoP/5.x

Postby Thels » Thu Jan 30, 2014 9:11 am

theckhd wrote:Have to run, but the blog post will be going up at 11:30 EST today. Most of the changes you mentioned are alreadymade, with the exception of the last HW+FW>ES>HW rotation (which isn't likely to be a gain anyway, since ES>HW is inferior to HW>ES).


"since ES>HW is inferior to HW>ES". You have not actually established in the test above that HW>ES outside of execute phase, only as a whole with the inclusion of Final Wrath. Though I agree that it's likely a DPS loss.

theckhd wrote:I'll have to think about Jw, that might be worth considering for the future. I'm not sure if we have cases where we're doing CS-X-Y-CS-J- instead of CS-X-J-CS-X-X- because of the GCD coming up early enough that we cast Y even though J will be available <0.2 seconds later.


Do you still have the reports of each individual priority test? If so, it should be easy to check if J is cast 2/3th as often as CS. If so, everything should be ok, and Jw is not required. Let's hope so, as Jw would be tricky. (IE, J on 0.25sec, CS on 0.5sec. Simply CSw>Jw>AS would delay AS so we can cast J, except that we won't cast J, as we'll wait for CS.)

theckhd wrote:It's possible to make a boss that's in execute range 100% of the fight (or, conversely, for 0% of the fight) in simc, but it's tricky/annoying. In the end, I'm not sure it's much more accurate than what we're doing either. I'll look into it, but realistically I think we should be satisfied with knowing that two rotations differ by less than e.g. 0.5% overall DPS.


It should be slightly more accurate as we have less noise in comparison. Though I agree that the difference in DPS output is minimal, especially if you consider that this would be most likely a theoretical case, as a more complex rotation might be taxing on the player's focus. I'm happy with a simple conditionless priority :)
ImageImage
User avatar
Thels
 
Posts: 1314
Joined: Tue May 15, 2007 7:30 am
Location: The Netherlands

Re: Theck's MATLAB thread - MoP/5.x

Postby Thels » Thu Jan 30, 2014 11:03 am

I've checked http://matlabadin.googlecode.com/svn/trunk/simc/io/rotation_paladin_protection_CSw_J_AS_HW_ES_HoW_Cons.html, and noticed 136.4 CS casts and 81.5 J casts. It should've resulted in (or at least close to) 90.9 J casts, so we're definitely missing out on Judgments.

Also, when checking out http://matlabadin.googlecode.com/svn/trunk/simc/io/rotation_paladin_protection_ES_CSw_J_AS_HW_HoW_Cons.html and http://matlabadin.googlecode.com/svn/trunk/simc/io/rotation_paladin_protection_LH_CSw_J_AS_HW_HoW_Cons.html, there seems to be quite a difference in Stormlash DPS, which is 11789 for the ES version, and 14251 for the LH version.

If we check http://matlabadin.googlecode.com/svn/trunk/simc/io/rotation_paladin_protection_CSw_J_AS_HW_ES_HoW_Cons.html and http://matlabadin.googlecode.com/svn/trunk/simc/io/rotation_paladin_protection_CSw_J_AS_HW_LH_HoW_Cons.html, more or less the same results apply. Also worth of note is that Stormlash has about two extra casts with LH than with ES.

I'm not sure if Simcraft is accurate here on Stormlash, but if so, that reduces the difference in DPS between ES and LH somewhat. However, that's not enough to explain the weird result.

I did some further digging, and I think the problem is with ES, and not LH. If you compare http://matlabadin.googlecode.com/svn/trunk/simc/io/rotation_paladin_protection_CSw_J_AS_HW_ES_HoW_Cons.html to http://matlabadin.googlecode.com/svn/trunk/simc/io/rotation_paladin_protection_ES_CSw_J_AS_HW_HoW_Cons.html, you notice that CSw>J>AS>HW>ES>HoW>Cons does MORE ES damage than ES>CSw>J>AS>HW>HoW>Cons.

Have I mentioned that I don't like vengeance one bit? Well, I'm glad when it's gone in 6.0, because it seems to be the culprit here.

ES>CSw>J>AS>HW>HoW>Cons increases the number of ES casts from 7.6 to 7.9. However, it guarantees that the first ES cast will be right off the bat, when we have zero vengeance, whereas CSw>J>AS>HW>ES>HoW>Cons doesn't pop the first ES until at least the 8th GCD (CS-J-AS-CS-HW-J-CS-ES), possibly later, if we get a GC proc. By that time we have already built up a decent amount of vengeance, which allows it to hit for nearly as much as the other ES.
ImageImage
User avatar
Thels
 
Posts: 1314
Joined: Tue May 15, 2007 7:30 am
Location: The Netherlands

Re: Theck's MATLAB thread - MoP/5.x

Postby theckhd » Thu Jan 30, 2014 12:40 pm

Thels wrote:"since ES>HW is inferior to HW>ES". You have not actually established in the test above that HW>ES outside of execute phase, only as a whole with the inclusion of Final Wrath. Though I agree that it's likely a DPS loss.

Sorry, what I meant by that was exactly what you said: we've established that the HW>ES queue outperforms ES>HW when averaged over an entire fight. I'm inferring from that result that HW+FW>ES>HW isn't likely to be a large gain, if a gain at all. Not an iron-clad case by any stretch, but probably accurate. It was too late to add HW+FW>ES>HW to the list before the post went live, but it's added in my local copy, so we'll have that data for next time.

Thels wrote:
theckhd wrote:It's possible to make a boss that's in execute range 100% of the fight (or, conversely, for 0% of the fight) in simc, but it's tricky/annoying. In the end, I'm not sure it's much more accurate than what we're doing either. I'll look into it, but realistically I think we should be satisfied with knowing that two rotations differ by less than e.g. 0.5% overall DPS.


It should be slightly more accurate as we have less noise in comparison. Though I agree that the difference in DPS output is minimal, especially if you consider that this would be most likely a theoretical case, as a more complex rotation might be taxing on the player's focus. I'm happy with a simple conditionless priority :)


It's not clear we'll actually have less noise, though. In theory noise should decrease as 1/sqrt(N), so more iterations should be better, as should having an entire fight of execute range rather than just 20% of it.

In practice, we're hitting some sort of error minimum in SimC, as far as I can tell. For anything over 50k iterations, the noise seems to floor out at about a 200-300 DPS 95% CI. I'm currently investigating why that's happening, but for the moment I wouldn't expect to get a smaller 95% CI even if I ran for 500k iterations using execute range for the entire fight.
"Theck, Bringer of Numbers and Pounding Headaches," courtesy of Grehn|Skipjack.
Simcraft 6.x, Call to Arms 6.0, Talent Spec & Glyph Guide 6.x, Blog: Sacred Duty
User avatar
theckhd
Moderator
 
Posts: 8031
Joined: Thu Jul 31, 2008 3:06 pm
Location: Harrisburg, PA

Re: Theck's MATLAB thread - MoP/5.x

Postby theckhd » Thu Jan 30, 2014 12:45 pm

Thels wrote:
theckhd wrote:I'll have to think about Jw, that might be worth considering for the future. I'm not sure if we have cases where we're doing CS-X-Y-CS-J- instead of CS-X-J-CS-X-X- because of the GCD coming up early enough that we cast Y even though J will be available <0.2 seconds later.


Do you still have the reports of each individual priority test? If so, it should be easy to check if J is cast 2/3th as often as CS. If so, everything should be ok, and Jw is not required. Let's hope so, as Jw would be tricky. (IE, J on 0.25sec, CS on 0.5sec. Simply CSw>Jw>AS would delay AS so we can cast J, except that we won't cast J, as we'll wait for CS.)


You've already sort of answered this, but it seems clear we need a Jw. That said, I don't think the situation you're describing is a problem because I don't think it should occur. Either CS and J are both becoming available in the next "slot", in which case we'd want to wait for CS anyway, or else one of the "w" conditionals won't be satisfied.

And by "slot" here I'm meaning "melee-hasted GCD time intervals." We can probably just use the same "w" conditional for J that we use for CS. Also, I need to increase the CS one eventually, as we talked about before, so we may just go whole-hog and call it a 0.5-second wait for both.
"Theck, Bringer of Numbers and Pounding Headaches," courtesy of Grehn|Skipjack.
Simcraft 6.x, Call to Arms 6.0, Talent Spec & Glyph Guide 6.x, Blog: Sacred Duty
User avatar
theckhd
Moderator
 
Posts: 8031
Joined: Thu Jul 31, 2008 3:06 pm
Location: Harrisburg, PA

Re: Theck's MATLAB thread - MoP/5.x

Postby theckhd » Thu Jan 30, 2014 12:50 pm

Thels wrote:Also, when checking out http://matlabadin.googlecode.com/svn/trunk/simc/io/rotation_paladin_protection_ES_CSw_J_AS_HW_HoW_Cons.html and http://matlabadin.googlecode.com/svn/trunk/simc/io/rotation_paladin_protection_LH_CSw_J_AS_HW_HoW_Cons.html, there seems to be quite a difference in Stormlash DPS, which is 11789 for the ES version, and 14251 for the LH version.

If we check http://matlabadin.googlecode.com/svn/trunk/simc/io/rotation_paladin_protection_CSw_J_AS_HW_ES_HoW_Cons.html and http://matlabadin.googlecode.com/svn/trunk/simc/io/rotation_paladin_protection_CSw_J_AS_HW_LH_HoW_Cons.html, more or less the same results apply. Also worth of note is that Stormlash has about two extra casts with LH than with ES.

I'm not sure if Simcraft is accurate here on Stormlash, but if so, that reduces the difference in DPS between ES and LH somewhat. However, that's not enough to explain the weird result.

Maybe Stormlash is proccing off of LH (either the cast or the ticks)? I'll look into it.

Thels wrote:I did some further digging, and I think the problem is with ES, and not LH. If you compare http://matlabadin.googlecode.com/svn/trunk/simc/io/rotation_paladin_protection_CSw_J_AS_HW_ES_HoW_Cons.html to http://matlabadin.googlecode.com/svn/trunk/simc/io/rotation_paladin_protection_ES_CSw_J_AS_HW_HoW_Cons.html, you notice that CSw>J>AS>HW>ES>HoW>Cons does MORE ES damage than ES>CSw>J>AS>HW>HoW>Cons.

Have I mentioned that I don't like vengeance one bit? Well, I'm glad when it's gone in 6.0, because it seems to be the culprit here.

ES>CSw>J>AS>HW>HoW>Cons increases the number of ES casts from 7.6 to 7.9. However, it guarantees that the first ES cast will be right off the bat, when we have zero vengeance, whereas CSw>J>AS>HW>ES>HoW>Cons doesn't pop the first ES until at least the 8th GCD (CS-J-AS-CS-HW-J-CS-ES), possibly later, if we get a GC proc. By that time we have already built up a decent amount of vengeance, which allows it to hit for nearly as much as the other ES.


I agree with your interpretation here. ES snapshots everything, including AP. LH does not, it updates dynamically. So LH at the front of the queue is still getting some of the effect of Vengeance as it ramps up to full value, while ES is getting shafted.

There are a number of ways we could fix this, including a simple forced delay (i.e. don't cast LH or ES until at least 10 seconds have elapsed). That's probably the easiest way to fix it. Downside is potential clipping (cast at ~10s, ~70s, ~130s, ~190s, ~250s, ~310s, ~370s, ~430s, combat ends at 450s but can vary by up to 20%). It'll ensure that the first one does full damage, but might clip the last (and largest) tick of the final cast in a lot of cases. That might come out in the wash though via the central limit theorem.
"Theck, Bringer of Numbers and Pounding Headaches," courtesy of Grehn|Skipjack.
Simcraft 6.x, Call to Arms 6.0, Talent Spec & Glyph Guide 6.x, Blog: Sacred Duty
User avatar
theckhd
Moderator
 
Posts: 8031
Joined: Thu Jul 31, 2008 3:06 pm
Location: Harrisburg, PA

Re: Theck's MATLAB thread - MoP/5.x

Postby theckhd » Thu Jan 30, 2014 1:48 pm

Yeah, Jw makes a difference:

Code: Select all
|                Rotation |    DPS |    HPS |   DTPS |    TMI |   Var |  SotR |  Wait |
|         CS>J>AS>Cons>HW | 373603 | 160013 | 160353 |   6212 |  2062 | 71.0% | 14.5% |
|         CS>J>AS>HW>Cons | 379608 | 159521 | 159854 |   4287 |   971 | 71.4% | 13.2% |
|    CS+W0.3>J>AS>HW>Cons | 373814 | 157738 | 158054 |    533 |   117 | 73.2% | 12.7% |
|        CSw>J>AS>Cons>HW | 368204 | 157862 | 158182 |    460 |    47 | 73.1% | 13.8% |
|       CSw>Jw>AS>Cons>HW | 369814 | 154062 | 154360 |    201 |    18 | 76.6% | 12.9% |
|        CSw>J>AS>HW>Cons | 373591 | 157666 | 157983 |    427 |    55 | 73.2% | 12.7% |


Now to decide whether to make CS=CSw & J=Jw and use "CSnw" and "Jnw" as shorthands instead of using Jw everywhere...
"Theck, Bringer of Numbers and Pounding Headaches," courtesy of Grehn|Skipjack.
Simcraft 6.x, Call to Arms 6.0, Talent Spec & Glyph Guide 6.x, Blog: Sacred Duty
User avatar
theckhd
Moderator
 
Posts: 8031
Joined: Thu Jul 31, 2008 3:06 pm
Location: Harrisburg, PA

Re: Theck's MATLAB thread - MoP/5.x

Postby Thels » Fri Jan 31, 2014 4:44 am

theckhd wrote:It's not clear we'll actually have less noise, though. In theory noise should decrease as 1/sqrt(N), so more iterations should be better, as should having an entire fight of execute range rather than just 20% of it.

In practice, we're hitting some sort of error minimum in SimC, as far as I can tell. For anything over 50k iterations, the noise seems to floor out at about a 200-300 DPS 95% CI. I'm currently investigating why that's happening, but for the moment I wouldn't expect to get a smaller 95% CI even if I ran for 500k iterations using execute range for the entire fight.


Let me word it a little differently. Say CS>J>AS>HW>HoW>ES>Cons is a 1000 DPS increase over CS>J>AS>HW>ES>HoW>Cons during execute phase. Over the course of an entire fight, that would mean a 200 DPS increase, which we either would not notice, or would doubt, considering the margin of error. If the entire fight was used in Execute phase, it would be a 1000 DPS increase, which would stick out of the margin of error more clearly.

Though even with so much as a 1k DPS difference, that's practically negligible.

theckhd wrote:You've already sort of answered this, but it seems clear we need a Jw. That said, I don't think the situation you're describing is a problem because I don't think it should occur. Either CS and J are both becoming available in the next "slot", in which case we'd want to wait for CS anyway, or else one of the "w" conditionals won't be satisfied.

And by "slot" here I'm meaning "melee-hasted GCD time intervals." We can probably just use the same "w" conditional for J that we use for CS. Also, I need to increase the CS one eventually, as we talked about before, so we may just go whole-hog and call it a 0.5-second wait for both.


Hmyeah. I was thinking about Ovale scripts, which are quite similar to SimC action lists, only you can't assume that the player has been performing a perfect rotation up to now. As long as the "player" is performing a perfect rotation, it shouldn't be an issue, unless at very high (well beyond 50%) haste levels. Heroism or Kafa Press might screw up the rotation. I guess Kafa Press is not included in Simcraft, but does it use Heroism?

theckhd wrote:Yeah, Jw makes a difference:

Code: Select all
|                Rotation |    DPS |    HPS |   DTPS |    TMI |   Var |  SotR |  Wait |
|         CS>J>AS>Cons>HW | 373603 | 160013 | 160353 |   6212 |  2062 | 71.0% | 14.5% |
|         CS>J>AS>HW>Cons | 379608 | 159521 | 159854 |   4287 |   971 | 71.4% | 13.2% |
|    CS+W0.3>J>AS>HW>Cons | 373814 | 157738 | 158054 |    533 |   117 | 73.2% | 12.7% |
|        CSw>J>AS>Cons>HW | 368204 | 157862 | 158182 |    460 |    47 | 73.1% | 13.8% |
|       CSw>Jw>AS>Cons>HW | 369814 | 154062 | 154360 |    201 |    18 | 76.6% | 12.9% |
|        CSw>J>AS>HW>Cons | 373591 | 157666 | 157983 |    427 |    55 | 73.2% | 12.7% |


Now to decide whether to make CS=CSw & J=Jw and use "CSnw" and "Jnw" as shorthands instead of using Jw everywhere...


Hmm, that's a pretty large difference. With the downtime going down from 28.9% to 23.4%, that's a decrease in downtime of 19%. :o

The funny thing is, DPS went up!
ImageImage
User avatar
Thels
 
Posts: 1314
Joined: Tue May 15, 2007 7:30 am
Location: The Netherlands

PreviousNext

Return to Advanced Theorycraft and Calculations

Who is online

Users browsing this forum: No registered users and 1 guest

Who is online

In total there is 1 user online :: 0 registered, 0 hidden and 1 guest (based on users active over the past 5 minutes)
Most users ever online was 380 on Tue Oct 14, 2008 6:28 pm

Users browsing this forum: No registered users and 1 guest