9th Ed playtesting flaw? (Just a hypothesis)

[HighMarshalAmp]

Now that the point values are sout... well, it seems to me there might have been one major flaw in the playtesting of 9th.

 

We know they had competitive 'professional' players join in.

 

What if they just brought already competitive lists and so the armies and units that were popular already got more balanced?

And nobody would use the units that would really have needed an update - 'cause they weren't competitive and so, by virtue of survivorship bias, those kinda stayed bad because nobody knew what their actual value was since nobody ever put them on the table?

 

Look at Sanguinary Guard and Death Company, for example. They kicked butt in many 8th Ed lists, now they've gone up. That might be good for everyone else. That's an arguably good change.

Look at Thousand Sons Scarab terminators. Those were overcosted and unpopular. They still are overcosted. They might not become popular.

Look at Tyranids as a whole. Not many people play them compared to other factions, they got a lazy increase across the board and nothing to help the faction.

 

What do you think?

Edited July 13, 2020 by HighMarshalAmp

[Brother Tyler]

Are you basing this conclusion on observation of the actual playtesting methodology and execution? Or are you merely presenting a hypothesis?

[HighMarshalAmp]

Are you basing this conclusion on observation of the actual playtesting methodology and execution? Or are you merely presenting a hypothesis?

Just a hypothesis based on my very subjective impressions from reading through the leaked changes.

I've editet the OP to make this even clearer.

 

Anyway, what's your take? Might involving playtesters from the competitive community have skewed the balance in favor of what was already not garbage?

Edited July 13, 2020 by HighMarshalAmp

[BluejayJunior]

Your whole hypothesis rests on the notion that all playtesters only ever played lists that were top table competitive in 8th. I think that is a completely unreasonable assumption. There are lots of playtesters, not just top tournament players. And as far as I know, even a lot of the top ranked ITC players weren't playtesters. They have developed a large group of playtesters from a wide variety of player types. And I'm not sure how much say playtesters have in points values anyway, other than saying this seems too cheap/expensive. I would expect that playtesters are given guidance to play a variety of lists to gauge how well the changes affect different units. I think you need some kind of evidence to back up your claim. 

[HighMarshalAmp]

Your whole hypothesis rests on the notion that all playtesters only ever played lists that were top table competitive in 8th. I think that is a completely unreasonable assumption. There are lots of playtesters, not just top tournament players. And as far as I know, even a lot of the top ranked ITC players weren't playtesters. They have developed a large group of playtesters from a wide variety of player types. And I'm not sure how much say playtesters have in points values anyway, other than saying this seems too cheap/expensive. I would expect that playtesters are given guidance to play a variety of lists to gauge how well the changes affect different units. I think you need some kind of evidence to back up your claim. 

 

I'd like to add again that it's merely an impression, not a claim.

What gives me that impression is that units that saw more playtime in 'top'-lists have undergone some changes while those absent from most tables were mostly increased by the average 15%.

Sure, I'd expect a guaranteed variety during playtests as well, but in what way it was given and what it focussed on, we don't know one way or the other.

 

I'll take it you don't agree with my impression then?

[Commander Dawnstar]

 

Anyway, what's your take? Might involving playtesters from the competitive community have skewed the balance in favor of what was already not garbage?

 

Personally I'd go with Goonhammer's take: GW applied a pretty standard model for points changes across the board, with some targeted nerfs here and there, and subsequently did essentially nothing to help units that were already in an unfavourable place or factions with little in the way of build flexibilty to help them limit the impact of points increases.

 

Essentially it's your argument from the other direction - GW were only interested in bespoke nerfs to outliers rather than meaningfully tailoring the balance of their points.

Edited July 13, 2020 by Commander Dawnstar

[Brother Tyler]

My take is one of neutrality. I won't be able to form an opinion [on the rules] until I have time to read the rules en toto. I won't be able to form an opinion on the playtesting methodology until I have time to study the reports on the playtesting (which isn't likely to happen).

 

We know that GW involved a lot of competitive players in the playtesting. However, the assumption appears to be that they either involved only/primarily competitive players or that the competitive players were one cross-section of the overall playtester group, but they had undue influence on the outcomes, overriding the conclusions of other segments of the playtest group. Neither seems likely to me.

 

Realistically, any playtesting methodology has its flaws, and no edition of the game has been perfect. One of GW's survival methods is to change the dynamic of the game periodically, primarily through new editions and rules changes. These shift the game sufficiently that they drive hobbyists to buy new/more models. They also keep the game from getting stale, hopefully keeping hobbyists in the hobby longer before they get bored. So I always expect there to be some "flaws" or areas where things don't work the way I think they should, and that these shift around from edition to edition.

 

Ironically, one of the complaints the community has voiced in the past has been the lack of competitive players in rules development, the theory being that getting competitive players involved would help GW to get the wording more precise and balance things better across the range of units and rules.

 

From a theoretical standpoint, I think that including competitive players in the playtesting is a good thing. They provide a valuable perspective. Their input has to be contextualized and synthesized with other segments of the community, but if they are held to a proper method (i.e., testing the rules across the broad range of units and situations), the outcomes should be favorable.

 

Ultimately I think that even if there are flaws, it would be presumptuous of anyone that wasn't involved in the playtesting to ascribe those flaws to any aspect of the playtesting methodology, especially where they are speculating on the allegedly flawed methodology. I can only say that X method led to Y flaw if I'm able to study the data. If I don't know the specifics of X method, I can't connect the dots to Y flaw with any degree of reliability or confidence.

[HighMarshalAmp]

[...]

We know that GW involved a lot of competitive players in the playtesting. However, the assumption appears to be that they either involved only/primarily competitive players or that the competitive players were one cross-section of the overall playtester group, but they had undue influence on the outcomes, overriding the conclusions of other segments of the playtest group. Neither seems likely to me.

[...]

Just to avoid looking like I'm demonizing competitive players, I really meant (and failed to say) large parts of the community with certain affinities towards some units who, in building their lists where they could, might have opted for uits with a good track record more often that not.

 

Also, giving the player bade a part in designing the next iteration of their game is something I applaud - not that GW would care about me.

 

I do concede however that just from the points alone, not much can be said about the final game and how it will play.

It's just that more popular factions and units, it appears to me (and I might certainly be wrong), have gotten more intent changes than less popular ones.

It might still very well lead to those unpopular units become viable alternatives to the popular ones now. Personally though, I doubt it. But again, I'm not claiming to know anything (anything) for sure

[Dark Shepherd]

Problem with testing for 8th was they werent competitive enough, and ran every faction mono, so I dont think theyve gone too far over. In house testers probably taking fluffy lists

[Vermintide]

Raising the points on strong units is a buff to overlooked units. Terminators, for example, stayed the same price, while everything else went up- So in relative terms, Terminators became more points efficient.

[MegaVolt87]

Raising the points on strong units is a buff to overlooked units. Terminators, for example, stayed the same price, while everything else went up- So in relative terms, Terminators became more points efficient.

 

That's a good point, a lot of the underperformers have seen only token points increases in comparison, however you could also argue its easier to just stack your army with what traditionally works and you can get it on the table faster fully painted due to reduced model count and still have an effective army though. 

[Vanger]

You are giving the GW devs too much credit and assume that the external playtesters have more control or influence in decisions as they actually have.

The 9th edition "starter points" are NOT a balancing act. They had the following objectives in mind:
1. Speed up the game. Less models on the board, faster games. This was by far the biggest complaint of everyone in 8th edition
2. De-emphasizing hordes because they would have to big of an advantage in an edition where Board Control is king. (Remember, you are not uspposed to max out those secondaries easily)
3. Making point costs more firendly to use. The only reason why a lot of units and wargear are now multiples of 5.
Aside from these major guidelines we can see 2 another factors:
4. Punish unballanced units. That's why the Leviathan costs 350 points now.
5. Making units they feel are underutilised more attractive by not raising their points.

However as they have said time and time again (and the FAQs just reinforce this) they are first and foremost a narrative minded bunch. Try as they might to balance the game and pay more heed to the competitive crowd, they lack the vision for a comprehensive balancing of the game. I know harsh words, but all the misery with Sotrm Shields, that we did not get anything instead of PotMS (and similar abilities), they did not standardise how plasma weaponry works, erasing Stratagems instead of reworking them . Or the fact that Space Marines can again do a null deploy army just reinforces this impression.

[A.T.]

A lot of the points changes appear to have been done via a very rough formula with limited concern for balance - you can see a pattern of flat or percentage based increases in points based on unit type and cost and rounding values to multiples of 5.

 

While there are units and items that have been clearly costed under or over this there are also a lot of balance changes that appear to be simply a result of rounding or falling on one side or another of an arbitrary points increase line with seemingly no regard for balance.

[Black Knight]

Also don't forget there are going to be errors in the sheets too, a few people that have gone over the lists have noticed things that probably have been missed, like vehicles that have had their chassis price increased similar to the cost of adding the weapons but not made the weapons 0, Orks and Admech for example.

[Lord Marshal]

We also have to remember that there's every chance GW just did not listen or didn't care for playtesting reports. If I remember right, a few of them said they raised concerns regarding the Iron Hands supplement prior to publication and they were shrugged off. 

 

The whole 'we brought in playtesters!' always came off as more like marketing speak to backup the "we playtest now, that means the rules MUST be better!" line, rather than GW suddenly giving that much of a damn about actual balance - beyond shifting copies of CA anyway.

Edited July 14, 2020 by Lord Marshal

[USNCenturion]

Do we know the actual play testing method laid out by GW? Do they tell groups “here are the rules we’re thinking of implementing, play a couple (hundred?) games and let us know what you think” or do they say, “use every unit in X codex in these situations, then in these other 9 situations and tell us what interactions you discovered and your opinions on the results”.

 

If it’s the latter, I’d say each unit could be balanced against the rule set and each other, but that would take a ridiculous amount of time, treasure, and man hours I think. The former is probably like some of the above stated; they were focused on reeling in the outliers and getting the new edition out from a rules focus.

 

I’d say wait for the individual codices to see real change and balance to units.

[Xenith]

This is absolutely true and happened between 7th and 8th also. The tournament playtesters judge the codex based on their proclivities.

 

Case in point being the Blood Angels - at the start of 8th the 'BA in 8th' article said they would be super powerful, hold ther own, etc, but the armies used were things like razorback spam - the thing that BA were best at in 7th. The strength of the codex was based on a single build that tourney players had/used, and the rest of the army was left by the wayside until the codex.

 

I hope they changed things in 9th, but I feel that this is still the case. GW should organise a weekend and fly everyone in to go nuts with the GW studio armies, and then do the balance pass. 

[Medjugorje]

I think they are playtesting with codicies which will be released later AND I think they do this different then you think. 

[Tyberos the Red Wake]

If casual bystanders such as ourselves can easily tell which units are obviously underpowered, it would stand to reason that competitive playtesters would be able to tell as well. After all, these are the units that are to be avoided at all costs when creating competitive netlists. That makes singling them out for testing and buffing extremely easy. It's the middling units that are in most danger of neglect, and they were already average to begin with so there's no dire emergency.

Edited July 15, 2020 by Tyberos the Red Wake

[Captain_Krash]

After watching a ton of bat reps. Is it just me or do transports seem 100% useless in 9th with how small and cramped the games are now...armies look so cramped on the boards now. It really looks like AOS with just two armies slamming into each other by turn two. Almost doesn’t feel like 40k anymore...

 

Krash

Edited July 16, 2020 by Captain_Krash

[Medjugorje]

If casual bystanders such as ourselves can easily tell which units are obviously underpowered, it would stand to reason that competitive playtesters would be able to tell as well. After all, these are the units that are to be avoided at all costs when creating competitive netlists. That makes singling them out for testing and buffing extremely easy. It's the middling units that are in most danger of neglect, and they were already average to begin with so there's no dire emergency.

but maybe we dont have the rules for the next codex. And the playtesters used to.

[MegaVolt87]

 

If casual bystanders such as ourselves can easily tell which units are obviously underpowered, it would stand to reason that competitive playtesters would be able to tell as well. After all, these are the units that are to be avoided at all costs when creating competitive netlists. That makes singling them out for testing and buffing extremely easy. It's the middling units that are in most danger of neglect, and they were already average to begin with so there's no dire emergency.

but maybe we dont have the rules for the next codex. And the playtesters used to.

 

 

Then why nerf IH then, it was written with the "next codex" in mind wasn't it, to be released in some indeterminate time in the future ? Playtesters can only give feedback on what they are given to use, it was busted + the "next codex" never came to justify why IH were so strong compared to the other SM supplements. Also- your BT update, they were written with the next edition rules in mind by that line of logic? So they don't need changing yes, because of the "next codex" ? 

Edited July 16, 2020 by MegaVolt87

[Vermintide]

After watching a ton of bat reps. Is it just me or do transports seem 100% useless in 9th with how small and cramped the games are now...armies look so cramped on the boards now. It really looks like AOS with just two armies slamming into each other by turn two. Almost doesn’t feel like 40k anymore...

 

Krash

Nah, I feel like 40k has always been (or was always supposed to be) about closing distance and charging to dislodge an enemy. Armies that were meant as gunlines, and armies that were meant as assault were designed that way intentionally for a different flavour.

 

But it seems like melee focus is still at a pretty consistent disadvantage with the new rules- The best play is still blowing what you can apart from a distance and using surgical charges with specific one-and-done units to get rid of key targets. Transports are useful for putting your objective holders where you want them now, but we'll not see the return of glorious Terminator charges out of the front hatch of a Land Raider any time soon.

 

Which kinda sucks. I really think they should bring back assaulting from a transport, or else I don't see how stuff like assault intercessors will be any good. They'll be stuck in the same limbo as Berzerkers.

 

This is absolutely true and happened between 7th and 8th also. The tournament playtesters judge the codex based on their proclivities.

 

Case in point being the Blood Angels - at the start of 8th the 'BA in 8th' article said they would be super powerful, hold ther own, etc, but the armies used were things like razorback spam - the thing that BA were best at in 7th. The strength of the codex was based on a single build that tourney players had/used, and the rest of the army was left by the wayside until the codex.

 

I hope they changed things in 9th, but I feel that this is still the case. GW should organise a weekend and fly everyone in to go nuts with the GW studio armies, and then do the balance pass.

And don't forget the fact you could deep strike T1 at the start of 8th. I honestly think the entire BA codex was balanced around that, and when that got FAQ'd the army itself was just crippled. They were genuinely dangerous back then and one of the reasons the early meta had all those silly screening bubblewrap congalines.

Edited July 20, 2020 by Vermintide