A Case Study Against Deck Based Matchmaking

I've decided to post this deck based matchmaking case study here on Reddit as a means to have a link associated to the long article of text. Use this post to form your own opinion, and to provide relevant information to others posing similar questions here on Reddit, or elsewhere on social media.

In this case study, we are going to look over a large array of available information to draw from, and form a conclusion based on the information compiled here...

Does Marvel Snap take into account your deck when attempting to match you against other players?

Despite the possibility of a card based matchmaking system been integrated into Marvel Snap, there is no proof of matchmaking that attributes the cards in your deck and pairs them against a deck of your opponents, or a deck that mimics your own or "mirrors" it. All proof of the contrary offered thus far is anecdotal evidence and conjecture.

So what do we know? Here are the facts...

Marvel Snap runs on Amazon server less architecture in 3 regions (currently as of the 10th of December) around the globe. You can find a case study describing their goals and design philosophy here...
https://aws.amazon.com/solutions/case-studies/second-dinner-nuverse-case-study/

The case study specifies the usage of Amazon's service: Gamelift. And more specifically, Fleximatch...
https://docs.aws.amazon.com/gamelift/latest/flexmatchguide/match-intro.html

Using Gamelift, and Fleximatch, Marvel Snap resulted in the highest volume of matches ever for a game using this service (to date). Second Dinner specifies the need for quality and seamless matchmaking, and the importance of costs in order to achieve this. So how does Fleximatch solve these issues?

What is Fleximatch?

Amazon's Fleximatch service is the customisable matchmaking service that Marvel Snap utilises to pair you up against opponents. It is written as JSON text , and is the player ticket creation and player batching solution that is used as part of Second Dinners server less architecture.

Fleximatch determines player attributes through values and strings, in order to apply those attributes to rulesets. Which then places your ticket into one or more batches, ready to then be selected for a match. These are governed by these rulesets properties...
https://docs.aws.amazon.com/gamelift/latest/flexmatchguide/match-ruleset-property-definitions.html

Rules govern the distance between two attributes (i.e, the difference between CL 1000 and CL 1500), comparisons between attributes (i.e, comparing an opponents Rank to an opponents MMR), collections of attributes (a Players Rank, Collection Level, MMR and 12 card selection are all viable attributes to compare), acceptable latency values, and finally the compound logical statements used...
https://docs.aws.amazon.com/gamelift/latest/flexmatchguide/match-rules-reference-ruletype.html

The last one, compound rules is important. You can use multiple compound rules in a match, but they must all be true in order to pair players in a match in the first place. For each card selected as part of a sum of a 12 card deck. The statements used combine the individual rules above to form the compound rule. And all statements of the rule must be true in order for a ticket in the pool to be put into batches.

To state that Marvel Snap pairs players based on Region, MMR, Collection Level, Ladder Ranking, card attribute strings of “counters" and card attribute trings of “mirrors”, one can expect the actual process to narrow down a significant level to finally match two players against each other. As shown by AWS pricing structures, this is not a cheap endeavor to undertake.

What is the cost?

The first cost needed to be considered is the spot pricing, for each instance used per hour. This for each of the current 3 Regions Second Dinner has available. Secondly comes the pricing of using the Fleximatch service. The cost is based on the amount of player packages submitted, and the entire matchmaking hours used...
https://aws.amazon.com/gamelift/pricing/

This is where the concept of a deck based matchmaking system begins to falls apart. FlexiMatch counts a Player Package each time you submit a player’s request to play a game session. This package also includes player attributes used to evaluate a match. Matchmaking Hours is calculated as the duration FlexiMatch is executing a matchmaking evaluation, rounded up to the nearest millisecond.

What does this mean? It means for each attempt to submit a ticket into the pool for batching, the request is a charge of service. That is every successful request, and every denied request.

If a players attribute of Rank, MMR, Collection Level, or the String lists of cards, comparisons or untrue logic statements: The request is still charged regardless of the failed attempt. This is $0.00002 per player package. A successful request, or not. And with even more added rulesets: the failed attempts begin to add up drastically compared to the successful ones.

We then move onto Matchmaking Hours. The total time it takes for a player for their request for player package, ticket into the pool, then batching takes time. Time that is charged per hour spent, rounded up to the nearest millisecond. The cost is $1 per matchmaking hour.

What drives matchmaking hours? The complexity of rulesets, player arrival, and most importantly the distribution and range of player attributes. The higher the complexity, the more costly it becomes to run a matchmaking service.

A package request that compares, Rank, MMR and Collection Level would be drastically cheaper, compared to one that compares the attributes of strings associating cards as values to further scrutinize in compound rule validation, as well as comparing a specific card as part of a string, that is in direct correlation to a card in another players attributes...
https://docs.aws.amazon.com/gamelift/latest/flexmatchguide/match-client.html

The amount of time it takes to match against an opponent with specific cards as part of a string would be astronomically longer than what we as players currently experience when playing Marvel Snap. And if the latency rule were relaxed gradually to expand the available pool of tickets, ask yourself: What did additional rulesets of “counters” and “mirrors” serve in the first place if the end result is players are matched regardless based on the time taken to attempt a match based on “counters” or “mirrors”?

Furthermore, the continuous counter claims of a deck based match making system factoring both “counters” to cards included in your deck AND cards “mirroring” the ones included in your deck results in strings of attributes that need to factor in both of these factors when attempting to form a player package request. As stated above, a successful attempt requires all IF statements to have true results. We will look into an example of this dilemma later in the case study.

This brings us onto our next topic...

Ruleset service quotas, upload practice, and design philosophy.

Roughly every 2 days, Marvel Snap graces us with a new hot location, an entirely new location, a brand new card, weekly spotlight releases, over the air update, new patch, weekend mission events and season pass mission event . All these have drastic influence on deck popularity, and have the possibility to shift the meta in some capacity for a time.

The implication that deck based matchmaking exists, also implies that every 48 hours at a minimum, a new ruleset is uploaded to the AWS architecture, with not only new attributes, new card based strings and logic statements, but also an omniscient precognition from Second Dinner to know which attributes to compare and compound before players have a chance to try out these in game changes themselves, disingenuously developing a meta before a community has a chance to develop it through actual play.

Say this was possible. And Second Dinner without fail, has a ruleset up and running every 2 days...

AWS Fleximatch has a service quota tied to their rulesets. Although we cannot determine the amount their license allows, it is a finite amount, per Region. Thus references used for backlogs are readily available based on changes made before uploading the new ruleset...
https://docs.aws.amazon.com/gamelift/latest/flexmatchguide/match-create-ruleset.html

The caveat here is that rulesets cannot be changed once created. Once they are uploaded, they must be deleted, and new ones created. Which also implies that the rulesets are saved as part of AWS service quotas ready to be uploaded when ready.

Based on how many 2 day intervals we have had in Marvel Snap's lifetime, that is an approximate amount of over 900 instances of a ruleset required to keep a backlog of version control, since the inception of this games release. Now again, we cannot determine the entire amount of rulesets as part of a Regions Service Quota, and JSON text is easy stored in formats to be copied.

But to suggest that each month, an exceptionally large amount of rulesets are required to properly maintain a deck based matchmaking system and they must be routinely re uploaded every 48 hours, ask yourself: Is having Engineers persistently uploading and removing AWS rulesets every 48 hours a feasible proposition? Is the concept of a ruleset been created by Second Dinner ahead of schedule based on a preconception of what exactly is a “counter” or “mirror” a feasible proposition? Does Second Dinner's design philosophy of removing engineers from backend management not contradict the fallacy that is the re upload of rulesets every 2 days?

Arguments Against Deck Based Matchmaking.

Based on the above information provided, here are two arguments that do not support the theory that deck based matchmaking exists...

- The Jean Grey / Super Skrull Argument.

for over 9 weeks between the months of September and November of 2023, when playing a Super Skrull against an opponent's Jean Grey: it would result in neither player been able to play cards in any lane, for the rest of the game, due to an unresolved bug...
https://discord.com/channels/978545345715908668/1168814143445413918

If decks are "matched" against one another based on cards included, the same could definitely be said of the opposite. Second Dinner had nothing to gain in not preventing pairing of a deck with Super Skrull against a deck with Jean Grey for almost 2 and a half months, if the end result is neither player been able to play any cards for the rest of the game, and forcing one player to retreat or run the game time down. Unless that option is not feasible or too costly by comparison.

During this stage of the above bug's existence: On the 4th of October, Second Dinner encountered an issue server side that prevented users from been matched against other players AT ALL. With queues up to 5 minutes or longer. This was tasked for the engineers to resolve and was fixed within 3 hours...
https://discord.com/channels/978545345715908668/1158862471734505553/1158897494739124275

So Although a server side matchmaking issue is feasible to fix in under 3 hours, ask yourself: Why is a prevention of a match up unfeasible to implement at any point over a 9 week period?

- The Daredevil / Daredevil Argument.

Because of the nature of Daredevil's ability. One could construe that Daredevil is both a “counter” to a deck with daredevil, as well as been included in a deck as potential "mirror" to Daredevil.

If you have a Daredevil on your side of the board, and your opponent has a Daredevil, his ability does not resolve in either instance...
https://discord.com/channels/978545345715908668/978547819214434304/1007666190556086312

We have established earlier that an if statement must be true in order for compound rules to resolve and form player tickets.

When attempting to batch players, to what degree would two decks with Daredevil fail to match? Is every pairing of a deck with Daredevil doomed to run into latency wait time rules, and force Second Dinner to spent more in matchmaking hours and failed player ticket submissions? An argument could be made that “Daredevil is the exception to this rule”. If that is the case, ask yourself: Do more exceptions exist? Is the concept of a card based ruleset redundant if the end result was NOT based on the cards included in your deck? In order to counter deck based matchamking rulesets, do you truly beleive that based on this information: an option to prevent deck based matchmaking is to include Daredevil in your deck to push out latency wait times, and result in consitent untrue compound rules?

Disputed Player Community Claims.

For over 1 year now, the development team have consistently stated the opposite of baseless claims that some in the community promote, with no evidence to back these said claims up. Players have also put in small, albeit dedicated sample sizes of data in an effort to disprove these claims with tracked game match ups. You can see some examples here...
https://postimg.cc/gallery/JnS1Wvd

Ben Brode, the CEO of Second Dinner has also weighed in on some ridiculous notions on stream, as well as providing examples of the scope of difficulty in further increasing matchmaking even slightly, with rulesets based on cards you had not seen before in your collection...
https://www.youtube.com/live/coDDgWaVlKY?si=HvfzUCTRBsHhJaEL&t=5325

Marvel Snapzone also has a dedicated source of references. And have come to the same result...
https://marvelsnapzone.com/matchmaking/

Conclusion.

Whether it be a counter to your deck, a mirrored copy of your deck, or both concurrently: The cost of feasibly managing a system of this scale does not outweigh the simple truth that games with more complex rulesets take longer than ones without. And this runs up the costs when attempting to batch players by quite substantial amounts based on the amount of concurreny levels attempted per minute. Including every millisecond of matchmaking time, as well as all successful and failed attempts to batch players.

Second Dinner's design philosophy goes against the concept of any additional rulesets pertained to matchmaking ontop of the ones currently used to match players. It provides diminishing returns, provide little benefit and does little to serve their needs in maintaining quality and seamless matchmaking without additional costs associated to providing this service. The amount of work involved in consitently maintaining these rulesets does not contradict the fact that these rulesets would need to be created in advance by teams of Engineers and Dvelopers on a consistent basis with omnisicence and predicition of a cards "counters" , and the popularity of decks the card will be included in to form "mirrors".

Using the information provided above, we can come to a very concrete conclusion...

There is no deck based matchmaking.

Decks are not prevented from pairing based on cards included. And decks are not accepted for pairing based on cards included.