This is basically an article describing why you can’t just look at an event after it occurs, see that it has some extremely rare characteristics, and then determine it was unlikely to happen by chance.
It is like asking someone to pick a random number between 1 and 1 million and then saying, “oh my god, it must not actually be random… the chances of choosing the exact number 729,619 is 1 in a million! That is too rare to be random!”
“You know, the most amazing thing happened to me tonight. I was coming here, on the way to the lecture, and I came in through the parking lot. And you won’t believe what happened. I saw a car with the license plate ARW 357. Can you imagine? Of all the millions of license plates in the state, what was the chance that I would see that particular one tonight? Amazing!”
Feynman is one of the best ever at explaining complicated concepts in ways almost everyone can understand. That is a very rare skill for the super intelligent to have.
Agreed on Feynman, but not necessarily on the generalization that it being a rare skill to simplify things. When you understand a thing so well, you can simplify it enough.
I think it also takes a certain humility of character (which can coexist with tremendous self-esteem and even ego; see Feinman, Richard for an example).
I know plenty of smart people that know a topic well and are still not great at simplifying it in a way that can be understood well by laymen. Separate skill imo
Your comment made me think of an interesting story and a funfact.
During WW2, allies tried to guess the number of German tanks by observing the serial numbers on captured tanks.
https://en.m.wikipedia.org/wiki/German_tank_problem
If, say, the serial numbers are unique, and come in sequence, if the five first numbers you see are all less than 100, it's a far chance that there aren't produced 200 tanks. (Provided some assumptions, of course.)
The funfact is that you get different results if you follow the frequentist or the Bayesian approach.
The Bayesian results will depend on the prior. They use a uniform distribution over # tanks produced, in the limit of the distribution's maximum -> infinity. Is that reasonable? Something more constrained might be better, maybe a gamma-Poisson prior with gamma mean based on some plausible estimate of production rate.
(The frequentist/Bayesian estimates should converge as you collect more observations.)
Yeah tbh it doesn't really go into chess-specific stats either
You could look at a bunch of other metrics to identify cheating: how many errors/perfect moves^ and whether that's within the usual range. How well were the opponents playing? Etc
If you consider that Nakamura might have been having a good day/week, was already stronger than his opponents, and some of them may have had bad games/days, you can change something from "extremely unlikely" to "about a dice roll"
not really. this may be true for the average player, but as Magnus has explained multiple times, all he or another top GM would need to be near-unbeatable would be to check an engine in 1 or 2 critical positions per game. this essentially impossible to detect statistically. even if a cheater were to use an engine on every move, it would be trivial to just vary the engine used for each turn, vary the number of moves picked, sometimes play a slightly worse move to evade detection, etc etc
Is that bit in The Queen’s Gambit about chess players coaching each other between matches complete bullshit? Or should one expect a player to occasionally play uncharacteristically when the stakes are high because they would seek out advice which skews their play?
Also psychological games fall neatly into the scenario you describe. I play better and you play worse because I got into your head, or sent the noisy people to be across the hall from you instead of from me, so I slept like a baby and you didn’t.
The adjournments in The Queen's Gambit were rendered obsolete after chess engines became strong enough to be useful in analysis. The last year that they were permitted was 1996.
Match play at the World Championship (where the two players play each other repeatedly for many games) involves a ton of inter-game coaching and work as each player's team goes over what went well, what went wrong, and how the next game should be approached.
Round robin play in small fields also has a significant amount of preparation because the schedule is known in advance, so players will know whom they have to play the following morning and will prepare accordingly.
I'm not comfortable saying that Hikaru does exactly 0 preparation for 3-minute Chess.com blitz games, but it's probably pretty close to 0.
The article itself states that this is not really a pattern of nature, but just a feature of log-normal distributions that sometimes do occur naturally.
This article feels like an illustration of how easy it is to fool top chess players. For example, if the accusation was against Hans Niemann, top chess players and their fans would be eating it up.
Not the same though because we aren't talking about random events. If a player with a significantly lower ELO than Hikaru got the same winning streak against the same tier of players then you could absolutely conclude that it was cheating.
Yeah, but say 1,000 people each flipped a coin 10,000 times and one of them once got a streak of 29 heads out of 30 flips. Can we assume anything then?
You can calculate the probability of having a fair coin and as N(Heads) increases that probability goes down. Each flip is indeed independent but the distribution of flips tells you something about the coin.
We aren't predicting flips based on "a" previous flip. We're predicting them based on the set of ALL KNOWN previous flips, which allows a statistical model.
'If you flipped a coin 100 times and all you got are heads you'
By starting the sentence with if, you are selecting the occurrence to look at.
If you said I am about to look at the results of this coin toss that happened yesterday, if it is all heads then I am going to assume it was not random, then you are making the claim before you have seen the results. You can still be wrong, but the chances of you being wrong is the rarity of the event.
My favorite way to describe this is in the context of predictions. It's the difference between throwing a dart to hit a target and throwing a dart to paint a target around where it lands.
>This is basically an article describing why you can’t just look at an event after it occurs, see that it has some extremely rare characteristics, and then determine it was unlikely to happen by chance.
No. That's not it. In this case, if you properly control for all the factors, it turns out that the odds of Nakamura having that kind of a win-streak (against low-rated opponents) was in fact high.
> This is basically an article describing <snip hot take>
This is entirely wrong and missing basic high school mathematics for non-theater kids.
The original claim is not archived, if you can be bothered you can track it down and do the correct 'hot take'. You can't just grab the first statistical principal you think of even if everyone else on Hacker News does.
Article - "it violates the likelihood principle", this seems wrong and Nakamura seems right, but you'd have to look at the original claim.
They were finding patterns in a long biased list of numbers, probably.
> Nakamura responded to Kramnik’s allegations by arguing that focusing on a particular streak while ignoring other games was cherry-picking. The researchers note that there’s a problem with this argument, too, as it violates the likelihood principle. This principle tells us the interpretation should only rely on the actual data observed, not the context in which it was collected.
I don't quite understand this objection? If I won the lottery at odds of 10 million to 1, you'd say that was a very lucky purchase. But if it turned out I bought 10 million tickets, then that context would surely be important for interpreting what happened, even if the odds of that specific ticket winning would be unchanged?
I believe they're speaking within the scope of the Bayesian analysis. We could interpret games outside of the winning streak as evidence to whether he's a cheater or not. Instead, I believe they are looking at the question of "given this winning streak in particular, what's the probability of him cheating in this set of games"?
They start with a prior (very low probability), I'm assuming they use the implied probabilities from the Elo differences, and then update that prior based on the wins. That's enough to find the posterior they're interested in, without needing to look outside the winning streak.
> "given this winning streak in particular, what's the probability of him cheating in this set of games"
I think the problem lies in the antecedent. Given all chess tournaments played, how often would we observe such a winning streak on average? If the number of winning streaks is near the average, we have no indication of cheating. If it is considerably lower or higher, some people were cheating (when lower, than the opponents).
Then the question is, whether the numbers of winning streaks of one person are unusually high. If we would for example expect aprox. 10 winning streaks, but observe 100, we can conclude that aprox. 90 were cheating. The problem with this is that the more people cheat, the more likely we are to suspect an honest person of cheating as well.
Again, this would be different if the number of winning streaks for a particular person were unusually high.
His performance in games outside the streak is relevant to the prior of his being a cheater, which in turn is highly relevant to how calculate p(cheater | this streak).
Indeed. I'd say that the issue is that they are misinterpreting the word "collecting". The principle is true if you are collecting or observing data live, but this data was collected long ago and with a much wider scope: when the games were recorded.
What they are doing here is sampling the data after the fact, and obviously one needs to take a uniformly random sample of a dataset for any statistical analysis done on it to be representative.
It is well known that Kramnik baselessly accuses everyone. The article seems to be more about statistics than chess, and doesn’t make any accusations. Kind of a click bait title IMO.
The title is “Did a US Chess Champion Cheat?” and the text of the article uses statistical analysis to show that the person most likely did not cheat. What would you consider to be misleading between the title and the article?
There's a YouTube video where he orders a match with someone and insists that they order brand new in box laptops and a locally hosted chess server (I think the hypothesis was testing if in person games were any different than online. The other player was in the same room). But they ran into technical difficulties when windows began auto-updates.
He's accused Luis Paulo Supi (Brazillian grandmaster) a few times after losing to him, and he accused Andrew Tang after losing to him. The latter was criticized in some online circles because it was seen as bullying a then 14 year old.
I don't know many other notable cases of Nakamura accusing players of cheating. Many players dislike how Nakamura conducts himself on stream and how he interacts with the chess community and this leads to exaggeration. It's simply wrong to compare him to Kramnik, who has dedicated many hours over the last couple years to accusing players.
Hikaru accused Luis Paulo Supi of cheating at least twice.
From his Wikipedia article:
```
In an online blitz tournament hosted by the Internet Chess Club in May 2015, American Grandmaster Hikaru Nakamura accused Supi of cheating (Supi had defeated Nakamura).[2] The tournament judges accepted Nakamura's accusation, reverted the match's result, and banned Supi from the tournament. Brazilian Grandmaster Rafael Leitão wrote in his personal website, "Accusing him of using an engine in this match is absurd. The match is full of tactical mistakes. Nakamura played extremely poorly and, honestly, wouldn't have survived long against any engine given his terrible opening.".
```
Some years later Nakamura lost 4-0 and again insinuated that GM Supi used an engine.
Despite all that, Nakamura still published a video calling him a "legend" for once beating Magnus in 18 moves
I'm just trying to figure out how you even cheat on chess, the only thing that comes to mind is moving pieces, and sneaking new ones on the board, but if there's enough cameras, how do you get away with it, eventually someone WILL notice, highlight it, point it out, and you will be shamed.
Just having someone who is following the game with a chess engine and who has a way to get a single message to you telling you that your opponent's last move was a serious blunder would be enough to give you a noticeable advantage.
For example look at the position in this video [1] from a recent game on Chess.com between Hikaru Nakamura and Fabiano Caruana (the title of the video says Magnus vs Hikaru because the video covers 3 of Hikaru's games in the tournament).
I linked to a spot in the video a little before the part where one simple message could changed the game because the host is explaining what Hikaru is going to be trying to do. Briefly, trading pieces off is good for Hikaru, and that's what he starts to do.
You can see from the evaluation bar this Stockfish says he is slightly better.
Then he plays Bg5 which looks like an easy way for force a pair of bishops off, continuing the plan. But look at the evaluation bar! It quickly swings from 0.2 in favor of white to 1.7 in favor of black. But black can only realize that advantage by playing RxN, a move that Fabiano did not even consider. He went on to lose the game.
A prearranged signal from a confederate that meant "Hikaru just made a game changing blunder" would very likely have resulted in Fabiano seeing RxN. It's a move that many would spot if they were given the position as a puzzle and so knew there was a tactic somewhere.
> I'm just trying to figure out how you even cheat on chess,
You use a chess engine to tell you the best move - you can run a chess engine on a modern phone that will easily best the world's top human chess players.
The simplest forms of this are things like: "play online, chess engine open in another window", "use your phone hiding in a bathroom cubicle" and "member of the audience follows your game with a chess engine and signals you somehow"
There are also rumoured to be very subtle ways of doing this - like playing unassisted for most of the game, but an engine providing 'flashes of genius' at one or two crucial moves of the game.
Major competitions have things like metal detectors and time-delay video feeds hoping to make cheating harder.
Since even a phone has enough processing power to make Stockfish play better than a super-GM, the Faraday cage isn't enough to prevent, say, someone tapping the position into a computer on their person and feeling for some sort of vibration[1] in response. It takes very little information to represent a position, and commentators have pointed out that the minimum amount of information required to produce a decisive advantage is 1 bit ("A winning move exists").
This makes me want to cheat just as a technical challenge. Could I hide a computer in my hair? Could I ingest a capsule computer and communicate with it using the resonance of my teeth chattering? (No, I would not insert one in an inappropriate place).
I'm sure it would be a downer that I cheated but it would do them a favor by saying: "look, you cannot stop it. Time for something new".
There is a Java script plugin for lichess that verbally tells you the best move in each position. I installed it (only for eval) and won my game (so unfair! But it was a random and not rated game). I removed the script. So it would be easy to use this or something like this to announce game changing situations.
"... But had I started cheating in a clever manner, I am convinced no one would notice. I would've just needed to cheat one or two times during the match, and I would not even need to be given moves, just the answer on which was way better. Or, here there is a possibility of winning and here you need to be more careful. That is all I would need in order to be almost invicible."
Getting any kind of information from a chess engine would be sufficient to gain an edge for a good player. Even something as simple as a nudge that there is a high value move in a position with no information about what the actual move is could be enough. Big chess tournaments tightly control phones and other devices for this reason. That's on a single-match level. On a tournament level there have been allegations of collusion where players will intentionally arrange their own matches to either be quick draws (to get a break to focus on other matches) or to give points to a designated player to help them win the tourney, Fischer famously accused Soviet chess players of doing this.
In the old Soviet/US rivalry days there was an accusation of cheating that I thought was novel. The accusation was that the Soviet players in the middle rounds were doing subtle not-right moves with the US #1. This forced the lead US player to put way too much effort into figuring out if it was some new line that he didn't know about and tiring him out. Then by the time he got to the final he was exhausted and confused.
No? It's a technique that could readily be done by one person, and teams are allowed to strategize. Bluffing/deception is kosher in chess, just harder as the key elements of the game are all public.
> During adjournments, players could count on the help of other strong masters, called seconds. These seconds would analyze the position and tell the player what they should play when the game resumed.
In parallel to this (and Bobby Fischer explicitly accused them of this), the Soviet players had already decided who would be the champion amongst themselves, and subtly let that player win his matches so that he was fresh and well-rested when he ended up playing non-Soviet players.
The vast, overwhelming majority of chess games are not played in front of cameras or even in-person. The accusation in the article was about online play, and specifically blitz which is played online even more commonly than slower formats of chess because moving quickly is easier for many people with a mouse than a physical board.
The way people cheat online is by running a chess engine that analyzes the state of the board in their web browser/app and suggests moves and/or gives a +/- rating reflecting the balance of the game. Sometimes people run it on another device like their phone to evade detection, but the low-effort ways are a browser extension or background app that monitors the screen. The major online chess platforms are constantly/daily banning significant amounts of people trying to cheat in this way.
Chess.com and Lichess catch these cheaters using a variety of methods, some of which are kept secret to make it harder for cheaters to circumvent them. One obvious way is to automatically compare people's moves to the top few engine moves and look for correlations, which is quite effective for, say, catching people who are low-rated but pull out the engine to help them win games occasionally. It's not that good for top-level chess because a Magnus or Hikaru or basically anyone in the top few hundred players can bang out a series of extremely accurate moves in a critical spot - that's why they're top chess players, they're extremely good. Engine analysis can still catch high-level cheaters, but it often takes manual effort to isolate moves that even a world-champion-class human would not have come up with, and offers grounds for suspicion and further investigation rather than certainty.
For titled events and tournaments, Chess.com has what's effectively a custom browser (Proctor) that surveils players during their games, capturing their screen and recording the mics and cameras that Chess.com requires high-level players to make available to show their environment while they play. This is obviously extremely onerous for players, but there's often money on the line and players do not want to play against cheaters either so they largely put up with the inconvenience and privacy loss.
Despite all of the above, high-level online cheating still happens and some of it is likely not caught.
> It's not that good for top-level chess because a Magnus or Hikaru or basically anyone in the top few hundred players can bang out a series of extremely accurate moves in a critical spot - that's why they're top chess players, they're extremely good.
Interesting; I thought I'd read that even the very best players only average ~90% accuracy, whereas the best engines average 99.something%?
Top-level players regularly are in the 90-95% range aggregated over many games, with spikes up to 98-99%. If you have 98 or 99% accuracy over the course of an entire game (which happens sometimes!), it's either very short or you had significant sequences where you were 100% accurate. If that happened in one of my games it'd be clear evidence I was cheating, if it happens in a Magnus game it's him correctly calculating a complex line and executing it, which he does pretty often.
Edit: Even lower-level cheated games are rarely 100% accurate for the whole game, cheaters usually mix in some bad or natural moves knowing that the engine will let them win anyways. That's why analysis is usually on critical sections, if someone normally plays with a 900 rating but spikes to 100% accuracy every time there's a critical move where other options lose, that's a strong suggestion they're cheating. One of the skills of a strong GM is sniffing out situations like that and being able to calculate a line of 'only moves' under pressure, so it's not nearly as surprising when they pull it off.
To compute accuracy, you compare the moves which are made during the game with the best moves suggested by the engine. So, the engine will evaluate itself 100%, given its settings are the same during game and during evaluation.
You get 99.9something% when you evaluate one strong engine by using another strong engine (they're mostly aligned, but may disagree in small details), or when the engine configuration during the evaluation is different from the configuration used in a game (e.g. engine is given more time to think).
Accuracy is a poor measure for cheating since better chess players will put you in a more complicated position. I'm not especially good but I've played some games with high accuracy just because I just did some book moves and the opponent makes a mistake. Accuracy was high but the correct moves were never especially hard to see.
reading your description of the "invasiveness" of chess.com's surveillance of high level tournament play, I realized that chess.com could issue their own anal probe, a sonar listening device to check that there aren't any other anal probes in use. finally! we can be assured of a good clean game played fairly from both seats!
Getting tips from another person or a computer on what best move to make. This could be as simple as a compatriot in the audience giving you hand signals.
Kramnik has proven himself a troll and did a lot of damage to the reputation of honest and otherwise wholesome people and this without credible proof at all. That those allegations persist under the form of news article is very unfortunate to those victims of his smearing.
>>The researchers note that there’s a problem with this argument, too, as it violates the likelihood principle. This principle tells us the interpretation should only rely on the actual data observed, not the context in which it was collected.
and then in the publication itself:
>>The likelihood principle [Edwards et al., 1963]
is a fundamental concept in Bayesian statistics that states that the evidence from an experiment is
contained in the likelihood function. It implies that the rules governing when data collection stops
are irrelevant to data interpretation. It is entirely appropriate to collect data until a point has been
proven or disproven, or until the data collector runs out of time, money, or patience
Surely there is a difference when you look at someone who played 46 games online in his life and scored 45.5 and when you look at someone who played 46000 games and scored 45.5/46 once.
The difference is that Kramnik wasn't "collecting the data" but looked at the whole Nakamura's playing history and found a streak.
Another example would be looking at coinflips and discarding everything before and after you encounter 10 heads in a row to claim you have solid evidence that the coin is biased.
They are misapplying the principle here. If what they wrote was correct then someone claiming: "Look, Nakamure won 100 out of 100 if you just look at games 3, 17, 21, 117...." would be proving Nakamura cheated if they applied methodology from the paper even assuming one in 10000 guilty players.
Just because you can choose sampling strategy and stopping rules (what the likelyhood principle states) doesn't mean you can discard data you collected or cherry pick parts that support your hypothesis.
How the data is collected is absolutely relevant and Nakamura is right to point it out.
General statistical question. If we say extend the coin flip example distribution to say 10B times. Should/would we expect to see a streak of 100 or even 1000 in the distribution somewhere? Intuition alone tells me probably not for 1000 but a smallish chance for 100 (even if 10B in a row i would think a streak of 100 would be unlikely)
Your intuition's not bad. The expected value for the longest run of heads in N total flips of a fair coin is around log2(N) - 1 with a standard deviation that's approximately 1.873 plus a term that vanishes as N grows large. log2(10B) - 1 is approximately 32 and with that standard deviation, even a run of 100 in 10B flips is incredibly unlikely. For more info see Mark F. Schilling's paper, "The Longest Run of Heads" available here https://www.csun.edu/~hcmth031/tlroh.pdf.
The underlying flaw in this analysis is it assumes ratings reflect actual performance in a given game. A long winning streak becomes far more likely if one of the players is part of several matches while tired, drunk, etc. Similarly a players peak performance is going to be higher than their ELO because that ELO includes games played under less ideal conditions.
ELO is presumably more accurate for over the board games at tournaments where players bring their A game than low stakes online games where someone may be less engaged. That’s IMO more worth testing.
While it is good to see some Bayesian statistics in use, I wouldn't in this case put so much emphasis on an exact calculated probability that he did or din not cheat, the prior in this case is simply too wishy-washy for that.
The sound conclusion is that this is not evidence of cheating, but it is not evidence of the contrary either.
I find it counter intuitive that the frequency of cheating matters. It's not something that happens randomly, people choose to. And if the #2 in the world decides to cheat it may be for different reasons than other players.
But, of course he doesn't. He streams all his games and gives constant stream of consciousness commentary. If you can explain your top level moves live with seconds per move, you aren't cheating.
> It's not something that happens randomly, people choose to.
In Bayesian analysis, probability does not refer to the long-term frequency but instead to the subjective credence given to the event. Otherwise the probability of any one-off event would be undefinable. Therefore it follows that you need to have a prior over possible hypothesis in order to update your beliefs systematically according to the laws of probability theory. If it were known that Hikaru had cheated in the past, but typically does not, we might use a different prior (e.g. a Laplacian prior in this case); if we knew cheating to be dependent on some other measurable variable (e.g. the emotional state of the player), we would incorporate this into our evidence.
If you want a deep dive into chess cheating, including a lot of wild stories, Sarah Z put out an entertaining Youtube video [1] a couple of months ago that explores the concept. It's a long video, but well worth the watch.
There is probably 1000 hours of videos online of Hikaru talking through games, literally exhibiting his skill in full transparency. Hard for me to even understand what it would mean for him to cheat, his brain at times feels like the cheat.
It's frustrating that their entire analysis is based on the claim that cheating occurs in maybe 1 out of 10,000 games; they got this from a quote in an interview with the deputy president of the World Chess Federation after he had been beaten in a charity match by someone who admitted cheating. To their credit they also ran the analysis assuming cheating is 1/500 and the odds rose to 7%. I suppose it makes sense that they are merely rebutting the accusations based on the same methodology but it's still frustrating.
Part of Bayesian analysis is choosing your prior probabilities. Luckily, with enough data the priors become less and less important, but you do need to choose them.
I'd be much more suspicious if his online performance didn't track with his professional over-the-board performance, where cheating would be much more difficult.
What are the odds that a cheating accusation accurately identifies an instance of cheating?
I don't say this lightly: Kramnik very likely has some sort of untreated psychiatric disorder. He is effectively a lolcow in the chess community because he regularly (as in, almost daily) accuses much better chess players of cheating.
It's honestly a bit undignified to treat his accusation against Nakamura as anything other than a man yelling at the sky.
There’s some strange inverse, where situations we KNOW are rife with cheating have few accusations thereof (bike racing perhaps), and those that are almost certainly very rare may also have few accusations (because it’s rare).
Sometimes you have to treat any accusation as “real” just to keep the cheaters at bay. (Cheating at online bridge is rampant, and cheating at bridge competitions was and perhaps still is fraught with many scandals).
It’s often common that the cheaters really ARE very good players - they’re just looking for less work, not a goal they couldn’t obtain otherwise.
Tangently related, reading this I couldn't help but think about the biological passports for professional cyclist. It tracks blood and other values overtime, so that anomalies will jump out.
Online chess is nonsense. One can beat all detection of using engines. Use 10 different engines and take their moves in random order. To avoid getting same move from all engines, make them to have varying strength.
> Kramnik pointed out the statistical improbability of Nakamura’s streak and stated that such a winning run would require the chess prodigy to play at a level higher than his current Elo rating (an estimate of a player’s skill level based on their historical play).
While ELO ratings are a probabilistic model, who said wins and losses have to be randomly distributed, there can be bad days and good days, for example if you haven't slept or if you are at the peak combination of study and cognitive, say because you are well rested on a monday and have been studying on the weekend.
This is basically an article describing why you can’t just look at an event after it occurs, see that it has some extremely rare characteristics, and then determine it was unlikely to happen by chance.
It is like asking someone to pick a random number between 1 and 1 million and then saying, “oh my god, it must not actually be random… the chances of choosing the exact number 729,619 is 1 in a million! That is too rare to be random!”
“You know, the most amazing thing happened to me tonight. I was coming here, on the way to the lecture, and I came in through the parking lot. And you won’t believe what happened. I saw a car with the license plate ARW 357. Can you imagine? Of all the millions of license plates in the state, what was the chance that I would see that particular one tonight? Amazing!”
-Feynman, from Six Easy Pieces
Feynman is one of the best ever at explaining complicated concepts in ways almost everyone can understand. That is a very rare skill for the super intelligent to have.
Agreed on Feynman, but not necessarily on the generalization that it being a rare skill to simplify things. When you understand a thing so well, you can simplify it enough.
I think it also takes a certain humility of character (which can coexist with tremendous self-esteem and even ego; see Feinman, Richard for an example).
I know plenty of smart people that know a topic well and are still not great at simplifying it in a way that can be understood well by laymen. Separate skill imo
yea i always remember this when people put on their tinfoil hats about some rare event.
Your comment made me think of an interesting story and a funfact.
During WW2, allies tried to guess the number of German tanks by observing the serial numbers on captured tanks.
https://en.m.wikipedia.org/wiki/German_tank_problem If, say, the serial numbers are unique, and come in sequence, if the five first numbers you see are all less than 100, it's a far chance that there aren't produced 200 tanks. (Provided some assumptions, of course.)
The funfact is that you get different results if you follow the frequentist or the Bayesian approach.
The Bayesian results will depend on the prior. They use a uniform distribution over # tanks produced, in the limit of the distribution's maximum -> infinity. Is that reasonable? Something more constrained might be better, maybe a gamma-Poisson prior with gamma mean based on some plausible estimate of production rate.
(The frequentist/Bayesian estimates should converge as you collect more observations.)
Yeah tbh it doesn't really go into chess-specific stats either
You could look at a bunch of other metrics to identify cheating: how many errors/perfect moves^ and whether that's within the usual range. How well were the opponents playing? Etc
If you consider that Nakamura might have been having a good day/week, was already stronger than his opponents, and some of them may have had bad games/days, you can change something from "extremely unlikely" to "about a dice roll"
^ according to stockfish
not really. this may be true for the average player, but as Magnus has explained multiple times, all he or another top GM would need to be near-unbeatable would be to check an engine in 1 or 2 critical positions per game. this essentially impossible to detect statistically. even if a cheater were to use an engine on every move, it would be trivial to just vary the engine used for each turn, vary the number of moves picked, sometimes play a slightly worse move to evade detection, etc etc
What I don’t understand is that Hikaru can visualise in his head 30+ moves ahead from both plays, and yet he’s not better than Magnus?
??
I'm all for having a Hikaru/Magnus discussion--one of my favourite topics--but this just doesn't make sense
Is that bit in The Queen’s Gambit about chess players coaching each other between matches complete bullshit? Or should one expect a player to occasionally play uncharacteristically when the stakes are high because they would seek out advice which skews their play?
Also psychological games fall neatly into the scenario you describe. I play better and you play worse because I got into your head, or sent the noisy people to be across the hall from you instead of from me, so I slept like a baby and you didn’t.
The adjournments in The Queen's Gambit were rendered obsolete after chess engines became strong enough to be useful in analysis. The last year that they were permitted was 1996.
Match play at the World Championship (where the two players play each other repeatedly for many games) involves a ton of inter-game coaching and work as each player's team goes over what went well, what went wrong, and how the next game should be approached.
Round robin play in small fields also has a significant amount of preparation because the schedule is known in advance, so players will know whom they have to play the following morning and will prepare accordingly.
I'm not comfortable saying that Hikaru does exactly 0 preparation for 3-minute Chess.com blitz games, but it's probably pretty close to 0.
There are actually some freaky patterns in nature (including how people think) that can help identify fake data...
https://en.wikipedia.org/wiki/Benford%27s_law
The article itself states that this is not really a pattern of nature, but just a feature of log-normal distributions that sometimes do occur naturally.
This article feels like an illustration of how easy it is to fool top chess players. For example, if the accusation was against Hans Niemann, top chess players and their fans would be eating it up.
Not the same though because we aren't talking about random events. If a player with a significantly lower ELO than Hikaru got the same winning streak against the same tier of players then you could absolutely conclude that it was cheating.
I had a math teacher in college that told a funny joke:
"For safety, I always bring a bomb with me when I fly"
"Why?"
"Because the odds of two people bringing a bomb on the same plane are so low"
You can and you should.
If you flipped a coin 100 times and all you got are heads you really should assume it didn't happen by chance.
Yeah, but say 1,000 people each flipped a coin 10,000 times and one of them once got a streak of 29 heads out of 30 flips. Can we assume anything then?
Nope. A previous flip has no bearing on the next flip.
You can calculate the probability of having a fair coin and as N(Heads) increases that probability goes down. Each flip is indeed independent but the distribution of flips tells you something about the coin.
We aren't predicting flips based on "a" previous flip. We're predicting them based on the set of ALL KNOWN previous flips, which allows a statistical model.
The odds of that happening by chance is so astronomical it involves a number you've probably never even heard of.
'If you flipped a coin 100 times and all you got are heads you'
By starting the sentence with if, you are selecting the occurrence to look at.
If you said I am about to look at the results of this coin toss that happened yesterday, if it is all heads then I am going to assume it was not random, then you are making the claim before you have seen the results. You can still be wrong, but the chances of you being wrong is the rarity of the event.
My favorite way to describe this is in the context of predictions. It's the difference between throwing a dart to hit a target and throwing a dart to paint a target around where it lands.
>This is basically an article describing why you can’t just look at an event after it occurs, see that it has some extremely rare characteristics, and then determine it was unlikely to happen by chance.
No. That's not it. In this case, if you properly control for all the factors, it turns out that the odds of Nakamura having that kind of a win-streak (against low-rated opponents) was in fact high.
> This is basically an article describing <snip hot take>
This is entirely wrong and missing basic high school mathematics for non-theater kids.
The original claim is not archived, if you can be bothered you can track it down and do the correct 'hot take'. You can't just grab the first statistical principal you think of even if everyone else on Hacker News does.
Article - "it violates the likelihood principle", this seems wrong and Nakamura seems right, but you'd have to look at the original claim.
They were finding patterns in a long biased list of numbers, probably.
> Nakamura responded to Kramnik’s allegations by arguing that focusing on a particular streak while ignoring other games was cherry-picking. The researchers note that there’s a problem with this argument, too, as it violates the likelihood principle. This principle tells us the interpretation should only rely on the actual data observed, not the context in which it was collected.
I don't quite understand this objection? If I won the lottery at odds of 10 million to 1, you'd say that was a very lucky purchase. But if it turned out I bought 10 million tickets, then that context would surely be important for interpreting what happened, even if the odds of that specific ticket winning would be unchanged?
I believe they're speaking within the scope of the Bayesian analysis. We could interpret games outside of the winning streak as evidence to whether he's a cheater or not. Instead, I believe they are looking at the question of "given this winning streak in particular, what's the probability of him cheating in this set of games"?
They start with a prior (very low probability), I'm assuming they use the implied probabilities from the Elo differences, and then update that prior based on the wins. That's enough to find the posterior they're interested in, without needing to look outside the winning streak.
> "given this winning streak in particular, what's the probability of him cheating in this set of games"
I think the problem lies in the antecedent. Given all chess tournaments played, how often would we observe such a winning streak on average? If the number of winning streaks is near the average, we have no indication of cheating. If it is considerably lower or higher, some people were cheating (when lower, than the opponents).
Then the question is, whether the numbers of winning streaks of one person are unusually high. If we would for example expect aprox. 10 winning streaks, but observe 100, we can conclude that aprox. 90 were cheating. The problem with this is that the more people cheat, the more likely we are to suspect an honest person of cheating as well.
Again, this would be different if the number of winning streaks for a particular person were unusually high.
His performance in games outside the streak is relevant to the prior of his being a cheater, which in turn is highly relevant to how calculate p(cheater | this streak).
The issue here is that the events are not independent. Because of that, the other games surely provide useful data
Indeed. I'd say that the issue is that they are misinterpreting the word "collecting". The principle is true if you are collecting or observing data live, but this data was collected long ago and with a much wider scope: when the games were recorded.
What they are doing here is sampling the data after the fact, and obviously one needs to take a uniformly random sample of a dataset for any statistical analysis done on it to be representative.
Or similarly I flip a coin a thousand times, but only tell you when it's heads and don't tell you how many flips I did.
I agree. The problem when you look only at some "interesting" data even has a name https://en.wikipedia.org/wiki/Look-elsewhere_effect
Or if Bob Barker has opened door number 3 and there's a goat behind it.
Monty Hall.
It is well known that Kramnik baselessly accuses everyone. The article seems to be more about statistics than chess, and doesn’t make any accusations. Kind of a click bait title IMO.
The title is “Did a US Chess Champion Cheat?” and the text of the article uses statistical analysis to show that the person most likely did not cheat. What would you consider to be misleading between the title and the article?
When I opened the article I thought it was going to be about someone cheating at the US Chess Championship.
"Statistical analysis shows US Chess Champion most likely did not cheat, despite recent claims" would be nice
The headline also complies with Betteridge's Law of Headlines. It's entirely legal.
This is a perfect example of https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...
Headlines following Betteridge's law were the original clickbait, and this definitely fits.
There's a YouTube video where he orders a match with someone and insists that they order brand new in box laptops and a locally hosted chess server (I think the hypothesis was testing if in person games were any different than online. The other player was in the same room). But they ran into technical difficulties when windows began auto-updates.
It’s using a random accusation as a starting point for explaining Bayesian analysis.
Hikaru is also notable for quickly accusing players baselessly
Whom did he accuse? Kramnik is known to accuse other players. I've never heard this about Nakamura.
He's accused Luis Paulo Supi (Brazillian grandmaster) a few times after losing to him, and he accused Andrew Tang after losing to him. The latter was criticized in some online circles because it was seen as bullying a then 14 year old.
I don't know many other notable cases of Nakamura accusing players of cheating. Many players dislike how Nakamura conducts himself on stream and how he interacts with the chess community and this leads to exaggeration. It's simply wrong to compare him to Kramnik, who has dedicated many hours over the last couple years to accusing players.
Hikaru accused Luis Paulo Supi of cheating at least twice.
From his Wikipedia article:
``` In an online blitz tournament hosted by the Internet Chess Club in May 2015, American Grandmaster Hikaru Nakamura accused Supi of cheating (Supi had defeated Nakamura).[2] The tournament judges accepted Nakamura's accusation, reverted the match's result, and banned Supi from the tournament. Brazilian Grandmaster Rafael Leitão wrote in his personal website, "Accusing him of using an engine in this match is absurd. The match is full of tactical mistakes. Nakamura played extremely poorly and, honestly, wouldn't have survived long against any engine given his terrible opening.". ```
Some years later Nakamura lost 4-0 and again insinuated that GM Supi used an engine.
Despite all that, Nakamura still published a video calling him a "legend" for once beating Magnus in 18 moves
Nakamura was sued for $100 million by Hans Niemann.
https://www.chess.com/news/view/hans-niemann-lawsuit-dismiss...
And what was the result? Weren't all counts dismissed?
Vlad the Implier
It's to be expected per https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...
Which is itself intuitive if you have the prior that “making the claim is the stronger headline, so if the claim is true, it’ll be in the headline”
I'm just trying to figure out how you even cheat on chess, the only thing that comes to mind is moving pieces, and sneaking new ones on the board, but if there's enough cameras, how do you get away with it, eventually someone WILL notice, highlight it, point it out, and you will be shamed.
Just having someone who is following the game with a chess engine and who has a way to get a single message to you telling you that your opponent's last move was a serious blunder would be enough to give you a noticeable advantage.
For example look at the position in this video [1] from a recent game on Chess.com between Hikaru Nakamura and Fabiano Caruana (the title of the video says Magnus vs Hikaru because the video covers 3 of Hikaru's games in the tournament).
I linked to a spot in the video a little before the part where one simple message could changed the game because the host is explaining what Hikaru is going to be trying to do. Briefly, trading pieces off is good for Hikaru, and that's what he starts to do.
You can see from the evaluation bar this Stockfish says he is slightly better.
Then he plays Bg5 which looks like an easy way for force a pair of bishops off, continuing the plan. But look at the evaluation bar! It quickly swings from 0.2 in favor of white to 1.7 in favor of black. But black can only realize that advantage by playing RxN, a move that Fabiano did not even consider. He went on to lose the game.
A prearranged signal from a confederate that meant "Hikaru just made a game changing blunder" would very likely have resulted in Fabiano seeing RxN. It's a move that many would spot if they were given the position as a puzzle and so knew there was a tactic somewhere.
[1] https://youtu.be/acjI2KqQ0gI?si=qkfkL6i53UDcBOQd&t=752
> I'm just trying to figure out how you even cheat on chess,
You use a chess engine to tell you the best move - you can run a chess engine on a modern phone that will easily best the world's top human chess players.
The simplest forms of this are things like: "play online, chess engine open in another window", "use your phone hiding in a bathroom cubicle" and "member of the audience follows your game with a chess engine and signals you somehow"
There are also rumoured to be very subtle ways of doing this - like playing unassisted for most of the game, but an engine providing 'flashes of genius' at one or two crucial moves of the game.
Major competitions have things like metal detectors and time-delay video feeds hoping to make cheating harder.
Future chess games will have to be played as Faraday cage matches. Two men enter, one man leaves.
Since even a phone has enough processing power to make Stockfish play better than a super-GM, the Faraday cage isn't enough to prevent, say, someone tapping the position into a computer on their person and feeling for some sort of vibration[1] in response. It takes very little information to represent a position, and commentators have pointed out that the minimum amount of information required to produce a decisive advantage is 1 bit ("A winning move exists").
[1] Yes, the ribald jokes have already been made
This makes me want to cheat just as a technical challenge. Could I hide a computer in my hair? Could I ingest a capsule computer and communicate with it using the resonance of my teeth chattering? (No, I would not insert one in an inappropriate place).
I'm sure it would be a downer that I cheated but it would do them a favor by saying: "look, you cannot stop it. Time for something new".
You'd love NASCAR then.
It's not really cheating in NASCAR, but rather, "it wasn't in the rulebook".
Example - https://www.youtube.com/watch?v=GnZ4nBrp6mo
Hide a computer in your chess clock. A small camera would view the board and somehow flash a code on the lcd display of the clock.
There is a Java script plugin for lichess that verbally tells you the best move in each position. I installed it (only for eval) and won my game (so unfair! But it was a random and not rated game). I removed the script. So it would be easy to use this or something like this to announce game changing situations.
Magnus Carlsen (2021)
"... But had I started cheating in a clever manner, I am convinced no one would notice. I would've just needed to cheat one or two times during the match, and I would not even need to be given moves, just the answer on which was way better. Or, here there is a possibility of winning and here you need to be more careful. That is all I would need in order to be almost invicible."
https://www.youtube.com/watch?v=VcbHmHHwlUQ&t=313s
Getting any kind of information from a chess engine would be sufficient to gain an edge for a good player. Even something as simple as a nudge that there is a high value move in a position with no information about what the actual move is could be enough. Big chess tournaments tightly control phones and other devices for this reason. That's on a single-match level. On a tournament level there have been allegations of collusion where players will intentionally arrange their own matches to either be quick draws (to get a break to focus on other matches) or to give points to a designated player to help them win the tourney, Fischer famously accused Soviet chess players of doing this.
Makes sene! Thanks, I dont play much chess so its a bit out of my wheelhouse.
Bluetooth buttplug (you can get such things on Amazon; https://en.wikipedia.org/wiki/Lovense) and an observer in the audience tapping out Morse code?
Or, more mundanely, bathroom breaks. https://www.nytimes.com/2024/10/16/sports/kirill-shevchenko-...
Your iPhone can reliably beat the best chess players in the world.
In the old Soviet/US rivalry days there was an accusation of cheating that I thought was novel. The accusation was that the Soviet players in the middle rounds were doing subtle not-right moves with the US #1. This forced the lead US player to put way too much effort into figuring out if it was some new line that he didn't know about and tiring him out. Then by the time he got to the final he was exhausted and confused.
I'm not inclined to see that as cheating.
Right, it's collusion.
No? It's a technique that could readily be done by one person, and teams are allowed to strategize. Bluffing/deception is kosher in chess, just harder as the key elements of the game are all public.
Before computers put an end to the practice, long games used to adjourn overnight. https://www.chess.com/terms/chess-adjournment
> During adjournments, players could count on the help of other strong masters, called seconds. These seconds would analyze the position and tell the player what they should play when the game resumed.
Agreed, it's not collusion if it's only done by one person.
That sounds like strategy, not cheating.
In parallel to this (and Bobby Fischer explicitly accused them of this), the Soviet players had already decided who would be the champion amongst themselves, and subtly let that player win his matches so that he was fresh and well-rested when he ended up playing non-Soviet players.
The vast, overwhelming majority of chess games are not played in front of cameras or even in-person. The accusation in the article was about online play, and specifically blitz which is played online even more commonly than slower formats of chess because moving quickly is easier for many people with a mouse than a physical board.
The way people cheat online is by running a chess engine that analyzes the state of the board in their web browser/app and suggests moves and/or gives a +/- rating reflecting the balance of the game. Sometimes people run it on another device like their phone to evade detection, but the low-effort ways are a browser extension or background app that monitors the screen. The major online chess platforms are constantly/daily banning significant amounts of people trying to cheat in this way.
Chess.com and Lichess catch these cheaters using a variety of methods, some of which are kept secret to make it harder for cheaters to circumvent them. One obvious way is to automatically compare people's moves to the top few engine moves and look for correlations, which is quite effective for, say, catching people who are low-rated but pull out the engine to help them win games occasionally. It's not that good for top-level chess because a Magnus or Hikaru or basically anyone in the top few hundred players can bang out a series of extremely accurate moves in a critical spot - that's why they're top chess players, they're extremely good. Engine analysis can still catch high-level cheaters, but it often takes manual effort to isolate moves that even a world-champion-class human would not have come up with, and offers grounds for suspicion and further investigation rather than certainty.
For titled events and tournaments, Chess.com has what's effectively a custom browser (Proctor) that surveils players during their games, capturing their screen and recording the mics and cameras that Chess.com requires high-level players to make available to show their environment while they play. This is obviously extremely onerous for players, but there's often money on the line and players do not want to play against cheaters either so they largely put up with the inconvenience and privacy loss.
Despite all of the above, high-level online cheating still happens and some of it is likely not caught.
Edit: More information on Proctor here: https://www.chess.com/proctor
> It's not that good for top-level chess because a Magnus or Hikaru or basically anyone in the top few hundred players can bang out a series of extremely accurate moves in a critical spot - that's why they're top chess players, they're extremely good.
Interesting; I thought I'd read that even the very best players only average ~90% accuracy, whereas the best engines average 99.something%?
Top-level players regularly are in the 90-95% range aggregated over many games, with spikes up to 98-99%. If you have 98 or 99% accuracy over the course of an entire game (which happens sometimes!), it's either very short or you had significant sequences where you were 100% accurate. If that happened in one of my games it'd be clear evidence I was cheating, if it happens in a Magnus game it's him correctly calculating a complex line and executing it, which he does pretty often.
Edit: Even lower-level cheated games are rarely 100% accurate for the whole game, cheaters usually mix in some bad or natural moves knowing that the engine will let them win anyways. That's why analysis is usually on critical sections, if someone normally plays with a 900 rating but spikes to 100% accuracy every time there's a critical move where other options lose, that's a strong suggestion they're cheating. One of the skills of a strong GM is sniffing out situations like that and being able to calculate a line of 'only moves' under pressure, so it's not nearly as surprising when they pull it off.
> whereas the best engines average 99.something%?
To compute accuracy, you compare the moves which are made during the game with the best moves suggested by the engine. So, the engine will evaluate itself 100%, given its settings are the same during game and during evaluation.
You get 99.9something% when you evaluate one strong engine by using another strong engine (they're mostly aligned, but may disagree in small details), or when the engine configuration during the evaluation is different from the configuration used in a game (e.g. engine is given more time to think).
Accuracy is a poor measure for cheating since better chess players will put you in a more complicated position. I'm not especially good but I've played some games with high accuracy just because I just did some book moves and the opponent makes a mistake. Accuracy was high but the correct moves were never especially hard to see.
Well accuracy is measured against the chess engine’s moves so it would be 100% by definition.
reading your description of the "invasiveness" of chess.com's surveillance of high level tournament play, I realized that chess.com could issue their own anal probe, a sonar listening device to check that there aren't any other anal probes in use. finally! we can be assured of a good clean game played fairly from both seats!
Well that's one hole plugged
Getting tips from another person or a computer on what best move to make. This could be as simple as a compatriot in the audience giving you hand signals.
Pulling a "hand of god" [1] in chess is unlikely to be as successful as it was in soccer.
Cheating is as simple as having somebody feed you chess engine moves from a nearby laptop running stockfish.
[1] https://en.wikipedia.org/wiki/The_hand_of_God
The chess hustlers in parks and beachside tables will take a pawn and the piece next to it with slight of hand. Or nudge it to a worse square.
Kramnik has proven himself a troll and did a lot of damage to the reputation of honest and otherwise wholesome people and this without credible proof at all. That those allegations persist under the form of news article is very unfortunate to those victims of his smearing.
Being a "troll" connotes bad faith, i.e. not believing one's own accusations.
From the footage I've seen of Kramnik, I think he does believe himself, and is just generally very "salty" about losing (as the kids say).
I don't believe the strength of ones convictions factors into whether they are a troll or not. Take patent trolls for example.
>>The researchers note that there’s a problem with this argument, too, as it violates the likelihood principle. This principle tells us the interpretation should only rely on the actual data observed, not the context in which it was collected.
and then in the publication itself:
>>The likelihood principle [Edwards et al., 1963] is a fundamental concept in Bayesian statistics that states that the evidence from an experiment is contained in the likelihood function. It implies that the rules governing when data collection stops are irrelevant to data interpretation. It is entirely appropriate to collect data until a point has been proven or disproven, or until the data collector runs out of time, money, or patience
Surely there is a difference when you look at someone who played 46 games online in his life and scored 45.5 and when you look at someone who played 46000 games and scored 45.5/46 once.
The difference is that Kramnik wasn't "collecting the data" but looked at the whole Nakamura's playing history and found a streak.
Another example would be looking at coinflips and discarding everything before and after you encounter 10 heads in a row to claim you have solid evidence that the coin is biased.
They are misapplying the principle here. If what they wrote was correct then someone claiming: "Look, Nakamure won 100 out of 100 if you just look at games 3, 17, 21, 117...." would be proving Nakamura cheated if they applied methodology from the paper even assuming one in 10000 guilty players. Just because you can choose sampling strategy and stopping rules (what the likelyhood principle states) doesn't mean you can discard data you collected or cherry pick parts that support your hypothesis.
How the data is collected is absolutely relevant and Nakamura is right to point it out.
General statistical question. If we say extend the coin flip example distribution to say 10B times. Should/would we expect to see a streak of 100 or even 1000 in the distribution somewhere? Intuition alone tells me probably not for 1000 but a smallish chance for 100 (even if 10B in a row i would think a streak of 100 would be unlikely)
Your intuition's not bad. The expected value for the longest run of heads in N total flips of a fair coin is around log2(N) - 1 with a standard deviation that's approximately 1.873 plus a term that vanishes as N grows large. log2(10B) - 1 is approximately 32 and with that standard deviation, even a run of 100 in 10B flips is incredibly unlikely. For more info see Mark F. Schilling's paper, "The Longest Run of Heads" available here https://www.csun.edu/~hcmth031/tlroh.pdf.
Neat! I guess this is a common thing to wonder about :)
That’s a cool result, thanks for the link!
The underlying flaw in this analysis is it assumes ratings reflect actual performance in a given game. A long winning streak becomes far more likely if one of the players is part of several matches while tired, drunk, etc. Similarly a players peak performance is going to be higher than their ELO because that ELO includes games played under less ideal conditions.
ELO is presumably more accurate for over the board games at tournaments where players bring their A game than low stakes online games where someone may be less engaged. That’s IMO more worth testing.
While it is good to see some Bayesian statistics in use, I wouldn't in this case put so much emphasis on an exact calculated probability that he did or din not cheat, the prior in this case is simply too wishy-washy for that.
The sound conclusion is that this is not evidence of cheating, but it is not evidence of the contrary either.
I find it counter intuitive that the frequency of cheating matters. It's not something that happens randomly, people choose to. And if the #2 in the world decides to cheat it may be for different reasons than other players.
But, of course he doesn't. He streams all his games and gives constant stream of consciousness commentary. If you can explain your top level moves live with seconds per move, you aren't cheating.
> It's not something that happens randomly, people choose to.
In Bayesian analysis, probability does not refer to the long-term frequency but instead to the subjective credence given to the event. Otherwise the probability of any one-off event would be undefinable. Therefore it follows that you need to have a prior over possible hypothesis in order to update your beliefs systematically according to the laws of probability theory. If it were known that Hikaru had cheated in the past, but typically does not, we might use a different prior (e.g. a Laplacian prior in this case); if we knew cheating to be dependent on some other measurable variable (e.g. the emotional state of the player), we would incorporate this into our evidence.
If you want a deep dive into chess cheating, including a lot of wild stories, Sarah Z put out an entertaining Youtube video [1] a couple of months ago that explores the concept. It's a long video, but well worth the watch.
[1]: https://www.youtube.com/watch?v=ZtN-i-IkRWI
There is probably 1000 hours of videos online of Hikaru talking through games, literally exhibiting his skill in full transparency. Hard for me to even understand what it would mean for him to cheat, his brain at times feels like the cheat.
This leaves out the extremely important detail that many if not all of these 46 games, Hikaru was actively streaming on Twitch.
The actual chess community's takeaway from this (if consensus is important to you) is that Kramnik (the accuser) has lost it a bit.
Kramnik went from chess champion who really came up with new lines in the Berlin defense, beating Kasparov at his peak: to now becoming a troll :(
It's frustrating that their entire analysis is based on the claim that cheating occurs in maybe 1 out of 10,000 games; they got this from a quote in an interview with the deputy president of the World Chess Federation after he had been beaten in a charity match by someone who admitted cheating. To their credit they also ran the analysis assuming cheating is 1/500 and the odds rose to 7%. I suppose it makes sense that they are merely rebutting the accusations based on the same methodology but it's still frustrating.
Part of Bayesian analysis is choosing your prior probabilities. Luckily, with enough data the priors become less and less important, but you do need to choose them.
I think if Kramnik accuses someone of cheating it might actually drop the posterior probability that they cheated.
I'd be much more suspicious if his online performance didn't track with his professional over-the-board performance, where cheating would be much more difficult.
Here's a better question:
What are the odds that a cheating accusation accurately identifies an instance of cheating?
I don't say this lightly: Kramnik very likely has some sort of untreated psychiatric disorder. He is effectively a lolcow in the chess community because he regularly (as in, almost daily) accuses much better chess players of cheating.
It's honestly a bit undignified to treat his accusation against Nakamura as anything other than a man yelling at the sky.
There’s some strange inverse, where situations we KNOW are rife with cheating have few accusations thereof (bike racing perhaps), and those that are almost certainly very rare may also have few accusations (because it’s rare).
Sometimes you have to treat any accusation as “real” just to keep the cheaters at bay. (Cheating at online bridge is rampant, and cheating at bridge competitions was and perhaps still is fraught with many scandals).
It’s often common that the cheaters really ARE very good players - they’re just looking for less work, not a goal they couldn’t obtain otherwise.
Tangently related, reading this I couldn't help but think about the biological passports for professional cyclist. It tracks blood and other values overtime, so that anomalies will jump out.
A simple solution is to hold all chess matches in a SCIF.
Can anyone tell me why numbers in this article are being rendered higgledy-piggledy on my browser, Firefox / Windows 11?
It uses old style figures rather than lining, if that’s what you mean by “higgledy-piggledy”. See https://practicaltypography.com/alternate-figures.html#oldst...
ta
Anyone who does competitive gaming or sports knows that the greatest compliment is to be called a hacker or cheater when not actually cheating.
Title is: Did a US Chess Champion Cheat?
why was it changed? This isn't a 'recent' story, it's from January.
Online chess is nonsense. One can beat all detection of using engines. Use 10 different engines and take their moves in random order. To avoid getting same move from all engines, make them to have varying strength.
> Kramnik pointed out the statistical improbability of Nakamura’s streak and stated that such a winning run would require the chess prodigy to play at a level higher than his current Elo rating (an estimate of a player’s skill level based on their historical play).
While ELO ratings are a probabilistic model, who said wins and losses have to be randomly distributed, there can be bad days and good days, for example if you haven't slept or if you are at the peak combination of study and cognitive, say because you are well rested on a monday and have been studying on the weekend.
Once against, (ab)using Bayes is the favored tool for Technical Boys when they want to bullshit.
I thought this chass cheating story was going to be about Hans Niemann's (alleged) vibrating anal beads. I'm slightly dissapointed.