Malarkey in the NFL: Science and facts are not subjective
First things first: I'm not a scientist. I'm a Patriots fan. You can say a quite enthusiastic one too. However, I have certain principles that I would never, ever cross just because I happen to support a team. Fact-checking is one of them: in fact I run such a website for Turkish media and politics. Along with that comes an obsession with basic journalistic standards. You know, like the keyword of the week: integrity. Respecting basic scientific principles and statistical methodology is also on the list. Oh, and as a civil libertarian, add "innocent until proven guilty" there too. That's why this week has been a quite infuriating one for me so far: I saw every item on the list above being blatantly disregarded by respected or not-so-respected media members, fans of all colors, random people on the internet etc. Thus I wanted to explain how a proper scientific investigation can be conducted in regards to the "DeflateGate" and how the media have mostly failed to report on this matter so far.
Part I. Science
I'll first say this for the fellow Pats fans out there: Belichick did not "debunk" DeflateGate with his scientific explanations yesterday. He provided an explanation for what might have happened as their own study indicated. One thing that remained a bit vague was whether they have handed the referees under-inflated balls and told them to inflate them to/make sure they were at 12.5 PSI, or that the balls were at 12.5 PSI when they handed them but that was an artificially high value due to the ball preparation procedure of the Pats. Belichick, one can say rightfully, did not provide all the details about what that procedure is because teams are not required to do that for national media, but he made it clear that they will tell the NFL about it when/if asked.
What we know, what Belichick reiterated, and what no one can deny is this: The weather affects the PSI values of footballs. That is a fact. In fact, a company named Head Smart Labs have filed a report about this issue. In their experiment, they have tried to recreate the conditions on the field, and their results indicate that the PSI values could have dropped 1.95.
However, also making rounds this morning was Bill Nye's "refutation" of Belichick's explanations. So, how did this guy, this great authority on scientific matters refute yesterday's presser?
Are you kidding me?!? "I don't think you can [do it]" is not a scientific explanation! Anybody who says "Oh, Bill Nye explained..." is committing a basic logical fallacy that is aptly named "appeal to authority." Head Smart Labs's analysis, whether flawed or not, is a valid attempt at being "scientific". Bill Nye's 5-second-long declaration is not. It is as simple as that. (Update: Boston Globe has talked to several scientists and they all agree that Belichick's explanation is technically possible. Does that un-refute what Belichick said? No, because neither "I think it could work" nor "I don't think that could work" are scientific statements.)
Now, let's talk about this other "gotcha" statement that is making rounds: "How come the Colts' balls were not affected?" There are a few problems with that:
1. The NFL has not officially confirmed yet that the Colts' balls were not affected. We have seen reports that they were within legal limits, but again we have seen a lot of reports since Monday that were later denied. (more on this later)
2. Even if that were true, it does not necessarily mean that the balls were not affected. If the Colts' balls were initially at 13.5 PSI and later dropped to 12.5 PSI, they would still be within the limits and be affected by the weather too.
3. Most importantly, the NFL has not provided the details as to how and when the balls were measured. Everything we know so far is based on data presented to us under the "DeflateGate" light.
At this point I will stop talking about every mistaken assumption I've heard so far, and try to answer the following, essential question: Do you want this investigation to be scientific, and the data to be unbiased? Well, this is what should have been done.
1. Both the Patriots' and the Colts' balls were measured in the same environment and their PSI values were recorded.
2. They were measured again at half time under the same circumstances: Same gauge, same room, same (or at least close) time of wait between the two measurements etc.
3. The change in PSI values were compared via a statistical test (T-test, for example) to see if there were significant differences between the set of balls both teams provided.
So far, we do not know how under-inflated the Patriots' balls were. We do not know any values whatsoever. We have hearsay that conflict each other: Heck, it was just reported by PFL that only one ball was under-inflated by 2 PSI, and the rest were under-inflated by close to 1 PSI!
In conclusion: Give all the news articles published this week to any unbiased scientist, and I'm pretty sure they would either laugh their ass off or be depressed about the state of affairs in the media today. Speaking of which:
Part II. Media coverage and reaction
Mike Saver had summarized all the faulty reports thus far on Thursday already, but let me briefly recap: We don't know how this investigation started. Despite initial reports, we know that it wasn't Jackson's interception that triggered it since Jackson himself denied it. That threw the "the CB realized the ball was under-inflated" argument away. As I have quoted before, the "2 PSI under-inflated" data that the media gave us could be mistaken. The only official statement so far has not included anything to clarify any lingering questions, yet that did not deter every single talking head on the media to have precise theories about what happened. The ESPN guys were quite sure that Brady lied, yet they did not suspect for one second that the reports leaked by anonymous sources could be the ones providing misinformation. People trusted "data", but they did not care about whether that data was sound.
How do we know so much when we actually know so little? That I just cannot understand.
Here's the thing: You might believe that the Patriots "cheated". You can also believe that they did not. No one can blame you for the way you feel. If anything, feelings are an important part of sports fandom. However, when you report on incomplete investigative results or hearsay and mix your feelings in it too, then you cross a line as a media member that you should never cross. Responsible reporting about this matter would have clarified at every step that the NFL has not officially stated anything. Responsible reporting would not have uttered words like "cheat" and "lie" as easily. Responsible reporting would have checked at least two sources before going with it.
***Spoiler alert: If you have not listened to all the episodes of Serial but you want to, skip this paragraph now.*** Do you believe Adnan Syed committed the crime he is accused of? Well, whether you say yes or no, that's your view, and I cannot argue with it. However, what I can and should argue with you is whether Adnan Syed should have been found guilty by the court or not. Because that is not a question about feelings, and the collective answer the society gives to that question affects all of us.
Have the Patriots cheated? I don't know. I don't have to believe in Belichick or Brady or any other report as of now. I choose to trust the investigation and the data yet to be released. If the NFL's investigation finds any wrongdoing, as a Patriot fan I will be the first in line to demand accountability from this organization. However, I will not form my opinions based on some pseudoscience. I don't have to think Belichick offered a sound rebuttal yesterday, but he offered a somewhat plausible explanation, supported by a scientific study. He adamantly denied any wrongdoing. Until I see precise, irrefutable evidence, I will take his word on it only because that is the sole "coherent" word I have heard so far. I believe in "innocent until proven guilty", I believe in science and experiments, and I know that the media has published a zillion reports that have contradicted each other so far. It is not the end of the story, but this story should be on pause until we see valid data, evidence or anything that resembles that. Enough with the rumors.
Photo credit: Stew Milne-USA TODAY Sports
Part I. Science
I'll first say this for the fellow Pats fans out there: Belichick did not "debunk" DeflateGate with his scientific explanations yesterday. He provided an explanation for what might have happened as their own study indicated. One thing that remained a bit vague was whether they have handed the referees under-inflated balls and told them to inflate them to/make sure they were at 12.5 PSI, or that the balls were at 12.5 PSI when they handed them but that was an artificially high value due to the ball preparation procedure of the Pats. Belichick, one can say rightfully, did not provide all the details about what that procedure is because teams are not required to do that for national media, but he made it clear that they will tell the NFL about it when/if asked.What we know, what Belichick reiterated, and what no one can deny is this: The weather affects the PSI values of footballs. That is a fact. In fact, a company named Head Smart Labs have filed a report about this issue. In their experiment, they have tried to recreate the conditions on the field, and their results indicate that the PSI values could have dropped 1.95.
However, also making rounds this morning was Bill Nye's "refutation" of Belichick's explanations. So, how did this guy, this great authority on scientific matters refute yesterday's presser?
Rubbing the football I don't think you can change the pressure. To really change the pressure you need one of these, the inflation needle. (...) GO SEAHAWKS!
Are you kidding me?!? "I don't think you can [do it]" is not a scientific explanation! Anybody who says "Oh, Bill Nye explained..." is committing a basic logical fallacy that is aptly named "appeal to authority." Head Smart Labs's analysis, whether flawed or not, is a valid attempt at being "scientific". Bill Nye's 5-second-long declaration is not. It is as simple as that. (Update: Boston Globe has talked to several scientists and they all agree that Belichick's explanation is technically possible. Does that un-refute what Belichick said? No, because neither "I think it could work" nor "I don't think that could work" are scientific statements.)
Now, let's talk about this other "gotcha" statement that is making rounds: "How come the Colts' balls were not affected?" There are a few problems with that:
1. The NFL has not officially confirmed yet that the Colts' balls were not affected. We have seen reports that they were within legal limits, but again we have seen a lot of reports since Monday that were later denied. (more on this later)
2. Even if that were true, it does not necessarily mean that the balls were not affected. If the Colts' balls were initially at 13.5 PSI and later dropped to 12.5 PSI, they would still be within the limits and be affected by the weather too.
3. Most importantly, the NFL has not provided the details as to how and when the balls were measured. Everything we know so far is based on data presented to us under the "DeflateGate" light.
At this point I will stop talking about every mistaken assumption I've heard so far, and try to answer the following, essential question: Do you want this investigation to be scientific, and the data to be unbiased? Well, this is what should have been done.
1. Both the Patriots' and the Colts' balls were measured in the same environment and their PSI values were recorded.
2. They were measured again at half time under the same circumstances: Same gauge, same room, same (or at least close) time of wait between the two measurements etc.
3. The change in PSI values were compared via a statistical test (T-test, for example) to see if there were significant differences between the set of balls both teams provided.
So far, we do not know how under-inflated the Patriots' balls were. We do not know any values whatsoever. We have hearsay that conflict each other: Heck, it was just reported by PFL that only one ball was under-inflated by 2 PSI, and the rest were under-inflated by close to 1 PSI!
In conclusion: Give all the news articles published this week to any unbiased scientist, and I'm pretty sure they would either laugh their ass off or be depressed about the state of affairs in the media today. Speaking of which:
Part II. Media coverage and reaction
Mike Saver had summarized all the faulty reports thus far on Thursday already, but let me briefly recap: We don't know how this investigation started. Despite initial reports, we know that it wasn't Jackson's interception that triggered it since Jackson himself denied it. That threw the "the CB realized the ball was under-inflated" argument away. As I have quoted before, the "2 PSI under-inflated" data that the media gave us could be mistaken. The only official statement so far has not included anything to clarify any lingering questions, yet that did not deter every single talking head on the media to have precise theories about what happened. The ESPN guys were quite sure that Brady lied, yet they did not suspect for one second that the reports leaked by anonymous sources could be the ones providing misinformation. People trusted "data", but they did not care about whether that data was sound.How do we know so much when we actually know so little? That I just cannot understand.
Here's the thing: You might believe that the Patriots "cheated". You can also believe that they did not. No one can blame you for the way you feel. If anything, feelings are an important part of sports fandom. However, when you report on incomplete investigative results or hearsay and mix your feelings in it too, then you cross a line as a media member that you should never cross. Responsible reporting about this matter would have clarified at every step that the NFL has not officially stated anything. Responsible reporting would not have uttered words like "cheat" and "lie" as easily. Responsible reporting would have checked at least two sources before going with it.
***Spoiler alert: If you have not listened to all the episodes of Serial but you want to, skip this paragraph now.*** Do you believe Adnan Syed committed the crime he is accused of? Well, whether you say yes or no, that's your view, and I cannot argue with it. However, what I can and should argue with you is whether Adnan Syed should have been found guilty by the court or not. Because that is not a question about feelings, and the collective answer the society gives to that question affects all of us.
Have the Patriots cheated? I don't know. I don't have to believe in Belichick or Brady or any other report as of now. I choose to trust the investigation and the data yet to be released. If the NFL's investigation finds any wrongdoing, as a Patriot fan I will be the first in line to demand accountability from this organization. However, I will not form my opinions based on some pseudoscience. I don't have to think Belichick offered a sound rebuttal yesterday, but he offered a somewhat plausible explanation, supported by a scientific study. He adamantly denied any wrongdoing. Until I see precise, irrefutable evidence, I will take his word on it only because that is the sole "coherent" word I have heard so far. I believe in "innocent until proven guilty", I believe in science and experiments, and I know that the media has published a zillion reports that have contradicted each other so far. It is not the end of the story, but this story should be on pause until we see valid data, evidence or anything that resembles that. Enough with the rumors.
Photo credit: Stew Milne-USA TODAY Sports
