This is the story of how I found what I believe to be scientific misconduct and what happened when I reported it. Science is supposed to b...| crystalprisonzone.blogspot.com
One of my new favorite games is the lyrically-named Potato Flowers in Full Bloom. Let me tell you why I love this game so much and why I hope you will also play it.| Crystal Prison Zone
A few weeks ago, I started a new job as a data scientist at the logistics division of a pharmaceutical company. This was primarily motivated by family reasons. My wife works at an art museum in Chicago and could not find satisfactory work in my small college town. We'd been living apart while I struggled to find a new professorship that might put us in a city she'd like. The academic job market is lousy enough that I was lucky to have a job anywhere -- I was not the kind of research superstar...| Crystal Prison Zone
Killing time in the UChicago stacks in the summer of 2019, I found a book from 1995 called Fraud and Erroneous Judgment in the Social Sciences. It's been an interesting read, because despite having been written nearly 25 years ago, much of it reads like it was written today. Specifically, there is very little substance about actually preventing, detecting, or prosecuting fraud, presumably because all these things are very difficult to do. | Crystal Prison Zone
Nick Brown asks:| Crystal Prison Zone
It's been a rich week of readings for wondering just what the hell we're doing. Loyka et al. (2019) present a framework for considering external validity, and this framework reminds us just how poorly we are doing at considering actual real-world human behavior. Tal Yarkoni has a preprint up that describes how implausible it is that the situations and stimuli we study will generalize to other situations and stimuli. Danielle Navarro has clarified her stance on preregistration by elaborating o...| Crystal Prison Zone
Recent research by Chang & Bushman (2019) reports how video games may cause children to be more likely to play with a real handgun. In this experiment, children participate in the study in pairs. They play one of three versions of Minecraft for 20 minutes. One version has no violence (control), another has monsters that they fight with swords (sword violence), and another has monsters that they fight with guns (gun violence). | Crystal Prison Zone
I’m approaching the end of my first semester teaching Intro to Social Psychology. As someone who came of age during the peak of the replication crisis (Bem, Stapel, Reproducibility Project), studies publication bias, and has had a hard time finding statistically significant results, I generally have a dim view of big chunks of the literature. I was worried that we would have very little to talk about given all the uncertainty, but we’ve made a good semester of it by talking about the gene...| Crystal Prison Zone
The prediction market is a way to try to assign probabilities to events. Bettors buy YES bets on things they think are likely to happen (relative to the market price) and NO bets on things they think are unlikely to happen (relative to the market price). Market dynamics lead the market price to settle on what is, across the bettors, the best subjective probability of the event. This is useful if you are trying to assign probabilities to one-off future events.| Crystal Prison Zone
I'm a new assistant professor trying to set up my research laboratory. I thought I'd try making the jump to PsychoPy as a way to make my materials more shareable, since not everybody will have a $750+ E-Prime or DirectRT license or whatever. (I'm also a tightwad.)| Crystal Prison Zone
At long last, our article "Overstated Evidence for Short-Term Effects of Violent Games on Affect and Behavior: A Reanalysis of Anderson et al. (2010)" is released from its embargo at Psychological Bulletin. (Paywalled version here.)| Crystal Prison Zone
The last couple years have seen an exciting explosion in new techniques for publication bias. If you're on the cutting edge of meta-analysis, you now can choose between p-curve, p-uniform, PET, PEESE, PET-PEESE, Top-10, and selection-weight models. If you're not on the cutting edge, you're probably just running trim-and-fill and calling it a day.| Crystal Prison Zone
The reliability of scientific knowledge can be threatened by a number of bad behaviors. The problems of p-hacking and publication bias are now well understood, but there is a third problem that has received relatively little attention. This third problem currently cannot be detected through any statistical test, and its effects on theory may be stronger than that of p-hacking.| Crystal Prison Zone
In DataColada [58], Simonsohn argues that funnel plots are not useful. The argument is, for true effect size δ and sample size n:| Crystal Prison Zone
It is a common goal of meta-analysis to provide not only an| Crystal Prison Zone
A few months ago, I had the opportunity to attend a symposium on research integrity. The timing was interesting because, on the same day, Retraction Watch ran a story on two retractions in my research area, the effects of violent media. Although one of these retractions had been quite swift, the other retraction had been three years in coming, which was a major source of heartache and frustration among all parties involved.| Crystal Prison Zone
Growing up, I played a lot of role-playing games for the Super Nintendo. One trope of late-game design for role-playing games are rare drops -- highly desirable items that have a low probability of appearing after a battle. These items are generally included as a way to let players kill an awful lot of time as they roll the dice again and again trying to get the desired item.| Crystal Prison Zone
Some months ago, a paper argued for the validity of an unusual measurement of aggression. According to this paper, the number of pins a participant sticks into a paper voodoo doll representing their child seems to be a valid proxy for aggressive parenting.| Crystal Prison Zone
Yesterday, Perspectives on Psychological Science published a 17-laboratory Registered Replication Report, totaling nearly 1900 subjects. In this RRR, researchers replicated an influential study of the Facial Feedback Effect, showing that being surreptitiously made to smile or to pout could influence emotional reactions.| Crystal Prison Zone
Fail-Safe N is a statistic suggested as a way to address publication bias in meta-analysis. Fail-Safe N describes the robustness of a significant result by calculating how many studies with effect size zero could be added to the meta-analysis before the result lost statistical significance. The original formulation is provided by Rosenthal (1979), with modifications proposed by Orwin (1983) and Rosenberg (2005).| Crystal Prison Zone
Inspired by a recent excellent lecture by Nick Brown, I decided to finally sit down and read Diederik Stapel's confessional autobiography, Ontsporing. Brown translated it from Dutch into English; it is available for free here.| Crystal Prison Zone
Brent Roberts suggests the replication movement solicit federal funding for the organization of federally-funded replication daisy chains. James Coyne suggests that the replication movement has already made a grave misstep by attempting to replicate findings that were always hopelessly preposterous. Who is in the right?| Crystal Prison Zone
Everyone seems to agree with the saying "extraordinary claims require extraordinary evidence." But what exactly do we mean by it?| Crystal Prison Zone
Last post, I talked about the benefits a manuscript enjoys in the process of scientific publication. To me, it seems that the main benefits are that an editor and some number of peer reviewers read it and give edits. Somehow despite this part coming from volunteer labor, it still manages to cost $1500 an article.| Crystal Prison Zone