Eat bigger animals. Save more welfare per calorie.| chris-said.io
This weekend the New York Times published an essay by Emma Goldberg titled, What if Charity Shouldn’t Be Optimized?. It argued that effective altruism (EA) – which uses cost/benefit analysis to fund easily quantified causes like malaria prevention and lead abatement – might detract from “squishier” causes like museums and community groups.| Chris Said
Have you ever felt a little hazy on what the Air Quality Index (AQI) actually means? I have, which is why I decided to deep dive on it. What I found was a bizarre Frankensteinian metric emerging from a web of federal judges, EPA employees, interest groups, and random members of the public.| Chris Said
When people first hear about the risk of AI running amok, they sometimes ask how so much damage could come from a piece of software that is trapped inside a computer.| Chris Said
Think of a view you hold politically. Bear with me, and ask yourself a few questions. How well do you really understand the issue? Have you thought through all of the second and third order consequences? Did you consider perverse incentives? What will happen when your plan is played out for multiple generations? Did you apply a discount factor? Do you know what the discount factor should be? Are there risks to your plan that you haven’t thought of? How well can you articulate the beliefs of...| Chris Said
[Code]| Chris Said
Disclaimer: I’m not an epidemiologist, so adjust your priors accordingly. Code available here.| Chris Said
Starting around 2012, teen loneliness and depression increased dramatically. A good explanation is social media, which became more ubiquitous and addictive around that time. Teens now spend an average of seven hours per day online, not including time spent doing homework. Social media is simply a massive intervention in teen culture, and no other development has so profoundly changed the way teens socialize. When you consider widespread observations of how social media can drive envy, anxiety...| Chris Said
What are some ways to reduce the risk of autism? According to published research, one of the most powerful interventions is for mothers to take folic acid around the time of conception. A recent meta-analysis of 10 studies found that folic acid supplementation reduced the odds of autism by about 42%. A large cohort study from Norway found an odds reduction of 49%, and some studies have even found reductions of more than 80%. These are massive effects.| Chris Said
Every once in a while, I try to better understand cross-entropy by skimming over some Medium posts and StackExchange answers. I always come away only half-understanding it.| Chris Said
Anybody looking at coronavirus data right now must feel very confused. The UK has a daily case count 60 times higher than Australia. Italy has a case fatality rate 3 times higher than nearby Greece and 12 times higher than Pakistan. These heterogeneities seem massive and have the potential to teach us critical insights about the disease. But as any epidemiologist will readilyacknowledge, these statistics are terribly confounded by inconsistent reporting protocols and variable testing ability,...| Chris Said
Attention mathematicians and computer scientists: I’ve got a problem for you, and I don’t know the solution.| Chris Said
This is Part III of a three part blog post on how to optimize your sample size in A/B testing. Make sure to read Part I and Part II if you haven’t already.| Chris Said
This is Part II of a three-part blog post on how to optimize your sample size in A/B testing. Make sure to read Part I if you haven’t already.| Chris Said
A special thanks to John McDonnell, who came up with the idea for this post. Thanks also to Marika Inhoff and Nelson Ray for comments on an earlier draft.| Chris Said
What do \(R^2\), laboratory error analysis, ensemble learning, meta-analysis, and financial portfolio risk all have in common? The answer is that they all depend on a fundamental principle of statistics that is not as widely known as it should be. Once this principle is understood, a lot of stuff starts to make more sense.| Chris Said
I recently came across a magazine that helps pastors manage the financial and operational challenges of church management. The magazine is called Church Executive.| Chris Said
Keyboard shortcuts are interesting. Even though I know they are almost always worth learning, I often find myself shying away from the uncomfortable task of actually learning them. But after years of clumsily reaching for the mouse while my colleagues looked at me with a kindly sense of pity, I have slowly accumulated enough keyboard tricks that I’d like to share them. This set is probably far from optimal, and different people have their own solutions, but it has worked well for me. Before...| Chris Said
I recently came across Artir’s Pyramid of Economic Insight and Virtue. It’s not actually a pyramid, but is instead a riff on the Expanding Brain meme. Check it out:| Chris Said
Here’s a data problem I encounter all the time. Let’s say I’m running a website where users can submit movie ratings on a continuous 1-10 scale. For the sake of argument, let’s say that the users who rate each movie are an unbiased random sample from the population of users. I’d like to compute the average rating for each movie so that I can create a ranked list of the best movies.| Chris Said
As a data scientist, a big part of my job involves picking metrics to optimize and thinking about how to do things as efficiently as possible. With these types of questions on my mind, I recently discovered a totally fascinating book about about economic problems in the USSR and the team of data-driven economists and computer scientists who wanted to solve them. The book is called Red Plenty. It’s actually written as a novel, weirdly, but it nevertheless presents an accurate economic histor...| Chris Said
One of the great developments in product design has been the adoption of A/B testing. Instead of just guessing what is best for your customers, you can offer a product variant to a subset of customers and measure how well it works. While undeniably useful, A/B testing is sometimes said to encourage too much “hill climbing”, an incremental and short-sighted style of product development that emphasizes easy and immediate wins.| Chris Said
How to make polished Jupyter presentations with optional code visibility.| chris-said.io
I often need to jump between different directories with very deep paths, like this:| Chris Said
As of yesterday I thought the debate about replication in psychology was converging on consensus in at least one respect. While there was still some disagreement about tone, basicallyeveryone agreed that there was value in failed replications. But then this morning, Jason Mitchell posted this essay, in which he describes his belief that failed replication attempts can contain errors and therefore “cannot contribute to a cumulative understanding of scientific phenomena”. It’s hard to kn...| Chris Said
I just finished an excellent fellowship at Insight Data Science. During our first few weeks there, each of us designed a website to demo at Insight’s sponsor companies. My website is called DealSpotter.| Chris Said
[Continuously updated. Last update December 14th, 2015]| Chris Said
Binocular rivalry is a visual illusion that occurs when the two eyes are presented with incompatible images. Instead of perceiving a mixture of the two images, most people experience alternations in which only one image is visible at a time. Binocular rivalry works best under controlled laboratory conditions with prisms or mirrors, but if you are lucky you might be able to experience it in the figure below. Try crossing your eyes to align the left boxes and right boxes, so that three boxes ar...| Chris Said
The US and Canada have very different systems for funding science. To compare them, I found some of the publicly available data on NIH R01s (USA) and NSERC Individual Discovery Grants (Canada), and plotted them below. Before describing the results, I should say that comparing NIH to NSERC is a bit like comparing apples to oranges, since NSERC is probably closer to the NSF than to the NIH. Nevertheless, the cross-country trends hold up across agencies, and in any case my goal is not to compare...| Chris Said
Let’s get one thing out of the way first: There is not a single scientist or science journalist who doesn’t know that correlation does not equal causation. Most have probably known it since high school.| Chris Said
Harvard, MIT, Stanford, and several other elite universities have all recently announced that they will be be offering free online courses. The courses will be massively open, taught by star professors, and supplemented with video lessons, embedded testing, and realtime feedback. This is surely good news for students who might not be able to access these resources otherwise, and it is an overall positive development for education. But what are the implications for scientists who conduct resea...| Chris Said
This is a continuously updated list of responses to questions I get about the main post and Top 8 List, which should be read first.| Chris Said
Bounty hunters can take care of what science undersupplies| chris-said.io
How the U.S. financial system is a model of integrity that science could learn from| chris-said.io
For a crime that could cost millions of lives, universities have failed to police themselves| chris-said.io
[Notebook]| Chris Said
In machine learning, double descent is a surprising phenomenon where increasing the number of model parameters causes test performance to get better, then worse, and then better again. It refutes the classical overfitting finding that if you have too many parameters in your model, your test error will always keep getting worse with more parameters. For a surprisingly widerange of models and datasets, you can just keep on adding more parameters after you’ve gotten over the hump, and performa...| Chris Said
Across the world, thousands of labs perform research on infectious pathogens. How risky is this research?| Chris Said
At this moment, a growing number of scientists are studying viruses that have not yet infected human populations, seeking to understand and publish information about how deadly they are. Some of these viruses are found in animal populations. Others are partially engineered by scientists, often specifically to make them more deadly or more transmissible. Scientists who conduct this type of research, which has been called pandemic virus prediction, say it could help us get ahead of the next pan...| Chris Said
Covid-19 vaccinations are a triumph of science. Every major public health official in America recommends them.| Chris Said
A special thanks to Scott Cunningham and Dean Eckles for helping me clear up my own confusion and inspiring me to write this blog post, and to Scott for his excellent book, which taught me a lot of background for this.| Chris Said
Instrumental Variables estimation is one of the most popular techniques in causal inference. But it can be a little hard to understand, especially when it is presented with terms mostly familiar to economists, such as “endogeneity” and “correlated with the error term”. In this blog post, I share some visualizations that gave me better intuitions about it. The post is designed for scientists and engineers who are familiar with causal inference, but who have a visual way of thinking tha...| Chris Said
Disclaimer: I’m not a medical professional, but I’ve been working closely with testing experts since July.| Chris Said
If you run a Github Pages blog that uses MathJax for formulas, you’ve probably had some problems recently. Most likely, you pushed some changes and now your math isn’t rendering correctly. The math looks fine locally, but it’s all messed up remotely. What happened?| Chris Said
UPDATE: After I posted this, I came across a recent preprint from Larremore et al. that makes this point much better than I do. The preprint emphasizes that antigen tests have high sensitivity when viral load is high, corresponding to when people are contagious. See rapidtests.org for more info.| Chris Said
A few months ago, I came across one of the most extraordinary papers I have ever read.| Chris Said
One of coolest things I ever learned about sensory physiology is how the auditory system is able to locate sounds. To determine whether sound is coming from the right or left, the brain uses inter-ear differences in amplitude and timing. As shown in the figure below, if the sound is louder in the right ear compared to the left ear, it’s probably coming from the right side. The smaller that difference is, the closer the sound is to the midline (i.e the vertical plane going from your front to...| Chris Said
When I was in grad school I occasionally overheard people talk about how humans do something called “hyperbolic discounting”. Apparently, hyperbolic discounting was considered irrational under standard economic theory.| Chris Said
This isn’t news to anybody, but the internet is changing our culture. Recently, I’ve been thinking about how it has changed our moral culture, and I realized that most of our beliefs on this topic are weirdly in tension with one another. Below are three questions that I feel are very much unresolved. I don’t have any good answers to them, and so I think they might be good topics for social science research.| Chris Said
Last week on Marginal Revolution, there was a link to a wonderful paper comparing the policy opinions of economic experts to those of the general public. The paper, by Paola Sapienza and Luigi Zingales, found some pretty significant discrepancies between the two groups. The authors attributed this difference to the degree of trust each group put in the implicit assumptions embedded into the economists’ answers. It’s an excellent and fascinating read. However, like pretty much all academic...| Chris Said
Welcome to the new location for The File Drawer! This blog is now hosted on Github Pages and powered by Jekyll. My old blog at filedrawer.wordpress.com will be shutting down soon.| Chris Said
Some videos on the internet are so good that I’ve watched them twice. Below is a list of 10 of my favorite interviews and dialogues. Obviously this isn’t an endorsement of all the positions taken. I just think they are very well done and fun to watch. The last four are best watched on 1.4x speed.| Chris Said
Everywhere you look, people are optimizing bad metrics. Sometimes people optimize metrics that aren’t in their self interest, like when startups focus entirely on signup counts while forgetting about retention rates. In other cases, people optimize metrics that serve their immediate short term interest but which are bad for social welfare, like when California corrections officers lobby for longer prison sentences.| Chris Said
Like most people who have analyzed data using frequentist statistics, I have often found myself staring at error bars and trying to guess whether my results are significant. When comparing two independent sample means, this practice is confusing and difficult. The conventions that we use for testing differences between sample means are not aligned with the conventions we use for plotting error bars. As a result, it’s fair to say that there’s a lot of confusion about this issue.| Chris Said
Yesterday I posted a very unscientific survey asking researchers to describe how failed replications changed their subjective estimates of effect sizes. The main survey asked for “ballpark estimates” of effect sizes, but an alternative interactive version allowed researchers to also report their uncertainty by specifying both the mean and variance of their posterior distributions. Thanks to everyone who participated. I won’t be analyzing any new data after this, but it’s never too la...| Chris Said
Next week I’m going to start a new job as a data scientist at Twitter and I am thrilled. Aside from Google search, no other website has had a more positive impact on my life than Twitter. Twitter is just so much fun, and I have learned so much from it. | Chris Said
If I had a chance to remake the high school STEM curriculum to reflect the skills that are actually needed in today’s world, my changes might look something like this:| Chris Said
National unemployment is high, but business is booming in some states. Vermont needs teachers. Nevada needs bartenders. North Dakota needs truck drivers and just about everything else.| Chris Said
There is a reproducibility crisis in psychology.| Chris Said
If there is one thing I have learned recently, it is that autism is a really, really complicated disorder. Autism is best known for causing repetitive behaviors and problems with social communication, but it is also known to cause issues in sensory perception. Many hypotheses for the underlying neurophysiological basis have been proposed. Among these is the excitation/inhibition (E/I) imbalance hypothesis, which states that levels of cortical excitation and inhibition are disrupted in autism....| Chris Said
Contrast response functions (CRFs) describe how a neuron’s firing rate depends on the contrast, or intensity, of a visual stimulus. CRFs are really important for testing theories about how the visual system works, and I’ve spent a lot of time over the past few years trying to indirectly measure them in humans, using EEG and fMRI. The problem, however, is that EEG and fMRI give me very different CRFs.| Chris Said
It’s the incentive structure, people! Why science reform must come from the granting agencies.| chris-said.io
How a small correlation with social media can drive a 50% increase in teen depression| chris-said.io
Infectiousness is mostly linear with viral count and slightly sublinear with observed RNA count| chris-said.io