Readers may be aware of a study published earlier this year which seemed to suggest a link between Internet use and depression. “The Relationship Between Excessive Internet Use and Depression” was published by researchers at the University of Leeds. A number of news outlets picked the story up, with a variety of sensational headlines. Given that news outlets have been publishing scare articles for years about how the Internet makes everyone isolated and depressed, I’m always curious to delve a little bit more deeply into the studies cited to see if they actually came to the conclusions reported in the media.
Science reporting in the mainstream media can be pretty dubious. Journalists exaggerate conclusions or massage the numbers so that they can have a more extreme headline or so that they can make stronger claims in their articles. And few articles examine or question the methodology of the studies they report on. I don’t think this is because journalists reporting on science don’t know how to question methodology. I think it’s because they think that the general public is not interested or isn’t capable of following.
I think that’s an erroneous assumption. Depriving members of the public of deeper analysis on the assumption that they aren’t interested or can’t follow is just insulting. And since this study is being bandied about a lot, I thought it might be worth discussing it here. Please note that I am not a scientist, so I’m not really qualified to critique scientific methodologies in-depth, at all; other contribs and readers are scientists and may have additional thoughts which I would love to read in comments.
The first question to ask when looking at a study is how the sample was selected. In this case, it was a convenience sample, which makes the conclusions of the study less firm. The study was based on an online questionnaire, which 1,319 people responded to. However, respondents were sought out with advertisements on social networking sites based in the United Kingdom. That’s immediately self-selecting and limiting; respondents were limited not to the entire pool of people using the Internet, but to English speaking people utilizing social networking sites in the United Kingdom who clicked on ads. And, of course, people who don’t use the Internet could not be used as a basis for comparison in the study because they weren’t recruited.
Over at , Tal Yarkoni points out that using this sample methodology is “…the equivalent of trying to establish cell phone usage patterns by randomly dialing only land-line numbers” in his critique “internet uses causes depression! or not.” (By the way, I would highly recommend reading his critique of the study, which gets into much more detail about the methodology and includes things like numbers! And citations!)
Participants were asked to fill out three separate surveys: the Internet Function Questionnaire, which is designed to assess the proportion of time online spent doing various activities; the Internet Addiction Test, used to assess whether or not someone might be classified with Internet addiction; and the Beck Depression Inventory, a popular metric for depression assessment. Using the responses, the researchers found that there appeared to be a correlation between Internet addiction and depression.
But does that correlation really hold up?
There are some serious problems with the framing of the questionnaires used in this study. For example, people are asked to respond to questions like “do you think you spend too much time online,” but the researchers did not directly ask users to report on how much time they spend online. “Too much time” for one person might be five hours a week, for someone else it might be 72 hours a week. Self reporting in general is also flawed because respondents may be primed to respond in one way or another. If self reporting cannot be backed up with additional data, it can be a bit dodgy. How do we decide when Internet use becomes “excessive,” especially since that probably varies from person to person?
And let’s examine the questions used in the Internet Addiction Test a bit more closely, because some of them might surprise you.
“How often do you form new relationships with fellow online users?” Hi, FWD readers! This site exists solely because of the formation of new relationships with online users; all of the FWD contributors met online. Indeed, FWD is structured around the formation of new relationships with online users, as is a lot of the blogosphere. Especially for people with disabilities, the Internet can actually be a really valuable source of interpersonal connections. Several FWD contributors rarely leave their homes and use the Internet as a primary resource for socialisation, and it would not surprise me at all to learn that some FWD readers fall into the same category. For me, for example, forming new relationships with fellow online users is a source of enrichment.
“How often do you fear that life without the Internet would be boring, empty, and joyless?” That’s a pretty loaded question. Empty and joyless? Not so much, for me, but perhaps for some. Boring? Yes. Although I can’t say that I “fear” this, it’s more something that I note. As I discussed above, for people with disabilities, the Internet provides opportunities for socialisation which would not be available otherwise. Both of these questions seem to assume that the only people who use the Internet are able and that there’s no way that Internet relationships and communities could replace real life ones. Or that for people who do the majority of their socialisation online, their socialisation somehow is not real, or is lesser than real life socialisation.
“How often do you feel preoccupied with the Internet when off-line, or fantasize about being on-line?” Fairly frequently, given that I have the keys to several servers and I fret about them when I’m not around. Which is fairly understandable, just like it’s understandable for people to worry about the welfare of things they are responsible for in the real world. Like other questions in the test, this privileges outside world concerns over online ones, and seems to ignore the role that the Internet can play in some lives.
I’m not going to go through all of the questions; I just wanted to give you a small sampling. Some of these questions are rather leading, and they don’t fully consider the fact that people communicate and socialise in different ways. I am apparently “experiencing occasional or frequent problems because of the Internet [and] should consider their full impact on [my] life,” according to my responses.
Correlation is not causation, as I frequently mutter to myself when reading or listening to science reporting. When you get down to it, the sample size was too small and too flawed to be meaningfully applied to the population, and it ignored an important question: If there is a correlation between Internet use and depression in the general population, is it because the Internet depresses people? Or is it because people with depression are more likely to use the Internet? And if that’s the case, is that a bad thing? I don’t think so.