Use of e-cigarettes in two different groups leads to….headlines

Unsurprisingly there has been yet another study published making a big song and dance about the “gateway” theory, claiming that “teens that use e-cigarettes are three times more likely to smoke” (taken from a headline), or that teens that use e-cigarettes are more likely to smoke.

A quick Google news search for e-cigarettes gives you 92 articles, each with a variation of the same headline and all citing the same study, or to be more precise the same press release. Some of the journalists did seek comments from Cancer Research UK, but not it seems Professor Linda Bauld who has already criticised the study here, and Professor Kevin Fenton from Public Health England.

The hashtag #ecigarettes on Twitter is also full of the same articles. Disappointing, but not entirely unexpected.

So what did this study set out to find?

Objective:

Use of electronic cigarettes (e-cigarettes) is prevalent among adolescents, but there is little knowledge about the consequences of their use. We examined, longitudinally, how e-cigarette use among adolescents is related to subsequent smoking behaviour.

It is known, thanks to various surveys globally that the use of e-cigarettes has risen among teenagers, so the researchers decided to select a representative sample, follow that sample for the duration of the study (the very definition of longitudinal) to see whether any of those using e-cigarettes move on to tobacco.

It is of course known from surveys that ask the right questions that the prevalence of e-cigarette use among youth is  predominantly confined to those who are (or were) tobacco users, with a minimal proportion (less than 1%) having never smoked. There are also very few signs, and very little robust evidence supporting the theory that e-cigarette use leads to tobacco use.

Methods:

Longitudinal school-based survey with a baseline sample of 2338 students (9th and 10th graders, mean age 14.7 years) in Hawaii surveyed in 2013 (time 1, T1) and followed up 1 year later (time 2, T2). We assessed e-cigarette use, tobacco cigarette use, and psychosocial covariates (demographics, parental support and monitoring, and sensation seeking and rebelliousness). Regression analyses including the covariates tested whether e-cigarette use was related to the onset of smoking among youth who had never smoked cigarettes, and to change in smoking frequency among youth who had previously smoked cigarettes.

Selecting 2,338 students in Hawaii in 2013, with a follow-up one year later (2014); which does beg the question, if this study began in 2013, and the initial completion was in 2014 why has it taken so long to release this paper?

Either way, at both intervals the researchers quizzed the participants on the use of e-cigarettes and tobacco, they asked a few questions related to other societal factors (such as “do your parents smoke”), attempted to determine if said teen was “rebellious” (really?). They also used statistics to investigate the “relationships” between one or more factors (regression analysis) – in this case whether the use of an e-cigarette was related in any way to the onset of smoking tobacco among “never smokers” and whether or not the use of an e-cigarette affected the frequency of youth who did smoke.

Seems to be a logical idea, grab a bunch of data , crunch some numbers and try to determine if cause equals effect. But there’s a few problems.

At “T1” the total number of participants was 2338, but at “T2” it was 2339. Why the difference?

Sadly, the reason for that is buried with the section “Analysis Methods” spanning pages 2 and 3 of the study:

We conducted parallel analyses for persons having complete data at T1 and T2 (complete-case analysis; N=1302), and also including those who did not (full-information analysis; N=2772)

So, complete-case analysis (completing the questionnaire at both intervals T1 and T2 only totalled 1302. So where did the other 1036 from T1 disappear to?

Well according to the study, those that disappeared were more likely to anyway because statistically those with “higher rebelliousness” or “lower parental support scores” were not going to complete the study at T2. But they did add 434 results from T2 only, and of course using computational models and statistics predicted the results of the missing 1036.

So a large proportion of the baseline (T1) participants did not complete T2. The major flaw here is that by adding an entirely new set of participants (which of course begs the question why were these additional 434 not in T1 in the first place) the researchers didn’t exactly follow the same sample throughout the course of the study.

Other flaws include, multiple testing – a lot of parameters were compared, each looking for a statistically significant result, which on such a fractured sample would only inflate the likelihood of finding a result when there isn’t one.

The authors note that the measure of e-cigarette “use” is relatively simple by defining “use” as used “once or twice” and they also state:

As in most longitudinal studies, there was attrition from the baseline sample

A substantial portion of the T1 participants were not present or included in the T2 interval (~44% missing) but

we demonstrated that findings were similar for complete-case and full-information analyses that included variables related to attrition.

By computational models to predict the likely answers from the missing T2 data and including an additional 434 sets of answers from T2 only. In effect, this study isn’t essentially a longitudinal study it’s more along the lines of a snapshot (cross-sectional) study with only 1302 participants common to both data sets.

The authors also note that there were errors in the data itself, specifically in classification of “smoking reports”, and as noted:

the rate of cigarette smoking was relatively low in this population, and the majority of T1 e-cigarette ever-users did not transition to cigarette smoking over a 1-year period.

So, the sample had a “relatively low” smoking prevalence and that the majority of e-cigarette users present at T1 did not transition to tobacco, yet by adding other parameters such as age, ethnic origin and rebelliousness the authors could “predict” that e-cigarette users would indeed transition.

It is worth noting that Hawaii also has a higher than average e-cigarette prevalence rate among youth (~31%) which is probably why the sample for this study was taken from there.

What the paper adds is insignificant as it cannot truly be classed as longitudinal in nature, had they reported only on the 1302 participants that remained, the figures would have been substantially different – around 8% for e-cigarettes and 4% for tobacco at both time points.