SciELO - Scientific Electronic Library Online

 
vol.31 número1Un programa personalizado de parentalidad a distancia mediante feedback por vídeo: un ensayo controlado aleatorizado índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados

Revista

Articulo

Indicadores

Links relacionados

  • En proceso de indezaciónCitado por Google
  • No hay articulos similaresSimilares en SciELO
  • En proceso de indezaciónSimilares en Google

Compartir


Psychosocial Intervention

versión On-line ISSN 2173-4712versión impresa ISSN 1132-0559

Psychosocial Intervention vol.31 no.1 Madrid ene. 2022  Epub 17-Ene-2022

https://dx.doi.org/10.5093/pi2021a8 

Articles

A decade in review: a systematic review of virtual reality interventions for emotional disorders

Revisión de una década: una revisión sistemática de las intervenciones mediante realidad virtual en trastornos emocionales

Dale P Rowland1  2  , Leanne M Casey1  2  , Aarthi Ganapathy1  2  , Mandy Cassimatis1  2  , Bonnie A Clough1  2 

1Griffith University, Queensland, Australia

2Menzies Health Institute Queensland, Australia

ABSTRACT

Research is increasingly demonstrating the therapeutic benefits of virtual reality interventions for various mental health conditions, though these rarely translate from research to application in clinical settings. This systematic review aims to examine the efficacy of current virtual reality interventions for emotional disorders, with a focus on clinical and technological features that influence translation of treatments from research to clinical practice. A comprehensive systematic literature search was conducted following PRISMA guidelines, for studies including the application of a virtual reality intervention to a clinical population of adults with an emotional disorder. Thirty-seven eligible studies were identified, appraised, and assessed for bias. Treatment effects were typically large across studies, with virtual reality being considered an efficacious treatment modality for various anxiety disorders and post-traumatic stress disorder. Virtual reality interventions were typically used for delivering exposure in cognitive behavioural therapy approaches. Considerable variability was seen in cost, technological specifications, degree of therapist involvement, delivery format, dosage, duration, and frequency of treatment. Suboptimal methodological rigour was identified in some studies. Remote use of virtual reality was rare, despite increasing options for in home use. Virtual reality interventions have the potential to overcome barriers to care and better meet the needs of consumers. Future research should examine the efficacy of virtual reality for treatment of depressive disorders and obsesive compulsive disorder. Improved methodological reporting and development of transdiagnostic and remotely delivered virtual reality interventions, will likely increase the translation of this treatment modality.

Keywords Digital mental health; Emotional disorders; Virtual reality; Transdiagnostic treatment; Remote interventions

RESUMEN

Cada vez más la investigación demuestra las ventajas terapéuticas de las intervenciones mediante realidad virtual en distintos estados de salud mental, aunque esta investigación raramente se traslada a la aplicación en el contexto clínico. Esta revisión sistemática pretende analizar la eficacia de las intervenciones actuales de realidad virtual en trastornos emocionales, centrándose en las características clínicas y tecnológicas que afectan a la transferencia de los tratamientos desde la investigación hasta la práctica clínica. Se llevó a cabo una amplia búsqueda bibliográfica sistemática de acuerdo con las directrices PRISMA para estudios que abarcan la aplicación de la intervención mediante realidad virtual a población clínica adulta con trastornos emocionales. Se consideraron elegibles 37 estudios, que fueron valorados y revisados para descartar sesgos. Los efectos de los tratamientos eran normalmente grandes, siendo considerada la realidad virtual como una modalidad de tratamiento eficaz para diversos trastornos de ansiedad y el de estrés postraumático. Las intervenciones de realidad virtual normalmente se han utilizado para la exposición en los enfoques de terapia cognitivo-conductual. Se observó una considerable variabilidad en coste, especificaciones tecnológicas, grado de implicación del terapeuta, formato de presentación, dosificación, duración y frecuencia del tratamiento. En algunos estudios se observó que el rigor metodológico estaba por debajo de los niveles óptimos. La utilización remota de realidad virtual no era frecuente a pesar de las posibilidades que ofrece para usarse en casa. Las intervenciones de realidad virtual tienen el potencial de superar barreras en los cuidados y cubrir mejor las necesidades de los consumidores. La investigación futura debería analizar la eficacia de la realidad virtual para tratar los trastornos depresivos y el desorden obsesivo compulsivo. Mejorar los informes metodológicos y el desarrollo de las intervenciones mediante realidad virtual transdiagnósticas y practicadas a distancia podría facilitar la transferencia de esta modalidad de tratamiento.

Palabras clave Salud mental digital; Trastornos emocionales; Realidad virtual; Tratamiento transdiagnóstico; Intervenciones a distancia

Introduction

Virtual reality (VR) technology delivers sensory stimuli to generate a perceptual and interactive experience of a realistic and immersive environment (Rizzo et al., 2019). VR is defined as a computer generated, three-dimensional environment that is viewed using dual-display stereoscopic technology such as head mounted displays (HMDs) (Clough & Casey, 2011; Riva et al., 2016; Turner & Casey, 2014). The past decade has seen unprecedented industrial developments and advancements in the technological specifications of VR. Improvements in computing power, graphics, advanced processing speed, and greater internet connectivity has resulted in increased access and consumer demand for VR (Rizzo et al., 2019). VR is now widely and easily available in the home, resulting in greater potential reach and accessibility for VR interventions (VRIs) for mental health. However, despite earlier predictions that VR would revolutionise mental health care (Newman et al., 2011), many VRIs are yet to reach their full potential (Botella et al., 2017; Serrano et al., 2019).

Research is increasingly demonstrating the therapeutic benefits of VRIs for various mental health conditions, though interventions rarely translate from research to application in clinical settings (Cipresso et al., 2018; Mishkind et al., 2017). Establishing efficacy does not guarantee uptake and routine use, with only 50% of clinical innovations translating to routine care (Bauer & Kirchner, 2019; Kirchner et al., 2020; Wiederhold & Riva, 2019). VRIs offer great promise for overcoming barriers to care, reducing unmet client need, enhancing existing preventative approaches, and improving service delivery (Clough & Casey, 2011; Clough, Eigeland, et al., 2019). A lack of commercially available VRIs highlights the need for a better understanding of the clinical features associated with translation and uptake. The current review provides a synthesis of research in this field, particularly regarding clinical features of VRIs for emotional disorders (EDs).

Virtual Reality Interventions for Mental Health

VRIs apply VR technology to promote psychological and/or behavioural change to achieve symptom reduction in clinical, subclinical, and non-clinical populations (Turner & Casey, 2014). VRIs can create environments suitable for training and treatment in both controlled and multi-sensory environments where dynamic clinical presentations can be observed and recorded (Rizzo et al., 2019). Like many digital interventions for mental health, VRIs can be used for promotion, case management, coaching, symptom-focused treatment, and comprehensive therapy (Reynolds et al., 2015). VRIs vary across theoretical orientation (e.g., cognitive behavioural therapy, dialectical behaviour therapy etc.), dosage (e.g., number of sessions), duration (e.g., time), and degree of therapist involvement (e.g., completely involved to no involvement). Some VRIs are deployed in traditional face-to-face therapy as an adjunct to existing treatment while others may be designed to be self-guided or therapist assisted. Yet despite this variability across VRIs, one commonality can be seen in the frequent application of VRIs for disorders with strong cue-exposure components such as anxiety disorders (Mishkind et al., 2017).

Features of Virtual Reality Interventions

Key features of VR include immersion, presence, and simulator sickness. Immersion refers to the stimulation of senses within a virtual environment that mimics interactions and experiences in reality (Cipresso et al., 2018). Immersion is the extent to which experiences in virtual environments achieve realism. The more extensive, surrounding, inclusive, vivid, and matching reality the virtual environment, the higher the degree of immersion (Slater et al., 1995). In this way, immersion can be considered as the objective level of sensory fidelity that a VR system can provide through its virtual environment (Bowman & McMahan, 2007). Presence is a complex state of feeling that is elicited by a virtual environment, and is described as a state of feeling that is ‘life-like' or ‘being there' (Cipresso et al., 2018). Previous research has attempted to examine the relationship between presence and cybersickness (Weech et al., 2019). Cybersickness is an adverse physiological response to a virtual environment that results in bodily discomfort (i.e., nausea, vertigo, dizziness, etc.) (Weech et al., 2019). Presence and cybersickness are negatively related in that virtual environments that are more realistic tend to elicit a greater sense of presence and immersion, which corresponds with fewer reports of cybersickness (Weech et al., 2019). Intrusive factors that decrease presence and increase the likelihood of cybersickness include sensory mismatch, VR display factors (i.e., less than 20 frames per second, low field-of-view, etc.), vection (i.e., the illusion of self-motion in relation to the brain's sensorimotor control system), intuitiveness of interactivity with the virtual environment, navigation control, and gaming experience. Immersion, presence, and cybersickness appear to be important components that influence engagement (Ling et al., 2014; Weech et al., 2019), but how these influence various clinical outcomes remain poorly understood.

Efficacy of Virtual Reality Interventions

The capacity to create tailored, stepped, and safe environments through VR has resulted in substantial evidence focussing on the use of this technology for the treatment of anxiety, phobias, and PTSD (Arroll et al., 2017; Botella et al., 2017; Carl et al., 2019; Fodor et al., 2018; Goncalves et al., 2012; Valmaggia et al., 2016). A number of comprehensive reviews (Clough & Casey, 2011; Clough & Casey, 2015b), systematic reviews (Botella et al., 2017; Goncalves et al., 2012; Valmaggia et al., 2016), and meta-analyses (Carl et al., 2019; Fodor et al., 2018) have demonstrated the clinical efficacy of VRIs for many psychopathologies (Carl et al., 2019; Valmaggia et al., 2016; Wiederhold & Riva, 2019). However, these have mainly been for the treatment of unidimensional disorders (e.g., anxiety disorders), as opposed to related disorders of emotion (Arroll et al., 2017; Botella et al., 2017; Carl et al., 2019; Fodor et al., 2018). The potential of VRIs to be utilised in the treatment of emotional disorders (e.g., anxiety, anxiety related, and unipolar mood disorders) remains a sparsely researched area.

Novel Virtual Reality Interventions

Single-disorder treatment protocols often lack external validity, do not provide guidance on how to address co-occurring disorders, and require clinicians to demonstrate expertise in administering various manual-based protocols (Cassiello-Robbins et al., 2020). Transdiagnostic treatment protocols are increasingly being adopted in clinical research and have translated into clinical practice (Cassiello-Robbins et al., 2020), but whether this has also occurred in VRIs is unknown. VRIs have the capacity to target core processes that are relevant across a variety of clinical domains (Rizzo et al., 2019). VRIs can be tailored to higher order dimensions of similar disorders, making it an efficacious modality that may achieve improvements in translation efforts. Indeed, in the broader digital mental health field, it has been identified that translation and long-term consumer use of mental health technologies, such as mobile applications, may only be achieved if a transdiagnostic approach is utilised (Chandrashekar, 2018).

Novel elements such as transdiagnostic VRIs, remote, or self-guided VRIs are some ways in which this modality of treatment can be optimised to achieve greater translation from research to clinical practice. Transdiagnostic VRIs could meet clients' needs in co-morbid populations, whilst remote or self-guided VRIs can overcome access barriers and increase the capacity of this technology to be used outside of the laboratory. Indeed, the technological advancements observed in untethered HMDs, such as the Oculus Quest, make this a particularly important time to review the evidence base and clinical features of VRIs for EDs.

Emotional Disorders

Consistent with previous research (Barlow et al., 2004; Brown & Barlow, 2009; Bullis et al., 2019), EDs are defined as a psychological disorder that is associated with the experience of frequent and intense negative emotions, elicits an aversive reaction to the negative emotional experience that is driven by the individual's diminished sense of control and negative appraisal of the emotion, and results in the individual avoiding or lessening their emotional experience in reaction to the onset of a negative emotion state (Bullis et al., 2019; Cassiello-Robbins et al., 2020). It has been argued that such disorders share common aetiology and maintaining mechanisms, and as such may be treated via the same approaches (Barlow, 2004). EDs included in the current review were: anxiety disorders, such as generalised anxiety, panic disorder, social anxiety; specific phobias; agoraphobia; obsessive-compulsive disorders; unipolar depressive disorders; mood disorders; and PTSD. Emerging research suggests the inclusion of related disorders in the classification of EDs (i.e., insomnia, eating disorders, and borderline personality disorders) (Bullis et al., 2019); however, more research examining the relationships between higher order dimensions of these related disorders is needed. Therefore, insomnia, eating disorders, and associated disorders of emotion are not included in this review. Within the face-to-face psychological literature, considerable focus has been given in recent years to understanding treatment response across the emotional disorders, with two recent systematic reviews focussing on these issues (Gonzalez-Robles et al., 2018; Sakiris & Berle, 2019). However, the extent to which VR may be a useful treatment approach across the emotional disorders remains unknown.

Study Aim

The current review set out to provide an updated synthesis of research within this field. To date, systematic reviews and meta-analytic studies have focussed on the efficacy of VRIs for unidimensional classifications of disorders as opposed to broader disorders of emotion (Botella et al., 2017; Carl et al., 2019; Fodor et al., 2018). In addition, reviews to date have primarily focussed on the efficacy of VRIs, with only limited attention given to factors related to translation. As such, the primary aim of this systematic review was to concurrently examine the efficacy of VRIs for EDs and investigate select clinical and technological features (such as dosage, degree of therapist guidance, treatment approach, VR specifications) that may influence translation of treatments from research to clinical practice. It is anticipated that examining these issues concurrently will provide a discussion for improving uptake and implementation of efficacious VRIs for EDs.

Method

The systematic review was conducted and reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA; Moher et al., 2010) guidelines. The protocol for the review was pre-registered with PROSPERO (CRD42019135795).

Literature Search and Search Strategy

Studies were sourced by conducting a comprehensive systematic literature search in the electronic databases of Medline, PsycINFO, Embase, and Cumulative Index of Nursing and Allied Health Literature (CINAHL). The search strategy consisted of the following search terms:

  1. “virtual reality” OR “virtual” OR “VR” OR “augmented reality” OR “AR” OR “altered reality”

  2. “anxiety disorder*” OR “depressive disorder*” OR “mood disorder*” OR “generalised anxiety disorder*” OR “social anxiety disorder*” OR “panic disorder*” OR “obsessive compulsive disorder*” OR “OCD” OR “post-traumatic stress disorder*” OR “PTSD” OR “major depressive disorder*”

  3. Exploded and mesh terms for (i)

  4. Exploded and mesh terms for “anxiety disorder*” AND “depressive disorder*” AND “mood disorder*”

  5. i OR iii

  6. ii OR iv

  7. v AND vi

The peer review of electronic search strategies guidelines and checklist for systematic reviews (McGowan et al., 2016) was applied to the search strategy, and appraised by independent reviewers before conducting the final search.

Inclusion and Exclusion Criteria

A comprehensive search was conducted to obtain articles published from 1989 to 2019. Experimental studies were deemed eligible for inclusion based on the following criteria: a treatment study that included participants over the age of 18, with a primary diagnosis of an ED, that was confirmed using validated self-report or clinician rated measure/s or by way of diagnostic interviewing. Full text studies needed to be available in English and published in the most recent decade (2010-2019). Publication date was kept as the last step in the exclusion of studies so that the authors could better understand advancements in VR technology, contextualise historical trends of research within this field, and source studies for preliminary checks in consistency of raters (see Risk of Bias and Quality Assessment section below).

Studies were excluded if they did not adequately describe or include a virtual reality intervention (e.g., virtual experiences through computer displays, projectors such as cave automatic virtual environment) that utilised an HMD (e.g., VR headset, 3D glasses, goggles). Further exclusions were made if studies failed to provide enough information about the design, intervention, VR apparatus, or clinical characteristics of participants (Figure 1). Non-experimental study designs, studies lacking empirical data, single case studies, and study protocols were excluded as they did not meet the criteria for evaluation and quality assessment. Studies containing psychopharmacological agents or disorders beyond those of emotion, or presence of comorbidity with a disorder not considered an emotional disorder, were also omitted (e.g., psychotic disorders, neurodevelopmental disorders) as treatment effects could not be examined independently.

Screening, Study Selection and Data Extraction

The process of study selection is displayed in Figure 1. The initial search was conducted in March 2019 and resulted in 4,425 non-duplicated articles, 3,629 of which were published between 1989 and 2019 (updated search conducted and independently reviewed in January 2020). Following removal of duplicates and studies published in languages other than English, titles and abstracts were screened. The removal of studies was then guided by the inclusion and exclusion criteria. Screening and data extraction were conducted independently by the lead author [DR] and an independent reviewer. Criteria for data extraction were determined prior to commencing the review. To ensure accuracy of data extraction, authors [DR, AG] recorded data using a standardised form. Initial data collected at screening included sample size, study design, mean age, diagnostic assessment, intervention features (i.e., use of HMD, dosage, duration), main findings, and limitations. Disagreement and discrepancies between reporting were resolved through discussion. Extracted data were synthesised descriptively. The rate of agreement was calculated at 86.1%. Consensus was reached through discussion and input from members of the research team [BC, LC]. Following full-text screening and data extraction, a total of 37 studies remained.

Figure 1.  PRISMA Flow Diagram of Study Selection Process 

Risk of Bias and Quality Assessment

The ROBIS risk of bias assessment tool (Whiting et al., 2016) was selected to assess the methodological quality of the included studies. Assessment of bias consisted of random sequence generation (D1), allocation concealment (D2), blinding of participants and personnel (D3), blinding of outcome assessment (D4), incomplete outcome data (D5), selective reporting (D6), and other bias (D7). The Critical Appraisal Skills Programme, 2018 (CASP) for RCTs checklist was also utilised to assess the methodological quality of studies (2018). In ensuring the criteria were applied consistently, the quality of assessments for risk of bias and CASP was tested on three studies (Banos et al., 2002; Coelho et al., 2008; Rothbaum et al., 2000) published before 2010. These articles were randomly sampled from a pool of excluded studies (k = 33), using a random number generator. The studies met all other inclusion criteria except publication date. This procedure was employed to ensure consensus could be reached between the raters [DR, AG] prior to, and throughout the assessment of the final articles (n = 37). CASP and risk of bias were examined for all articles by the first author [DR], with a subset of studies (60%) selected at random for independent review [AG]. The inter-rater agreement for the preliminary assessment of risk of bias and CASP was 85.7% (k = 3). For the final articles (n = 37) the inter-rater agreement for risk of bias was 77.9% (n = 22) and 80.2% for the CASP (n = 23). Discrepancies were resolved through discussion between raters and authors [LC, BC].

Results

Overview of Included Studies (n = 37)

Most studies were conducted in Europe (k = 18), with specific phobias being the most commonly researched disorder (k = 12). Only one study examined a VRI for more than one type of phobia (Moldovan & David, 2014), namely social, height, and flying phobias. The most common therapeutic approach was CBT, with virtual reality exposure therapy (VRET) being the most frequent technique applied in treatment (k = 35). Interestingly, none of the resultant articles examined the effect of VRIs for depressive disorders, a combination of disorders/transdiagnostic application, nor for OCD. Few VRIs could be used with minimal guidance by a therapist or utilised remotely (k = 5), with interventions primarily focused on coaching, symptom-focused treatment, or comprehensive therapy (Reynolds et al., 2015).

Table 1.  Overview of Included Studies by Disorder (n = 37)  

Note. ABS II = Attitudes and Beliefs Scale II; ACQ: Agoraphobia Cognitions Questionnaire; ADIS-IV = Anxiety Disorders Interview Schedule for DSM-IV; AQ = Acrophobia Questionnaire; ASC-P = Appraisal of Social Concerns – Probability subscale; ASC-C = Appraisal of Social Concerns – Consequence subscale; ASI = Anxiety Sensitivity Index; ATHQ = Attitude toward Height Questionnaire; ATQ30 = Automatic Thoughts Questionnaire; BAT = Behavioural Approach Task; BAI = Beck Anxiety Inventory; BBQ = Brunnsviken Quality of Life Scale. BDI = Beck Depression Inventory; BFNE = Brief Fear of Negative Evaluation Scale; BSQ = Bodily Sensation Questionnaire; CAPS-5 = Clinician-administered PTSD Scale for DSM-5; CAS = Chambless Agoraphobic Questionnaire; CGI = Clinical Global Impressions Scale; CSQ-8 = Client Satisfaction Questionnaire; CTQ = Childhood Trauma Questionnaire Short Form; DAS = Dysfunctional Attitude Scale; DASS 21 = Depression Anxiety Stress Scale; DEFAS = Danger Expectations and Flying Anxiety Scales; DERS = Difficulties of Emotion Regulation Scale; DES = Dissociative Experience Scale; DFS = Dental Fear Scale; ERS = Experience Rating Scale; EUROHIS-QOL = Eurohis Quality of Life Scale; FAM = Flight Anxiety Modality Questionnaire; FAQ = Focus of Attention Questionnaire; FAS = Flight Anxiety Situation Questionnaire; FFMQ = Five Facets of Mindfulness Questionnaire; FFQ = Fear of Flying Questionnaire; FFS = Fear of Flying Scale. FNE-B = Fear of Negative Evaluation Brief Form; FSQ = Fear of Spiders Questionnaire – French version. FQ = Fear Questionnaire; GAD-7 = General Anxiety Disorder 7 items. GAF = Global Assessment of Functioning; GPQ = Gatineau Presence Questionnaire GSE = General Self-Efficacy Questionnaire; HAMA = Hamilton Anxiety Rating Scale. HADS = Hospital Anxiety and Depression Scale; HIQ = Heights Interpretation Questionnaire; IAPT = Improving Access to Psychological Therapies Scale; IASMHS = Inventory of Attitudes toward Seeking Mental Health Services; IPQ = Igroup Presence Questionnaire; ITC-SOPI = Independent Television Company SOP Inventory; ITQ = Immersive Tendency Questionnaire; LIS = Life Interference Scale; LSAS-SR = Liebowitz Social Anxiety Scale-Self Reported Version; MAIA = Multidimensional Assessment of Interoceptive Awareness; M-FAST = Miller-Forensic Assessment of Symptoms; MIA: Mobility Inventory for Agoraphobia; MINI = Mini International Neuropsychiatric Interview; MSAQL = Manchester Short Assessment of Quality of Life; NEQ-32 = Negative Effects Questionnaire; PCL-5 = PTSD Checklist for DSM-5; PCL-C = PTSD Checklist Civilian Version; PCL-M = PTSD Checklist – Military Version; PDBQ = Personality Disorder Belief Questionnaire; PDSS = Panic Disorder Severity Scale; PHQ-9 = Patient Health Questionnaire; PPGAS = Panic, Phobia and Generalized Anxiety Scale; PRCS = Personal Report of Confidence as a Speaker; PRJQ = Presence and Reality Judgment Questionnaire; PQ = Presence Questionnaire; PSE-SQ = Perceived Self-Efficacy towards Spiders Questionnaire; PTS = Paranoid Thoughts Scale; QMI = Questionnaire upon Mental Imagery; SAS = Social Adjustment Scale; SBQ = Spider Beliefs Questionnaire; SCID-I = Structured Clinical Interview for Axis-1 Disorders; SCID-II: Structured Clinical Interview for Axis-1 Disorders; SCQ: Social Costs Questionnaire; SDS: Sheehan Disability Scale; SESS: Self-Efficacy for Social Situations Scale; SF-36 = The Medical Outcome Studies 36-item Short Form Health Survey; SPRS = Social Performance Rating Scale; SPQ = Spider Phobia Questionnaire; SPS = Social Phobia Scale; SIAS = Social Interaction Anxiety Scale; SoPQ = Sense of Presence Questionnaire; SOQ = Session Opinion Questionnaire; SSPS = Self Statements during Public Speaking Scale; SSQ = Simulation Sickness Questionnaire; SSRPH = Stigma Scale for Receiving Psychological Help; STAI = State Trait Anxiety Inventory; SUD = Subjective Units of Discomfort; SWEAT = Specific Work for Exposure Applied in Therapy; TC/E = Treatment Credibility/Expectancy Questionnaire; TES = Therapist Evaluation Scale; TPQ = Treatment Preference Questionnaire; TRES = Therapeutic Relationship Evaluation Scales; VAS = Visual Analogue Scale; WAI-SF = Working Alliance Inventory Short Form; WSA = Work and Social Adjustment Scale.

A variety of recruitment strategies were employed to develop and test VRIs, these included but were not limited to, self and clinician referrals through universities, health services, workplaces, social media, and other online advertising. VRIs were typically tested in research settings (k = 35) over primary health care settings (k = 1). All VRIs were delivered individually, with therapists or research personnel tailoring treatments to clients during exposure based VRI (k = 35). Studies frequently reported that the VRI was adapted from traditional treatment protocols (k = 29). Most research utilised VR HMDs that were tethered, costly, and manufactured prior to 2010. Most VR experimental conditions were efficacious when compared to waitlist and control conditions with mostly large treatment effects (Table 1). The precision of treatment effects was infrequently reported and only a small number of studies (k = 6) included at least one or more VR specific measures (i.e., presence, immersion, cybersickness) and one or more client centric measures of treatment (i.e., treatment satisfaction, preference, acceptability). A more detailed summary of the included studies is provided in Table 1.

Efficacy of VRIs

The majority of VRIs outperformed waitlist controls on various self-report measures of social anxiety (Anderson et al., 2013; Bouchard et al., 2017; Kampmann et al., 2016; Kim et al., 2017; Lindner, Miloff, Fagernas, et al., 2019; Robillard et al., 2010), panic disorder with/without agoraphobia (Meyerbröker et al., 2011; Meyerbroeker et al., 2013; Pelissolo et al., 2012); PTSD (Reger et al., 2016), and various specific phobias (Freeman et al., 2018; Michaliszyn et al., 2010; Raeder et al., 2019; Shiban, Brutting, et al., 2015). Many VRIs were either equally effective or not statistically different from other active treatment conditions, such as for social anxiety disorder (Anderson et al., 2013; Robillard et al., 2010), panic disorder and agoraphobia (Malbos et al., 2013; Pelissolo et al., 2012; Quero et al., 2013), PTSD (Beidel et al., 2019; Norr et al., 2019; Reger et al., 2019), and various specific phobias (Lindner, Miloff, Fagernas, et al., 2019; Michaliszyn et al., 2010; Raeder et al., 2019; Rus-Calafell et al., 2013; Tortella-Feliu et al., 2011). Some VRIs were more effective than active treatments for social anxiety (Bouchard et al., 2017), PTSD (McLay et al., 2011), and specific phobia (Gujjar et al., 2019). Conversely, active treatment conditions in three studies outperformed VRIs across measures of social anxiety (Kampmann et al., 2016) and PTSD symptomology (McLay et al., 2017; Reger et al., 2016).

Methodological Features of Studies

Of the final studies, 19 were randomised control trials (RCTs), with the remainder of studies consisting of two open trials, one multiple baseline design, one parallel case series, and the remainder (k = 14) consisting of other observational analytic study designs. All studies clearly addressed a focussed issue, with most randomising participants to conditions (k = 29). Few studies adequately reported on information pertaining to randomisation method, type of randomisation, and how random allocation sequencing occurred. Fourteen studies explicitly reported on statistical power, nine of which met power estimates. Most studies comprised small samples, with intervention groups consisting of 54 participants or less. Only two studies were conducted in hospital settings (Levy et al., 2016; McLay et al., 2011), with the remainder taking place in research laboratories or tertiary institutions. Dropout rates in the current systematic review were similar to those reported in meta-analytic studies (Carl et al., 2019; Turner & Casey, 2014). It is important to note that some studies had minimal attrition (Freeman et al., 2018; Geraets et al., 2019; Kim et al., 2017; Tortella-Feliu et al., 2011) (Table 1). High incidences of attrition appeared to be more common in VRIs for PTSD, though attrition rates were still similar to those seen in traditional face-to-face interventions (Beidel et al., 2019; Deng et al., 2019) and other meta-analytic studies of VRIs for anxiety disorders (Beidel et al., 2019; Benbow & Anderson, 2019; Deng et al., 2019). Assessment of risk of bias domains revealed that concealment of allocating participants to groups was problematic, as was reporting of incomplete outcome data (Figure 2). Interestingly, many VRIs achieved clinical significance in the presence of sub-optimal methodological rigour, and these previous observations were independent of effect size (Turner & Casey, 2014). A full appraisal of each study against the CASP is displayed in Appendix.

Figure 2.  Summary of ROBIS Risk of Bias for Included Studies (N = 37). 

Clinical Features of VRIs

Dosage, duration, and frequency. A high degree of variability in dosage, duration, and frequency of VRIs was evident (Table 2). Frequency of treatment ranged from one-session treatment to twenty sessions of VR and duration of VRIs varied from 5 minutes to 120 minutes. Some studies scheduled breaks for participants, with most studies immersing participants sequentially for shorter intervals to mitigate adverse effects such as simulator sickness. This variability makes it difficult to distinguish optimal dosage, duration, and frequency of VRI sessions by disorder.

Table 2.  Clinical Characteristics of VRI Studies by Disorder  

Note. 1Approximate. Hyphen (-) indicates data not available.

Presence, immersion, and simulator sickness. VR specific measures, such as presence, immersion, simulator sickness, visual analogue, and body sensation, were included in a small number of studies (Table 1). Of the studies that included measures of presence, mixed findings were reported. Non-significant relationships were found between perceived presence and treatment efficacy in Malbos et al. (2013). Based on the difference between scores between pre-test and post-test on various dependent variables, small insignificant positive correlations were found between the presence questionnaire and the agoraphobia cognitions questionnaire (r = .15, p = .56), depression anxiety stress scales (r = .36, p = .15), and the mobility inventory for agoraphobia (r = .10, p = .36) (Malbos et al., 2013). No significant differences were observed in a behavioural approach task for spider phobia, with scores at post-treatment for those with high-presence compared to those with low presence scores (ß = .90, 95% CI = -0.32 to 2.13, p = .14) suggesting perceived levels of presence in a VRI compared to in-vivo exposure did not influence improvements in a behaviour task (Lindner, Milhoff, Fagernas, et al., 2019).

Navarro-Haro et al. (2019) found a gradual decrease in presence scores as participants progressed through VR sessions for flying phobia, whereas mean scores on the presence questionnaire increased between and across treatment sessions for social anxiety disorder (Bouchard et al., 2017). However, these increases in scores across sessions did not significantly predict decreases in symptomology. Results from one study revealed that presence did not moderate pre-treatment and post-treatment anxiety scores for various phobias (i.e., social, flying, and height phobia) (Moldovan & David, 2014), nor did presence predict reduction of fear in a VRI for spider phobia (Tardif et al., 2019). Two studies included the immersive tendency questionnaire by Witmer and Singer (1998). Neither study found a significant relationship between immersion and improvements in treatment outcome (Moldovan & David, 2014; Tardif et al., 2019). These findings indicate that the role of presence and immersion in treatment outcome in VR remains unclear. In reference to translation, VRIs with sub-optimal immersive environments can still be effective in the treatment of certain psychological disorders. Consumers and clinicians have the option to forgo more costly HMDs with greater technical specifications (i.e., FOV, frames per second, etc.) for those that create a more affordable immersive experience.

The effects of simulator sickness on treatment outcomes were also mixed, despite all using the simulator sickness questionnaire (SSQ) by Kennedy et al. (1993). Freeman et al. (2018) reported consistently low levels of simulator sickness across the treatment period for flying phobia (first treatment session M = 2.21, p < .001 and last treatment session M = 1.35, p = .002). Scores on the SSQ did not significantly predict attrition in the treatment of SAD (Bouchard et al., 2017), but scores were shown to be significantly higher in a clinical sample of SAD comparted to a non-clinical sample (Kim et al., 2017) and this was consistent across treatment sessions (t50 = -3.12, p = .003). Scores on the SSQ varied among other studies examining VRET for agoraphobia (M = 14.0, SD = 8.50) (Malbos et al., 2013) and spider phobia (M = 7.80, SD = 6.51) (Tardif et al., 2019). These findings indicate that simulator sickness may be more common at the start of treatment and may influence early termination of treatment, particularly in clinical populations. However, no clear link to other treatment outcomes was observed in the included studies.

Therapist involvement. As reported elsewhere (Newman et al., 2011), most VRIs tend to include a high degree of therapist involvement. Some studies reported the role of the therapist as being a point of contact for questions pertaining to treatment or in assisting with troubleshooting of VR technology (Freeman et al., 2018; Kampmann et al., 2016). Others reported minimal therapist involvement in VR treatment, suggesting that some VRIs may have potential for remote use in the physical absence of a therapist. Certain studies emphasised the role of the therapist as an active agent in the design and development of virtual environments that could be tailored to the client's therapeutic needs, such as personalized exposure scenarios (Freeman et al., 2018; Loucks et al., 2019). None of the studies quantified the extent of therapist involvement (i.e., minutes per session), and few clearly reported on the primary role of the therapist in each respective VRI. The impact of therapist involvement in many of the included studies on outcome remains inconclusive. Table 2 demonstrates the extent of therapist involvement across studies. Few studies reported minimal to no therapist guidance and the capacity of VRI to be used remotely as a standalone or adjunctive consumer-led intervention remains largely unexplored.

Remote VRIs. Five of the included studies had no or minimal therapist guidance (Freeman et al., 2018; Kim et al., 2017; Levy et al., 2016; Lindner, Miloff, Fagernas, et al., 2019). Levy et al. (2016) treated hospital outpatients with acrophobia from a remote distance using VRET. Three sessions consisted of remote VRET, while the remaining three were conducted in a traditional therapy office in the presence of the therapist. Findings revealed no significant differences in experiences of anxiety, presence, or working alliance between the remote VRET and guided VRET sessions, suggesting that participants had similar experiences remotely to traditional VRET. Freeman et al. (2018) used a virtual coach to guide participants through an inexpensive VRET (“Now I Can Do Heights”) for fear of heights. This VRI was designed to be used without a therapist; however, it could be delivered by a therapist in a traditional clinical setting. The virtual coach provided psychoeducation from a cognitive perspective and asked participants to rate their fear on a series of questions. The underlying mechanism of treatment was for individuals to determine how accurate their fears were, identify and challenge safety-seeking behaviors, and mitigate avoidance towards challenging scenarios. Uptake and completion of the full course of treatment were high (n = 44, 90%) and the authors noted that treatment effects (d = 2.0) far exceeded those seen in a meta-analysis of therapist assisted exposure treatments (d = 1.1) (Wolitzky-Taylor et al., 2008).

Promising findings also have been observed for remote VRIs for social anxiety disorder (Kim et al., 2017; Lindner, Miloff, Fagernas, et al., 2019). Kim et al. (2017) sought to examine whether mobile-based VR self-training was as valid and effective as VRET based on previous studies (Anderson et al., 2013; Kampmann et al., 2016; Robillard et al., 2010). Significant treatment effects were observed for VR self-training and for group (non-clinical vs. clinical social anxiety disorder) on the HADS, LSAS, and the SIAS. Improvements were reported for both conditions for general anxiety, performance anxiety, and social interaction anxiety, as well as depression.

In Lindner, Milhoff, Fagernas, et al.'s (2019) study, treatment for fear of public speaking comprised either one session of exposure therapy with a therapist or one session of self-led VRET. Both conditions demonstrated strong treatment effects, with the therapist-led condition more efficacious (d = 1.67) than the self-led VRET condition (d = 1.38). Participants from both conditions subsequently completed a four week, online modular program that promoted transition to in-vivo exposure. This study provides an example of the capacity for VR to be combined with other digital mental health interventions for efficacious remote delivery of treatments. Similarly in Lindner, Miloff, Fagernas, et al.'s (2019) subsequent study relating to treatment of spider phobia, one-session treatment with a therapist was compared to one session of self-led VRET, with both conditions demonstrating strong treatment effects. Therapists in both studies (Lindner, Miloff, Fagernas, et al., 2019; Lindner, Miloff, Zetterlung, et al., 2019) took on a “computer technician” role and were to provide care if a serious emotional response was elicited. One commonality that can be seen in these studies is their application of consumer available VR hardware (e.g., HTC Vive, Samsung Gear VR). Such technology would allow for easy remote delivery of self-led interventions, although this would need to be explored further in future research. Although minimal studies have been conducted to date, early findings are promising for the delivery of in home and remote interventions.

Treatment Experience

Few studies included one or more measures of user engagement, treatment satisfaction, or attitudes towards VRIs. Studies that did include secondary outcome measures relating to user engagement included client satisfaction (Anderson et al., 2013), session opinion and treatment preference (Botella et al., 2014), working alliance (Anderson et al., 2013; Moldovan & David, 2014), mental health associated stigma (Mishkind et al., 2017), access and attitudes towards psychological treatment (Freeman et al., 2018; Reger et al., 2016), and the therapeutic relationship (Michaliszyn et al., 2010; Pelissolo et al., 2012). Scores from the experience rating scale-therapist version demonstrated that prior to administering either treatments therapists held more pessimistic views towards VRET than traditional face-to-face CBT for panic disorder with agoraphobia (p = .01). However, patients' expectancies of either treatment did not differ (Pelissolo et al., 2012). No differences were observed for ratings of the therapeutic relationship between VRET or CBT.

Other Features of VRI Technology

Most studies utilised HMDs and other VR equipment that was manufactured prior to 2010 (Table 3). Compared with the advanced VR technology currently available to consumers, the technological specifications of VR equipment used in the studies reviewed were substandard. According to a multi-user study by Mehrfard et al. (2019), comparative analysis revealed that three specific HMDs (i.e., Samsung Odyssey+, Oculus Rift S, and the HTC Vive) performed better across various metrics such as comfort, image quality, weight, and tracking stability. These HMDs were not common among studies in this review, with most of the HMDs having poorer technical specifications, such as lower display resolutions, narrower field of view (FOV), less intuitive head tracking degrees of freedom (DOV), and were tethered to a computer, which incurred additional costs. These were the eMagin z800 and the nVisor sx60, and have been reported to be the two most commonly used HMDs in mental health research (Jerdan et al., 2018).

Table 3.  Specifications for Head Mounted Displays (Jerdan et al., 2018; Snoswell & Snoswell, 2019; Wiederhold & Riva, 2019)  

Note. 1DOF = degrees of freedom; 2FOV = field of view; 3approximate cost; 4discontinued VR system. Hyphen (-) indicates data not available.

Studies (k = 7) that specified use of a HMD but not system (Anderson et al., 2013; Gebara et al., 2016; McLay et al., 2014; McLay et al., 2017; McLay et al., 2012; McLay et al., 2011; Moldovan & David, 2014; Norr et al., 2018).

The affordability of recent VR technologies is widely reported (Bouchard et al., 2017; Kim et al., 2017; Mehrfard et al., 2019), yet no study in the current review reported the exact cost of VR HMDs or accompanying equipment. Internet searches and previously reported price points of HMDs revealed that most of the studies used costly and outdated HMDs (Table 3). Despite the ubiquity of mobile phones, few of the studies included them as a cost effective platform for delivering VRIs via HMDs (e.g., Samsung gear VR, Google Cardboard VR etc.) (Kim et al., 2017; Lindner, Miloff, Fagernas, et al., 2019). Mid-range VR HMDs, such as the Oculus Quest, Oculus Go, and HTC Vive, were also underutilised. None of the included studies reported on the commercial availability of their VRI, and it was unclear if any of the VRIs from the included studies were to be made available to consumers after publication.

Discussion

The aim of this review was to examine efficacy of VRIs for EDs in relation to not only outcome, but also various translational factors that influence the availability of VRIs in clinical and consumer settings. This review provides an updated synthesis of research on VRIs for psychological treatment and extends upon previous systematic reviews and meta-analytic studies, which have examined the efficacy of VRIs for unidimensional classifications of disorders (Botella et al., 2017; Carl et al., 2019; Fodor et al., 2018). The review highlights the extent to which VRIs have been effective for single disorder treatment, predominantly in the application of exposure techniques and as an adjunct to traditional treatments for anxiety disorders and PTSD. However, key gaps relating to transdiagnostic and remote application of VRIs for the treatment of EDs have also been identified. At the start of this decade it was predicted that “as technological innovations continue to develop, the efficacy, portability, and feasibility of the use of technology should continue to grow” (Newman et al., 2011, pp. 100-101). However, across digital mental health technologies, difficulties with translation into clinical practice have been observed, including in the translation of VRIs. We argue that the gaps identified by this review help to explain these translation difficulties.

Main Findings

The efficacy of VRIs for anxiety disorders and PTSD has been widely reported and are supported in the current review (Botella et al., 2017; Carl et al., 2019; Deng et al., 2019; Fodor et al., 2018), yet few have become available outside of research settings with little translation into clinical contexts (Mishkind et al., 2017; Wiederhold & Riva, 2019). To the best of the authors' knowledge, none of the included studies had effectively translated treatment from research to a commercially available VR program, ready for consumer use in the home or clinician uptake in practice settings. Furthermore, none of the included studies reported that the VRI was to be made available to consumers or clinicians, despite promising rates of compliance and completion of these treatments. Research is emerging for remote or minimally guided VRIs, though translation of these into the home remains poorly understood. In addition, although transdiagnostic VRIs offer great promise for overcoming translational issues, research is yet to thoroughly explore this.

Methodological Rigour

While VRIs may yield significant treatment effects in the face of suboptimal reporting, translational research is often outpaced by the ongoing development, updates, and iterations of VR technologies (Mishkind et al., 2017). Rapid prototyping and iterations of VRIs are often not possible due to the significant time delays that exist between demonstration of clinical efficacy in RCTs, to implementation and routine use of evidence-based treatment in practice (Clough & Casey, 2015a; Kirchner et al., 2020). There is considerable room for improvement in the methodological reporting of VRI studies as many studies did not adequately describe the VR apparatus, cost of HMDs, or include measures of engagement, immersion, or presence. As such, the quality of the associated VR software programs was unable to be assessed. This lack of detail in reporting limits implementation efforts and complicates replication of studies.

Similar to findings reported elsewhere (Fodor et al., 2018), results from the risk of bias assessment and study appraisal revealed various methodological considerations related to appropriate and insufficient reporting of randomisation, group allocation procedures, reasons for participant attrition, and data management. Indeed, much of the research reported on originally developed, idiosyncratic VRIs, without reference to broader dissemination or commercial availability. Most of the included studies applied costly (> $1,000 USD), outdated (manufactured prior to 2010), and discontinued HMDs that are not accessible to consumers. Mobile phone-compatible VRIs are an accessible treatment modality that take advantage of the untethered and portable nature of mobile phones. However, this review revealed only three studies in which a mobile HMD was used (Kim et al., 2017; Lindner, Miloff, Fagernas, et al., 2019). VRIs were identified as an efficacious adjunct to traditional treatment, in combination with other technology-based interventions such as video-conference therapy via Skype (Pedram et al., 2020), and as a top-up to one-session VRET with online modules for computerised exposure therapy for public speaking anxiety (Lindner, Miloff, Fagernas, et al., 2019). Taken together, these studies demonstrate the capacity of VRIs to be used for remote, in person, and blended psychological treatment approaches, although the area is still in its infancy.

Implications

VRIs are well established as an efficacious treatment modality, particularly for disorders of anxiety and fear. They are a compelling modality for delivering transdiagnostic skills training such as mindfulness, psychoeducation, and cognitive restructuring, although further research in needed to explore these intervention options in VR. Better translation of VRIs for EDs could provide consumers with the opportunity to engage in client-centred, self-guided therapeutic interventions (i.e., through interactive and immersive: mindfulness exercises, skill practice, distraction tasks, psychoeducation, and feedback) in the home, and could be an effective avenue for the prevention of mild to moderate presentations. Furthermore, an in home, transdiagnostic VRI that simulates client-tailored scenarios could be useful in enhancing dosage effects of treatment between traditional face-to-face appointments. Self-guided or remote VRIs could further promote client autonomy with treatment, improve adherence to homework activities between sessions, and may be effective in overcoming barriers to traditional treatment such as stigma and access.

The recent production of more affordable (less than $1,000 USD), high quality, and portable VR hardware (e.g., Oculus Rift, HTC Vive, Google Cardboard) such as those applied by Donker et al. (2019), is promising for consumers and clinicians. The capacity for VRIs to meet client need through the continued provision of structured instructions and immediate feedback is a compelling solution to the translation of treatment into the home. Remotely accessible and self-guided VRIs have the potential to further improve treatment coverage by circumnavigating travel barriers and costly logistics of exposure treatments (Newman et al., 2011).

Other factors that have been reported to limit translation include selecting appropriate, efficacious and evidence-based VRIs (Mehrfard et al., 2019) which may be due to knowledge deficits, limited awareness of VRIs, misconceptions about their cost, clinical efficacy, utility, concerns for working alliance, and deficits in competence with VR technology (Clough, Eigeland, et al., 2019; Clough, Rowland, et al., 2019; Lindner, Miloff, Zetterlund, et al., 2019). In achieving greater translation of VRIs, there is a need for standardized clinical training and education in available and validated VR treatment protocols with explicit directions on how to implement them (Lindner, Miloff, Zetterlund, et al., 2019; Mishkind et al., 2017; Wiederhold & Riva, 2019).

Recommendations and Limitations

VRI studies have been outpaced by rapid advancements in technology and as such it is imperative that innovative research designs be considered. Research designs such as sequential multiple assignment randomized trials (Almirall et al., 2014) or multiphase optimisation strategy trials (Collins et al., 2007) have the capacity to bridge the digital divide and keep pace with ongoing technological iterations and updates required by many third party VR market places (such as the App Store, SteamVR, and the Oculus store). Dismantling studies could be an efficacious strategy for determining the active technological and therapeutic components of treatment that are needed to optimise translation (Newman et al., 2011; Rizzo et al., 2019). Such approaches may also help to improve the quality of research within this field. Future research should clearly report data on the software used, level of engagement and immersion, and quantify the extent of therapist involvement in minimally guided VR treatments.

Studies typically consisted of small samples that lacked statistical power and did not provide adequate detail on VR apparatus, cost-effectiveness, inclusion of a control group and transparent and complete reporting (Fodor et al., 2018). Similar to previous reviews (Fodor et al., 2018; Jerdan et al., 2018), the current review cannot conclude which VR program or HMD is most optimal for clinical use, although this review was limited in its exclusions of non-three-dimensional VR equipment and interventions that did not explicitly use a HMD for the treatment of an ED in a clinical sample of adults.

Finally, although considerable attention has been given to issues of efficacy within this field, it is recommended that future research focus on issues of dissemination and translation. This may include development of clinician training programs to promote adjunctive use of VRIs in practice, a focus on development of commercially available VRIs, and development of repositories or databases for publicly available programs. The latter already exists in many countries, with a focus on online programs and mobile apps (e.g., www.emhprac.org.au). Inclusion of VRIs in these databases would allow clinicians and consumers to easily search for appropriate programs and review the evidence and technical specifications of each, facilitating easier translation for clinical practice and remote use.

Conclusion

Although VR is becoming more accessible in the consumer market, few VRIs for psychological disorders have translated from research to practice. This systematic review provides an update on the current status of VRIs for EDs. The potential of VRIs to improve mental health is continuously being showcased by innovative research and technological transformation. To the best of the authors' knowledge, no VRI has been explicitly designed for the transdiagnostic treatment of EDs. Given the overlap in the aetiology of mood and anxiety disorders and their prevalence in comorbid and co-occurring clinical presentations, a shift away from disorder specific treatment protocols towards a transdiagnostic approach may enhance translation efforts, as is the current focus in many face-to-face treatments. VRIs can target core processes that are relevant across a variety of clinical domains and greater attention should be given to the capacity of VRIs to be utilised for more than just the exposure component of treatments. It is crucial that commercially available HMDs are used with VRIs so that accessible self-help, self-guided, and remote options for care can be used in the home. Further, VRIs that are tailored to higher order dimensions of similar disorders, such as EDs, and which may be delivered remotely, may considerably enhance the translation and reach of these interventions.

Acknowledgments

The authors wish to acknowledge Bonnie Dixon and Katrina Henderson for completing the PRESS checklist, Makayla Jennings for her research assistance and for independently screening articles for inclusion, and Sarah Macdonald for independently screening updated articles published in 2019.

Cite this article as: Rowland, D. P., Casey, L. M., Ganapathy, A., Cassimatis, M., & Clough, B. A. (2021). A decade in review: A systematic review of virtual reality interventions for emotional disorders. Psychosocial Intervention, 31(1), 1-20. https://doi.org/10.5093/pi2021a8

References

Almirall, D., Nahum-Shani, I., Sherwood, N. E., & Murphy, S. A. (2014). Introduction to SMART designs for the development of adaptive interventions: With application to weight loss research. Translational Behavioral Medicine, 4(3), 260-274. https://doi.org/10.1007/s13142-014-0265-0 [ Links ]

Anderson, P. L., Price, M., Edwards, S. M., Obasaju, M. A., Schmertz, S. K., Zimand, E., & Calamaras, M. R. (2013). Virtual reality exposure therapy for social anxiety disorder: A randomized controlled trial. Journal of Consulting & Clinical Psychology, 81(5), 751-760. https://doi.org/10.1037/a0033559 [ Links ]

Arroll, B., Wallace, H. B., Mount, V., Humm, S. P., & Kingsford, D. W. (2017). A systematic review and meta-analysis of treatments for acrophobia. The Medical Journal of Australia, 206(6), 263-267. https://doi.org/10.5694/mja16.00540 [ Links ]

Banos, R. M., Botella, C., Perpina, C., Alcaniz, M., Lozano, J. A., Osma, J., & Gallardo, M. (2002). Virtual reality treatment of flying phobia. Ieee Transactions on Information Technology in Biomedicine, 6(3), 206-212. https://doi.org/10.1109/Titb.2002.802380 [ Links ]

Barlow, D. H., Allen, L. B., & Choate, M. L. (2004). Toward a unified treatment for emotional disorders. Behavior Therapy, 35(2), 205-230. https://doi.org/https://doi.org/10.1016/S0005-7894(04)80036-4 [ Links ]

Bauer, M. S., & Kirchner, J. (2019). Implementation science: What is it and why should I care? Psychiatry Research, 283, 112376. https://doi.org/10.1016/j.psychres.2019.04.025 [ Links ]

Beidel, D. C., Frueh, B. C., Neer, S. M., Bowers, C. A., Trachik, B., Uhde, T. W., & Grubaugh, A. (2019). Trauma management therapy with virtual-reality augmented exposure therapy for combat-related PTSD: A randomized controlled trial. Journal of Anxiety Disorders, 61, 64-74. https://doi.org/10.1016/j.janxdis.2017.08.005 [ Links ]

Benbow, A. A., & Anderson, P. L. (2019). A meta-analytic examination of attrition in virtual reality exposure therapy for anxiety disorders. Journal of Anxiety Disorders, 61, 18-26. https://doi.org/10.1016/j.janxdis.2018.06.006 [ Links ]

Botella, C., Breton-Lopez, J., Serrano, B., Garcia-Palacios, A., Quero, S., & Banos, R. (2014). Treatment of flying phobia using virtual reality exposure with or without cognitive restructuring: Participants' preferences. Revista de Psicopatologia y Psicologia Clinica, 19(3), 157-169. https://doi.org/10.5944/rppc.vol.19.num.3.2014.13898 [ Links ]

Botella, C., Fernandez-Alvarez, J., Guillen, V., Garcia-Palacios, A., & Banos, R. (2017). Recent progress in virtual reality exposure therapy for phobias: A systematic review. Current Psychiatry Reports, 19(7), 42. https://doi.org/10.1007/s11920-017-0788-4 [ Links ]

Bouchard, S., Dumoulin, S., Robillard, G., Guitard, T., Klinger, E., Forget, H., Loranger, C., & Roucaut, F. X. (2017). Virtual reality compared with in vivo exposure in the treatment of social anxiety disorder: A three-arm randomised controlled trial. British Journal of Psychiatry, 210(4), 276-283. https://doi.org/10.1192/bjp.bp.116.184234 [ Links ]

Bowman, D. A., & McMahan, R. P. (2007). Virtual reality: How much immersion is enough? Computer, 40(7), 36-43. https://doi.org/10.1109/MC.2007.257 [ Links ]

Brown, T. A., & Barlow, D. H. (2009). A proposal for a dimensional classification system based on the shared features of the DSM-IV anxiety and mood disorders: Implications for assessment and treatment. Psychological Assessment, 21(3), 256-271. https://doi.org/10.1037/a0016608 [ Links ]

Bullis, J. R., Boettcher, H., Sauer-Zavala, S., Farchione, T. J., & Barlow, D. H. (2019). What is an emotional disorder? A transdiagnostic mechanistic definition with implications for assessment, treatment, and prevention. Clinical Psychology: Science and Practice, 26(2), 1-19. https://doi.org/10.1111/cpsp.12278 [ Links ]

Carl, E., Stein, A. T., Levihn-Coon, A., Pogue, J. R., Rothbaum, B., Emmelkamp, P., Asmundson, G. J. G., Carlbring, P., & Powers, M. B. (2019). Virtual reality exposure therapy for anxiety and related disorders: A meta-analysis of randomized controlled trials. Journal of Anxiety Disorders, 61, 27-36. https://doi.org/10.1016/j.janxdis.2018.08.003 [ Links ]

Cassiello-Robbins, C., Southward, M. W., Tirpak, J. W., & Sauer-Zavala, S. (2020). A systematic review of Unified Protocol applications with adult populations: Facilitating widespread dissemination via adaptability. Clinical Psychology Review, 78, 101852. https://doi.org/10.1016/j.cpr.2020.101852 [ Links ]

Chandrashekar, P. (2018). Do mental health mobile apps work: Evidence and recommendations for designing high-efficacy mental health mobile apps. Mhealth, 4(3). https://doi.org/10.21037/mhealth.2018.03.02 [ Links ]

Cipresso, P., Chicchi Giglioli, I. A., Alcañiz Raya, M., & Riva, G. (2018). The past, present, and future of virtual and augmented reality research: A network and cluster analysis of the literature. Frontiers in Psychology, 9, 2086. https://doi.org/https://doi.org/10.3389/fpsyg.2018.02086 [ Links ]

Clough, B. A., & Casey, L. M. (2011). Technological adjuncts to enhance current psychotherapy practices: A review. Clinical Psychology Review, 31(3), 279-292. https://doi.org/10.1016/j.cpr.2010.12.008 [ Links ]

Clough, B. A., & Casey, L. M. (2015a). Smart designs for smart technologies: Research challenges and emerging solutions for scientist-practitioners within e-mental health. Professional Psychology: Research and Practice, 46(6), 429-436. https://doi.org/10.1037/pro0000053 [ Links ]

Clough, B. A., & Casey, L. M. (2015b). The smart therapist: A look to the future of smartphones and mHealth technologies in psychotherapy. Professional Psychology: Research and Practice, 46(3), 147-153. https://doi.org/10.1037/pro0000011 [ Links ]

Clough, B. A., Eigeland, J. A., Madden, I. R., Rowland, D., & Casey, L. M. (2019). Development of the eTAP: A brief measure of attitudes and process in e-interventions for mental health. Internet Interventions, 18, 100256. https://doi.org/10.1016/j.invent.2019.100256 [ Links ]

Clough, B. A., Rowland, D. P., & Casey, L. M. (2019). Development of the eTAP-T: A measure of mental health professionals' attitudes and process towards e-interventions. Internet Interventions, 18, 100288. https://doi.org/10.1016/j.invent.2019.100288 [ Links ]

Coelho, C. M., Silva, C. F., Santos, J. A., Tichon, J., & Wallis, G. (2008). Contrasting the effectiveness and efficiency of virtual reality and real environments in the treatment of acrophobia. PsychNology Journal, 6(2), 203-216. www.psychnology.org [ Links ]

Collins, L. M., Murphy, S. A., & Strecher, V. (2007). The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): New methods for more potent eHealth interventions. American Journal of Preventive Medicine, 32(5), S112-S118. https://doi.org/10.1016/j.amepre.2007.01.022 [ Links ]

Critical Appraisal Skills Programme. (2018). CASP randomised controlled trial checklist. https://casp-uk.net/wp-content/uploads/2018/01/CASP-Randomised-Controlled-Trial-Checklist-2018.pdfLinks ]

Deng, W., Hu, D., Xu, S., Liu, X., Zhao, J., Chen, Q., Liu, J., Zhang, Z., Jiang, W., & Ma, L. (2019). The efficacy of virtual reality exposure therapy for PTSD symptoms: A systematic review and meta-analysis. Journal of Affective Disorders, 257, 698-709. https://doi.org/10.1016/j.jad.2019.07.086 [ Links ]

Donker, T., Cornelisz, I., Van Klaveren, C., Van Straten, A., Carlbring, P., Cuijpers, P., & van Gelder, J.-L. (2019). Effectiveness of self-guided app-based virtual reality cognitive behavior therapy for acrophobia: A randomized clinical trial. JAMA Psychiatry, 76(7), 682-690. https://doi.org/10.1001/jamapsychiatry.2019.0219 [ Links ]

Fodor, L. A., Cote, C. D., Cuijpers, P., Szamoskozi, S., David, D., & Cristea, I. A. (2018). The effectiveness of virtual reality based interventions for symptoms of anxiety and depression: A meta-analysis. Scientific Reports, 8(1), 10323. https://doi.org/10.1038/s41598-018-28113-6 [ Links ]

Freeman, D., Haselton, P., Freeman, J., Spanlang, B., Kishore, S., Albery, E., Denne, M., Brown, P., Slater, M., & Nickless, A. (2018). Automated psychological therapy using immersive virtual reality for treatment of fear of heights: A single-blind, parallel-group, randomised controlled trial. The Lancet Psychiatry, 5(8), 625-632. https://doi.org/10.1016/s2215-0366(18)30226-8 [ Links ]

Gebara, C. M., Barros-Neto, T. P., Gertsenchtein, L., & Lotufo-Neto, F. (2016). Virtual reality exposure using three-dimensional images for the treatment of social phobia. Brazilian Journal of Psychiatry, 38(1), 24-29. https://doi.org/10.1590/1516-4446-2014-1560 [ Links ]

Geraets, C. N. W., Veling, W., Witlox, M., Staring, A. B. P., Matthijssen, S., & Cath, D. (2019). Virtual reality-based cognitive behavioural therapy for patients with generalized social anxiety disorder: A pilot study. Behavioral and Cognitive Psychotherapy, 1-6. https://doi.org/10.1017/S1352465819000225 [ Links ]

Gonçalves, R., Pedrozo, A. L., Coutinho, E. S., Figueira, I., & Ventura, P. (2012). Efficacy of virtual reality exposure therapy in the treatment of PTSD: A systematic review. PLoS One, 7(12), e48469. https://doi.org/10.1371/journal.pone.0048469 [ Links ]

Gonzalez-Robles, A., Diaz-Garcia, A., Miguel, C., Garcia-Palacios, A., & Botella, C. (2018). Comorbidity and diagnosis distribution in transdiagnostic treatments for emotional disorders: A systematic review of randomized controlled trials. PLoS One, 13(11), e0207396. [ Links ]

Gujjar, K. R., van Wijk, A., Kumar, R., & de Jongh, A. (2019). Efficacy of virtual reality exposure therapy for the treatment of dental phobia in adults: A randomized controlled trial. Journal of Anxiety Disorders, 62, 100-108. https://doi.org/10.1016/j.janxdis.2018.12.001 [ Links ]

Jerdan, S. W., Grindle, M., van Woerden, H. C., & Boulos, M. N. K. (2018). Head-mounted virtual reality and mental health: Critical review of current research. JMIR Serious Games, 6(3). https://doi.org/doi:10.2196/games.9226 [ Links ]

Kampmann, I. L., Emmelkamp, P. M., Hartanto, D., Brinkman, W. P., Zijlstra, B. J., & Morina, N. (2016). Exposure to virtual social interactions in the treatment of social anxiety disorder: A randomized controlled trial. Behaviour Research Therapy, 77(2), 147-156. https://doi.org/10.1016/j.brat.2015.12.016 [ Links ]

Kampmann, I. L., Emmelkamp, P. M. G., & Morina, N. (2019). Cognitive predictors of treatment outcome for exposure therapy: Do changes in self-efficacy, self-focused attention, and estimated social costs predict symptom improvement in social anxiety disorder? BMC Psychiatry, 19(1), 80. https://doi.org/10.1186/s12888-019-2054-2 [ Links ]

Kennedy, R. S., Lane, N. E., Berbaum, K. S., & Lilienthal, M. G. (1993). Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The International Journal of Aviation Psychology, 3(3), 203-220. https://doi.org/10.1207/s15327108ijap0303_3 [ Links ]

Kim, H. E., Hong, Y.-J., Kim, M.-K., Jung, Y. H., Kyeong, S., & Kim, J.-J. (2017). Effectiveness of self-training using the mobile-based virtual reality program in patients with social anxiety disorder. Computers in Human Behavior, 73, 614-619. https://doi.org/10.1016/j.chb.2017.04.017 [ Links ]

Kirchner, J. E., Smith, J. L., Powell, B. J., Waltz, T. J., & Proctor, E. K. (2020). Getting a clinical innovation into practice: An introduction to implementation strategies. Psychiatry Research, 283, 112467. https://doi.org/10.1016/j.psychres.2019.06.042 [ Links ]

Levy, F., Leboucher, P., Rautureau, G., & Jouvent, R. (2016). E-virtual reality exposure therapy in acrophobia: A pilot study. Journal of Telemedicine and Telecare, 22(4), 215-220. https://doi.org/10.1177/1357633X15598243 [ Links ]

Lindner, P., Miloff, A., Fagernas, S., Andersen, J., Sigeman, M., Andersson, G., Furmark, T., & Carlbring, P. (2019). Therapist-led and self-led one-session virtual reality exposure therapy for public speaking anxiety with consumer hardware and software: A randomized controlled trial. Journal of Anxiety Disorders, 61, 45-54. https://doi.org/10.1016/j.janxdis.2018.07.003 [ Links ]

Lindner, P., Miloff, A., Zetterlund, E., Reuterskiöld, L., Andersson, G., & Carlbring, P. (2019). Attitudes toward and familiarity with virtual reality therapy among practicing cognitive behavior therapists: A cross-sectional survey study in the era of consumer VR platforms. Frontiers in Psychology, 10, 176. https://doi.org/10.3389/fpsyg.2019.00176 [ Links ]

Ling, Y., Nefs, H. T., Morina, N., Heynderickx, I., & Brinkman, W. P. (2014). A meta-analysis on the relationship between self-reported presence and anxiety in virtual reality exposure therapy for anxiety disorders. PloS One, 9(5), e96144. https://doi.org/10.1371/journal.pone.0096144 [ Links ]

Loucks, L., Yasinski, C., Norrholm, S. D., Maples-Keller, J., Post, L., Zwiebach, L., Fiorillo, D., Goodlin, M., Jovanovic, T., Rizzo, A. A., & Rothbaum, B. O. (2019). You can do that?!: Feasibility of virtual reality exposure therapy in the treatment of PTSD due to military sexual trauma. Journal of Anxiety Disorders, 61, 55-63. https://doi.org/10.1016/j.janxdis.2018.06.004 [ Links ]

Malbos, E., Rapee, R. M., & Kavakli, M. (2013). A controlled study of agoraphobia and the independent effect of virtual reality exposure therapy. Australian and New Zealand Journal of Psychiatry, 47(2), 160-168. https://doi.org/10.1177/0004867412453626 [ Links ]

McGowan, J., Sampson, M., Salzwedel, D. M., Cogo, E., Foerster, V., & Lefebvre, C. (2016). PRESS peer review of electronic search strategies: 2015 guideline statement. Journal of Clinical Epidemiology, 75, 40-46. https://doi.org/10.1016/j.jclinepi.2016.01.021 [ Links ]

McLay, R. N., Baird, A., Webb-Murphy, J., Deal, W., Tran, L., Anson, H., Klam, W., & Johnston, S. (2017). A randomized, head-to-head study of virtual reality exposure therapy for posttraumatic stress disorder. Cyberpsychology, Behavior and Social Networking, 20(4), 218-224. https://doi.org/10.1089/cyber.2016.0554 [ Links ]

McLay, R. N., Graap, K., Spira, J., Perlman, K., Johnston, S., Rothbaum, B. O., Difede, J., Deal, W., Oliver, D., Baird, A., Bordnick, P. S., Spitalnick, J., Pyne, J. M., & Rizzo, A. (2012). Development and testing of virtual reality exposure therapy for post-traumatic stress disorder in active duty service members who served in Iraq and Afghanistan. Military Medicine, 177(6), 635-642. https://doi.org/10.7205/milmed-d-11-00221 [ Links ]

McLay, R., Ram, V., Murphy, J., Spira, J., Wood, D. P., Wiederhold, M. D., Wiederhold, B. K., Johnston, S., & Reeves, D. (2014). Effect of virtual reality PTSD treatment on mood and neurocognitive outcomes. Cyberpsycholology, Behavior & Social Networking, 17(7), 439-446. https://doi.org/10.1089/cyber.2013.0383 [ Links ]

McLay, R. N., Wood, D. P., Webb-Murphy, J. A., Spira, J. L., Wiederhold, M. D., Pyne, J. M., & Wiederhold, B. K. (2011). A randomized, controlled trial of virtual reality-graded exposure therapy for post-traumatic stress disorder in active duty service members with combat-related post-traumatic stress disorder. Cyberpsychology, Behavior and Social Networking, 14(4), 223-229. https://doi.org/10.1089/cyber.2011.0003 [ Links ]

Mehrfard, A., Fotouhi, J., Taylor, G., Forster, T., Navab, N., & Fuerst, B. (2019). A comparative analysis of virtual reality head-mounted display systems. arXiv preprint. https://doi.org/arXiv:1912.02913 [ Links ]

Meyerbröker, K., Morina, N., Kerkhof, G., & Emmelkamp, P. M. G. (2011). Virtual reality exposure treatment of agoraphobia: A comparison of computer automatic virtual environment and head-mounted display. Annual Review of CyberTherapy and Telemedicine, 9(1), 41-45. http://www.embase.com/search/results?subaction=viewrecord&from=export&id=L363088696Links ]

Meyerbroeker, K., Morina, N., Kerkhof, G. A., & Emmelkamp, P. M. (2013). Virtual reality exposure therapy does not provide any additional value in agoraphobic patients: A randomized controlled trial. Psychotherapy & Psychosomatics, 82(3), 170-176. https://doi.org/10.1159/000342715 [ Links ]

Michaliszyn, D., Marchand, A., Bouchard, S., Martel, M. O., & Poirier-Bisson, J. (2010). A randomized, controlled clinical trial of in virtuo and in vivo exposure for spider phobia. Cyberpsychology, Behavior & Social Networking, 13(6), 689-695. https://doi.org/10.1089/cyber.2009.0277 [ Links ]

Miloff, A., Lindner, P., Dafgard, P., Deak, S., Garke, M., Hamilton, W., Heinsoo, J., Kristoffersson, G., Rafi, J., Sindemark, K., Sjolund, J., Zenger, M., Reuterskiold, L., Andersson, G., & Carlbring, P. (2019). Automated virtual reality exposure therapy for spider phobia vs. in-vivo one-session treatment: A randomized non-inferiority trial. Behaviour Research & Therapy, 118, 130-140. https://doi.org/10.1016/j.brat.2019.04.004 [ Links ]

Mishkind, M. C., Norr, A. M., Katz, A. C., & Reger, G. M. (2017). Review of virtual reality treatment in psychiatry: Evidence versus current diffusion and use. Current Psychiatry Reports, 19(11), 80. https://doi.org/10.1007/s11920-017-0836-0 [ Links ]

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Group, P. (2010). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. International Journal of Surgery, 8(5), 336-341. https://doi.org/10.1016/j.ijsu.2010.02.007 [ Links ]

Moldovan, R., & David, D. (2014). One session treatment of cognitive and behavioral therapy and virtual reality for social and specific phobias. Preliminary results from a randomized clinical trial. Journal of Evidence-Based Psychotherapies, 14(1), 67-83. https://www.researchgate.net/publication/267842953_One_session_treatment_of_cognitive_and_behavioral_therapy_and_virtual_reality_for_social_and_specific_phobias_Preliminary_results_from_a_randomized_clinical_trialLinks ]

Navarro-Haro, M. V., Modrego-Alarcon, M., Hoffman, H. G., Lopez-Montoyo, A., Navarro-Gil, M., Montero-Marin, J., Garcia-Palacios, A., Borao, L., & Garcia-Campayo, J. (2019). Evaluation of a mindfulness-based intervention with and without virtual reality dialectical behavior therapy mindfulness skills training for the treatment of generalized anxiety disorder in primary care: A pilot study. Frontiers in Psychology, 10, 55. https://doi.org/10.3389/fpsyg.2019.00055 [ Links ]

Newman, M. G., Szkodny, L. E., Llera, S. J., & Przeworski, A. (2011). A review of technology-assisted self-help and minimal contact therapies for anxiety and depression: Is human contact necessary for therapeutic efficacy? Clinical Psychology Review, 31(1), 89-103. https://doi.org/10.1016/j.cpr.2010.09.008 [ Links ]

Norr, A. M., Bourassa, K. J., Stevens, E. S., Hawrilenko, M. J., Michael, S. T., & Reger, G. M. (2019). Relationship between change in in-vivo exposure distress and PTSD symptoms during exposure therapy for active duty soldiers. Journal of Psychiatric Research, 116, 133-137. https://doi.org/10.1016/j.jpsychires.2019.06.013 [ Links ]

Norr, A. M., Smolenski, D. J., Katz, A. C., Rizzo, A. A., Rothbaum, B. O., Difede, J., Koenen-Woods, P., Reger, M. A., & Reger, G. M. (2018). Virtual reality exposure versus prolonged exposure for PTSD: Which treatment for whom? Depress Anxiety, 35(6), 523-529. https://doi.org/10.1002/da.22751 [ Links ]

Pedram, S., Palmisano, S., Perez, P., Mursic, R., & Farrelly, M. (2020). Examining the potential of virtual reality to deliver remote rehabilitation. Computers in Human Behavior, 105, 106223. https://doi.org/https://doi.org/10.1016/j.chb.2019.106223 [ Links ]

Pelissolo, A., Zaoui, M., Aguayo, G., Yao, S. N., Roche, S., Ecochard, R., Gueyffier, F., Pull, C., Berthoz, A., Jouvent, R., & Cottraux, J. (2012). Virtual reality exposure therapy versus cognitive behaviour therapy for panic disorder with agoraphobia: A randomized comparison study. Journal of CyberTherapy & Rehabilitation (JCR), 5(1), 35-43. http://libraryproxy.griffith.edu.au/login?url=https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=103940278&site=ehost-live&scope=siteLinks ]

Quero, S., Pérez-Ara, M. Á., Bretón-López, J., García-Palacios, A., Baños, R. M., & Botella, C. (2013). Acceptability of virtual reality interoceptive exposure for the treatment of panic disorder with agoraphobia. British Journal of Guidance & Counselling, 42(2), 123-137. https://doi.org/10.1080/03069885.2013.852159 [ Links ]

Raeder, F., Woud, M. L., Schneider, S., Totzeck, C., Adolph, D., Margraf, J., & Zlomuzica, A. (2019). Reactivation and evaluation of mastery experiences promotes exposure benefit in height phobia. Cognitive Therapy and Research, 43(5), 1-11. https://doi.org/10.3389/fpsyg.2019.01191 [ Links ]

Reger, G. M., Koenen-Woods, P., Zetocha, K., Smolenski, D. J., Holloway, K. M., Rothbaum, B. O., Difede, J., Rizzo, A. A., Edwards-Stewart, A., Skopp, N. A., Mishkind, M., Reger, M. A., & Gahm, G. A. (2016). Randomized controlled trial of prolonged exposure using imaginal exposure vs. virtual reality exposure in active duty soldiers with deployment-related posttraumatic stress disorder (PTSD). Journal of Consulting and Clinical Psychology, 84(11), 946-959. https://doi.org/10.1037/ccp0000134 [ Links ]

Reger, G. M., Smolenski, D., Norr, A., Katz, A., Buck, B., & Rothbaum, B. O. (2019). Does virtual reality increase emotional engagement during exposure for PTSD? Subjective distress during prolonged and virtual reality exposure therapy. Journal of Anxiety Disorders, 61, 75-81. https://doi.org/10.1016/j.janxdis.2018.06.001 [ Links ]

Reynolds, J., Griffiths, K. M., Cunningham, J. A., Bennett, K., & Bennett, A. (2015). Clinical practice models for the use of e-mental health resources in primary health care by health professionals and peer workers: A conceptual framework. JMIR Mental Health, 2(1). https://doi.org/10.2196/mental.4200 [ Links ]

Riva, G., Wiederhold, B. K., & Gaggioli, A. (2016). Being different: The transformative potential of virtual reality. Annual Review of CyberTherapy and Telemedicine, 14, 3-7. [ Links ]

Rizzo, A., Thomas Koenig, S., & Talbot, T. B. (2019). Clinical results using virtual reality. Journal of Technology in Human Services, 37(1), 51-74. https://doi.org/10.1080/15228835.2019.1604292 [ Links ]

Robillard, G., Bouchard, S., Dumoulin, S., Guitard, T., & Klinger, E. (2010). Using virtual humans to alleviate social anxiety: Preliminary report from a comparative outcome study. Annual Review of CyberTherapy and Telemedicine, 8(1), 46-48. http://www.embase.com/search/results?subaction=viewrecord&from=export&id=L358907240Links ]

Rothbaum, B. O., Hodges, L., Smith, S., Lee, J. H., & Price, L. (2000). A controlled study of virtual reality exposure therapy for the fear of flying. Journal of Consulting and Clinical Psychology, 68(6), 1020-1026. https://doi.org/10.1037//0022-006X.68.6.1020 [ Links ]

Rus-Calafell, M., Gutierrez-Maldonado, J., Botella, C., & Banos, R. M. (2013). Virtual reality exposure and imaginal exposure in the treatment of fear of flying: A pilot study. Behavior Modification, 37(4), 568-590. https://doi.org/10.1177/0145445513482969 [ Links ]

Sakiris, N., & Berle, D. (2019). A systematic review and meta-analysis of the unified protocol as a transdiagnostic emotion regulation based intervention. Clinical Psychology Review, 72, 101751. https://doi.org/10.1016/j.cpr.2019.101751 [ Links ]

Serrano, B., Botella, C., Wiederhold, B. K., & Baños, R. M. (2019). Virtual reality and anxiety disorders treatment: Evolution and future perspectives. In A. Rizzo & Bouchard, S. (Eds.) Virtual reality for psychological and neurocognitive interventions (pp. 47-84). Springer. [ Links ]

Shiban, Y., Brutting, J., Pauli, P., & Muhlberger, A. (2015). Fear reactivation prior to exposure therapy: does it facilitate the effects of VR exposure in a randomized clinical sample?. Journal of Behavior Therapy and Experimental Psychiatry, 46, 133-140. https://doi.org/10.1016/j.jbtep.2014.09.009 [ Links ]

Shiban, Y., Schelhorn, I., Pauli, P., & Muhlberger, A. (2015). Effect of combined multiple contexts and multiple stimuli exposure in spider phobia: A randomized clinical trial in virtual reality. Behaviour Research & Therapy, 71, 45-53. https://doi.org/10.1016/j.brat.2015.05.014 [ Links ]

Snoswell, A. J., & Snoswell, C. L. (2019). Immersive virtual reality in health care: Systematic review of technology and disease states. JMIR Biomedical Engineering, 4(1), e15025. https://doi.org/10.2196/15025 [ Links ]

Tardif, N., Therrien, C. E., & Bouchard, S. (2019). Re-examining psychological mechanisms underlying virtual reality-based exposure for spider phobia. Cyberpsychology, Behavior & Social Networking, 22(1), 39-45. https://doi.org/10.1089/cyber.2017.0711 [ Links ]

Tortella-Feliu, M., Botella, C., Llabrés, J., Bretón-López, J. M., del Amo, A. R., Baños, R. M., & Gelabert, J. M. (2011). Virtual reality versus computer-aided exposure treatments for fear of flying. Behavior Modification, 35(1), 3-30. https://doi.org/10.1177/0145445510390801 [ Links ]

Turner, W. A., & Casey, L. M. (2014). Outcomes associated with virtual reality in psychological interventions: Where are we now? Clinical Psychology Review, 34(8), 634-644. https://doi.org/10.1016/j.cpr.2014.10.003 [ Links ]

Valmaggia, L. R., Latif, L., Kempton, M. J., & Rus-Calafell, M. (2016). Virtual reality in the psychological treatment for mental health problems: An systematic review of recent evidence. Psychiatry Research, 236, 189-195. https://doi.org/10.1016/j.psychres.2016.01.015 [ Links ]

Weech, S., Kenny, S., & Barnett-Cowan, M. (2019). Presence and cybersickness in virtual reality are negatively related: A review. Frontiers in Psychology, 10, 158. https://doi.org/10.3389/fpsyg.2019.00158 [ Links ]

Whiting, P., Savovi, J., Higgins, J. P., Caldwell, D. M., Reeves, B. C., Shea, B., Davies, P., Kleijnen, J., & Churchill, R., & the ROBIS Group (2016). ROBIS: A new tool to assess risk of bias in systematic reviews was developed. Journal of Clinical Epidemiology, 69, 225-234. https://doi.org/10.1016/j.jclinepi.2015.06.005 [ Links ]

Wiederhold, B. K., & Riva, G. (2019). Virtual reality therapy: Emerging topics and future challenges. Cyberpsychology, Behavior, and Social Networking, 22(1), 3-6. https://doi.org/10.1089/cyber.2018.29136.bkw [ Links ]

Witmer, B. G., & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence, 7(3), 225-240. https://doi.org/10.1162/105474698565686 [ Links ]

Wolitzky-Taylor, K. B., Horowitz, J. D., Powers, M. B., & Telch, M. J. (2008). Psychological approaches in the treatment of specific phobias: A meta-analysis. Clinical Psychology Review, 28(6), 1021-1037. https://doi.org/10.1016/j.cpr.2008.02.007 [ Links ]

Appendix

CASP Evaluation

Note. 1 Did not include one or more measures of user acceptance, satisfaction, or usability.2 Did not include one or more outcome measures of presence, immersion or cybersickness.

Received: April 16, 2021; Accepted: July 22, 2021; pub: September 14, 2021

Correspondence: dale.rowland@griffithuni.edu.au (D. Rowland).

Conflict of Interest

The authors of this article declare no conflict of interest.

Creative Commons License This is an Open Access article distributed under the terms of the Creative Commons Attribution-Noncommercial No Derivative License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium provided the original work is properly cited and the work is not changed in any way.