A pilot shows that an exercise works. Whether it lasts is decided afterwards.
Visualization: An employee practices at a pump with a headset and controllers while a colleague watches, people and plant are freely designed | AI-assisted illustration, not a product photo | XR Stager Online Magazine | VISORIC GmbH
A maintenance team practices a rare, safety-critical shutdown sequence without stopping the real plant. The colleague wearing the headset works with concentration, his coworker watches, and after the exercise the feedback is enthusiastic. The pilot is a success. Six months later, the headset sits in a cupboard. The technology did not fail. Nobody put the training into the shift schedule, updated the content, charged the device or enabled access.
This pattern is so common that it deserves a name of its own: the pilot that wins the day and loses everyday life. This article accompanies the maintenance team through eight steps, from the right use case to scaling. The team and the plant are fictional, an editorial condensation and not a real customer case. The statements are based on studies and review articles named in the text. Recommendations, metric proposals and time frames are marked as such, they are not standardized industry norms.
At the end stands a question every pilot should answer before it starts: Who runs this training in month seven, when the project team has long since moved on? Whoever answers it in advance no longer judges a pilot by its wow effect alone, but by the conditions for lasting operation.
- A successful pilot initially shows only that one group could complete the exercise in one setting.
- Whether a headset is the right choice depends on the learning goal and not on the device.
- Reaction, learning, transfer and business impact should be measured, each with its own method.
- Regular operation needs names, budgets and a procedure for updates.
- Scaling starts with a second, similar site and not with everyone at once.
The first step clarifies what a pilot actually proves.
What the pilot has really proven
Step 1 of 8 · Classify
In the maintenance area, the team sits in the virtual plant for the first time. It opens a cover, names components and practices the shutdown sequence. Everyone is engaged, the feedback is positive, and the participants ask when it continues. The pilot feels like a success, and it is one, but its statement reaches less far than the impression suggests. It shows that this group could complete this experience in this setting.
That is valuable, but narrower than proof of effect. A good reaction is not the same as lasting learning. A knowledge test right after the training does not show whether someone still masters the sequence weeks later. And a successful headset experience says nothing about whether shift planning, IT, device management and course responsibility work in everyday life.
How large this gap is in research is shown by a systematic review of safety-related VR training. It analyzed 136 studies from 2016 to August 2021. Reaction and learning were measured frequently, behavior at the workplace in not a single study and business results in three.[1] These two levels, however, decide whether a pilot moves into operation.
The first management question is therefore not whether people liked it. It is: Which problem was the training meant to solve, which change in work behavior do we expect, and which data would show that it has occurred?

Between pilot and routine stands a proof that still has to be delivered.
Image: Step 1 of the process, the same exercise in the pilot and in regular operation, with the proof of skill in between[1] | AI-assisted illustration, not a product photo | XR Stager Online Magazine | VISORIC GmbH
The image shows it in three stations. On the left stands the pilot, on the right the routine, and in the middle glows a green check mark labeled Skill Proof. This check mark is the part a pilot alone does not deliver. A pilot shows an experience, the proof of skill comes from operations.
- A pilot initially shows that one group could complete the exercise in one setting.
- A good reaction is not the same as lasting learning.
- A knowledge test right afterwards says little about retention after weeks.
- In 136 studies on safety-related VR training, measuring behavior at the workplace was missing entirely.[1]
- The first question is which change in work behavior is expected.
Whether a headset is the right tool for this change at all is clarified in the next step.
Which task really deserves a headset
Step 2 of 8 · Select
VR is particularly plausible when the learning subject is spatial, procedural or risky: walking through a plant, recognizing a rare fault, practicing a sequence under time pressure, or making decisions in an environment that would be expensive or dangerous in reality. There, the simulation makes situations repeatable and allows mistakes without real follow-up costs.
For pure factual instruction or short standard information, a headset is often a detour. A browser lesson, a video, a real practice device or a supervised demonstration can be more efficient. The medium should result from the learning task and not from the availability of a device.
A randomized study with 95 employees of a biotechnology company in Brazil, Denmark and the USA compared an immersive VR experience with a video presentation of the same content. VR achieved moderate advantages in conceptual knowledge (effect size d = 0.41), spatial knowledge (d = 0.61) and the intention to apply what was learned (d = 0.57). For factual knowledge there was no difference.[2] The study supports a selection based on learning goals. It does not prove an actual change of behavior in operations, because what was measured was intention, not action.
A practical selection rule follows: VR makes the shortlist when spatial understanding, execution of actions or safe repetition are central and the training is needed often enough to justify setup and maintenance. Otherwise, the simpler format should be examined first.

The learning format follows the task, not the device.
Image: Step 2 of the process, VR practice, video and real tool compared by fit to the task[2] | AI-assisted illustration, not a product photo | XR Stager Online Magazine | VISORIC GmbH
- VR suits spatial, procedural and risky tasks.
- For pure facts, video, a browser lesson or a real device are often sufficient.
- In a study with 95 employees, VR outperformed video in conceptual and spatial knowledge.[2]
- For factual knowledge there was no difference.[2]
- The study measures the intention to apply and does not prove a change of behavior.[2]
Once the format is set, the question follows what learners are actually supposed to do in it.
The learning goal comes before the 3D world
Step 3 of 8 · Design
Good VR training starts with an observable action. Should a person correctly secure a plant, detect a deviation or justify a decision? Scene, interaction and feedback arise from this action. Without this chain, what often remains is an impressive environment in which learners see a lot and practice little in a targeted way.
For the maintenance team, the learning goal could read: The technician carries out the five steps of the shutdown in the correct order and recognizes a given deviation. From this, tasks, error situations, hints and evaluation can be derived. Every interaction step needs a function: it directs attention, triggers a decision or reports back an action.
The image shows the loop this rests on. First the person acts, then comes the feedback, then the repetition follows until the action is secure. The feedback can come from an experienced colleague or from the application itself, what matters is that it arrives quickly and concretely.
A systematic review of procedural VR training reports overall positive effects compared with less immersive environments, but points to a heterogeneous body of studies and differences in design and measurement.[3] This argues for clean instructional design and a check in one’s own context, not for a blanket superiority of VR. For the first prototype, one representative procedure, one typical error and one clear feedback are often enough. Only when experts and learners confirm that the exercise reflects the real task in a meaningful way is it worth extending to further variants.

Action, feedback and repetition form the loop in which skill is built.
Image: Step 3 of the process, the learning loop of action, feedback and repetition[3] | AI-assisted illustration, not a product photo | XR Stager Online Magazine | VISORIC GmbH
- At the start stands an observable action and not the 3D scene.
- Scene, interaction, error situations and feedback follow from the action.
- Every interaction step should direct, trigger a decision or report back an action.
- Studies on procedural VR training show overall positive but inconsistent effects.[3]
- A first prototype needs one procedure, one typical error and one clear feedback.
How to find out whether this loop works is the question of the fourth step.
Four levels separate enthusiasm from competence
Step 4 of 8 · Measure
Anyone who wants to prove effect should keep four levels apart, which go back to the model by Donald Kirkpatrick.[4] Reaction asks whether the experience was understandable and acceptable. Learning asks whether knowledge or skill has changed. Behavior, called Transfer in the image, asks whether the action is later carried out correctly at the workplace. Results, called Impact in the image, asks whether a business metric improves, such as error rate, downtime or safe process execution.
Each level requires its own measurement. A satisfaction questionnaire answers no question about work performance. A knowledge test right after the training proves no long-term retention. And a lower error value can have many causes, which is why every statement needs a suitable measurement and, where possible, a comparison with the previous training. The table shows the levels and how often they were measured in research on safety-related VR training.
| Level | Key question | Example of a measurement | Measured in 136 studies |
|---|---|---|---|
| Reaction | Was the experience understandable and acceptable? | Short survey, dropout rate | 90 studies (66 percent) |
| Learning | Has knowledge or skill changed? | Knowledge test, performance test with baseline measurement | 98 studies (72 percent) |
| Transfer (Behavior) | Is the action carried out correctly at the workplace? | Observation, inspection record some weeks later | 0 studies |
| Impact (Results) | Does a business metric improve? | Error rate, downtime, safe process execution | 3 studies (about 2 percent) |
The closer a measurement is to everyday work, the less often it exists.
Table: Four levels of effect according to Kirkpatrick and their frequency in 136 studies on safety-related VR training, 2016 to August 2021[1][4] | XR Stager Online Magazine | VISORIC GmbH
The authors of the review recommend choosing learning measurements to fit the training goal: knowledge tests for information, transfer tests for practical skills. This includes a baseline measurement, if possible a comparison with a control condition, and follow-up tests for retention.[1]
For regular operation, a simple dashboard should contain at least five items: usage rate, completed trainings, reasons for dropout, technical availability and a work-related performance metric. The selection depends on the process. Usage data show that an offering is used, they do not explain whether it works.

Acceptance, learning, transfer and business benefit are four different statements.
Image: Step 4 of the process, four levels of effect from reaction to business benefit[1][4] | AI-assisted illustration, not a product photo | XR Stager Online Magazine | VISORIC GmbH
- Reaction, learning, transfer and impact each need their own measurement.[4]
- In 136 studies, reaction was measured 90 times and learning 98 times.[1]
- Behavior at the workplace did not appear in any study, business results in three.[1]
- Learning measurements should fit the goal and include a baseline and follow-up tests.[1]
- Usage data show reach, not effect.
For such an offering to last in everyday work, someone has to stand behind it. That is covered in the fifth step.
Operations need names
Step 5 of 8 · Assign
Six months after the pilot, the project group has moved on. Nobody knows who charges the headsets, who manages access, when the training goes into the shift schedule or who updates an outdated process version. The problem is rarely the VR technology. It is an operating process that was assigned to nobody.
Before the pilot is approved, there should therefore be a concrete operating model with four roles, as the image shows. The content owner is responsible for the learning goal and currency. The training lead integrates the unit into curriculum and scheduling. IT support checks devices, accounts, network, data protection and the support path. The site operator organizes issue, return, hygiene and charge status. In addition, there is a clear escalation path for failures on the training day.
At every process change there must be a simple way to review, release and version content: a named content owner, a date for the next review and a marking of the valid version. Whether the solution is organized centrally, by site or with an external operator is secondary. What matters is that responsibility does not end with the project plan.
An overview article on the implementation of VR in healthcare names acceptance, available resources and a systematic implementation process as central conditions.[6] The work refers to healthcare and cannot automatically be transferred one to one to every industry, but it is still useful as orientation. In Germany, a practical point comes on top: if devices or software record data about employees’ behavior or performance, the co-determination of the works council under Section 87(1) no. 6 of the German Works Constitution Act may apply.[7] This should be clarified with the works council and data protection before the start. The note is not legal advice.

A training operation consists of four roles, and each needs a name.
Image: Step 5 of the process, four operating roles from content responsibility to site operation[6][7] | AI-assisted illustration, not a product photo | XR Stager Online Magazine | VISORIC GmbH
- The operating model is in place before the pilot is approved and not afterwards.
- Four roles carry the operation: content, training, IT and site.
- Content needs an owner, a review date and a marked valid version.
- Acceptance, resources and a systematic implementation process are considered central conditions.[6]
- The co-determination of the works council may apply to data-recording systems.[7]
With the roles in place, the costs that continue after the pilot are also settled. They are the topic of the sixth step.
The costs continue after the pilot
Step 6 of 8 · Calculate
A robust business case looks at the total costs over the period of use. Besides concept and development, devices, accessories, replacement, software operation, device management, content maintenance, support, onboarding, cleaning, site logistics and downtime belong in the calculation. With distributed sites, transport, network conditions and local support are added. The image summarizes it in four items: devices, support, content care and, at the end, the total costs.
The previous training must also be recorded fairly: trainer time, travel, plant downtime, consumables, repetitions and possible risks. Both calculations should cover the same period and the same target group. A one-time pilot price is not a sensible comparison to a learning offering that is used permanently.
A fictional numerical example shows what matters. Assume a learning offering causes total costs of 90,000 euros over three years, and 180 people go through it. If 150 of them pass the transfer test, the costs are 600 euros per qualified person. If only 100 pass, it is 900 euros. The figures are freely chosen and not market values, they illustrate the yardstick: the relevant comparison is not whether VR is cheaper than development. It is which total costs arise per demonstrably qualified person.
The calculation also includes the risks: data processing, accounts, security updates, accessibility, hygiene and the question of what happens when a device fails. A meta-analysis of 55 studies with 3,016 participants found increased symptoms of so-called VR sickness on average for headset applications, with differences depending on content, type of movement and duration of use.[9] For some groups, an alternative format is therefore needed. A good operating design leaves nobody without a learning path just because one person cannot or does not want to use the headset.
A review of 201 studies on VR training in organizations names, among other barriers, high costs for infrastructure and content development, required specialist knowledge, simulator sickness and quickly aging hardware. At the same time, it stresses the need for research on long-term effects.[8] Expected savings should therefore be stated as hypotheses until your own data support them.

Care and time come on top of the device, and only everything together makes the total cost.
Image: Step 6 of the process, devices, support and content care as building blocks of total costs[8][9] | AI-assisted illustration, not a product photo | XR Stager Online Magazine | VISORIC GmbH
- The calculation covers the entire period of use, not only development and pilot.
- The previous training is recorded with the same criteria, the same period and the same target group.
- The yardstick is total cost per demonstrably qualified person.
- Headset use can cause discomfort, so an alternative format is needed.[9]
- Expected savings count as hypotheses until your own data support them.[8]
From roles and costs, a pilot can be built that really answers these questions. The seventh step shows how.
Piloting like an operational trial
Step 7 of 8 · Test
A pilot that shows only technical feasibility does not answer the question of introduction. That needs a small operational trial: a real target group, a realistic process, defined responsible persons and measurements before and after the training. Where possible, it is compared with the existing format, with the same learning goal and comparable practice time. In the image, this is the second path: next to the VR exercise stands a colleague practicing the same task in the previous form on a real housing.
A practical evaluation plan first measures the starting point, then the performance directly after the training and the retention after a set period. For procedural tasks, sequence errors, skipped inspection steps, processing time and required help can be recorded. A transfer test should contain a real or realistic task. Usage logs add availability, repetitions and dropouts.
Why the waiting period matters is shown by a classic of memory research. A replication of the Ebbinghaus forgetting curve had one subject learn lists of nonsense syllables and relearn them after 20 minutes up to 31 days. After 20 minutes, he still saved about 47 percent of the learning time when relearning, after one day about 32 percent, after 31 days only about 4 percent.[5] That was a single subject with syllables, no proof for maintenance procedures, but a hint why a test directly after the training can turn out too optimistic.
Before the start, decision thresholds are set: what counts as sufficient, which problems lead to revision, when does it end? These thresholds are not general norm values, they result from risk, learning goal and operating requirement. The sample should also fit the decision, because a small pilot can uncover problems but rarely estimate rare effects precisely. As an editorial proposal, a review period of four to eight weeks for first transfer and operational observations is suitable, provided the training rhythm allows it. For rare events or long-term metrics, a longer observation is needed. This is a design proposal and not a scientifically validated period.

A decision becomes robust when it is measured before and after.
Image: Step 7 of the process, baseline measurement, two practice formats, follow-up measurement and decision[1][5] | AI-assisted illustration, not a product photo | XR Stager Online Magazine | VISORIC GmbH
- An operational trial uses a real target group, a realistic process and named responsible persons.
- The starting point, the performance directly afterwards and retention after a fixed period are measured.[1]
- Comparison with the previous format requires the same learning goal and similar practice time.
- Decision thresholds for stop, revision and extension are fixed before the start.
- Four to eight weeks is an editorial proposal, not a verified norm.
If the operational trial shows effect and feasibility, the question of extension arises. The eighth step answers it.
Scaling starts with the next site
Step 8 of 8 · Scale
If the operational trial shows effect and feasibility, scaling does not follow immediately at all sites. First comes a second, preferably similar area of use. There it becomes clear which parts are reusable and which depend on plant, language, safety rules or shift model. In the image, this is the Site 2 station, between Pilot and Scale.
Often reusable are learning logic, interaction patterns, evaluation model, device standard and support process. Site-specific usually remain plant models, process variants, language, technical approvals and local IT requirements. In the image, the reusable building blocks lie as a stack in the middle and feed all three stations. A modular structure with traceable versions makes changes visible and prevents outdated content from running on unnoticed.
For roll-out, a staged decision path is advisable: first one stable use case, then a further site, then a portfolio of similar trainings. At each stage, usage, transfer, support effort and total costs are checked again. Where a headset offers no sufficient added value, a browser or tablet variant can complement access. Such alternatives should preserve the learning goal and not deliver a stripped-down copy. A review explicitly names standardized programs across several sites as an opportunity of VR training.[8]
Success is reached when the training fits into the normal working rhythm, is reliably supported and its content stays current. A compact checklist for approval serves as the conclusion.
| Area to check | Key question before approval |
|---|---|
| Learning goal | Which concrete action or decision should become safer, faster or more reliable? |
| Medium fit | What added value does VR offer compared with video, browser, real device or classroom training? |
| Operations | Who is responsible for curriculum, devices, support, IT approval and content updates? |
| Effect | Which baseline, learning, transfer and operational data are collected, and when? |
| Cost-effectiveness | What total costs arise per qualified person over the planned period of use? |
| Scaling | Which criteria lead to stop, revision, second site or broad roll-out? |
Six questions are enough to judge a pilot by its viability and not by its wow effect.
Table: Checklist for approving a VR training, editorial proposal based on steps 1 to 8 | XR Stager Online Magazine | VISORIC GmbH
The yardstick for VR training is therefore not enthusiasm in the pilot, but a professionally justified learning advantage that is translated into an operable process and checked in the work context. The operations question belongs before the pilot and not only where the project team has moved on.

A shared toolkit of learning logic and modules carries every further site.
Image: Step 8 of the process, from the pilot via a second site to scaling with reusable modules[8] | AI-assisted illustration, not a product photo | XR Stager Online Magazine | VISORIC GmbH
- The pilot is followed by a second, preferably similar site and not by a broad roll-out.
- Often reusable are learning logic, interaction patterns, evaluation model, device standard and support process.
- Site-specific usually remain plant models, language, technical approvals and local IT requirements.
- At each stage, usage, transfer, support effort and total costs are checked anew.
- Browser or tablet variants complement access as long as they preserve the learning goal.
How you can place your own project is shown by the following check.
How ready is your VR project? The practice check
The eight steps can be applied to your own project. The VR Practice Check asks 16 short questions, two for each step. Each question is a practical situation asking how you would proceed. Several answers are plausible, the most sound one counts most. After each answer you get a short note. At the end, the evaluation shows your overall score, strengths and gaps in the eight areas and the two steps that pay off next.
The check is a judgment test based on this article and not a scientifically validated test. No data is stored. It takes about five minutes and works on a computer, tablet and smartphone.
Interactive check: VR Practice Check | Concept, research and analysis: Ulrich Buckenlei | Implementation: VISORIC GmbH | Basis: studies and review articles, see sources | XR Stager Online Magazine | VISORIC GmbH
- The check has 16 questions in eight areas, matching the eight steps of the article.
- Each question is a practical situation in which you have to weigh how you would proceed.
- After each answer, a short note puts the trade-off and its limits into context.
- The evaluation shows the overall score, the strengths and the two areas with the biggest leverage.
- The result can be discussed by email, nothing is stored.
Between a successful pilot and a lasting learning offering lies an operating plan, and it starts with a clearly defined first step.
From pilot to a learning offering that stays in everyday work
Somewhere in every company there is a headset in a cupboard that once inspired everyone. With the right preparation, this does not have to happen: a VR training stays in everyday work when three things come together. It needs a learning goal that can be measured on an observable action, an operation with named responsible persons, and a test that checks learning and transfer before the decision on extension is made.
The expert team of VISORIC GmbH from Munich combines over 15 years of experience in 3D, AI and XR with practical experience in spatial computing, real-time 3D and digital twins. VISORIC develops training and learning environments for technical processes and accompanies companies from the first idea to a working application. Together with you, the team clarifies which use case suits the start, how operations are organized and how success can be measured.[10][11]

15 years of experience in 3D, AI and XR: the VISORIC expert team from Munich.
Image: © Ulrich Buckenlei | XR Stager Online Magazine | VISORIC GmbH
A well-conceived pilot project with a single use case and a clearly defined learning environment can often be realized much faster and more cost-effectively than many companies expect. The start consists of three steps:
- Sharpen use case and learning goal, including the question whether VR adds value over video or a real device.
- Set up the pilot as an operational trial, with named roles, a measurement plan and decision thresholds.
- Plan the path to scaling in stages, from the second site to a training portfolio.
This is exactly the starting point for a conversation: not the big, company-wide vision, but a clearly defined first step that can be implemented quickly.
Who runs this training in your company in the seventh month, when the project team has long since moved on?
Talk to the VISORIC expert team from Munich about spatial computing, digital twins and XR training. Together we will turn your use case into a precise, traceable next step.
Contact:
Email: info@visoric.com
Phone: +49 89 21552678
Sources and References
- Stefan, Mortimer, Horan. Evaluating the effectiveness of virtual reality for safety-relevant training, a systematic review. Virtual Reality 27, 2023, pp. 2839 to 2869. DOI 10.1007/s10055-023-00843-7.
- Baceviciute et al. Investigating the value of immersive virtual reality tools for organizational training, an applied international study in the biotech industry. Journal of Computer Assisted Learning 38(2), 2022, pp. 470 to 487. DOI 10.1111/jcal.12630.
- Jongbloed, Chaker, Lavoué. Immersive procedural training in virtual reality, a systematic literature review. Computers and Education, 2024. DOI 10.1016/j.compedu.2024.105124.
- Kirkpatrick and Kirkpatrick. Evaluating Training Programs, The Four Levels. 3rd edition, Berrett-Koehler, 2006.
- Murre and Dros. Replication and Analysis of Ebbinghaus’ Forgetting Curve. PLOS ONE 10(7), 2015, e0120644.
- Kouijzer et al. Implementation of virtual reality in healthcare, a scoping review on the implementation process of virtual reality in various healthcare settings. Implementation Science Communications 4, 2023, article 67. DOI 10.1186/s43058-023-00442-2.
- Betriebsverfassungsgesetz (German Works Constitution Act), Section 87(1) no. 6, co-determination on technical devices for monitoring behavior or performance. gesetze-im-internet.de/betrvg/__87.html.
- Carvalho, Conceição, Marques. The Impact of Virtual Reality on Employee Training and Learning in Organisations, A Systematic Literature Review. Applied Sciences 15(19), 2025, 10459. DOI 10.3390/app151910459.
- Saredakis et al. Factors Associated With Virtual Reality Sickness in Head-Mounted Displays, A Systematic Review and Meta-Analysis. Frontiers in Human Neuroscience 14, 2020, article 96. DOI 10.3389/fnhum.2020.00096.
- VISORIC practice projects in the areas of digital twins, real-time 3D and spatial computing.
- XR Stager platform for real-time 3D, digital twins, knowledge AI and industrial spatial computing applications.
Contact Persons:
Ulrich Buckenlei (Creative Director)
Mobile: +49 152 53532871
Email: ulrich.buckenlei@visoric.com
Nataliya Daniltseva (Project Manager)
Mobile: +49 176 72805705
Email: nataliya.daniltseva@visoric.com
Address:
VISORIC GmbH
Bayerstraße 13
D-80335 Munich