The Engagement Report - Detailed Information
This article is intended for instructors only.
This article is intended for instructors only.
Metrics Used to Calculate Engagement
Metric | Description |
Time Spent on Site (Minutes) | Measures the total time spent by a participant in the simulation tile only, excluding the practice zone. This metric reflects a participant's commitment to engaging with the simulation. |
Saved Decisions | The number of decisions saved by a participant, indicating their level of engagement with the simulation. |
Reports Consulted | The number of end-of-year reports read by a participant, demonstrating their interest in reviewing their progress. |
Indicators Consulted | Students who regularly check indicators show an interest in understanding the effects of their decisions on the simulation. |
Podiums Consulted | The more podiums a participant consults, the more they are engaging with the competitive aspect of the simulation. |
All metrics except "Time Spent on Site" count the number of individual
actions taken. For example, Saved Decisions might be 10 out
of a possible total of 12. This means that converting these metrics to a
percentage is a straightforward calculation (10/12 * 100). Calculating a percentage for "Time Spent on Site" is going to be a bit more involved.
Converting Time Spent on Site
We'll compare each participant's value to the values of all other participants in the same group and year of simulation. This will help us understand where each participant falls within the broader group. To convert each participant's time spent on site into a percentage, we'll use a special method to handle participants who spend an unusually long or short amount of time compared to their peers.
This approach can help to:
- Reduce subjectivity in scoring
- Increase fairness and equity among participants
- Provide a more nuanced understanding of participant performance
Step 1: Find the first and third quartiles (Q1 and Q3)
From this comparison, we'll find two key values:
- The first quartile (Q1) is the value below which 25% of participants fall.
- The third quartile (Q3) is the value above which 25% of participants fall.
Step 2: Find the interquartile range (IQR)
Next, we'll find the difference between Q3 and Q1. This will give us the IQR, which helps us understand the spread of values within the group.
Step 3: Determine the higher limit
Now, we'll add 1.5 times the IQR to Q3 ((Q3 + (1.5 * IQR)). This will give us a "higher limit" value that marks the boundary beyond which participants are considered outliers.
Based on where their value falls relative to these points, we have four options:
- If the participants spend more time than the higher limit, they'll be assigned 100% (since they're significantly above the rest of the group).
- If their value is between Q3 and the higher limit, we'll scale their result to a percentage between 85% and 100%.
- If their value falls within the interquartile range (Q1 to Q3), we'll scale their result to a percentage between 75% and 85%.
- If their value is lower than or equal to Q1, we'll scale their result to a percentage between 0% and 75%.
Weighting of Engagement Metrics - Final Score
The engagement metrics are weighted according to their relative importance in assessing participant engagement. The weights assigned to each metric can be adjusted as needed to reflect changes in the assessment criteria.
For example, you may decide that:
- Time Spent on Site is worth 40% of the total score
- Saved Decisions are worth 30%
- Reports Consulted is worth 20%
- Indicators Consulted is worth 5%
- Advice Read is worth 3%
- Podiums Consulted is worth 2%
Note that it is also possible to remove one or more of these metrics from the report if they are deemed unnecessary or irrelevant.
The final Engagement Score will be calculated by adding the weighted value of all selected metrics.
Related Articles
How to use the Engagement tool
This article is intended for instructors. This tool applies to only our Kayak, Golf, and Videogame simulations. It is possible to create a custom engagement report for a simulation. This report provides a detailed view of participant interaction in ...Instructor tools - How it works - Step-by-step Guide
This article is intended for instructors. In the Instructor Zone of your Praxar portal, you can access the following Instructor tools at the left of your screen. The objective of this article is to clarify: When to use a specific tool? Why use a ...Financial accounting: Using the Report tool to grade student results
This article is intended for instructors. This article concerns only the 3 components of the Financial Accounting simulation: financial statements, financial analysis and budgeting. Regardless of which component(s) you're working on, the reporting ...The Grading Report: How grades are calculated
This article is intended for instructors only. Calculating Grades To calculate grades, we follow these 4 steps: Step 1 - Converting Min/Max Fields to Percentage For Min/Max fields (identified by the presence of a "Minimum Score" option), we need to ...Creating Report Templates
This article is intended for instructors. The Report generator within the Instructor Zone is a highly useful tool to allow the customization of reports for our simulations. Using report templates to save the main parameters of a report can help speed ...