IMPORTANT: Only users with Observer privilege can do SP Performance Assessment (SPPA), and only if they are members of the Faculty group of the activity.
There are two methods of SP assessment in CAE LearningSpace:
a. Filling out the same checklist the SP has completed, then comparing the Observer's and the SP's answers, thus getting a difference value for the SP within a case.
NOTE: Only those SP checklists will be available for observation that have been checked as "Observed" in the Section editor of each SP section.
IMPORTANT: There has to be at least one SP section marked as "Observed" for a valid SP evaluation (i.e. observed SP checklist and SPPA part).
Section editor (top of the section panel)
b. Filling out the SP Performance Assessment case part of an activity.
Setting Up an SP Performance Assessment Part
- Within the Cases module, open the Edit Case window for the case you intend to use.
- Go to the Case Content page.
- Click the SP Performance Assessment tab.
TIP: Click the arrow on the right end of the tabs to have the SPPA tab displayed.
Tabs on the Case Content page
4. Create new or reuse already existing sections to put together the SPPA part as described in How to Create a Section and How to Reuse Sections.
NOTE: When evaluating SP performance, the checklist can be filled out while watching the learner-SP encounter live (running activities) or after the end of the activity (post-encounter observation).
Setting Up Data Entry Tools for SPPA
- Switch to Classic view.
- Expand the Data Entry Tools tab.
Data Entry Tools tab in Classic view
3. On the SP Performance Assessment - Live Observation panel, select a running activity from the drop-down menu, click Select, then narrow it down by using the Room, Activity, Case and Learner drop-downs.
SP Performance Assessment - Live Observation page
4. On the SP Performance Assessment - Post-Encounter Observations panel: select an activity from the Activity drop-down and click Select to access the Post-Encounter Observation page. Clicking Go next to the encounter intended to be observed will redirect to a page where the video recording of the encounter can be watched and the SP and/ or SPPA checklists, if any, can be filled out.
SP Performance Assessment - Post-Encounter Observation page
Setting Up SP Performance Assessment
IMPORTANT: When making a post-encounter observation, the system gives a randomly generated list of encounters that should be observed. These encounters are selected so that together with currently existing observations, they will satisfy the Observation Requirement, which refers to the number of encounters that should be observed.
- On the Preset Reports tab of the Reports module, select the activity you want to work with.
- Click the Activity Reporting Setup link at the bottom of the tab.
NOTE: The link only appears when an activity has been selected.
3. Click the SP Performance Assessment tab.
The SP Performance Assessment panel expands. SP Performance Assessment panel
4. Select the desired preferences from the encounter rate radio button options and specify the percentage/ number via the drop-down menu:
- % from all encounters - the required number of observations calculated based on all the encounters
- % from the encounters of each SP - the required number of observations calculated based on the number of encounters each SP
- encounter(s) per SP per day (event) - the required number of observations equal to the number of encounters each SP has a day/ event
5. To determine the threshold for when SPs are marked, enter percentage values in the threshold percentage fields.
NOTE: If the Total SP Performance Score or the lowest per-encounter SP Performance Score of an SP is lower than the percentage set here, the cell of the affected score will be highlighted in yellow.
6. On the show/ hide score & learner options, select whether section agreement scores and learner names would be visible.
Accessing SP Performance Assessment Report
IMPORTANT: The SP Performance Assessment Report is used to evaluate the performance of each SP for each assigned encounter and case. It shows a table listing all the cases of the activity with separate lines for each SP who participated in a certain case. Each row shows the main SP-evaluating scores and there are also tools to review the observation for each encounter observed.
- On the Preset Reports tab of the Reports module, select the activity you want to work with.
NOTE: The SP Performance Assessment report is only available when the activity has already been selected
2. Click the SP Performance Assessment button.
The SP Performance Assessment Report page opens. SP Performance Assessment Report
Inter-rater Agreement: The accordance of answers given by the SP and the observer for the same SP checklist expressed in percentage.
SP Performance Evaluation: The result of the SPPA checklist, if any, expressed in percentage.
Total SP Performance Score: The mean of the Inter-rater Agreement and the SP Performance Evaluation results.
3. When selecting an observation from the Observations drop-down, the SP Observation Comparison screen appears, where any difference between SP's and the Observer's answers is highlighted in yellow.
Overview
Content Tools