SP onboarding process:
- Set up user - Registration
- Do training - watch videos (SP Training)
- SP performance review - evaluation
- Observation - SPPA - if necessary
How to register your SPs with demographics? (USER MANAGER)
When the role is set to SP or the privileges include SP, three additional settings surfaces are displayed:
Personal Data settings: for entering customized data specific to and necessary for management of SP users.
WHEN CLOSING AND REOPENING USER ACCOUNT/EDIT USER, the two additional features reveal themselves as well:
Physical & Medical History settings: for registering medical history and conditions of the SP user.
Activity settings: for recording activities (with specific details) that the SP user has participated in.
Conditions listed here are real conditions of the SP Users from his/her own medical history.
SP SPECIFIC SEARCH
You can search for SP-s in the LS database in a more focused way based on the above added personal and physical data. For this, you can use the Batch User Create (BUC) feature of LS, by exporting an Excel file of a specific SP group which includes all above entered data in separate columns. (User manager > Batch User Create tab > Step 1 > Download template > Select SP group from dropdown > Download)
Another way to search for specified SP-s is using the “Search users, groups” field of User manager, by pre-selecting “SP” in the respective dropdown menu and selecting the checkboxes for the desired SP sections under “Which SP Sections?”:
Searches among ALL USERS!
**************************************************************************************************************************
2.Which section of LS can be used to build SP evaluation tools (with templates you can customize to a limited degree)? (CASE MANAGER)
(SP Part – SP’s evaluation on learner’s performance)
SP Training part: (INDEPENDENT FROM SPPA!) - serves for self-training/self-evaluation.
This part serves as practice for SPs, assessing other SPs' performance. It provides means for analyzing and measuring how standardized patients perform compared to other (more experienced) SPs. By using this tool, SPs can easily study web-based training material, view sample video streams, watch “gold standards” performances, practice checklist completion and check understanding by completing checklist and comparing their answers to others. You can save time, get equal or better results and provide self-directed learning for your SPs.
The SP Training results can be found in the SP Training Review on the Test Activity Management tab in the Reports.
General criteria for setting up and using the SP Training Part:
1) The activity used for SP Training has to be a running activity.
2) The case the SP Training is set up has to be assigned to the activity.
3) There must be a Learner's completed SP part (checklist) and a video recording for the case.
4) SP doing the SP Training has to be member of the activity's SP group.
Setting up the SP Training part in the Case Manager:
With clicking SP Training part on the Select Part page, you will find 3 types of sections: Practice, Evaluation and Materials (these sections are different from the regular SP part sections). The order of these sections is strict: a Practice section has to be set up before an Evaluation section, placing the Materials section is flexible. The SP Training part must have a Practice and an Evaluation sections, optionally it can have a Materials section.
1)Setting up the Practice section:
On the Practice section page, you can select the encounter for the Trainee.
From the 'Activity' drop-down select the activity the Trainee will do in the SP Training (all those activities are listed in this drop-down which include the current case).
Then you can select a learner from the 'Learner' drop-down (this drop-down includes only those learners who are members of the related activities and have a video recording for the case. After saving this page you can set up the Evaluation section.
2) Setting up the Evaluation section:
The Evaluation section shows the results of the Trainee for the selected encounter compared to this 'control' section. This evaluation is displayed for the SP Trainee upon submitting the Practice section checklist. On the Evaluation section page, you can select from the 'Selected section' drop-down the Practice section you have set up before. Then you can save this page.
3) Setting up the Materials section:
The Materials section is optional, you can use it for giving some instruction or additional information for the Trainee. In this section you can select only two types of items: Multimedia and/or Message Field.
With selecting the Message Field item, you can provide any textual information. With selecting Multimedia item, you can provide further information for the Trainee.
NOTE! SP Training is available when the activity is upcoming or running:
Upcoming activity - normal SP Data Entry is not available, hence the SP Training will be accessible and displayed under Current Activities in the Your Activities panel.
Running activity - SP Training access can be controlled via the 'Data entry only allowed from manually authorized workstations' setting in the Activities module. That is, when an SP logs in from an unauthorized computer, he/she will be able to access the SP Training and not the SP Data Entry. When an SP logs in from an authorized computer, he/she will be able to access the SP Data Entry.
SP Training results are displayed in the SP Training Review among the Activity Management reports. (Reports & Results > Test Activity Management > SP Training Review)
Calculating the score for the Trainee is based on the original case and Activity. When the SP Trainee (the SP doing an SP Training) submits responses for the checklist, his or her answers are compared to those of the SP originally filling in the SP part for the case. When an SP selects an Activity. For doing a training part, his or her submitted answers are always saved to the database of the selected event.
SP Performance Assessment (SPPA)
This checklist serves as an evaluation of SP performance. Users with Observer privilege can fill this part out and the results affect the SP performance score. Only users with Observer privilege can do SP Performance Assessment (SPPA), and only if they are members of the Faculty group of the activity.
There are two methods of SP assessment in CAE LearningSpace:
a.Filling out the same checklist the SP has completed, then comparing the Observer's and the SP's answers, thus getting a difference value for the SP within a case.
Note: Only those SP checklists will be available for observation that have been checked as "Observed" in the Section editor of each SP section.
Note 2: This part is obligatory if SPPA is being performed. The second method can provide extra information only.
b.Filling out the SP Performance Assessment case part of an activity. Note: This part is NOT OBLIGATORY!
OBSERVERS can access SPPA module via Classic view > Data entry tools > SPPA Live Observation OR SPPA Post-Enc. Observation
HERE IT IS IMPORTANT that the Observer is added in the FACULTY GROUP of the activity, (as it is faculty privilege) for being able to review SP-s videos (recording or live).
3. Which are the reports regarding SP grading (e.g. to see if any big discrepancies)?
SP Performance Assessment report
The SP Performance Assessment Report is used to evaluate the performance of each SP for each assigned encounter and case. It shows a table listing all the cases of the activity with separate lines for each SP who participated in a certain case. Each row shows the main SP-evaluating scores and there are also tools to review the observation for each encounter observed.
Inter-rater Agreement: The accordance of answers given by the SP and the observer for the same SP checklist expressed in percentage. (this value is the absolute value of the difference between the value of the answer
selected by the SP and the one selected by the Observer. If for an item, they both select the answer with the same value then the difference value for this item is 0. The difference value for an item is zero even if they selected different answers if those answers have the same point value. This is because the difference does not make any difference in the score of the learner: it would be the same if the SP did exactly as the Observer. (Of course, if the selection of the answer for an item is important and student scores depend on the SP’s choice, the answers should have different point values.)
The difference value calculation is only using responses to items with pre-defined answers, responses to open-ended items like text boxes are NOT used, not even if they are scored.
more on that here: https://help.caelearningspace.com/cf/display/EntHS/About+SP+Performance+Assessment
and here: https://help.caelearningspace.com/cf/display/EntHS/SP+Performance+Assessment+checklist
SP Performance Evaluation: The result of the SPPA checklist, if any, expressed in percentage. It refers to one certain encounter and it is calculated very similarly to the learner scores, but here we are scoring the SP, not the learner. The SP gets as many points for each section of a certain part of the case as the answer selected by the observer is worth. Then the system calculates the percentage score for each section of this part and get the SP Performance Evaluation value for this observation by calculating the weighted average of the section percentage scores using the section weights, as set in the Case Manager.
Total SP Performance Score: The mean of the Inter-rater Agreement and the SP Performance Evaluation results. (the average of the inter-rater agreement value and the SP performance evaluation value.)
Section agreement value: This percentage value refers to one section of the SP part, and is the difference of the total achievable non-open-ended point value in the section and the difference value, divided by the total achievable point value.
SP PERFORMANCE REVIEW
Shows the results of the SP checklists, by listing all the participating SP-s along with the number of learners they have evaluated (in brackets) and the scores they have given to each section of the SP checklist (abbreviations in the headings of the respective columns) expressed in percentages, and their Average value.
Name: (115) – for how many learners the SP has submitted Data entry; SP part sections and averages.
What is it used for: if the average score is remarkably different (shows big discrepancies) it might be due to the SP being less efficient, having a bad day or would need more training. It is NOT connected to SP Performance Assessment.
Would be handy for checking which SP-s did a specific case – but it only shows records for a specific Activity.
4. Which report shows how many times a specific SP has worked with a specific student?
- SP activity feature of User manager can be used for this.
- “SP-s who did this “- feature of Case manager
(No specific report can be run in the Reports module of LS.)
SP TRAINING: Set Data entry only allowed from manually authorized workstations and log in from a non-authorized workstation to access this:
AFTER filling in the same checklist the SP did (SP part of the case), Evaluation appears:
SP Training report shows the same Agreement value:
*********************************************************************************************
BEFORE AN OBSERVATION:
AFTER AN OBSERVATION:
LIVE OBSERVATION:
REPORTS SHOWING RESULT OF OBSERVATION:
SP Observation Comparison (SPPA Report > Observations dropdown)
Answers of SP (SP) vs that of Observer (O). If different – difference value is added ‘1’.
In the SP part those items are highlighted where the answers selected by the SP and the observer are different. (The items of the SPPA part is filled in only by observers, so here you will not see "differences", but the answers of the observer.)
Overview
Content Tools