SP Related Features in LearningSpace

SP onboarding process

1. Registration of the SP user in User Manager 

2. Training - Practice tool: watching videos and SP assessment - SP Training

3. Observation - SP Performance Assessment (SPPA)

4. Evaluation - SP Performance Review report

In the User Manager, when the role is set to SP or the Privileges include SP, three additional settings surfaces are displayed (See also Additional Settings for SPs):

1. Personal Details settings: for entering customized data specific to and necessary for the management of SP users.
   
   

   When closing and reopening User account/Edit user window, two additional features reveal themselves as well.

2. Physical & Medical History settings: for registering the medical history and conditions of the SP user. 
   The conditions listed here are real conditions of the SP users from their own medical history.
   
   

3. Activity settings: for recording Activities (with specific details) the SP user has participated in.
   
   

SP Specific Search

A specific way to search for specified SPs is using the Search users, groups field of User manager, by pre-selecting “SP” in the respective dropdown menu and selecting the checkboxes for the desired SP sections under Which SP sections?:

You can search for SPs in the LearningSpace database in a more focused way based on the above added Personal and Physical data.
For uploading this data, you can use the Batch User Create (BUC) feature of LearningSpace, by exporting an Excel file of a specific SP group which includes all above entered data in separate columns. (User manager > Batch User Create tab > Step 1 > Download template > Select SP group from dropdown > Download)

SP Evaluation

The SP checklist (SP Part) you can set up in the Case Manager serves for the SP’s evaluation of the Learners' performance. To evaluate SPs you can use the following tools (with templates you can customize to a limited degree) of LearningSpace:

 SP Training part

This part serves for self-training/self-evaluation, as a practice for SPs, assessing other SPs' performance. It provides means for analyzing and measuring how standardized patients perform compared to other (more experienced) SPs. By using this tool, SPs can easily study web-based training material, view sample video streams, watch “gold standards” performances, practice checklist completion and check understanding by completing the checklist and comparing their answers to others. You can save time, get equal or better results and provide self-directed learning for your SPs.

General criteria for setting up and using the SP Training Part

1) The Activity used for SP Training has to be a running or upcoming Activity.

2) The Case the SP Training is set up has to be assigned to the Activity.

3) There must be a completed SP part (checklist) and a video recording for the Case.

4) SP doing the SP Training has to be a member of the Activity's SP group.

Setting up the SP Training part in the Case Manager

 By selecting SP Training tab from the Case part, you will find 3 types of sections: Practice, Evaluation and Materials (these sections are different from the regular SP part sections). The order of these sections is strict: a Practice section has to be set up before an Evaluation section, and placing the Materials section is flexible. The SP Training part must have a Practice and an Evaluation section, optionally it can have a Materials section.

Setting up the Practice section

1. On the Practice section page, you can select the encounter for the Trainee.

2. From the 'Activity' drop-down select the Activity the Trainee will do in the SP Training (all those activities are listed in this drop-down which includes the current Case).

3. Then you can select a learner from the 'Learner' drop-down (this drop-down includes only those learners who are members of the related activities and have a video recording for the Case. After saving this page you can set up the Evaluation section.     

Setting up the Evaluation section

The Evaluation section shows the results of the Trainee for the selected encounter compared to this 'control' section. This evaluation is displayed for the SP Trainee upon submitting the Practice section checklist. On the Evaluation section page, you can select from the 'Selected section' drop-down the Practice section you have set up before. 

Setting up the Materials section

The Materials section is optional, you can use it for giving some instruction or additional information to the Trainee. In this section, you can select only Note item (or  Multimedia and/or Message Field in the Classic Case editor).

By selecting the Message Field item, you can provide any textual information. By selecting the Multimedia item, you can provide further information for the Trainee.

1. Upcoming Activity - normal SP Data Entry is not available, hence the SP Training will be accessible and displayed under Current Activities in the Your Activities panel.
   
   
2. Running Activity - SP Training access can be controlled via the 'Data entry only allowed from manually authorized workstations' setting in the Activities module.
   1. When an SP logs in from an unauthorized computer, they will be able to access the SP Training and not the SP Data Entry.
   2. When an SP logs in from an authorized computer, they will be able to access the SP Data Entry.

Reports

The SP Training results can be found in the SP Training Review on the Test Activity Management tab in the Reports.
 

Calculating the score for the Trainee is based on the original Case and Activity. When the SP Trainee (the SP doing the SP Training) submits responses for the checklist, their answers are compared to those of the SP originally filling in the SP part for the Case. When an SP selects an Activity for doing a training part, their submitted answers are always saved to the database of the selected event.

SP Performance Assessment (SPPA)

This checklist serves as an evaluation of SP performance. 
Users with Observer privilege can fill this part out and the results affect the SP performance score. Only users with Observer privilege can do SP Performance Assessment (SPPA), and only if they are members of the Faculty group of the Activity.

There are two methods of SP assessment in LearningSpace:

 a. Filling out the same checklist the SP has completed, then comparing the Observer's and the SP's answers, thus getting a difference value for the SP within a Case.

 b. Filling out the SP Performance Assessment Case part of an Activity, in addition to the SP checklist. 

Another important condition is to have the correct settings adjusted on the Acitvity Reporting setup page of the respective Activity. Here, the SP Performance Assessment table serves for setting how the required number of observations should be calculated and what the SPPA report should look like.

Observers can access the SPPA module via Classic view > Data entry tools > SPPA Live Observation or SPPA Post-Encounter Observation.

   SP Performance Assessment report

The SP Performance Assessment Report is used to evaluate the performance of each SP for each assigned encounter and Case. It is accessible under Prest Reports, Test Activity Management module.

It shows a table listing all the Cases of the Activity with separate lines for each SP who participated in a certain Case. Each row shows the main SP-evaluating scores and there are also tools to review the observation for each encounter observed.

Inter-rater Agreement

The accordance of answers given by the SP and the observer for the same SP checklist expressed in percentage. (this value is the absolute value of the difference between the value of the answer selected by the SP and the one selected by the Observer. If for an item, they both select the answer with the same value then the difference value for this item is 0. The difference value for an item is zero even if they selected different answers if those answers have the same point value. This is because the difference does not make any difference in the score of the learner: it would be the same if the SP did exactly as the Observer. (Of course, if the selection of the answer for an item is important and student scores depend on the SP’s choice, the answers should have different point values.)

The difference value calculation is only using responses to items with pre-defined answers, responses to open-ended items like text boxes are NOT used, not even if they are scored.

SP Performance Evaluation: The result of the SPPA checklist, if any, expressed in percentage. It refers to one certain encounter and it is calculated very similarly to the learner scores, but here we are scoring the SP, not the learner. The SP gets as many points for each section of a certain part of the Case as the answer selected by the observer is worth. Then the system calculates the percentage score for each section of this part and get the SP Performance Evaluation value for this observation by calculating the weighted average of the section percentage scores using the section weights, as set in the Case Manager.

Total SP Performance Score: The mean of the Inter-rater Agreement and the SP Performance Evaluation results. (the average of the inter-rater agreement value and the SP performance evaluation value.)

Section agreement Value: This percentage value refers to one section of the SP part, and is the difference of the total achievable non-open-ended point value in the section and the difference value, divided by the total achievable point value.   

SP Performance Review

Shows the results of the SP checklists, by listing all the participating SP-s along with the number of learners they have evaluated (in brackets) and the scores they have given to each section of the SP checklist (abbreviations in the headings of the respective columns) expressed in percentages, and their Average value.

Name: (115) – for how many learners the SP has submitted Data entry; SP part sections and averages.

What is it used for: if the average score is remarkably different (shows big discrepancies) it might be due to the SP being less efficient, having a bad day or would need more training. It is NOT connected to SP Performance Assessment.

Would be handy for checking which SP-s did a specific Case – but it only shows records for a specific Activity.

4. Which report shows how many times a specific SP has worked with a specific student?

 - SP Activity feature of User manager can be used for this.

• “SP-s who did this “- feature of Case manager

(No specific report can be run in the Reports module of LS.)

SP Training

 Set Data entry only allowed from manually authorized workstations and log in from a non-authorized workstation to access this:

AFTER filling in the same checklist the SP did (SP part of the Case), Evaluation appears:

SP Training report shows the same Agreement value:

REPORTS SHOWING RESULT OF OBSERVATION:

SP Observation Comparison (SPPA Report > Observations dropdown)

Answers of SP (SP) vs that of Observer (O). If different – difference value is added ‘1’.

In the SP part those items are highlighted where the answers selected by the SP and the observer are different. (The items of the SPPA part is filled in only by observers, so here you will not see "differences", but the answers of the observer.)