New Feature EP-4399 Results/Certifications - Result Releasing
Nov 12, 2024
Overview
An optional extra stage has been added to epaPRO in release 5.28.0
that will allow you to delay the releasing of results/certifications to the public. This work also comes with SLA enhancements.
Changes to Functionality
Provider screens
A new setting has been added into the edit/view Providers screen (Auto Release Outcomes).
This new setting will control whether results and certifications conducted against this provider will require manual release to external users (such as apprentices, providers, and employers). This step occurs either after QA, or after result submission (in the cases where a result is not sampled for QA).
Note that this new setting will default to the value of “Yes” - this maintains the functionality of epaPRO as it is now, meaning that results and certifications will automatically become available to external users once the QA process is complete (if applicable).
Result screen changes
The assessment result screen has been amended to now automatically release the result on submission, subject to the following criteria:
The result is not sampled for QA
The provider is configured to automatically release outcomes
If either of the above are not true, i.e.. the result needs QA and the provider isnt set to automatically release outcomes, then the result submission will instead place the result in a “pending outcome release” state
If the grade is not released at this stage, a new action will be created for any user that has the Process Outcome Releases permission to release the outcome.
Note that when a void result is added to the system via any method (via the results screen, or via the various external services available), this will automatically release regardless of the provider setting.
Resits and Retakes
The functionality surrounding resits and retakes has been amended to take the new outcome release step into consideration. From now on, resits and retakes will not be considered available, until the result is released.
Any failure type grade that requires a resit, will now not be possible to rebook, until the failed grade is released to the external users.
As mentioned above, void results will not require this manual releasing step.
In the case of retakes, an apprentice will not enter the retake review status until the retake is released.
Certification screen changes
The certification screen has been amended to now automatically release the certification on save, controlled by the provider being configured to automatically release outcomes.
If not, then the certification will instead be placed in a “pending outcome release” state.
If the grade is not released at this stage, a new action will be created for any user that has the “Process Outcome Releases” permission to release the outcome.
Release Outcomes screen
A new screen has been introduced within the Assessment → Release Outcomes menu, controlled by the new permissions (“Read” and “Process” Outcome Releases) - this new screen will list any results and/or certifications that are pending release to external users, providing the corresponding apprentice is not deleted, and is deemed “active” (apprentice statuses “In Assessment”, “Completed” and “Certified”).
The screen can be filtered using the normal screen filters (such as search, cohort ID, provider, employer, standard, etc.) along with entity type (result or certification), component type (in the case of results), and dates.
From within this screen, outcomes can be released either individually or in bulk, using two methods:-
“Now”
“Scheduled”
Selecting the “Now” option will release the outcome as soon as the Confirm button is pressed, allowing for immediate availability.
If you use the ellipsis menu for releasing the window appears as follows:
If the “Schedule” option is chosen, the outcome can be released at a date/time of the user’s choosing, providing the date/time selected is not in the past.
If an outcome is released immediately, the outcome should disappear from the screen and no longer be available.
If scheduled, the outcome should remain in the screen, along with it’s scheduled date. Until the outcome is released by the schedule, the schedule can be updated by editing the release (the outcome can also be released instantly by selecting the “Now” option and resubmitting).
QA listing screens
The QA listing functionality has been updated to consider results and certifications that have not been released as Pending/In QA, if the logged in user is an external user type.
Note that this will only be applicable if an external user type is given access to this listing screen – internal users should be unaffected.
QA show/edit screens
The QA screen has been updated to allow users with QA and Process Outcome Release permissions to release an outcome at the point that it is signed off at QA.
The declaration box has been updated within the QA screen to now show an additional toggle - “Release Outcome with QA?”. Setting this toggle to “on”, before submitting the QA, will cause the outcome to be released at the same time as the QA being completed.
Note that this will not display if the provider is set to automatically release outcomes, as this will occur automatically and therefore it does not require a choice.
Assessment Profile screen
The assessment profile screen has been enhanced to take into consideration the new “Pending outcome release” status of both results and certifications.
When logged in as an external user, an outcome will remain at status “pending QA” until the outcome is released - even when the grade has not been sampled for QA.
When logged in as an internal user, the grade details can be seen once the outcome is “complete” (i.e. past QA or not sampled), but will inform the user that the outcome is pending release via a badge. Users that have the permission to release outcomes will be presented with a new primary action button to take them to a, pre-filtered list based on the selected apprentice’s ULN.
In addition to this, internal users will also see the programme checklist items show as an amber warning instead of green, when a result or certification is complete, but has not been released yet. In the case of external users, they will continue to show as grey and incomplete.
Element Side-drawer
The element side-drawer has been updated in a similar way to the assessment profile, to show when a result or certification is pending outcome release. The grade will be hidden from external users until the outcome is released. Internal users will also see the result/certification status as an amber warning, instead of green, when a result or certification is complete but has not been released yet. In the case of external users, they will continue to show as grey and incomplete.
In addition to this, the results widget will also offer internal users with processing permissions a shortcut button to release the outcome - this works in a similar way to the primary action button mentioned within the assessment profile section.
Apprentice Profile screen
The apprentice profile has also been updated with additional enhancements to honour the new outcome release logic.
Viewing the apprentice profile as an external user prior to outcome release will hide result and certification grades. Internal users will see the grades, but will see a “pending outcome release” badge, to confirm this is not available to the public yet.
Action changes
As part of this work, the following actions have been added or amended to keep actions and SLAs in line:
Certification is pending outcome release
Is created when a certification QA is completed
Is assigned to any users that have the new Process Outcome Releases permission
Completes when the certification outcome is released (either automatically due to the provider setting, or via the “Now” or “Scheduled” methods from within the new Assessment → Release Outcomes screen)
Will pause when an apprentice enters an apprentice status not deemed “active” (such as “Back into Learning”, “Retake Review”, etc.)
Result is pending outcome release
Is created when either a result QA is completed, or a result is submitted and deemed to not require QA
Is assigned to any users that have the new Process Outcome Releases permission
Completes when the result outcome is released (either automatically due to the provider setting, or via the “Now” or “Scheduled” methods from within the new Assessment → Release Outcomes screen)
Will pause when an apprentice enters an apprentice status not deemed “active” (such as “Back into Learning”, “Retake Review”, etc.)
These actions (like other actions) will appear within the actions side-drawer, and within the “My To-Do’s” screen, with a shortcut button to the Assessment → Release Outcomes screen.
SLAs
As part of this work, various changes have been made to the way that SLAs track the apprentice journey. Though some of these changes are under the hood to improve the way epaPRO does this, there are also some frontend changes. These are:
A new SLA has been introduced to measure the Time from Result Completed to Outcome Released
This will record the time from the result being marked as “complete” (either by passing QA, or not requiring QA at the point of submission) to the time the result is released (either via the “now” or “schedule” options)
A new SLA has been introduced to measure the Time from Certification Completed to Outcome Released
This will record the time from the certification being marked as “complete” (by passing QA) to the time the result is released (either via the “now” or “schedule” options)
The pre-existing combined SLA of “Time from Element Scheduled to QA Published” has been extended to become “Time from Element Scheduled to Outcome Released”
This is so that the SLA can continue to monitor based on it’s original function (time from element scheduled to the outcome being available), but with additional benefits
Because Outcome Release will always happen regardless of the QA requirement, this grants a much more consistent endpoint
It also allows for the element to change between these stages, as the endpoint of the SLA is guaranteed, and will only happen once
This should not affect previous reporting data (regarding breaches vs non-breaches) as it is assumed that results were released at the point as they became available, though the columns in the report have been updated to reflect the new measuring point (outcome released date is shown, instead of the QA completion date)
The pre-existing SLA “Apprentice Time in EPA” has received various tweaks
Prior to this development, this SLA was introduced into the System Management screen, but had very little code to measure this information. There was also not a report available to see any data
This work will now (due to changes under the hood within epaPRO) correctly monitor the total time an apprentice is in EPA - from gateway approval through to certification outcome release
This SLA will pause when an apprentice enters an apprentice status not deemed “active” (such as “Back into Learning”, “Retake Review”, etc.)
Note that it has not been possible to retrospectively start these timers for apprentices that have either been through, or are currently going through, EPA - although the gateway approved date is known, it would not be possible to know of any change of circumstances that would have paused this timer during their EPA.
Reporting changes
If this work is not enabled for the provider, then the date that a result was released (e.g. QA published) will automatically behind the scenes, become the “outcome_release_date”.
The following reports have been created/amended as part of this development:
Assessment → Assessment Pass/Fail
Assessment results will now be added to the figures when the outcome has been released, not when the result was completed (QA’d or skipped)
Assessment → Assessment 1st Time Pass
Assessment results will now be added to the figures when the outcome has been released, not when the result was completed (QA’d or skipped)
Assessment → Result Tracking
Many fields will only be displayed if either:
The user is internal and the result has been completed
The user is external and the result outcome has been released
The list of fields is:
date_delivered
date_result_added
date_qa_added
date_qa_published
category
result
is_alternative
grade_type
result_next_action
submitted_date
submitted_by
available_date (now uses outcome released date, not the result completed date)
qa_required
qa_complete
outcome_released (new field)
outcome_additional_info
QA → Outcome Releases
A new report that shows the outcome releases that have taken place
This will include backdated entries for any results/certifications that have been completed up until the date this work is deployed
These will have an outcome release method of “Legacy”
The report offers all the standard filtering expected for result/certification style reports, along with some new ones:
Outcome Release Type
Results or Certifications
Outcome Release Method
System
Manual
Manual With Certification
Scheduled
Scheduled With Certification
Legacy
Released With QA
Certification → Certification Overview
Many fields will only be displayed if either:
The user is internal and the certification has been completed
The user is external and the certification outcome has been released
The list of fields is:
certification
certification_name
certification_date
certification_created
earliest_esfa_submission_date
qa_name
qa_created
epa_feedback
epa_next_steps
esfa_certificate_ref
esfa_submission_type
qa_complete
qa_published_date
outcome_release_date (new field)
Certification → Ofqual
Results now show show based on the outcome being released, not the result being completed (QA’d or skipped)
Certification → Certification Pass/Fail
Certifications will now be added to the figures when the outcome has been released, not when the certification was completed (QA’d)
Certification → Certification 1st Time Pass
Certifications will now be added to the figures when the outcome has been released, not when the certification was completed (QA’d)
Certification → Average Days in EPA
The average_time_in_epa will now calculate using the outcome release date and the apprentice audit date, not the qa published date as previously used
Financials → Result Outcome Actions
Results now show show based on the outcome being released, not the result being completed (QA’d or skipped)
Outcome release date has been added as an available date filter
Financials → Certification Billing
Certifications now show show based on the outcome being released, not the certification being completed (QA’d)
Outcome release date has been added as an available date filter
Financials → Advanced Certification Billing
Outcome Release date has been added as an available date filter
SLA → SLA Overview
A new certification_outcome_release_date column has been introduced that will show the certification outcome release date
Certification outcome release date has been added as an available date filter
SLA → SLA Detailed
Two new columns have been introduced:
result_outcome_released_date
certification_outcome_released_date
Result Released Date & Certification Released Date have been added as available date filters
SLA → SLA - Time from element scheduled to result outcome released
The pre-existing “SLA - Time from element scheduled to QA published” report has been renamed and updated to work based off the outcome release date, not the QA published date.
This reflects the changes to the actions and SLAs mentioned earlier
As part of these changes, the following fields have been renamed:
result_published_date → outcome_release_date
SLA → SLA - Time from result complete to outcome released
A new report to track the time taken from a result being completed (QA’d or skipped) to result outcome released
This report offers all of the normal filters shared with other SLA reports
SLA → SLA - Time from certification complete to outcome released
A new report to track the time taken from a certification being completed (QA’d) to certification outcome released
This report offers all of the normal filters shared with other SLA reports
SLA → SLA - Apprentice Time in EPA
A new report to track the time taken for an apprentice to go through EPA (gateway approval date to certification outcome release)
This report offers all of the normal filters shared with other SLA reports
API
The following API endpoints have been updated by this functionality. The details of this are below:
GET result/
&GET result/:id
Will now only return released results if accessing the API as an external user
Internal users will still get results if they are post QA, or were not selected for sampling
Note that AO users continue to get all submitted results, even if they are not through QA.
A new field “
outcome_release_datetime
" has been added to the return data - if populated for an internal user, they will know this is also available to external users
GET certification/
&GET certification/:id
Will now only return released certifications if accessing the API as an external user
Internal users will still get results if they are post QA
A new field “
outcome_release_datetime
" has been added to the return data - if populated for an internal user, they will know this is also available to external users
In addition to this, the API queue will now only publish the following entries when an outcome is released, instead of relying on the QA / skipping of QA as it did previously:
publish_certification
certification_result_available
publish_result
These can then be used to pull the new data mentioned above using the normal routes.
Impacted Areas
Provider
Assessment Results
Certification
QA
SLAs
Actions
Reporting
Element Side-drawer
Certification Side-drawer
Assessment Profile
Apprentice Profile
External Services
API
Permissions
Outcome Release → Process
Given to owner and awardingorganisation roles by default
Outcome Release → Read
Given to owner and awardingorganisation roles by default