New Feature EP-4399 Results/Certifications - Result Releasing

Nov 12, 2024

Overview

An optional extra stage has been added to epaPRO in release 5.28.0 that will allow you to delay the releasing of results/certifications to the public. This work also comes with SLA enhancements.

Changes to Functionality

Provider screens

A new setting has been added into the edit/view Providers screen (Auto Release Outcomes).

Open

Auto release outcome setting on a Provider.

This new setting will control whether results and certifications conducted against this provider will require manual release to external users (such as apprentices, providers, and employers). This step occurs either after QA, or after result submission (in the cases where a result is not sampled for QA).

Result screen changes

The assessment result screen has been amended to now automatically release the result on submission, subject to the following criteria:

• The result is not sampled for QA

• The provider is configured to automatically release outcomes

If either of the above are not true, i.e.. the result needs QA and the provider isnt set to automatically release outcomes, then the result submission will instead place the result in a “pending outcome release” state

If the grade is not released at this stage, a new action will be created for any user that has the Process Outcome Releases permission to release the outcome.

Resits and Retakes

The functionality surrounding resits and retakes has been amended to take the new outcome release step into consideration. From now on, resits and retakes will not be considered available, until the result is released.

As mentioned above, void results will not require this manual releasing step.

In the case of retakes, an apprentice will not enter the retake review status until the retake is released.

Certification screen changes

The certification screen has been amended to now automatically release the certification on save, controlled by the provider being configured to automatically release outcomes.

If not, then the certification will instead be placed in a “pending outcome release” state.

If the grade is not released at this stage, a new action will be created for any user that has the “Process Outcome Releases” permission to release the outcome.

Release Outcomes screen

A new screen has been introduced within the Assessment → Release Outcomes menu, controlled by the new permissions (“Read” and “Process” Outcome Releases) - this new screen will list any results and/or certifications that are pending release to external users, providing the corresponding apprentice is not deleted, and is deemed “active” (apprentice statuses “In Assessment”, “Completed” and “Certified”).

Open

New Release Outcomes Screen

The screen can be filtered using the normal screen filters (such as search, cohort ID, provider, employer, standard, etc.) along with entity type (result or certification), component type (in the case of results), and dates.

From within this screen, outcomes can be released either individually or in bulk, using two methods:-

• “Now”

• “Scheduled”

Open

Bulk Release Outcome Screen

Selecting the “Now” option will release the outcome as soon as the Confirm button is pressed, allowing for immediate availability.

Open

Selecting the Now option via Bulk Release

Open

Warning pop up.

If you use the ellipsis menu for releasing the window appears as follows:

Open

Release Outcome screen via ellipsis menu.

If the “Schedule” option is chosen, the outcome can be released at a date/time of the user’s choosing, providing the date/time selected is not in the past.

Open

Bulk Release Outcome via the scheduled option showing the date and time

• If an outcome is released immediately, the outcome should disappear from the screen and no longer be available.

• If scheduled, the outcome should remain in the screen, along with it’s scheduled date. Until the outcome is released by the schedule, the schedule can be updated by editing the release (the outcome can also be released instantly by selecting the “Now” option and resubmitting).

QA listing screens

The QA listing functionality has been updated to consider results and certifications that have not been released as Pending/In QA, if the logged in user is an external user type.

QA show/edit screens

The QA screen has been updated to allow users with QA and Process Outcome Release permissions to release an outcome at the point that it is signed off at QA.

Open

Edit QA Screen

Open

Once Complete Result QA clicked the following appears

The declaration box has been updated within the QA screen to now show an additional toggle - “Release Outcome with QA?”. Setting this toggle to “on”, before submitting the QA, will cause the outcome to be released at the same time as the QA being completed.

Assessment Profile screen

The assessment profile screen has been enhanced to take into consideration the new “Pending outcome release” status of both results and certifications.

When logged in as an external user, an outcome will remain at status “pending QA” until the outcome is released - even when the grade has not been sampled for QA.

Open

Provider users view showing pending QA as usual

When logged in as an internal user, the grade details can be seen once the outcome is “complete” (i.e. past QA or not sampled), but will inform the user that the outcome is pending release via a badge. Users that have the permission to release outcomes will be presented with a new primary action button to take them to a, pre-filtered list based on the selected apprentice’s ULN.

Open

Internal users view showing Release outcome option and badge post QA

In addition to this, internal users will also see the programme checklist items show as an amber warning instead of green, when a result or certification is complete, but has not been released yet. In the case of external users, they will continue to show as grey and incomplete.

Open

Internal Users view of checklist, with Amber indicator

Element Side-drawer

The element side-drawer has been updated in a similar way to the assessment profile, to show when a result or certification is pending outcome release. The grade will be hidden from external users until the outcome is released. Internal users will also see the result/certification status as an amber warning, instead of green, when a result or certification is complete but has not been released yet. In the case of external users, they will continue to show as grey and incomplete.

In addition to this, the results widget will also offer internal users with processing permissions a shortcut button to release the outcome - this works in a similar way to the primary action button mentioned within the assessment profile section.

Open

Internal User View of Element Side Drawer

Apprentice Profile screen

The apprentice profile has also been updated with additional enhancements to honour the new outcome release logic.

Viewing the apprentice profile as an external user prior to outcome release will hide result and certification grades. Internal users will see the grades, but will see a “pending outcome release” badge, to confirm this is not available to the public yet.

Open

Internal View of Apprentice Profile with new badge for “Pending Outcome Release”

Action changes

As part of this work, the following actions have been added or amended to keep actions and SLAs in line:

• Certification is pending outcome release

  • Is created when a certification QA is completed

  • Is assigned to any users that have the new Process Outcome Releases permission

  • Completes when the certification outcome is released (either automatically due to the provider setting, or via the “Now” or “Scheduled” methods from within the new Assessment → Release Outcomes screen)

  • Will pause when an apprentice enters an apprentice status not deemed “active” (such as “Back into Learning”, “Retake Review”, etc.)

• Result is pending outcome release

  • Is created when either a result QA is completed, or a result is submitted and deemed to not require QA

  • Is assigned to any users that have the new Process Outcome Releases permission

  • Completes when the result outcome is released (either automatically due to the provider setting, or via the “Now” or “Scheduled” methods from within the new Assessment → Release Outcomes screen)

  • Will pause when an apprentice enters an apprentice status not deemed “active” (such as “Back into Learning”, “Retake Review”, etc.)

These actions (like other actions) will appear within the actions side-drawer, and within the “My To-Do’s” screen, with a shortcut button to the Assessment → Release Outcomes screen.

SLAs

As part of this work, various changes have been made to the way that SLAs track the apprentice journey. Though some of these changes are under the hood to improve the way epaPRO does this, there are also some frontend changes. These are:

• A new SLA has been introduced to measure the Time from Result Completed to Outcome Released

  • This will record the time from the result being marked as “complete” (either by passing QA, or not requiring QA at the point of submission) to the time the result is released (either via the “now” or “schedule” options)

• A new SLA has been introduced to measure the Time from Certification Completed to Outcome Released

  • This will record the time from the certification being marked as “complete” (by passing QA) to the time the result is released (either via the “now” or “schedule” options)

• The pre-existing combined SLA of “Time from Element Scheduled to QA Published” has been extended to become “Time from Element Scheduled to Outcome Released”

  • This is so that the SLA can continue to monitor based on it’s original function (time from element scheduled to the outcome being available), but with additional benefits

    • Because Outcome Release will always happen regardless of the QA requirement, this grants a much more consistent endpoint

    • It also allows for the element to change between these stages, as the endpoint of the SLA is guaranteed, and will only happen once

  • This should not affect previous reporting data (regarding breaches vs non-breaches) as it is assumed that results were released at the point as they became available, though the columns in the report have been updated to reflect the new measuring point (outcome released date is shown, instead of the QA completion date)

• The pre-existing SLA “Apprentice Time in EPA” has received various tweaks

  • Prior to this development, this SLA was introduced into the System Management screen, but had very little code to measure this information. There was also not a report available to see any data

  • This work will now (due to changes under the hood within epaPRO) correctly monitor the total time an apprentice is in EPA - from gateway approval through to certification outcome release

  • This SLA will pause when an apprentice enters an apprentice status not deemed “active” (such as “Back into Learning”, “Retake Review”, etc.)

Reporting changes

The following reports have been created/amended as part of this development:

• Assessment → Assessment Pass/Fail

  • Assessment results will now be added to the figures when the outcome has been released, not when the result was completed (QA’d or skipped)

• Assessment → Assessment 1st Time Pass

  • Assessment results will now be added to the figures when the outcome has been released, not when the result was completed (QA’d or skipped)

• Assessment → Result Tracking

  • Many fields will only be displayed if either:

    • The user is internal and the result has been completed

    • The user is external and the result outcome has been released

  • The list of fields is:

    • date_delivered

    • date_result_added

    • date_qa_added

    • date_qa_published

    • category

    • result

    • is_alternative

    • grade_type

    • result_next_action

    • submitted_date

    • submitted_by

    • available_date (now uses outcome released date, not the result completed date)

    • qa_required

    • qa_complete

    • outcome_released (new field)

    • outcome_additional_info

• QA → Outcome Releases

  • A new report that shows the outcome releases that have taken place

  • This will include backdated entries for any results/certifications that have been completed up until the date this work is deployed

    • These will have an outcome release method of “Legacy”

  • The report offers all the standard filtering expected for result/certification style reports, along with some new ones:

    • Outcome Release Type

      • Results or Certifications

    • Outcome Release Method

      • System

      • Manual

      • Manual With Certification

      • Scheduled

      • Scheduled With Certification

      • Legacy

      • Released With QA

• Certification → Certification Overview

  • Many fields will only be displayed if either:

    • The user is internal and the certification has been completed

    • The user is external and the certification outcome has been released

  • The list of fields is:

    • certification

    • certification_name

    • certification_date

    • certification_created

    • earliest_esfa_submission_date

    • qa_name

    • qa_created

    • epa_feedback

    • epa_next_steps

    • esfa_certificate_ref

    • esfa_submission_type

    • qa_complete

    • qa_published_date

    • outcome_release_date (new field)

• Certification → Ofqual

  • Results now show show based on the outcome being released, not the result being completed (QA’d or skipped)

• Certification → Certification Pass/Fail

  • Certifications will now be added to the figures when the outcome has been released, not when the certification was completed (QA’d)

• Certification → Certification 1st Time Pass

  • Certifications will now be added to the figures when the outcome has been released, not when the certification was completed (QA’d)

• Certification → Average Days in EPA

  • The average_time_in_epa will now calculate using the outcome release date and the apprentice audit date, not the qa published date as previously used

• Financials → Result Outcome Actions

  • Results now show show based on the outcome being released, not the result being completed (QA’d or skipped)

  • Outcome release date has been added as an available date filter

• Financials → Certification Billing

  • Certifications now show show based on the outcome being released, not the certification being completed (QA’d)

  • Outcome release date has been added as an available date filter

• Financials → Advanced Certification Billing

  • Outcome Release date has been added as an available date filter

• SLA → SLA Overview

  • A new certification_outcome_release_date column has been introduced that will show the certification outcome release date

  • Certification outcome release date has been added as an available date filter

• SLA → SLA Detailed

  • Two new columns have been introduced:

    • result_outcome_released_date

    • certification_outcome_released_date

  • Result Released Date & Certification Released Date have been added as available date filters

• SLA → SLA - Time from element scheduled to result outcome released

  • The pre-existing “SLA - Time from element scheduled to QA published” report has been renamed and updated to work based off the outcome release date, not the QA published date.

    • This reflects the changes to the actions and SLAs mentioned earlier

  • As part of these changes, the following fields have been renamed:

    • result_published_date → outcome_release_date

• SLA → SLA - Time from result complete to outcome released

  • A new report to track the time taken from a result being completed (QA’d or skipped) to result outcome released

  • This report offers all of the normal filters shared with other SLA reports

• SLA → SLA - Time from certification complete to outcome released

  • A new report to track the time taken from a certification being completed (QA’d) to certification outcome released

  • This report offers all of the normal filters shared with other SLA reports

• SLA → SLA - Apprentice Time in EPA

  • A new report to track the time taken for an apprentice to go through EPA (gateway approval date to certification outcome release)

  • This report offers all of the normal filters shared with other SLA reports

API

The following API endpoints have been updated by this functionality. The details of this are below:

• GET result/ & GET result/:id

  • Will now only return released results if accessing the API as an external user

    • Internal users will still get results if they are post QA, or were not selected for sampling

    • Note that AO users continue to get all submitted results, even if they are not through QA.

  • A new field “outcome_release_datetime" has been added to the return data - if populated for an internal user, they will know this is also available to external users

• GET certification/& GET certification/:id

  • Will now only return released certifications if accessing the API as an external user

    • Internal users will still get results if they are post QA

  • A new field “outcome_release_datetime" has been added to the return data - if populated for an internal user, they will know this is also available to external users

In addition to this, the API queue will now only publish the following entries when an outcome is released, instead of relying on the QA / skipping of QA as it did previously:

• publish_certification

• certification_result_available

• publish_result

These can then be used to pull the new data mentioned above using the normal routes.

Impacted Areas

• Provider

• Assessment Results

• Certification

• QA

• SLAs

• Actions

• Reporting

• Element Side-drawer

• Certification Side-drawer

• Assessment Profile

• Apprentice Profile

• External Services

• API

Permissions

• Outcome Release → Process

  • Given to owner and awardingorganisation roles by default

• Outcome Release → Read

  • Given to owner and awardingorganisation roles by default