Utilizing remote assessments as a tool in the accreditation process

Introduction

Organizations around the world have pivoted to remote work amid the COVID-19 pandemic. And to a remarkable extent, many of them have been able to successfully carry out much of their business operations. Among those organizations are accreditation bodies that have chosen to replace a significant part of their onsite assessments by remote assessments. IAS had already implemented such practices in extraordinary circumstances for more than 5 years, accumulating significant experience. Still, it was only the last few months, during the COVID-19 pandemic, that IAS systematically utilized remote management system assessments as a standard practice for a considerable number of carefully selected assessments, based on the complexity of the scope of accreditation.

A remote management system assessment is when electronic means are used to collect assessment evidence by utilizing electronic communication tools. A remote assessment means assessing from a remote location. According to IAF ID 12:2015, remote assessment is defined as “…the facilitation of assessment of a Conformity Assessment Body from a location other than that being physically present”.

The methods used to converse from a remote location may include one or a combination of technologies such as telephone communication, videoconferencing, e-mailing, online chat rooms etc. The new ISO 19011:2018 “Guidelines for auditing management systems” has already incorporated language to address remote management system audits.

The obvious benefit of remote assessments is a more efficient use of resources. Remote assessment techniques can save assessor travel time and expenses. It will also facilitate situations where a site is not easily accessible, or when there is an urgent need for assessment, and in extraordinary situations such as medical emergencies-quarantines, pandemics, etc. On the other hand, onsite assessments need more time for preparation and implementation and in many cases, they don’t provide the same flexibility as the remote assessments.

Collecting data, information

It is IAS’ primary concern that when performing remote management system assessments, it is ensured that whatever means are used, the credibility of the assessment results is maintained. The outcomes/deliverables of the remote assessment are expected to be the same as the ones collected by an assessment in which on-site or direct means are used to identify evidence. The same agenda assessment report and checklist documents are also completed by IAS assessors and are submitted to IAS for review.

During the remote assessment, appropriate data is collected to verify conformity to the assessment criteria. The data must be enough to verify conformity, free of bias and representative of the current status of the area/activity being assessed.

During remote assessments and in order to achieve assessment efficiency by recording and analyzing information provided electronically the client is required to provide more information in advance of the assessment. In that case, the client is required to complete checklists and submit specific records, before the assessment, reporting on the controls in place or recent changes in their system or process.

Three elements of an assessment are affected when the assessment is performed remotely:

Interviewing

This can be conducted remotely using teleconference technology. In preparation for the remote interview, the lead assessor must communicate with the interviewees in order to agree on the arrangements, including software to be utilized, dates, time, etc. The time difference is always considered. When the assessor is planning to perform the interview from home then additional arrangements are expected to avoid possible distractions.

The assessor needs to check in advance to ensure that both the client and the assessor have access to compatible operational instrumentation, software/apps, microphones, speakers, video cameras, etc.

Assessors also need to submit a specially developed questionnaire for the client to answer prior to the interview. Based on the answers to the pre-interview questions, the assessor prepares the actual interview questions and they are determining the tools needed to collect assessment evidence.

Remote assessing issues to be taken under consideration by the assessment team members include the following:

• Preparation needs to ensure that interviews are punctual.
• The room to be used for the interview is suitable.
• Become familiar with the communication equipment/software/app.
• Beware of weak or unstable internet connection. (Assessor and client should be prepared to use the telephone if an internet connection is not stable).
• Considerations regarding the usage of cameras and video equipment that could be banned due to security issues. (Assessor must be sure beforehand that the client will agree to use video image capturing technologies).

Reviewing and verifying documents and records

This process can be performed off-site if the assessor has access to any kind of electronic document control system. Records can be forwarded to the assessor, as requested, using different methods such as mail, skype, WebEx, etc. The electronic document control system/app to be used must be agreed with the client before conducting the remote assessment.

The assessor will need to be prepared and able to select records to be verified during the interview, such as test reports, inspection reports or calibration reports, certificates, personnel files etc. In this way, off-site verification of documents and records could be as effective as on-site assessment and could save assessing time.

The potential issues to be considered and resolved before the remote assessment are the need for scanning equipment for paper copies, if any, gaining remote access to the documents used by the client, and the time it takes to be trained on accessing and navigating the document control software/app.

Witnessing

Remote assessment practices for actual witnessing (of a process such as a test, or inspection, or audit) can be used to collect data online. Collecting data remotely is more demanding when performed remotely in real-time. It is possible to use a digital camera to observe the process and review related evidence. Surveillance cameras could be used, but they should be avoided because their quality or functionality could be inadequate.

Each situation should be evaluated based on data access and the importance of the process or assessment risk. For some remote assessments, data collection may need to be skipped or verified during a later on-site assessment.

For remote assessments, observing specific processes may not be important for certain areas. For example, setting up cameras to watch the human resources or purchasing department at work is not going to yield better information than asking the auditee directly for the required data. But watching the lab’s specimen conditioning area or observing the testing area surroundings could be important because there may be physical signs affecting testing implementation. Similarly, when process implementation, ongoing process controls, or process outputs need to be observed, appropriate real-time video surveillance may be needed.

Remote assessment considerations

The following questions should be answered when preparing a remote assessment:

Assessment scope and objectives or purpose

Can the remote assessment be performed during an initial accreditation, surveillance, a reassessment and/or scope expansion?

IAS assessors are expected to consult the respective program manager for instructions per case.

Nature of the processes to be assessed

Does the process to be assessed involve oral communication or documentation, retrieval of records and document control?

IAS assessors are trained on how to understand the client processes before conducting the remote assessment (using checklists, client’s Quality Manual and/or other MS documentation).

Type of instrumentation, equipment and materials involved in the process to be assessed

Which parts of the (testing/calibration/inspection etc.) process should be demonstrated?

IAS assessors are expected to define which parts of the operation, that need to be observed, are critical for verification of conformity.

Number of client facility areas that the assessor wishes to observe

Which areas should be covered?

The IAS lead assessor is expected to define and agree with the client, in advance, the areas to observe.

Scheduling

What is the preferable time to schedule the remote assessment?

The IAS lead assessor is expected to acknowledge and manage time zone issues and try to coordinate reasonable and mutually agreeable convening times.

Time management

How long will the remote assessment take?

IAS has performed a series of pilot remote assessments in the past. It was found that additional time is required to perform a remote assessment at the same level of quality as a regular assessment. Therefore, it is recommended to schedule for additional time and be prepared well, before the on-site assessment, addressing the issues noted above.

Communication tools

Will communication tools be adequate?

The IAS lead assessor considers the availability of appropriate electronic communication equipment, as well as the capability of the IAS team assessors and client to operate electronic communication equipment and address any applicable security requirements.

A trial meeting with the client using the agreed-upon media platforms could be conducted to ensure that the scheduled assessment will perform as planned.

Conclusion

A remote assessment, for accreditation purposes, can be considered in many cases (depending on the applicable accreditation standard and the scope of accreditation) as an acceptable alternate to on-site assessments during extraordinary circumstances such as a pandemic. Accreditation bodies around the world have implemented such assessments during the COVID-19 pandemic, carrying out, successfully, much of their accreditation operations.

During that period IAS had also replaced a significant part of its onsite assessments of selected clients’ management systems with remote assessments. The experience of this process indicates that its success is heavily dependent on the availability of the appropriate instrumentation and software, the careful preparation and the adequate training of involved staff. It is the opinion of IAS personnel involved in this process, that the on-site assessments are, still, a superior auditing tool, for accreditation purposes.

Bibliography:

• IAS SOP 28, Conducting Remote Assessments, October 2019
• IAS Remote Assessment Guide, v1, April 2020
• IAF Informative Document Principles on Remote Assessment Issue 1 (IAF ID 12:2015)
• ISO 19011:2018 “Guidelines for auditing management systems”
• IPC Guidance Document BD-04-006, Performing A Management System Audit – Auditing Techniques, 2015
• Remote Control, is e-auditing the next logical step? by J.P. Russell
• Remote Audit: Out of Sight but Not Out of Mind, by David Ade