What is required for validation of software used in QMS processes?
If you are not quite sure what to do for software validation, you are not alone!
There are different standards explaining how to validate software products and some software seem impossible to validate at all. In this blog we will explain how to address this in a general matter and, as example, show you what it means when it comes to validating Matrix software.
Overview of regulations and standards
In the illustration above you see that the needs to validate software comes from mainly two places:
ISO 13485: Medical devices — Quality management systems — Requirements for regulatory purposes
21 CFR Part 820: Code of Federal Regulations Title 21--Food and Drugs Chapter I--Subchapter H--Medical Devices Part 820 Quality System Regulation
On the left hand side you see standards which propose a way of validating software used in quality systems:
ISO/TR 80002-2: Medical device software — Part 2: Validation of software for medical device quality systems
AAMI TIR 36: Validation Of Software For Regulated Processes
There are more and other methods you can use, e.g. General Principles of Software Validation; Final Guidance for Industry and FDA Staff
What you need to validate
You need to validate software used in quality systems, this covers software to manage the QMS itself (e.g. MatrixQMS), the documentation of the medical device itself (e.g. MatrixALM) as well as other software used during development (e.g. compilers) and production.
It does not even stop here as you might also use software for human resources, and other parts of the company which by default fall under these requirements.
It also includes proprietary software, which you create yourself to be more productive.
Why you need to validate
Both ISO 13485 and 21 CFR Part 820 are giving a framework for your Quality Management System from a regulatory point of view.
When it comes to software validation ISO 13485:2016 says:
The organization shall document procedures for the validation of the application of computer software used in the quality management system. Such software applications shall be validated prior to initial use and, as appropriate, after changes to such software or its application.
The specific approach and activities associated with software validation and revalidation shall be proportionate to the risk associated with the use of the software.
And
The organization shall document procedures for the validation of the application of computer software used in production and service provision. Such software applications shall be validated prior to initial use and, as appropriate, after changes to such software or its application. The specific approach and activities associated with software validation and revalidation shall be proportionate to the risk associated with the use of the software, including the effect on the ability of the product to conform to specifications.
In the 21 CFR Part 820, the following can be found:
Automated processes. When computers or automated data processing systems are used as part of production or the quality system, the manufacturer shall validate computer software for its intended use according to an established protocol. All software changes shall be validated before approval and issuance. These validation activities and results shall be documented.
Specifically for electronic signatures and records, you can read:
Persons who use closed systems to create, modify, maintain, or transmit electronic records shall employ procedures and controls designed to ensure the authenticity, integrity, and, when appropriate, the confidentiality of electronic records, and to ensure that the signer cannot readily repudiate the signed record as not genuine. Such procedures and controls shall include the following:
Validation of systems to ensure accuracy, reliability, consistent intended performance, and the ability to discern invalid or altered records.
How to do the validation
How exactly this should be done is not described in ISO 13485 nor in 21 CFR Part 820.
AAMI TIR 36 and ISO/TR 80002-2 give more explanation on things to be taken into account. As the ISO/TR 80002-2 is similar to AAMI TIR 36, we will focus on the ISO/TR 80002-2.
One thing they all methods applied to validate software should have in common is that it starts from a risk-based approach. This means that the depth of validation is based on the risks associate with the software. That also means that a minimum you should document which risks exists for a software and how much validation you conclude is necessary because of those,
Validation of QMS-related software using ISO/TR 80002-2
According to the standard, validation is split up into 3 main phases:
#1 Develop
The develop phase itself is subdivided into 4 phases:
Define: this phase contains the intended use, use requirements, risk analysis, validation plan
Implement: this phase contains the actual design and coding of the software, risk analysis on software level, trace matrix
Test: this phase is about software verification
Deploy: this phase contains the installation of the software on the target platform and the commissioning
#2 Maintain
This phase is about the maintenance of the software, meaning bugs analysis/fixes, risk analysis, system monitoring, backups, recovery process.
#3 Retire
The retire phase deals with the decommissioning of the software and making sure no critical data is lost in the process.
Example: Application of ISO/TR 80002-2
In this example we go through the phases above, showing how it applies to a software like MatrixALM or MatrixQMS
Develop phase - Define
Matrix is an Off-the-shelf (OTS) software. It is fully developed by Matrix Requirements GmbH, which has defined its intended use, has made a thorough risk analysis and created a validation and verification plan. For Matrix software, our release documentation contains all information regarding intended use, risk analysis and verfication/validation. It is made available to our customers. If you use software from other suppliers ask them about verification and validation documentation as well as ISO certifications (e.g. Matrix Requirements is ISO 13485 and ISO 27001 certified).
As you are using the software you need to know and document what you are going to use it for. For this there are a few things to take into consideration:
Which processes are linked to the use of the software (in this example Matrix Requirements)?
Which risks are involved?
Did I make any specific modifications or configuration changes (or have the Matrix team make them for me)?
Am I linking Matrix to any other software which is important to my QMS?
Are there any risks involved the latter?
To document your intended use of Matrix Requirements and the related risks, we provide you with a validation project, called TOOLS. This project is meant to validate Matrix Requirements software, but can as well be used for validating other software tools you might be using. We pre-populated already some of the user requirements and risks, which you can either accept, change or delete. You can also add more requirements and risks of your own.
Furthermore, this project allows you to describe how you are using the software and how frequently you are planning to validate it.
Develop phase – Implement
This phase is mainly important for people that are developing their own software tools. The software design, software risk analysis, coding and testing belongs to this phase. If you develop your own software you can actually use IEC 62304 or IEC 82304 to get idea what should be done - even though you don't implement a medical device.
We provide our validation and verification report as part of its release documentation in the TOOLS project.
Matrix also provides its certificates (EN ISO 13485 and IEC ISO 27001) as part of its documentation to show that we develop our software in a controlled way.
Based on the criticality of the software tool, the documentation that is provided by your suppliers and the risks you defined, you could as well decide to do a vendor audit (through questionnaires or other means).
Develop phase – Test
This phase is again mainly for people that are creating their own tools as this phase is about verification. For you own software you can again look at IEC 62304 / IEC 82304.
Matrix on the other hand provides their validation and verification report as part of its release documentation in the TOOLS project.
If you don't get this kind of documentation from other suppliers, you might need to audit the vendor (depending on the involved risks).
Develop phase – Deploy
This phase is where the software is installed on the target platform. This is indeed where validation on the customer’s site comes mostly into play as well. Even though ISO 80002-2 doesn’t require formal IQ/OQ/PQ activities, this is where you would find them.
For Matrix, there are two scenarios:
Instances hosted in the cloud
Instances hosted on an intranet
Both have their own particularities when it comes to validation.
IQ is to verify the process of software installation and its documentation.
For cloud installations, the IQ is fully done by Matrix. This is not something our customers need to do. We create an instance for our customers on one of our servers.
For intranet installations, Matrix sends the specifications, requirements and instructions for installations to the customer, who needs to perform the installation accordingly.
OQ is to verify that the system is functioning properly. Before releasing a version of our software, Matrix is going through several test rounds. The end result of our tests is documented in our Verification and Validation report. This is part of our release documentation and therefore available to our customers (see Develop phase -Test). There are however a few things that can be done on the customer’s side as well:
For cloud installations, what needs to be tested is the basic functionalities, confirm correct licenses in the admin client and check the Matrix release documentation. If you made specific configurations, you should as well verify that these are working.
For intranet installations, what needs to be tested are the same things as for cloud installation. Next to those, also specific features like creating pdf, sending emails, etc, together with the backup and restore mechanism should be tested as they depend on a correct installation.
The TOOLS project is pre-populated with test cases that cover the basic functionalities. These can be extended by the customer with test cases for their specific configurations. In order to execute the tests, a test project can be created, based on an existing project. The TOOLS project also contains a template for a validation plan and report so that the validation activities and results can be documented in a structured way.
PQ is to verify the proper functioning of the software in real life settings. When using Matrix software, OQ and PQ can be overlapping. The tests that are performed during validation activities by the customers are done in their own instances. All of the test results can be documented using the TOOLS project.
Additional validation activities that belong to PQ can be checking the data integrity after upgrades and/or looking at the behavior of the system when working with a full team in the project (real-life setting). For this additional tests can be created in the TOOLS project.
In general, one important aspect to remember is, that all the validation activities start with the intended use of the software tool and the risks that you defined. The validation activities should be proportional to this.
Maintain phase
As mentioned above, based on feedback, Matrix Requirements software will be improved. When bugs are fixed, an update will be released. This means the Matrix team goes again through test cycles, creates new release documentation and provides this to the customers.
Maximum twice a year Matrix releases a new version of the software with new features. Of course, this triggers again a full test cycle and new documentation. Customers are notified upfront and get the option to opt-out of these upgrades.
Depending on the risk analysis and the decision on when to re-do the validation activities (see Develop phase – Define), this might lead to a new validation cycle on the customer’s side, which means they would perform again the tests and create a new validation report for the new version of Matrix.
In case you are using software tools for which you are not notified about upgrades or you don't have the option to opt-out, you might want to consider doing a validation at a pre-determined interval (e.g. once a year).
Retire phase
This phase is the decommissioning of the software tool.
All information that is stored in Matrix software can be exported at any time.
For other tools, it's important to investigate upfront how information can be extracted.
Conclusion
Software validation is everywhere and it can become quite confusing to know what exactly you need to do and how to document this. This is why we have created a TOOLS project to facilitate the execution and documentation of the validation of Matrix software or other software tools.
It covers the release documentation, intended use, requirements, risk analysis, validation plan and validation tests for the use of Matrix, together with the possibility to perform the tests and create the according validation documentation for every release.