A structured 7 step process for risk assessment of a decentralized clinical trial
It is a given that the people charged with your clinical trial planning, regulatory affairs and operations are better at executing standard operating procedures then in performing risk analysis and thinking like attackers. D.LiebermanRisk assessment is a process that starts before you write the protocol, when you are writing the CRF (to determine what data to collect) and any time there are amendments to the study. See the below graphic from the Transcelerate Web site to see why procedures do not protect your clinical trial and why SDV does not assure patient compliance to the protocol. Note that “Material risk” is any threat to the success of the study from problems with study startup to problems with poor patient compliance.

Introduction
Does counting compliance activities secure the deliverables of your clinical trial? First define “secure”. Security is about reducing the impact of unpredictable attacks on assets – in your case, attacks on the 2 most critical assets of your decentralized clinical trial – the data and the subjects. Some examples of unpredictable attacks on your clinical trial: There may be multiple sources of data errors at sites, ranging from mistakes, misunderstandings, sloppiness and all the way to incompetence. There may data fraud – deliberate fabrication or falsification of data There are patients that comply and patients that take their treatment randomly and in strange and wonderful ways. There are patient reported outcomes that make sense and then there are the people who write War and Peace in the ePRO system and crash the SAS analysis program with special characters they used. Will compliance activity check-boxing mitigate ANY of the above attacks? No.How to mitigate unexpected attacks on your data and patients
Once we understand that check-box compliance procedures are not a good countermeasure for threats to your study deliverables (solid scientific data, patient safety, patient compliance with the clinical protocol) what are our options for mitigation? Consider your strengths and weaknesses. Starting with your weaknesses, it is a given that the people charged with your clinical trial planning,regulatory affairs and operations are better at executing standard operating procedures then in performing risk analysis and thinking like attackers. There is a fundamental divide, a metaphorical valley of death of mentality and skill sets between a regulatory-affairs and clinical operations mindset and a professional security mindset. This essay offers a systematic approach – if you will, a common language, a language of people-centric threat modeling that helps clinical managers cross the chasm between thinking like a regulatory affairs person and thinking like an attacker who wants to destroy your study.Start by thinking about how your study can be attacked.
Analyzing the impact of attacks on medtech studies requires hard work, hard data collection and hard analysis. It’s not a sexy, fun to use, feel-good application like Apple Music. Risk analysis may yield results that are not career enhancing, and as the threats get deeper and wider with bigger and more complex trials – so the security valley of death deepens and gets more untraversable. There is a joke about systems programmers – they have heard that there are real users out there, actually running applications on their systems – but they know it’s only an urban legend. Like any joke, it has a grain of truth. IT and security are primarily systems and procedures-oriented instead of customer-safety oriented. Similarly – clinical regulatory affairs are primarily paper and process-oriented instead of attack-oriented.Leave your paper and process comfort zone
If the essence of security is protecting the people who use a company’s products and services then the essence of security for a clinical trial is protecting patients and acquiring reliable data.A structured 7-step process for risk analysis of your clinical trial
We propose a structured process for risk analysis and ongoing risk management. No previous training is required and the process can become a key part of a medtech developer’s management toolkit. The risk analysis and management process has 7 steps as described in the below schematic (“the risk management loop”). The process uses threat modeling and quantitative risk assessment methods based on providing a financial value to assets (such as EDC systems and patient eCRF records) in order to determine value at risk and prioritize security countermeasures.
Read this if you are new to risk analysis
Choose one (1) question you want to answer. That’s it. Only one (1). For example – “what is the threat scenario for patients participating in the study and not passing inclusion/exclusion criteria”? After you have nailed the question, nail the threat scenario – i.e. how it can happen. After you nail the threat scenario – quantify the threat in terms of probability of occurrence and its impact and potential damage to your study.Read this if you are a medtech developer
In a medtech study which uses wearables, connected medical devices or mobile medical device apps (or any combination thereof), having up-to-date documentation of software functionality and architecture is required in order to correctly identify vulnerabilities and threat scenarios. The following documentation is required as part of the risk analysis process: 1. Functional description of the system including relevant use cases 2. Architectural diagram of the system 3. Documentation of sub-modulesHow to document the risk assessment for your medical device study
Up-to-date documentation of the study protocol and CRF is required in order to correctly identify vulnerabilities and threat scenarios. Historical records of protocol amendments is unnecessary. The following source documentation is required as part of the risk analysis process: Study protocol Treatment schedule and visit flow eCRF CRF edit checks These documents must be detailed enough to be used as reference for the decisions regarding the applicability of various threat scenarios to the analyzed system. Step 2 Identify assets of your study The correct mapping of assets (EDC database, patient safety, drug accountability data, etc), their financial value and the evaluation of financial loss to the sponsor when these assets are damaged or stolen, is one of the most critical tasks in the threat analysis process. The assets value is used as the basis for calculating threat risks and countermeasures priorities.Asset valuation is not a one-time activity
Due to the importance of asset valuation, the asset list and corresponding values should be reviewed once a year by the controller or CEO during the course of the study. Step 3 Identify the moving parts (components) in your study Using a systems approach to your study, map the moving parts in your study. This will include application software components (EDC, IWRS, ePRO, centralized monitoring systems etc), people functions (study monitors, site monitors, project manager, CRCs, principal investigators). Map the “moving part” entities to assets (for example patient records) and update the threat model with the components and functions. Tagging different components and functions in the system help the analyst in classifying the various data and software entities and relating them to specific vulnerabilities and safeguards such as protecting PHI processed by an outsourced call center.Step 4 Identify your study vulnerabilities
Identifying and classifying vulnerabilities requires the analyst to be intimate with the study primary and secondary endpoints, safety endpoints, protocol design, implementation and deployment details. The analyst should also be familiar with clinical operations procedures and the types of users, customers and patients that use the system or are involved with delivering services.Step 5 Build / update the threat model
Classifying attacker types The basic attacker types are: study user roles (site and study monitors,Pis, CRC, project managers, IT staff or cloud EDC providers) , malicious outsiders, trusted insiders and other site staff and outsourcing service providers. Additional attacker types (such as hacktivists) may be added when relevant. Different attacker types will have different motivations and different costs for mounting an attack. Attack motivation and cost are an important part in estimating threat probability since cheap attacks by highly motivated individuals are more likely than expensive attacks by attackers with little to gain. Identifying attack entry points The best strategy for this step is to review attacker types and document every possible way a potential attackers could access the system. The list of entry points may be refined in the course of the risk management loop.Step 6 Build your risk mitigation plan. Calculate residual risk
Risk assessment is not over until the fat lady sings. You walk away from the risk assessment table with a much deeper understanding of what threats count and how much residual risk you have after deploying controls – technical controls, monitoring of deliverables, patient safety monitoring This is the most important step of the risk analysis and management process. The outputs are: A map of the relationships between threats and area tags, assets, attacker types, entry points and vulnerabilities An evaluation of the total damage and risk parameters for each of the threats Write mitigation plans Calculate residual risk – i.e. how much risk exists after you implement your new controls. Since threats are the most complex entities in the model, the process of identifying and constructing threat’s elements and parameters has a ‘decomposition’ flavor. During this step the analyst(s) will have to return to previous analysis steps in order to create missing entities, such as assets and vulnerabilities that are referenced by the threat that is constructed.Step 7 Validate your findings
The accurate identification of countermeasures and their relations with vulnerabilities is the basis for validating the correctness of the risk mitigation plan. The best way in our experience of validating a risk analysis is to show it other people outside your office and ask them what they think.Validation output
A list of countermeasures that mitigate vulnerabilities: The list should include the implementation cost and an indication if the countermeasure is already implemented. A map of the relationships between countermeasures and vulnerabilities: This map shows which vulnerability is mitigated by which countermeasure(s). A validated risk mitigation plan will include the following management level reports: Threats ordered by risk Threats ordered by the financial damage Safeguards ordered by risk mitigation percentages Safeguards ordered by their effectiveness (mitigation/implementation cost) Asset value at risk before mitigation Residual value at risk after the mitigation planSummary
We have presented a systematic 7 step process for identifying and analyzing threats to the assets of your clinical trial – whether its unpredictable user behavior or patients at risk. Assessing the risk posture of any study will benefit from this proven systematic methodology and will help you take a paper and process-oriented study team from a place of weekly and monthly reports and activity-counting to a faster-moving, and vastly more effective place of risk understanding and mitigation.100X faster to deviation detection in medical device studies.
Automated Patient compliance deviation detection and response on the flaskdata.io platform for a connected medical device clinical trial is 100X faster than manual monitoring.
Automated compliance monitoring analytics and real-time alerts let you focus your site monitoring visits on work with the PI and site coordinators to take total ownership and have the right training and tools to meet their patient recruitment and patient compliance goals.
