When is patient compliance important in medical device clinical trials?

In this post, Danny Lieberman, founder of flaskdata.io , discusses when patient compliance is crucial for your medical device clinical trial and when patient compliance is a negligible factor to success of the study.

From adverse events to patient compliance

My original goal for Flaskdata.io was  to use machine learning to predict onset of adverse events during interventional medical device clinical trials. 

For that goal, we needed data, so we started by providing cloud EDC services for medical device clinical trials with high-touch personal service and attention to the quality of the data model.  Very quickly – it become apparent that we did not have enough data (and after 20 studies, hundreds of sites and thousands of patients), we still do not have enough data to predict adverse events.

However, after performing 6 digital clinical trials in 2 chronic disease indications (acute migraine and chronic constipation) we had an epiphany – “PATIENT COMPLIANCE IS KING”

Customers using the Flaskdata.io platform for digital clinical trials, collected data via the EDC from investigators, collected data from patients (via our ePRO app) and collected data from connected medical devices (via the Flaskdata.io medical device API). The evidence was overwhelming :

Patient compliance to the protocol is an acute issue and critical success factor to the success of a connected medical device clinical trial. 

Or is it.

Who owns patient compliance?  The sponsor, the CRO, the site or the subject?

This discovery of the importance of patient compliance made a profound impression on us because it came from customers and empirical data they collected in our EDC systems.   This impression would not change, although we began to hear dissenting opinion on the importance of, and responsibility for patient compliance in clinical trials.

Public discussion on trends in the clinical trials industry is heavily influenced by big pharmaceutical companies, big CROs like PPD and IQVia and a $70BN/year clinical operations services industry that deal largely with oncology and biotechnology.   When we spoke to biotech prospects about the ability of our digital clinical trials platform to accelerate time to regulatory submission and assure high levels of patient compliance – people smiled and said “Well automated compliance monitoring is an innovative approach, but in fact, patient compliance is not important to us”.   

We then spoke with the Israel country manager of one of top 3 global CROs – and they said “Interesting question.   We collect many clinical trials operations metrics, but patient compliance to the clinical protocol is not a metric we collect”.   I asked – “In that case, who is in charge of patient compliance? and the answer was – the sites”.  In this scheme of things, if patient compliance is not a CRO metric, then the sponsor has a blind-spot to what is possibly, the single most important factor to the success of his connected medical device clinical trial.  Or not.

After that, we spoke with the country manager of one of the top 3 pharmaceutical companies and Israel and he told us again “Patient compliance is a non-issue for us.  Patients come to the hospital and get treatment and there is no problem”. I asked him “What about psychiatry trials?”  He replied – “well yes, everyone knows that psychiatry trials have acute issues of patient compliance”

Hmm.

We then went back and did the most logical thing – searching in Google for “the critical success factors of clinical trials” and there are 290 million results and a ton of empirical evidence and academic and industry research on the importance of patient adherence in clinical trials.

And this vast body of empirical data is dealing primarily with drug trials, not medical device trials.   The VP Clinical of a gene therapy prospect (who had previously worked at a medical device company) told us that in gene therapy patient compliance is negligible while in medical device trials, patient compliance is acute.

Hmm again. So what does Google say?

The high failure rate of clinical trials has significant impact on providing potential curative treatments to patients in need….

One key factor that has been identified in the high failure rate of clinical trials is the adherence of patients participating in clinical trials to the dosing, treatment, and study procedures that are very carefully put in place in clinically rigorous protocols. Due to the rigor that is required in order to demonstrate an “effect” relative to a standard of care treatment, even a small deviation in medication adherence can result in a trial failing to meet its pre-specified clinical endpoint.

Additionally, the current nature of clinical studies include strict timelines and competition among sites to enroll eligible subjects which can many times result in the inclusion of subjects that are simply not “medication-compliant”. The issue of medication adherence is therefore one key factor sponsors should carefully look at monitoring closely when designing and planning the medical and operational oversight of their trials.

Unfortunately, the issue of medication adherence many times goes unmanaged and falls solely on site staff to oversee. As clinical sites are many times running multiple concurrent trials and are themselves pressed to remain productive, the one-on-one daily management of medication adherence of study subjects can many times be neglected. It is therefore in the best interest of the patients in need that sponsors look towards solutions that can help to support their clinical sites in providing additional resources to maintain close and frequent interactions with subjects enrolled in key studies. It simply is no longer sufficient to rely solely on very busy clinical practices to ensure successful adherence of patients in enrolled in trials.   

See Compliance – a key factor to a successful clinical trial.

I’m confused.  Is compliance the best interest of the patient or the best interest of the PI, or the sponsor or all of the above?  We know that the PI must monitor participants’ compliance with study requirements. Failure to monitor patients adequately can sabotage the entire study and damage the site’s reputation. 

CROS not collecting patient compliance metrics. Busy sites. Lack of tools. PIs who are generally not hands-on with the patients.   Sounds like a classic finger-pointing situation.  

We hear of the importance of site selection, but if patient compliance is not a CRO metric, then how do we measure site performance properly?  

The 4 quadrants of patient compliance

In fact, the question of HOW to measure the importance of compliance is intimately related to 4 factors – and interestingly enough is totally unrelated to the site or the PI.  The 4 factors of patient compliance are:

  1. How do you collect data?

  2. What is the indication?

  3. What does the product do?

  4. How involved is the patient in the treatment?

In order to understand why there is dissenting opinion on the topic of the importance of patient compliance – we can map life-science products into 4 quadrants:  (Patient-centric, Digital, Investigator-centric, Implanted).  The top right quadrant in green is a digital clinical trial for chronic disease, the top left is a traditional EDC operation with varying degrees of patient involvement, the bottom left is little patient involvement and EDC data collection from paper source and the bottom right is no patient involvement but with data collection from implanted devices (an interesting and extremely important use case in its own right).

The above picture tells the whole story.

Patient compliance in clinical trials is crucial in digital clinical trials and patient-centric trials using traditional EDC and patient reported outcomes.

In the end it is about the patient – not the PI, not the site operations team and their training, policies and procedures and not about the CRO.

But hey – this is something any sponsor worth their salt already knows.

Israeli Medical device innovation for high patient compliance

One of the most challenging problems in medical device clinical trials and in real-life is how to achieve high levels of patient compliance to the protocol.    
Automated patient compliance technology in medical device clinical trials is confronting CROs with an unpleasant status-quo of SDV as a low-value-add, high-cost, time-consuming activity for patient compliance assurance.  The approach that this company takes provides continuous patient monitoring without requiring patient compliance at all. 

EarlySense is an Israeli medical device that is based on a paddle placed under the patient’s mattress that continuously monitors patient movement, HR and RR trends.

The EarlySense device helps facilitate timely interventions for patients in non-ICU settings by adding a layer of care with continuous monitoring, drawing attention to those who may show early signs of deterioration and may require clinical intervention.

Since the EarlySense contact-free sensor (it looks like a small plate)  is placed under the mattress and there are no leads attached to the patient – there is no need for patient compliance.

EarlySense bedside unit

We spoke Dalia Argaman, VP Clinical & Regulatory at EarlySense to understand how to take medical devices from the design and engineering stages, navigate regulatory pathways and execute medical device clinical trials that receive FDA approval and save lives.

“I have a BSc in Chemistry and MSc in Chemical Physics and I was lucky enough to start my career immediately after completing my education. I joined Direx (a startup employing 6-7 people at that time) and became a part of developing an innovative medical device in the field of shockwave lithotripsy which is designed to break kidney stones without invasive procedure. Beforehand the standard of care was that patients with kidney stones had to undergo surgical procedure full of discomfort and further complications. I was the one who took the medical device through clinical trials  when the first prototype was created”.

It was back then that Dalia got introduced to the world of medical device clinical trials, submissions to regulatory authorities (FDA, CE, CFDA) and clinical data management.

Dalia has been on top of this innovative world through the 20 years of her career, working in several companies that develop different medical devices. Alongside success, she experienced a number of professional and personal challenges that shaped her career.

“You actually might be surprised to find out that the biggest challenge I had to overcome during my career was to try and balance family life with professional. I am a career person but I also have a family and I always had to juggle between being a mother, raising a family and moving forward on the career ladder. In addition, I think when you are a regulatory person, working with various people with different understanding, different backgrounds and fields and trying to get everyone at the same page is a huge professional challenge”, she said.

While working for Glucon (Developer of Non-Invasive Glucose Monitoring Devices in Israel) Dalia was involved in clinical trials with patients who had diabetes: that included children as well. She recalls that this experience was most memorable through her entire career as it required creating a medical device that would make patient compliance easy.

“People with diabetes are prone to numerous complications: patients have to monitor their blood glucose level constantly to avoid hyperglycaemia: the procedure is usually done by pricking the finger and drawing blood for analysis. You can only imagine how uncomfortable it is for young children or their parents who sometimes have to wake their child up several times a night to check it.  Being involved in a company that develops a device capable of making so many people’s lives easy is a subject of great pride for me”, she says.

Dalia Argaman is currently in charge of clinical regulatory affairs and quality assurance in Earlysense. The company develops contactless sensors that are placed under hospital bed mattresses and allow to monitor vital signs (heart rate, respiratory rate and other parameters) in a contactless way without making the patient feel uncomfortable.

“They can help the physicians, nurses to continuously supervise the patient and detect early signs of deteriorating in order to intervene early, thus reaching a better outcome”.   All of this is done using passive monitoring of the patient’s movement and without requiring patient compliance.

“There is currently a long delay between the time that a medical device is being developed by research and development teams, execution of medical device clinical trials, analysis of data received from clinical data management team, submission to FDA and the time that products get to the end users. The delay is connected with vigorous testing a product has to get through in order to be in compliance with standards and be approved. I think FDA understands well the importance of using automation to accelerate the process of executing clinical trials in order enable these medical devices to get to market and start saving lives sooner”.

Why paper is not an option for your medical device clinical trial

This is a piece David wrote a couple of years ago originally entitled “Why you cannot afford to use paper in your first Phase I efficacy trial for your medical device”.   David’s premise is that people do not like change.

Why you cannot afford to use paper in your first Phase I efficacy trial

In all walks of life, people do not like change.

We have heard the axiom change is good all throughout our lives, but the fact remains that people, as basic animals, are hesitant to embrace change and take on new endeavors. Human beings are creatures of habit, and are more often than not content within their comfort zones, regardless if they are losing out on valuable experiences, money, etc.

Studies have even been conducted revealing how opposed to change we creatures of habit are. People will sacrifice the opportunity to enhance their quality of life because it may require a change to their routine, or learning new habits, and humans hate that.

In the clinical research industry, paper-based data capture methods have been used effectively, and for decades. Paper is the norm, and many a successful study have been conducted using this such method. While paper is the traditional, tried-and-true method for data capture and management (especially during Phase I efficacy trials of medical devices, which typically have smaller subject counts and shorter study durations), that does not mean it is the best method available, or that it is the most cost efficient.

In fact, the last point is no longer true whatsoever.

There are small CROs and clinical study sponsors that are so used to paper data capture for small medical device clinical trials, that they oppose the change to electronic data capture (EDC). However, while some early objections were valid in opposition of EDC for Phase I, they no longer ring true.

EDC has been implemented for clinical studies, particularly in later-stage trials such as Phase III studies with thousands of patients, for a little over 15 years. By now, many of the concerns regarding the ample paper vs EDC debate at any clinical study stage are now moot.

Today we are going to touch upon why one cannot afford to use paper for Phase I efficacy trials for medical devices, and will greatly benefit from the change to cloud EDC.

Time savings on amendments

During Phase I efficacy testing, pharmaceutical companies are getting their feet wet for the first time while developing a new drug. This is the stage with the highest level of patient risk, and EDC quickly thwarts paper-based systems in this realm.

Phase I experiences the most amendments to drug administration frequency, dosing, and amendments to trial need to always be compliant with the FDA’s 21 CFR Part 11. Vendor validated EDC systems are easily augmented to comply with changes to FDA regulations, and have measures in place within the software, to monitor and ensure that study SOPs are compliant every step of the way.

Paper simply cannot do that.

Also, amendments, whether at the hands of a regulatory agency or medical device company, tack on months of extended study time and costs. According to a study done by Tufts University, a single amendment using paper-based systems increases study time by an average of 2 months and costs the study over $400,000. The study also showed that on average, each study experiences 2.3 amendments to protocol.

The time savings, and thus cost savings, on amending SOPs is enormous for studies conducted using EDC, as the software and eCRFs can be augmented in the blink of an eye. Also, if further amendments are required down the study chain, they are made just as quickly.

Real-time data monitoring of cleaner, faster study data

Using EDC instead of paper affords clinicians and data monitors real-time access to data capture. Also, cloud EDC like Clear Clinica is mobile accessible, so all members of the study team can remotely access data using mobile devices like smartphones or tablets, the very second it is entered. This is especially valuable for Phase I trials, because these have the highest risk to study subjects, as they are the first in line to test the drugs.

Even though Phase I efficacy trials do not typically involve hundreds of subjects, adjustment to treatment protocol and dosing need to be made with as little delay possible. Patient safety is a top priority for clinical studies.

For studies using patient reported outcomes (ePRO), EDC wins over paper-based systems every time. When a patient enters data into the system, risk-based monitoring protocols within the software inform study teams whether or not a patient is at risk for harming their health if they proceed at the administered dose. This allows clinicians to make adjustments to dosage, or cease the subjects participation in the study, before their health is harmed if the dosage is too high or if grossly adverse effects are experienced by the patient.

Again, paper simply cannot perform in this manner for Phase I efficacy testing.

Further, using EDC for Phase I is smarter than paper regarding cleaner, error-free data. Human error occurs. Even the brightest of clinicians and data monitors will make a mistake when entering data, or miss an incomplete form. Especially being that Phase I is the first stage for drug testing, the cleaner data is from the get-go the smoother it will be for conducting Phase II and III of the trial.

The EDC system software can be set so that eCRF values are standardized, so that when data is captured and entered into the eCRF fields error notifications are displayed when data is outside of the field parameters. The same goes for submission of eCRFs that are incomplete. EDC systems like Clear Clinica are programmed according to data parameters set by sponsor or CRO staff for each trial’s needs. Also, once the eCRF parameters are set, they can be modified if needed according to amendments, but otherwise they remain uniform, saving time during the entire study cycle’s lifetime.

You cannot afford to not use EDC

Nobody will deny that the up-front costs of implementing cloud EDC for Phase I will cost more than a paper-based system. However, not doing so because of that reason is myopic and short-sighted. The safety, risk, time, and data quality savings are well worth the initial investment, as the system is not going to be used for only one study.


Down the road, after incorporating an EDC for Phase I, and using it for II and III, the money spent is quickly offset by the costs saved on time, IT personnel expenses (EDC vendors have support staff to solve whatever issues may arise), and data assurance, amongst others. The sooner you switch to EDC, the sooner future studies will save your CRO or sponsor organization money, and mitigate Phase I patient risks.

A structured 7 step process for risk assessment of a medical device clinical trial

In this essay, I discuss a systematic methodology for evaluating risk in your medical device clinical trial. This is a methodology that has proven itself in hundreds of security and privacy compliance risk assessment projects in a wide variety of healthcare, clinical and IT scenarios. 

It is a given that the people charged with your clinical trial planning,regulatory affairs and operations are better at executing standard operating procedures then in performing risk analysis and thinking like attackers.

D.Lieberman

Risk assessment is a process that starts before you write the protocol, when you are writing the CRF (to determine what data to collect) and any time there are amendments to the study.

See the below graphic from the Transcelerate Web site to see why procedures do not protect your clinical trial and why SDV does not assure patient compliance to the protocol.  Note that “Material risk” is any threat to the success of the study from problems with study startup to problems with poor patient compliance.

Introduction

Does counting compliance activities secure the deliverables of your clinical trial?

First define “secure”.

Security is about reducing the impact of unpredictable attacks on assets – in your case, attacks on the 2 most critical assets of your clinical trial – the data and the subjects.

Some examples of unpredictable attacks on your clinical trial:

There may be multiple sources of data errors at sites, ranging from mistakes, misunderstandings, sloppiness and all the way to incompetence.

There may data fraud – deliberate fabrication or falsification of data

There are patients that comply and patients that take their treatment randomly and in strange and wonderful ways.

There are patient reported outcomes that make sense and then there are the people who write War and Peace in the ePRO system and crash the SAS analysis program with special characters they used.

Will compliance activity check-boxing mitigate ANY of the above attacks?

No.

How to mitigate unexpected attacks on your data and patients

Once we understand that check-box compliance procedures are not a good countermeasure for threats to your study deliverables (solid scientific data, patient safety, patient compliance with the clinical protocol) what are our options for mitigation?

Consider your strengths and weaknesses.

Starting with your weaknesses, it is a given that the people charged with your clinical trial planning,regulatory affairs and operations are better at executing standard operating procedures then in performing risk analysis and thinking like attackers.

There is a fundamental divide, a metaphorical valley of death of  mentality and skill sets between a regulatory-affairs and clinical operations mindset and a professional security mindset.

This essay offers a systematic approach – if you will, a  common language, a language  of people-centric threat modeling that helps clinical managers cross the chasm between thinking like a regulatory affairs person and thinking like an attacker who wants to destroy your study.

Start by thinking about how your study can be attacked.

Analyzing the impact of attacks on medtech studies requires hard work, hard data collection and hard analysis.  It’s not a sexy, fun to use, feel-good application like Apple Music.   Risk analysis  may yield results that are not career enhancing, and as  the threats  get deeper and wider  with  bigger and more complex trials – so the security valley of death deepens and gets more untraversable.

There is a joke about systems programmers – they have heard that there are real users out there, actually running applications on their systems – but they know it’s only an urban legend. Like any joke, it has a grain of truth. IT and security are primarily systems and procedures-oriented instead of  customer-safety oriented. Similarly – clinical regulatory affairs are primarily paper and process-oriented instead of attack-oriented.

Leave your paper and process comfort zone

If the essence of security is protecting the people who use a company’s products and services then the essence of security for a clinical trial is protecting patients and acquiring reliable data.

A structured 7-step process for risk analysis of your clinical trial

We propose a structured process for risk analysis and ongoing risk management.  No previous training is required and the process can become a key part of a medtech developer’s management toolkit.

The risk analysis and management process has 7 steps as described in the below schematic (“the risk management loop”). The process uses threat modeling and quantitative risk assessment methods based on providing a financial value to assets (such as EDC systems and patient eCRF records) in order to determine value at risk and prioritize security countermeasures.

The 7 step risk process provides a systematic way to manage risk while responding to changes in regulation, business environment and clinical research feature set/functionality. Let’s start with some basic definitions:

Definitions

Vulnerability is a weakness, limitation or a defect in one or more of the system’s elements that can be exploited to disrupt the normal functionality of the system. The weakness or defect may be either in specific areas of the system, its layout, its users, operators, and/or in its policies and procedures.

Countermeasure is a technical, physical or procedural safeguard that mitigates one or more vulnerabilities.

Asset – data, systems, physical assets or intellectual property of value to the organization.

Threat – action(s) that exploit vulnerabilities in order to damage assets.

Asset value – the financial value of an asset that is destroyed of stolen. Assets may be digital (software source, physical (a server) or commercial (a corporate brand).

Damage to Asset – damage to a physical asset or damage to a digital asset in terms of breach of confidentiality, impacted system availability or broken integrity of systems and/or data.  Damage is estimated in financial terms.

Threat probability is the likelihood that a threat will turn into a real attack. Threat probability can be described in terms of ARO – Annual Rate of Occurrence; i.e. how many times a year that the attack is forecasted to happen.

Threat risk is the likelihood of damage that may be caused to one or more assets by the threat. 
Recommended countermeasures the possible countermeasures that reduce the threat’s risk based on the countermeasures that mitigate the threat vulnerabilities.

Actual countermeasures (aka mitigation plan) is a subset of recommended countermeasures that is assumed to be the most effective for mitigating a specific threat. Choice of specific safeguards is often a judgment call of the threat analyst.

Countermeasure cost is the financial value that is associated with the implementation of a specific countermeasure.

Countermeasure cost effectiveness is the degree of mitigation introduced by a specific countermeasure to the overall risk in the system in relation with the cost of implementing this specific countermeasure.

Attacker is a person (or group of persons) that may perform the steps of a specific threat scenario.

Attacker Types are the various classes of attackers that are differentiated according to their motivation, qualification, available attack tools and their accessibility to the attacked system’s resources.

Entry Points are points of entry made by attackers into the system, for example doors in a building or users who have a login to your EDC system.

The 7 step risk analysis loop

Risk analysis is not a one-way, one-time process you do, report and file away.   Analyzing attacks and risk in your studies is an ongoing exercise always relying on quality human intel from the field – from CRCs, subjects and site monitors.

Step 1 Set scope

The threat analyst(s) will identify reasonable threat scenarios and their probability.

Read this if you are new to risk analysis

Choose one (1) question you want to answer.  That’s it. Only one (1). For example – “what is the threat scenario for patients participating in the study and not passing inclusion/exclusion criteria”?  After you have nailed the question, nail the threat scenario – i.e. how it can happen.  After you nail the threat scenario – quantify the threat in terms of probability of occurrence and its impact and potential damage to your study.    

Read this if you are a medtech developer

In a medtech study which uses wearables, connected medical devices or mobile medical device apps (or any combination thereof), having up-to-date documentation of software functionality and architecture is required in order to correctly identify vulnerabilities and threat scenarios. The following documentation is required as part of the risk analysis process:

 1. Functional description of the system including relevant use cases

 2. Architectural diagram of the system

 3. Documentation of sub-modules

How to document the risk assessment for your medical device study

Up-to-date documentation of the study protocol and CRF is required in order to correctly identify vulnerabilities and threat scenarios. Historical records of protocol amendments is unnecessary.

The following source documentation is required as part of the risk analysis process:

Study protocol

Treatment schedule and visit flow

eCRF

CRF edit checks

These documents must be detailed enough to be used as reference for the decisions regarding the applicability of various threat scenarios to the analyzed system.

Step 2 Identify assets of your study

The correct mapping of assets (EDC database, patient safety, drug accountability data, etc), their financial value and the evaluation of financial loss to the sponsor when these assets are damaged or stolen, is one of the most critical tasks in the threat analysis process. The assets value is used as the basis for calculating threat risks and countermeasures priorities.

Asset valuation is not a one-time activity

Due to the importance of asset valuation, the asset list and corresponding values should be reviewed once a year by the controller or CEO during the course of the study.

Step 3 Identify the moving parts (components) in your study

Using a systems approach to your study, map the moving parts in your study. This will include application software components (EDC, IWRS, ePRO, centralized monitoring systems etc), people functions (study monitors, site monitors, project manager, CRCs, principal investigators).

Map the “moving part” entities to assets (for example patient records) and update the threat model with the components and functions. 

Tagging different components and functions in the system help the analyst in classifying the various data and software entities and relating them to specific vulnerabilities and safeguards such as protecting PHI processed by an outsourced call center.

Step 4 Identify your study vulnerabilities

Identifying and classifying vulnerabilities requires the analyst to be intimate with the study primary and secondary endpoints, safety endpoints, protocol design, implementation and deployment details. The analyst should also be familiar with clinical operations procedures and the types of users, customers and patients that use the system or are involved with delivering services.

Step 5 Build / update the threat model

Classifying attacker types

The basic attacker types are: study user roles (site and study monitors,Pis, CRC, project managers, IT staff or cloud EDC providers) , malicious outsiders, trusted insiders and other site staff and outsourcing service providers. Additional attacker types (such as hacktivists) may be added when relevant.  Different attacker types will have different motivations and different costs for mounting an attack. Attack motivation and cost are an important part in estimating threat probability since cheap attacks by highly motivated individuals are more likely than expensive attacks by attackers with little to gain.

Identifying attack entry points

The best strategy for this step is to review attacker types and document every possible way a potential attackers could access the system. The list of entry points may be refined in the course of the risk management loop.

Step 6 Build your risk mitigation plan. Calculate residual risk

Risk assessment is not over until the fat lady sings.    You walk away from the risk assessment table with a much deeper understanding of what threats count and how much residual risk you have after deploying controls – technical controls, monitoring of deliverables, patient safety monitoring

This is the most important step of the risk analysis and management process. The outputs are:

A map of the relationships between threats and area tags, assets, attacker types, entry points and vulnerabilities

An evaluation of the total damage and risk parameters for each of the threats

Write mitigation plans

Calculate residual risk – i.e. how much risk exists after you implement your new controls.

Since threats are the most complex entities in the model, the process of identifying and constructing threat’s elements and parameters has a ‘decomposition’ flavor. During this step the analyst(s) will have to return to previous analysis steps in order to create missing entities, such as assets and vulnerabilities that are referenced by the threat that is constructed.

Step 7 Validate your findings

The accurate identification of countermeasures and their relations with vulnerabilities is the basis for validating the correctness of the risk mitigation plan. The best way in our experience of validating a risk analysis is to show it other people outside your office and ask them what they think.

Validation output

A list of countermeasures that mitigate vulnerabilities: The list should include the implementation cost and an indication if the countermeasure is already implemented.

A map of the relationships between countermeasures and vulnerabilities: This map shows which vulnerability is mitigated by which countermeasure(s).

A validated risk mitigation plan will include the following management level reports:

Threats ordered by risk

Threats ordered by the financial damage

Safeguards ordered by risk mitigation percentages

Safeguards ordered by their effectiveness (mitigation/implementation cost)

Asset value at risk before mitigation

Residual value at risk after the mitigation plan

Summary

We have presented a systematic 7 step process for identifying and analyzing threats to the assets of your clinical trial – whether its unpredictable user behavior or patients at risk.

Assessing the risk posture of any study will benefit from this proven systematic methodology  and will help you take a paper and process-oriented study team from a place of weekly and monthly reports and activity-counting to a faster-moving, and vastly more effective place of risk understanding and mitigation.

100X faster to deviation detection in medical device studies.

Patient compliance automation on the flaskdata.io platform for a medical device clinical trial is 100X faster than manual monitoring. Automated compliance monitoring analytics and real-time alerts let you focus your site monitoring visits on work with the PI and site coordinators to take total ownership and have the right training and tools to meet their patient recruitment and patient compliance goals.

What does risk-based monitoring mean for CROS?

What does Risk-Based Monitoring mean for CROs?

Contract research organizations (CROs) should implement risk-based monitoring (RBM) as a top priority for medical device clinical studies. Use of modern data technologies for remote risk-based monitoring can help reduce non-value added rework, and  dramatically improve patient compliance in medical device clinical trials and help speed up time to statistical report.

The goal of a medical device clinical trial is to test the efficacy and safety for the intended use of the device. In many cases, medical devices are used at home by patients and caregivers for chronic diseases such as diabetes, migraine, heart failure, chronic constipation.   Medical devices are connected wearables for treating chronic diseases (like the Theranica Therapeutics migraine wearable for neuromodulation therapy) or implanted connected devices (like  the Vectorious minimally-invasive implanted device that is packaged with an external home-unit; the in order to increase CHF patient quality of life and reduces readmission rates).

CROs are charged with the task of conducting medical device clinical trials that are valuable to the healthcare industry and produce near-real-life results. RBM helps assure that data collected is reliable, and is highly applicable to help develop innovative home-use medical devices by assuring high levels of patient compliance to the study protocol.

In this post, we are going to go over how remote risk-based monitoring can help your current study and also provide value for your medical device development.

What is risk-based monitoring?

Risk-based monitoring entails the mitigation of risks during a medical device clinical trial that a CRO is conducting through the process of identifying, assessing, and then monitoring the risks associated with patient safety throughout the course of the study.

In August 2013 the FDA issued draft guidance for “A Risk-based approach to monitoring”

“The overarching goal of this guidance is to enhance human subject protection and the quality of clinical trial data by focusing sponsor oversight on the most important aspects of study conduct and reporting”

The draft guidance includes three steps in a risk-based approach to monitoring:
1. Identifying critical data and processes. To accurately monitor the quality of a study and the safety of its patients, sponsors must know which elements are vital for each particular study, including informed consent to eligibility screening and tracking of adverse events.

2. Performing an assessment of risk. Risk assessments requires determining specific causes of risk and the effect of study errors pertaining to risk.

3. Developing a comprehensive monitoring plan. According to the FDA’s guidance regulations, RBM plans should “describe the monitoring methods, responsibilities, and requirements of the trial.” Planning is responsible for communicating risks and monitoring procedures to each party involved in trial RBM.

With Case Report Forms (CRFs) now being completed with cloud electronic data capture (EDC) software systems, (or entered directly using eSD (electronic source documents) or ePro (electronic patient reported outcomes),  it is possible to to collect CRF data into a centralized database, accessible by all parties, along with a full register of operational, quality controls, efficacy and clinical safety data from all sites, subjects and devices across multiple studies.

Remote risk-based monitoring algorithms are designed for scanning the data for specific and calculate specific metrics that trigger alerts to the clinical data management team – for example excessive query rates or the cycle time from patient visit complete to eCRF data entered both of which are indicative of data quality issues at sites

Remote risk-based monitoring metrics may be considered in two main categories:  patient safety risk (for example trends of vital signs) and data quality risk (for example long data entry cycle times).

When human study monitors visit sites once every 4-6 weeks, it is hard to catch slow-moving, high risk events.  Human brains are not good  at identifying and classifying  small numbers of slow-moving events, although they can easily identify a face moving quickly across a basketball court. 

Computer algorithms are outstanding at identifying anomalies in time-based clinical data found in clinical trials – and this is where RBM will have its biggest payback for you in your study.

What comes next?

What comes next is to make an accurate assessment of the data model. A single number, or a graph displaying data points, data in any form, has no meaning until the data is coherent and understood by a clinician with how the data is understood, and interpreted with how it can produce increased or mitigated risk to the patient, and the study as a whole.

Making RBM your study’s mantra

Consider this example: if a medical device study has a greater than expected negative-result reporting rate, are we talking about the great attention to detail of the study staff, or the negative performance of the interventional device? How do we determine this?

The answer is that you need to dive in deeper, and have monitors in place that are not only looking at the data, but observing whether the site is performing at the maximum study level, or whether the data is plainly reflecting that the device is effective for the study subjects.

An essential component in performing RBM is that it is required to guarantee that it can ensure that your data collected either indicates that risk is assessed, and that analysts can repeat, from site-to-site (if multiple sites are used) consistent performance that will protect study subjects, and reduce the time and effort it takes to identify risks.

The next crucial course of action during the study is to use the findings to take a direct course of action. With a clear understanding of a site’s specific risk level, CROs should be designated to visit the site(s) and take appropriate measures to alleviate site risks.

With a coherent and thorough understanding of study risks, CROs will take the necessary steps to not only reduce patient risks, but reduce future patient risks, helping your studies run more efficiently and save future costs.

What to look out for, and how to approach challenges

In our experience it has been proven that combined analysis, EDC system automation, visualisation tools, along with the monitors’ innate abilities, established processes for corrective actions have resulted in an efficient and effective uncovering of risks. This can, and will, put data and patients, and thus the entire study in jeopardy. What CRO monitors should look out for include:


– Fabricated, false or manipulated data (remember Theranos – where the financials were as cooked as the results?

– Missing data omissions

– Deviations from protocol, poorly trained clinician data submissions

Take for example if there was data that showed an alarmingly  high rate of a rise in blood pressure for study subjects. Monitors should not only be looking into the increase in blood pressure, but also examining whether the data being submitted is accurate

Monitors should consider whether the submitted data is wrong, or if the data is sound and the device is at fault or patients were taking conmeds that contraindicated the treatment.

RBM as your guiding light


RBM is not just second guessing every step of your clinical trial. It is a culture of study operations. The goal is to ensure that a CRO is approaching studies with not only patient care in mind, but also study efficiency and lower study costs. This is based upon past assessment of data monitoring and monitoring abilities. RBM is a CRO’s past, present, and future for data assurance and patient safety.

How to sustain high patient compliance in medical device trials

A comparison between pharma trials and medical device clinical trials

Truth or fiction - medical device trials are simpler than pharma

The differences between medical device trials and drug efficacy studies are similar to the differences between starting a law firm and starting a business training runners – while both are businesses in a basic sense, they each have specific criteria to adhere to regarding government regulation of standards and practices, and varying endpoints to determine whether the customer/client is satisfied and a task is finally complete.

There is also a vast difference in numbers and uses. More than 500,000 different types of medical devices are produced globally – compared to 20,000 medicinal products – each in service of vastly diverse indications.

For the law firm – regulatory compliance impinges on the lawyer and for running – compliance hinges almost totally on the runner.

But the analogy is not arbitrary.   Pharma trials are like law firms – where protocol compliance is primarily the responsibility of the principal investigator.     Trainers are like medical device trials – where protocol compliance is primarily the personal responsibility of the runner (the patient).

Before we examine and reveal whether or not clinical monitoring of medical device trials is easier than clinical trial monitoring for pharmaceutical trials, it is better to understand that first of all they are significantly different from one another.

Simpler, or just different?

During pharmaceutical testing, extensive clinical trials are necessary to evaluate the efficacy and safety of each and every drug, every single time pharma develops a drug in hopes of bringing it to market. With regards to medical devices, some do not even require a trial whatsoever. For example, a simple bandage is indeed considered a medical device, and subject to regulatory body regulation. Yet, a bandage for protecting a healing, minor laceration will not require a clinical trial during design and development.

When we refer to a “medical device”, it is a blanket term and can refer to everything from gauze or scalpel to a prosthetic limb or stent. For devices that pose a risk to patient safety, a trial must be conducted. That being said, while every drug needs to be tested through Phase I to Phase III, and monitored after it is brought to market during Phase IV, medical devices that pose minor risk to patient safety do not require the degree of testing that higher risk devices do.

While many medical devices are exempt from clinical trials pre-marketing, interventional medical devices for chronic diseases like migraine and chronic constipation require a series of clinical trials even for predicate devices.     Since chronic disease involve home use medical devices – the issue of patient compliance becomes acute and real-time clinical becomes a critical success factor for the medical device clinical trial. 

Given unique indications and interventions, for high-risk medical devices, such as powered units like pacemakers or radiation generators like x-rays, the clinical trials required before bringing them to market can rival pharmaceutical testing in scope and complexity. Both industries require regulatory compliance, for example with 21 CFR Part 11 for electronic record documentation and storage.

Again, it is like comparing a running trainer to a lawyer – the trainer needs to ensure that she keeps her runners on track with a safe but continuously-improving running schedule (volume and speed), and the lawyer needs to sustain a practice that meets regulatory and ethical standards. Even though each vocation presents their unique challenges, both the trainer and the lawyer need to file their taxes at the end of the year or answer to the IRS.

That being said, when all is accounted for, the trainer probably has a more challenging compliance situation than the lawyer. Let us examine this question by  comparing medical device clinical trials to drug trials.

Randomization vs fixed group

One key difference during the clinical trial of a drug or medical device may be the randomization of study subjects. Pharma demands that patients be randomized when monitoring and evaluating a drug’s efficacy and side-effects, and drugs are wildly less predictable than a device.

Placebo monitoring

When developing and testing a device, regardless of whether it is an MRI or a syringe, there is no placebo to test it against – the device is either performing as expected or needs adjustments/redesign. Drugs are metabolized by the body, whereas devices simply demonstrate their functionality or not.

In this case, medical device trials are measurably simpler than their pharmaceutical counterparts, requiring less study subjects and less long-term monitoring before making it to market.

Number of study sites and subjects

Pharma clinical studies, especially Phase II and beyond, involve a vastly greater number of study subjects (Phase III is typically in the thousands) across any number of study sites, often in multiple countries. Medical device clinical trials testing for efficacy may involve smaller numbers of sites and subjects – on the order of 10-20 sites and less than 1000 subjects. However, the somewhat lower number of subjects is more than offset by generally large numbers of variables that are collected per subject – typically as much as a 1000 variables not including repeat visits.

Blinding and clinical monitoring

Drug trials, in an effort to mitigate bias toward efficacy, are conducted using blind controls – where the patient knows not whether they are being administered an active drug or placebo, or double blind – where neither clinician nor subject are aware of placebo or drug during testing.

Medical device studies’ requirements for blinding  and clinical monitoring in safety and efficacy studies can be quite complex with requirements for CRF and user role level blinding, in order to prevent investigators from being biased during recruitment and in order to prevent blinded evaluators such as histologists from being influenced by site findings. In this respect, medical device trials may actually have a higher level of complexity for clinical trial monitoring due to blinding requirements. When patients are blinded, ensuring patient compliance becomes a difficult technical challenge for EDC systems and generally requires a separate CTMS and ePRO system which then cascades into HIPAA and GDPR privacy requirements. 

CRO staff training

With pharmaceutical clinical trials, sponsors need to ensure that staff are trained in the data capture system planned for the study, whether it is paper-based or EDC. Either way, except for brand new clinicians, monitors, managers, etc. it is far more likely than not that the CRO staff have experience with paper systems.

If switching to EDC from paper, the bulk of the data capture and monitoring logic stays the same, and minimal onboarding (2-4 weeks) is required. After being initially trained, CROs have an understanding of the system, and the knowledge can be applied to future studies without a hitch.

However, this is not the case for connected medical device studies, which are much more complex concerning CRO training. With drugs, pharmacologists do the legwork with regards to design – CROs simply run the trial, collect data, and monitor results. It is repetitive, study after study.

Medical devices are new technology each time they are studied, and the complexity and/or safety risks of some Class III devices can be staggering. A CRO with ample drug trial experience cannot apply their knowledge to a device study without first training clinicians, monitors, managers, and support staff with proper device operation. Especially for CROs not yet experienced with a device of similar nature, the device conduct training can be exhaustive and intensive, taking far more time and resources to amply familiarize staff with the device being tested.

So, truth or fiction?


The answer is – both. In many regards, device trials are simpler to conduct, but they present their own unique challenges for patient compliance that pharma often ignores – since the CRO is designating responsibility for patient compliance to the sites.

While both types of trials need to be equally compliant with regulatory guidelines, medical device trials pose sizable challenges due to patient compliance issues – whether compliance of patient-reported outcomes or whether compliance of patients to the treatment protocol.

Is social networking a threat or an opportunity for patient compliance?

Climbing the mountain of compliance

As CEO or VP Clinical and regulatory, do you see social networking as an opportunity or as a threat for your medical device clinical trials ?

From a cyber security and privacy perspective, an immediate reaction might be to focus on threats to privacy and intellectual property and answer no to the above question, but social networking is so part of our lives today, it is impossible to ignore. Your patients and your site coordinators are on Instagram whether you like it or not.

The challenge of privacy, GDPR and HIPAA Security Rule compliance for medical device clinical trials is no longer a problem to be solved by lawyers. The value and advantage of ePRO, connected medical devices, eSource and cloud EDC make these technologies a must for medical device clinical trials.

While privacy regulation are obvious constraints, there is a tremendous opportunity for social media and private social networking technologies to accelerate the patient recruitment process, support patient engagement and enrich the medical device clinical operations team with better information regarding patient compliance in medical device clinical trials.

For home-use and chronic disease medical device – the objectives of high levels of patient compliance are top priority for every medical device CEO.

The value of high levels of patient compliance to the clinical protocol are clear to medical device CEOs who embrace online  and modern cloud technologies.  But the importance of patient compliance in clinical trials appears less clear for big pharma who are reluctant adopters of private social networking and social media in general..

… So why do big pharma have a problem with social?

Big pharma are online with a wealth of information as a public service. Medical devices, esthetics and food supplements have run their business online for years. 

While big pharmaceutical companies are not major social networking players – they do know how a lot about online marketing.  Let’s talk about work that Sanofi-Aventis has been doing online since 2009.

Sanofi Aventis portfolio includes research and development and manufacturing of new medications. They cover 7 major therapeutically areas: cardio, thrombosis, oncology, diabetes, CNS, internal medicine and vaccines. The site has loads of information concerning the company such as: Press releases, material targeted to draw the attention of possible investors, access to the Research and development section where you can learn about the newest medications as well as their future plans and their clinical trails plus guests can apply for the different clinical research experiments.

On the other hand – it’s easy to try out social media and see if you get traction. The energy barrier is so low and the leverage on Youtube is so high, it’s an irresistible force pushing on a very heavy object like this:

Beautiful landscape
This is an immovable object.

Behind the science

Sanofi Aventis and AstraZeneca Launch YouTube Sites – Social media has been a buzzword in the pharmaceutical industry for the past few years (see ePharma Summit), but few companies have crossed the line into the world of social networking or conducting two-way conversations with patients online.

Feb 18, 2009 

The Sanofi channel is part of its integrated GoInsulin campaign, an unbranded health education program designed to give people more information about diabetes and serve as a launching pad to the Sanofi homepage. It features an array of patient videos and a link to an off-site, online game that separates the myths about insulin from reality. The channel has no branded drug material, but lists the company’s name below the top banner.

Although social media like Twitter is dominantly about personal opinions and experiences, social software such as blogs, micro-blogs and file sharing have important collaborative applications.

For example – like how to integrate all the information and care of a patient with multiple issues and care-givers (a typical MSA patient will have a GP, neurologist, speech therapist, physiotherapist, nutritionist and primary care giver at home who is usually the  husband or wife of the patient with problems of their own. Speaking before a conference of the Case Management Society of America in October 2007, Tim Rothwell from Sanofi Aventis discussed their commitment to help resolve problems of collaborative care

The issues and challenges of poor transitions of care, said Rothwell, are critically important to him personally and to Sanofi-Aventis as a company. ‘The problem, of course, is a healthcare system that, for many – particularly those who get bounced around within it – is fragmented and sometimes even frightening,’ Rothwell observed. ‘For those who have family members or friends who have experienced repeated encounters with the healthcare system, the only consistent thing they believe it delivers is confusion and, sometimes, flawed outcomes.’


Patient compliance automation

30x faster than PEOPLE

If you want to learn more about how to integrate your connected medical device, or if you are interested in helping with the project, give us a buzz and ask Batya or Danny for a demo.


Thanks for reading!

The role of your biostatistician for success in medical device clinical trials

medical device clinical trials

The importance of biostatistics in medical device clinical trials

In order to understand the importance of biostatistics in medical device clinical trials we talked to Dr. Lisa Deutsch.

Lisa Deutsch, Ph.D is a Managing Partner at BioStats Statistical Consulting Ltd. Lisa is one of the rare breed of biostatisticians that are specialized practitioners in biostatistics as consultants to sponsors in medical device clinical trials.

Starting off as a maths student with hopes of becoming an actuary one day, Lisa soon found out that statistics are much more interesting for her and moved into that field and as she says herself “stayed there forever”.

Her natural tendency towards all things scientific, especially in the medical field combined with the efforts of Master’s and Ph.D. supervisors (Linda Har-Nevo and Laurence Freedman respectively) effectively synthesized Lisa’s 2 passions- mathematics and biology, making her an exceptional professional in the unique space of statistical analysis planning and execution for clinical trials.

Lisa tells that while she was still at University, people started looking for consultancy and sought her advice, then got her first big time consultancy job from which eventually pulled into clinical trials and  consulting to Israeli medical device manufacturers in the are of applications for FDA approval.

The uniqueness of a medical device requires a rigorous scientific approach

Each medical device is unique in almost respects from perspective of intended use, engineering, data model, efficacy and safety.   Lisa started working in medical statistics and had to learn how to answer questions on her own and she developed her own skill-set.  That involved meticulous attention to details and thorough checking and developing a rigorous scientific approach to evaluating efficacy and safety of medical devices in clinical trials.

Trends in society drive innovative medical devices

According to Lisa, we are seeing rapid innovations and achievements in medical devices particularly in automated diagnosis and robotic procedures which are very hot fields.

People are trying to automate medicine using software as medical devices to enable quick and accurate diagnostics, to facilitate treatment, and to monitor the patients’ progress and improve clinical outcomes.

Other drivers of medical device innovations in Israel are connected with limited resources in terms of clinics and health care personnel and an aging population which create a booming market for medical device clinical trials in Israel.

As the population gets older, more people retire and the medical profession is no exception. Following the collapse of the Soviet Union, Israel experienced a huge influx of nurses and physicians. In about 20 years’ time, many of them will head towards retirement: this combined with aging population puts a great deal of pressure on Israeli healthcare.

Scientific testing in medical device clinical trials

As a result of an economy that does not have the economic resources of the US – Israeli medical device innovation is increasing turning towards automation – automation of patient adherence, automation of medical diagnosis and automation of patient monitoring.    All of this medical innovation needs to be tested scientifically in clinical trials.

The role of the biostatistician in medical device clinical trials

The role of the biostatistician in medtech clinical trials starts before the first patient enters the study with a SAP – a statistical analysis plan.  The SAP describes the planned analysis for a series of clinical trials and relates to the primary endpoints of the medical device trials, the safety endpoints and do that based on the medical device trial clinical protocol.   A SAP requires statistical expertise and abstract thinking skills.  During the medical device clinical trial, the biostatistician will be called upon to perform an interim analysis and at the end of the study a final statistical analysis to assess the efficacy and safety of the medical device.

Risk based monitoring. It’s about the people.

esource for people in clinical trials

It will come as no surprise to anyone who is actively involved in clinical trial operations that the heart of the matter in clinical trial monitoring is people.

I have written herehere and here about how people are the key assets and vulnerabilities in medical device clinical trials.

Understanding people and providing timely and high-quality communications rests is a key to data-driven decision making. There may be high value in the analysis of the data (for example – inconsistencies in dosing at sites) but if the data is not communicated in a timely and effective manner, risk based monitoring will not be effective or timely either!

MANA RBM risk based monitoring is a consultancy based in Denver CO, led by Penelope Manasco MD, that provides CRO and consulting services around risk-based monitoring. Dr. Manasco recently published an article  It Takes a Village to Achieve Risk-Based Monitoring that discusses the “people” components needed to successfully adopt Risk Based Monitoring. It also compares the roles of the Site Monitor and Central Monitor and discusses the role of risk assessment, training, and implementing Risk-Based monitoring solutions.

Good job!

There are sites who are outstanding, entering high quality data within 24 hours of the patient visit and then there are sites who take 2-3 weeks to enter data, enter future events and are not always careful regarding informed consent and participation in the trial.

This is a people issue not a technology issue.

Jenya Konikov-Rozenman is product lead at Flaskdata.io. Jenya has a masters degree in biotechnology from the Hebrew University and is a doctoral candidate at Tel Aviv in medical science. She is GCP and CRA certified and leads with super-human devotion to customer delivery.

Implementing an EDC to take advantage of risk-based monitoring

Clinical trial monitoring is 30-40% of your project costs.    At least half of that cost is manual work and SDV activities which can be eliminated with use of modern technologies like electronic source documents and remote risk-based monitoring.

In this post, we take a look at how you can automate routine data management activities and eliminate costly manual labor of your study monitors.  In order to eliminate costly people and paper processing, we need to  implement a strategy of “management by exception” or as the FDA calls it “remote risk-based monitoring“.

For the benefit of connected medical device developers who  are new to clinical trial operations and data management,  or new to electronic data capture (EDC) for clinical studies; this post starts with an introduction to FDA guidance on remote risk-based monitoring and goes on to provide some practical suggestions for using your EDC system within a remote risk-based monitoring plan for your connected medical device study.

(more…)