Living in an ideal world where the study nurse isn’t overwhelmed by IT

Tigran examines the idea of using EDC edit checks to assure patient compliance to the protocol.

How should I assure patient compliance to the protocol in a medical device trial?

I get asked sometimes whether automated patient compliance deviation detection and response  is not overkill.

After all, all EDC systems allow comparing input to preset ranges and data types (edit checks). Why not use this, already available off the shelf functionality, to catch non-compliance? As Phileas Fogg put it: “Learn to use what you have got, and you won’t need what you have not”.

Why edit checks are not enough

There are 4 issues with using EDC edit checks to enforce patient compliance:

Individual variations

The original purpose of edit checks is to catch data entry mistakes. As they are generated automatically, they need to be robust enough not to fire indiscriminately. The effect non-compliance has on clinical data can be far less clearcut. This is especially true when taking individual variation between patients into account.


Even if we were able to reliably catch non-compliance through clinical data alone, there’s the issue of timing.

Each hour of delay between non-compliance event and a prompt to return to compliance devalues the prompt. Delays could come from a) manually entering source data into EDC, b) edit check firing in batch mode rather than during data entry, c) the time needed to process the edit checks.  What’s the benefit of being told you were not compliant one week ago?

Talk of closing the stable door after the horse has bolted…

By the time the nurse contacts the patient, the damage has already been done. No reinforcement is possible, as a patient could (theoretically) be reminded about the need to be compliant with the interval of several weeks – in which case this will serve as a token reminder, nothing more.

The study nurse may not have spare time on her hands

Let’s assume we live in an ideal world, where the study nurse isn’t overwhelmed by thousands of edit checks firing for no reason, and where data flows into EDC with no delay.

Even if this is true, there’s still the small matter of actually reaching out to the patient. When compliance reaches 90% that’s considered a good result – so in the best case scenario, the nurse would need to reach out to patients in 10% of cases. Edit checks are meant to be resolved immediately. If the EDC used fires edit checks during data entry, then the data entry process will be paralyzed. If edit checks are fired in the background, then the whole data cleaning/query resolution process would stall.

Edit checks are not an operational tool

What would happen in reality, though, is that any edit checks introduced to monitor patient compliance would be overridden by site staff. Together with any legitimate edit checks designed to keep the errors out. Resulting in the same level of compliance and much dirtier database. And that’s best case scenario, if otherwise no data would be entered at all.

Tigran Arzumanov is an experienced business development/sales consultant running BD as a service, a Contract Sales Organization for Healthcare IT and Clinical development.

Killed by code in your connected medical device

patient compliance in medical clinical device trials

Are we more concerned with politicians with pacemakers or families with large numbers of connected medical devices?

Back in 2011, I thought it would only be a question of time before we have a drive by execution of a politician with an ICD (implanted cardiac device). May 2019, with mushrooming growth in connected medical devices (and after the Israeli 2019 elections), I am rethinking my risk analysis.

Consider this: If a typical family of 2 parents and 3 children have 5 mobile devices, it is a reasonable that this number will double with medical IoT and software as devices for diabetes management, asthma monitoring, fetal monitoring, remote diagnosis of children, home-based urine testing and more.

So far, it seems the politicians are still around, but the cybersecurity vulnerabilities for medical devices are growing in frequency and impacting big medical device vendors like Medtronic as reported by FDA in March 2019 – Cybersecurity Vulnerabilities Affecting Medtronic Implantable Cardiac Devices, Programmers, and Home Monitors

Audience: Patients with a Medtronic cardiac implantable cardioverter defibrillators (ICDs) or cardiac resynchronization therapy defibrillators (CRT-Ds)

-Caregivers of patients with a Medtronic ICD or CRT-D

-Cardiologists, electrophysiologists, cardiac surgeons, and primary care physicians treating or managing patients with heart failure or heart rhythm problems using a Medtronic ICD or CRT-D

-Medical Specialties

-Cardiac Electrophysiology, Cardiology, Cardiothoracic Surgery, Heart Failure

Purpose: The U.S. Food and Drug Administration (FDA) is issuing this safety communication to alert health care providers and patients about cybersecurity vulnerabilities identified in a wireless telemetry technology used for communication between Medtronic’s implantable cardiac devices, clinic programmers, and home monitors. The FDA recommends that health care providers and patients continue to use these devices as intended and follow device labeling.

Although the system’s overall design features help safeguard patients, Medtronic is developing updates to further mitigate these cybersecurity vulnerabilities. To date, the FDA is not aware of any reports of patient harm related to these cybersecurity vulnerabilities.

In Jan 9, 2017 FDA reported in a FDA Safety Communication on “Cybersecurity Vulnerabilities Identified in St. Jude Medical’s Implantable Cardiac Devices and Merlin@home Transmitter.

At risk:

-Patients with a radio frequency (RF)-enabled St. Jude Medical implantable cardiac device and corresponding Merlin@home Transmitter

-Caregivers of patients with an RF-enabled St. Jude Medical implantable cardiac device and corresponding Merlin@home Transmitter

-Cardiologists, electrophysiologists, cardiothoracic surgeons, and primary care physicians treating patients with heart failure or heart rhythm problems using an RF-enabled St. Jude Medical implantable cardiac device and corresponding Merlin@home Transmitter

Different classes of device. Different threat scenarios. A wellness app does not have the same threat model as implanted devices

I’ve been talking to our medical device customers about mobile security of implanted devices for over 7 years now.

I  gave a talk on mobile medical device security at the Logtel Mobile security conference in Herzliya in 2012 and discussed proof of concept attacks on implanted cardiac devices with mobile connectivity.

But – ICD are the edge, the corner case of mobile medical devices.

If a typical family of 2 parents and 3 children have 5 mobile devices, it is a reasonable scenario that this number will double withe devices for fetal monitoring, remote diagnosis of children, home-based urine testing and more.

Mobile medical devices are becoming a pervasive part of the Internet of things; a space of  devices that already outnumber workstations on the Internet by about five to one, representing a $900 billion market that’s growing twice as fast as the PC market.

There are 3 dimensions to medical device security – regulatory (FDA), political (Congress) and cyber (vendors implementing the right cyber security countermeasures)

The FDA is taking a tailored, risk-based approach that focuses on the small subset of mobile apps that meet the regulatory definition of “device” and that the software as a device mobile apps:

-are intended to be used as an accessory to a regulated medical device, or

-transform a mobile platform into a regulated medical device.

Mobile apps span a wide range of health functions. While many mobile apps carry minimal risk, those that can pose a greater risk to patients will require FDA review. The FDA guidance document  provides examples of how the FDA might regulate certain moderate-risk (Class II) and high-risk (Class III) mobile medical apps. The guidance also provides examples of mobile apps that are not medical devices, mobile apps that the FDA intends to exercise enforcement discretion and mobile medical apps that the FDA will regulate in Appendix AAppendix B and Appendix C.

Mobile and medical and regulatory is a pretty sexy area and I’m not surprised that politicians are picking up on the issues. After all, there was an episode of CSI New York  that used the concept of an EMP to kill a person with an ICD, although I imagine that a radio exploit of  an ICD or embedded insulin pump might be hard to identify unless the device itself was logging external commands.

See my presentation ‘Killed by code’

Congress is I believe, more concerned about the regulatory issues than the patient safety and security issues:

Representatives Anna Eshoo (D-CA) and Ed Markey (D-MA), both members of the House Energy and Commerce Committee sent a letter last August asking the GAO to Study Safety, Reliability of Wireless Healthcare Tech and report on the extent to which FCC is:

Identifying the challenges and risks posed by the proliferation of medical implants and other devices that make use of broadband and wireless technology.
Taking steps to improve the efficiency of the regulatory processes applicable to broadband and wireless enabled medical devices.
Ensuring wireless enabled medical devices will not cause harmful interference to other equipment.
Overseeing such devices to ensure they are safe, reliable, and secure.Coordinating its activities with the Food and Drug Administration.

At  Black Hat August 2011, researcher Jay Radcliffe, who is also a diabetic, reported how he used his own equipment to show how attackers could compromise instructions to wireless insulin pumps.

Radcliffe found that his monitor had no verification of the remote signal. Worse, the pump broadcasts its unique ID so he was able to send the device a command that put it into SUSPEND mode (a DoS attack). That meant Radcliffe could overwrite the device configurations to inject more insulin. With insulin, you cannot remove it from the body (unless he drinks a sugary food).

The FDA position that it is sufficient for them to warn medical device makers that they are responsible for updating equipment after it’s sold and the downplaying of  the threat by industry groups like The Advanced Medical Technology Association is not constructive.

Following the proof of concept attack on ICDs by Daniel Halperin from the University of Washington, Kevin Fu from U. Mass Amherst et al “Pacemakers and Implantable Cardiac Defibrillators:Software Radio Attacks and Zero-Power Defenses”  this is a strident wakeup call to medical device vendors  to  implement more robust protocols  and tighten up software security of their devices.

Improving patient compliance to medical device protocols with threat models

To paraphrase Lord Kelvin – “You cannot improve what you cannot measure”.

I have about 10′ before Shabbat and I wanted to offer 2 possible approaches for improving patient compliance to medical device clinical protocols.

One approach considers the patient as an attacker to the study data.  This approach considers social, cost, adverse events, personal, technical and privacy aspects as study data vulnerabilities.  The idea is to construct a prioritised countermeasure plan during the study and refine it with real-world data

The second approach uses a behavioral model as opposed to a threat model.

It assumes that patient compliance to a protocol in a trial will always be better than in real-life but that at the end of the day – people have various reasons sometimes not clearly known to themselves why the do not comply.

In this approach, a cost-effective strategy for assuring compliance post-marketing in the real-world  uses validated machine learning models of what affected patient compliance during the controlled clinical trial.   Reinforcement during the trial also reveals to the model what worked and what didn’t.

In order for a medical device company to decide what model works best for them – they must measure the movement and value of their data, and weigh that in terms of their data model.

How to become an insights-driven clinical operations manager

In my post Putting lipstick on a pig of eCRF I noted that good online systems do not use paper paradigms. In this post – I will develop the idea of using digital / mobile /automation to become an insights-driven clinical operations manager.

Insights-driven clinical operations practices are more important than ever if you want to operate a global multi-center medical device trial with speed at scale.

I’ll show you how the democratization of data insights with a comprehensive patient compliance monitoring (PCM) platform can help you improve:

1.Medtech Developer and clinical trial operations productivit

2.Site coordinator experiences

3.Treatment and patient reported outcome compliance

Democratization of data insights

In the old world of legacy clinical operations, you waited for site monitoring visits to get feedback on deviations. Remote monitoring and modern EDC systems have enabled us to take a big step forward, although it seems that CRO central monitoring activities move the analytics and deviation detection and response further away from the site operations team and sponsor project manager.  Central monitoring for clinical trials is like the old Soviet central planning committees.   The exact opposite of democratization. Medidata purchased Shyft Analytics last year.  The Shyft platform combines different data sources and a data preparation pipeline of data cleaning, data transformation, normalization, metadata. The idea as I understand it, is to use both clinical trial and real-world commercial data to share data and insights in pharma clinical development and commercial teams. It seems that the system is not appropriate for neither medtech developers nor site operations teams.

Shyft Analytics is a beautiful example of democratization of data insights inside a pharmaceutical company but it is several levels removed from site clinical operations.

Medtech Developer and clinical trial operations productivity

The medtech engineering teams are often bit actors in the medical device clinical trial. For connected devices, this is a mistake in our experience.   A connected medical device, software medical device app or home device uses APIs to external systems (which may fail) and embedded hardware and software (which may fail).     Feedback from the sites in real-time to the device engineers will help fix issues and improve availability and performance of the device or enable swapping out faulty devices.

Here is a screen shot from a Flaskdata instance clearly showing 2 patients and 2 different sites with device failure issues.

Site coordinator experiences

Here is another example of how integrating the device logs with the PCM platform helps the site understand quickly why the patient is having treatment compliance issues.

The site coordinator, study monitor and study PM can now see the big picture for this patient – in this case, the patient has a device issue which impacts treatment compliance and reporting issues.

Treatment and patient reported outcome compliance

The data is then aggregated using a generalized patient compliance model to provide an overview and comparison of how the sites are performing on their patient compliance metrics. The site coordinator can drill down to individual patients as we saw above and run a playbook response to resolve the deviations. In a glance, we see that PRO compliance is very high at almost all the sites while the treatment compliance is at 71%.

People time is real-time

The word “democracy” (Greek: δημοκρατία) combines the elements dêmos (δῆμος, which means “people”) and krátos (κράτος, which means “force” or “power”), and thus means literally “people power”.  With real-time PCM that is always accessible to all the members of the extended medical device clinical trial, we can achieve true democratization of data insights.




Israel Biomed 2019-the high-social, low stress STEM conference

Impressions from Biomed 2019 in Tel Aviv

This week was the annual 3 day Biomed/MIXiii (I have no idea what MIXiii means btw) conference in Tel Aviv.  The organizers also billed it as the “18th National Life Science and Technology Week” (which I also do not know what that means). This was a particular difficult time for a conference of medical device and pharma in Tel Aviv since it coincided with the Eurovision 2019 activities – and the traffic was tough.

There were a huge number of lectures and participants from all over the world and I suppose from that perspective, the conference is a success and tribute to the burgeoning Israeli biomed industry.  Forbes calls Biomed “The High-Paying, Low-Stress STEM Job You Probably Haven’t Considered”.  I think that this is probably a good description for the conference – high participation but low stress.

My colleagues and I come to the conference to network, schmooze, meet customers and suppliers.  It’s a good opportunity to take a few meetings, say hi to friends and hustle for new business.  Having said that, I did meet a few really interesting companies:

RCRI – is a Minneapolis MN based medical device CRO.  I met Todd Anderson and his boss Lisa Olson and pitched our approach for fast data in clinical trials to assure high levels of patient compliance to the protocol and submit faster to FDA.    Todd and Lisa get it and they were open about the CRO business model being more people-hours not speed.     They seemed genuinely interested in what we are working on but its hard to tell with Americans.

Docdok Health – is a startup founded by Yves Nordman, who is a Swiss MD living in Carmiel.  It’s a doctor-patient communications platform beginning to branch out into Post-marketing studies with RWD.    We shared demos and it seems that there is synergy between our regulatory platform and their post-marketing work.

Resbiomed – met Alex Angelov, the CEO.  Alex is leading a consortium including Flaskdata, Carl Zeiss, Collplant, PreciseBio and Pluristem for a Horizon2020 submission for an amazing project for an implant to the cornea.  Dan Peres from Pluristem got us together.   Cheer for us!

BSP Medical and ICB (Israel China Biotech investment) – my buddy Hadas Kligman literally took me by hand to visit to Yehuda Bruner and Andrew Zhang and I did my 60s elevator pitch on getting medical device companies to FDA/CFDA 6-12 months faster.   We agree to talk after the conference.

Butterfly Medical –  I met Idan Geva, the CEO last year at Biomed – we ate lunch at the same table.  I pitched him but he was uninterested – they were using EDC2Go – and he didn’t want to hear other options.     At the Minnesota pavilion talking to Todd Anderson from RCRI,   Idan shows up and looks at me and says “Heah – Hi Danny – I left a contact me request on your web site yesterday and no one got back to me. I said shame on us.  He says – he was referred to us by someone from Florida who used to use Medidata.  I asked where/who? was it Miami?  He says yeah it was Miami and checks his phone – says its someone from Precision Clinical Research that are using Flaskdata and recommended.    (Precision is one of our customer’s Miami sites).  I asked what happened to EDC2Go – he said well you know – they are end of life (I think this means the end of low-cost EDC) and we are now entering questionnaires manually on paper and it is driving us crazy.   He said – can you stick around and give us a demo at 15:00?  I said sure.  We met at 15:00 by the bar upstairs in the David Intercontinental and I demoed the system – he said “Show me the Forms designer”. I showed him.  He says “show me how CRC enters data” – I showed him.  He says “Show me how to extract data” – I showed him.  I think he actually did not believe how fast the Extract to CSV process was and asked me twice if that was the data.  In the end – the format of Mac Numbers was a bit strange for him. I showed him a quick presentation – and he saw that Serenno is a customer – and says – “Heah Tomer is a neighbor of ours in the incubator in Yokneam”.    He asked how much and I said $2K for a basic onboarding package and $1500 / month.  Or $10K and we will build the CRF (their CRF is super simple btw).  He wanted a discount, being Israeli.  I said – “lets meet with your clinical person and get her to buy-in to the solution.  If she buys in – you and I can talk business but before that, there is no point horse-trading.

Count the probabilities of this happening and you will see that it is an impossible event.

Thursday I went back to demo Todd and meet Dr Yael Hayun from Syqe Medical. Yael is one of the most impressive people I’ve met in a long time. She is an MD from Hadassah and one of the movers and shakers in LogicBio Therapeutics.    After we chatted – I told her that Syqe is lucky to have her onboard.   I did our Today is about Speed presentation and a short demo. She was suitably impressed and then mentioned they had met with a Danish EDC company called Smart Trial – which turns out is yet another low-cost eCRF provider.   I said look – eCRF is like 10% of the solution you need – in the case of Syqe, you have a digital inhaler and with cannabis, you are going to have a lot of concerns about patient compliance.

This is what we do – fast data collection from patients, investigators and digital inhalers and automated deviation detection and response.

On the way back – huge traffic from Eurovision.   Didn’t hear a single lecture but the meetings and people were outstanding.


Perverse incentives

The perverse incentive for the high costs of medical devices and delay to market

The CRO outsourcing model and high US hospital prices result in higher total CRO profits via higher costs to companies developing innovative medical devices.   These costs are passed down to consumers after FDA clearance.

We’ll take a look at the cost dynamics of medical device clinical trials and the clinical trial value chain.

We’ll then consider an alternative business model that changes the way medical device sponsors conduct clinical trials, reduce their costs by 70-80% and shortens time to FDA submission.

The high costs of US hospitals

By 2000, the US spent more on healthcare than any other country, whether measured per capita or a percentage of GDP.

U.S. per capita health spending was $4,631 in 2000, an increase of 6.3 percent over 1999. 4 The U.S. level was 44 percent higher than Switzerland’s, the country with the next-highest expenditure per capita; 83 percent higher than neighboring Canada; and 134 percent higher than the OECD median of $1,983. 5

It’s the prices, stupid.

In 2011, the US Affordable Care Act set a requirement for MLR (Medical Loss Ratio) that insurers must spend 80-85% of revenue on medical services.    This reduced insurer margins, and drove up hospital prices to make up for lower margin.

The CRO business model

CROS (clinical research organizations) are outsourcing businesses that provide an array of services for clinical trial management and monitoring, reporting and regulatory submission.   For medical device studies, CROS employ 2 basic outsourcing models, people sourcing and functional sourcing. In people out-sourcing, the medical device company is responsible for managing contractors. In functional outsourcing, the company may buy a set of functions, for example study monitoring and medical writing.

Neither CRO model has an explicit incentive to complete a study faster since that would reduce outsourcing revenue for the CRO. The more time a CRO spends on monitoring, site visits, SDV and study closeout, the more revenue it generates.

A medical device sponsor may elect to do it himself which shifts the CRO cost to an internal headcount cost supplemented with additional costs for consultants with risk and time delays by not having the CRO expertise and infrastructure. There is tacitly no free lunch, as we will discuss later in this article.

The result is a perverse incentive for delay and higher costs to bring innovative medical devices to market.

The CRO business model combined with higher hospital prices drive higher total profits via higher costs to customers. The higher cost of innovative medical devices is then passed down to consumers (patients) after FDA clearance.

Consumer value chains

A consumer value chain looks generically like this:

Suppliers -> Distributers -> Consumers

By the early 90’s, the PC industry led by Intel and Microsoft used a 2-tier value chain:


Resellers were further segmented according the customer size and industry segment – Retail, Large accounts, SMB and VARS (value-added-resellers) selling their own products and services to a particular industry vertical.   The PC industry value-chain model left Microsoft with 50% of the SRP (suggested retail price) and delivered products to customers that were 45-50% less than SRP, leaving the channel with 0-5%.

The channel was forced to implement extremely efficient operations and systems and sell value-added services and products in order to survive.

By the new millennium, Apple introduced a 1-tier model with a user-experience designed and controlled by Apple.

The Apple 1-tier channel looks like this:

Apple->Apple Stores->Consumers

Eventually the Apple channel model broadened to include a 2-tier model similar to PC industry:


By the mid-2000s, Amazon AWS (and generally the entire cloud service / SaaS industry) evolved the channel model to 0-tiers with a direct subscription and delivery model.


As AWS grew and introduced spot pricing, an aggregation sub-market developed, looking extremely similar to movie and TV distribution models.


AWS also became a distribution channel for other cloud products similar to content distribution (Think Netflix).

Third-party products->AWS->Consumers

Outstanding user-experience and aggregation are the hallmarks of companies like Airbnb, Netflix and Uber.    

The common thread is that AWS and Netflix deliver a digital product end-to-end, whereas Airbnb and Uber aggregate trusted suppliers inside the Airbnb and Uber brand environment and provide an outstanding and uniform user experience to all the consumers.  This is in contrast to the variegate user experience a customer got from the 90’s Microsoft channel. There are great resellers and terrible resellers.

We will return to user experience and aggregation later.

The medical device clinical trial value chain

The first published RCT (randomized clinical trial) in medicine appeared in the 1948 paper entitled Streptomycin treatment of pulmonary tuberculosis.

The clinical trial value chain for medical devices looks strange once after the historical perspective of how Intel, Microsoft, Amazon and Netflix evolved their value chains.

The medical device clinical trial value chain has 3 tiers with patients that are both suppliers and consumers.

Patients->Hospitals->CROS->Medical device companies->Patients

A dystopian user experience

Little has changed in the past 71 years regarding clinical trials.    Clinical trials and hospital operations now have a plethora of complex expensive, difficult-to-use IT with a value chain that provides a dystopian user experience for hospitals, patients and medical device companies.

HCOs (healthcare operators) rely on data collection technology procured by companies running clinical research (sponsors and CROs). This creates a number of inefficiencies:

1 – HCO staff are faced with a variety of systems on a study by study basis. This results in a large amount of time spent learning new systems, staff frustration and increased mistakes. This is passed on in costs and time to sponsors after CRO markup.

2 – The industry is trending towards the use of eSource and EMR to EDC data transfer. eSource/ePRO tools need to be integrated into the patient care process. Integration of EMR with EDC becomes logistically difficult due to the number of EDC vendors on the market (around 50 established companies).

3 – Siloed data collection in hospitals with subsequent manual data re-entry results in large monitoring budgets for Source Data Verification, and delays caused by data entry errors and related query resolution. Delays can be on the order of weeks and months.

4 – Use of multiple disconnected clinical systems in the hospital creates a threat surface of vendor risk, interface vulnerabilities and regulatory exposure.

Losing focus on patients

One of the consequences of the 3-tier medical device value chain is loss of focus on the patient user experience.  Upstream and to the left, patients are ‘subjects’ of the trial. The patient reported outcomes apps they use vary from study to study. Downstream and to the right (what FDA calls ‘post-marketing’), patients are consumers of the medical device and the real-world user experience is totally different than the UX in the study.   The real-world data of device efficacy and safety is disconnected from the clinical trial data of device efficacy and safety.

Clinical trial validation

Patient compliance is critical to clinical trial validation of medical device. Who owns patient compliance to the research protocol?  The medical device sponsor, the CRO, the hospital site or the subject?   The CRO may not collect a patient compliance metric since he outsources to the hospital. The hospital may not have the tools and the medical device company is outside the loop. My essay on determining when patient compliance is important in medical device trials goes into more detail on the problem of losing focus on the patient.

Vertical integration and aggregation

We previously made a qualitative claim that hospital site costs are high for medical device studies.  How high are they relative to consumer healthcare?

In a medical device trial recently done on the platform, the sponsor paid the hospital investigatory sites $700K for a 100 subject, 7 month multi-site study. (There were no medical imaging and blood test requirements).

In 2016, Medicare Advantage primary care spend was $83 PMPM (per member per month).      Let’s say that a premium service should cost $100 PMPM.    Let’s use that as a benchmark for the cost of processing a patient in a medical device trial.  Take this medical device Phase II medical device trial with 100 patients, running for 7 months:

That’s 100 x 7 x 100 = $70K for patients. Not $700K.

Perhaps the law of small numbers is killing us here.  The way to solve that is with aggregation and vertical integration. Let’s return to the medical device clinical trial value chain. As we can see, there are too many moving parts and a disconnect between patients in the clinical studies and consumers in the real world.

Patients->Hospitals->CROS->Medical device companies->Patients

One alternative is to integrate backwards and to the left.   This requires managing hospital site functions and to a certain degree is done in the SMO (site management organization model).

The other alternative is to integrate forward and to the right.   This is the path that Airbnb, Uber and Netflix took aggregating consumer demand with an outstanding user experience.  The aggregation gives Airbnb, Uber and Netflix buying power to the left, enabling them to choose the best and most cost-effective suppliers.

The value chain would then look like this:

Suppliers->Medical device companies->Patients

This is a model that we see increasingly with Israeli medical device vendors with limited budgets.   The Medical device company uses a cloud platform to collect digital feeds from investigators, patients and devices and automate monitoring for deviations. Focus on the patient user experience begins with design of the device and continues to post-marketing. Aggregation of patients enables purchasing power with suppliers – research sites, clinical consultants and study monitors.

Flaskdata - esource, ePro, patient compliance montoring,

Short-term versus long-term cost allocation

The reality is that using a technology platform for vertical integration is more expensive initially for implementation by the medical device company.   It should be.

Under-funding your infrastructure results in time delays and cost spikes to the medical device sponsor at the end of the study.

The current CRO methodology of study close-out at the end of a clinical trial lowers costs during the trial but creates an expensive catch-up process at the end of the study.

The catch-up process of identifying and closing discrepancies can take 2-6 months depending on the size and number of sites. The catch-up process is expensive, delaying submission to FDA and revenue since you have to deal with messy datasets.   The rule of thumb is that it costs 100X more to fix a defect after the product is manufactured than during the manufacturing process. This is true for clinical trials as well.   A real-time alert on treatment non-compliance during the study can be resolved in 5 minutes.   By waiting to the end of study it will take a day of work-flow, data clarifications and emails to the PI.


Vertical integration reduces costs and delay at study-end with continuous close. It is more expensive initially for the medical device company and it should be because it accelerates time to submission and reduces monitoring and close-out costs.


When is patient compliance important in medical device clinical trials?

In this post, Danny Lieberman, founder of , discusses when patient compliance is crucial for your medical device clinical trial and when patient compliance is a negligible factor to success of the study.

From adverse events to patient compliance

My original goal for was  to use machine learning to predict onset of adverse events during interventional medical device clinical trials. 

For that goal, we needed data, so we started by providing cloud EDC services for medical device clinical trials with high-touch personal service and attention to the quality of the data model.  Very quickly – it become apparent that we did not have enough data (and after 20 studies, hundreds of sites and thousands of patients), we still do not have enough data to predict adverse events.

However, after performing 6 digital clinical trials in 2 chronic disease indications (acute migraine and chronic constipation) we had an epiphany – “PATIENT COMPLIANCE IS KING”

Customers using the platform for digital clinical trials, collected data via the EDC from investigators, collected data from patients (via our ePRO app) and collected data from connected medical devices (via the medical device API). The evidence was overwhelming :

Patient compliance to the protocol is an acute issue and critical success factor to the success of a connected medical device clinical trial. 

Or is it.

Who owns patient compliance?  The sponsor, the CRO, the site or the subject?

This discovery of the importance of patient compliance made a profound impression on us because it came from customers and empirical data they collected in our EDC systems.   This impression would not change, although we began to hear dissenting opinion on the importance of, and responsibility for patient compliance in clinical trials.

Public discussion on trends in the clinical trials industry is heavily influenced by big pharmaceutical companies, big CROs like PPD and IQVia and a $70BN/year clinical operations services industry that deal largely with oncology and biotechnology.   When we spoke to biotech prospects about the ability of our digital clinical trials platform to accelerate time to regulatory submission and assure high levels of patient compliance – people smiled and said “Well automated compliance monitoring is an innovative approach, but in fact, patient compliance is not important to us”.   

We then spoke with the Israel country manager of one of top 3 global CROs – and they said “Interesting question.   We collect many clinical trials operations metrics, but patient compliance to the clinical protocol is not a metric we collect”.   I asked – “In that case, who is in charge of patient compliance? and the answer was – the sites”.  In this scheme of things, if patient compliance is not a CRO metric, then the sponsor has a blind-spot to what is possibly, the single most important factor to the success of his connected medical device clinical trial.  Or not.

After that, we spoke with the country manager of one of the top 3 pharmaceutical companies and Israel and he told us again “Patient compliance is a non-issue for us.  Patients come to the hospital and get treatment and there is no problem”. I asked him “What about psychiatry trials?”  He replied – “well yes, everyone knows that psychiatry trials have acute issues of patient compliance”


We then went back and did the most logical thing – searching in Google for “the critical success factors of clinical trials” and there are 290 million results and a ton of empirical evidence and academic and industry research on the importance of patient adherence in clinical trials.

And this vast body of empirical data is dealing primarily with drug trials, not medical device trials.   The VP Clinical of a gene therapy prospect (who had previously worked at a medical device company) told us that in gene therapy patient compliance is negligible while in medical device trials, patient compliance is acute.

Hmm again. So what does Google say?

The high failure rate of clinical trials has significant impact on providing potential curative treatments to patients in need….

One key factor that has been identified in the high failure rate of clinical trials is the adherence of patients participating in clinical trials to the dosing, treatment, and study procedures that are very carefully put in place in clinically rigorous protocols. Due to the rigor that is required in order to demonstrate an “effect” relative to a standard of care treatment, even a small deviation in medication adherence can result in a trial failing to meet its pre-specified clinical endpoint.

Additionally, the current nature of clinical studies include strict timelines and competition among sites to enroll eligible subjects which can many times result in the inclusion of subjects that are simply not “medication-compliant”. The issue of medication adherence is therefore one key factor sponsors should carefully look at monitoring closely when designing and planning the medical and operational oversight of their trials.

Unfortunately, the issue of medication adherence many times goes unmanaged and falls solely on site staff to oversee. As clinical sites are many times running multiple concurrent trials and are themselves pressed to remain productive, the one-on-one daily management of medication adherence of study subjects can many times be neglected. It is therefore in the best interest of the patients in need that sponsors look towards solutions that can help to support their clinical sites in providing additional resources to maintain close and frequent interactions with subjects enrolled in key studies. It simply is no longer sufficient to rely solely on very busy clinical practices to ensure successful adherence of patients in enrolled in trials.   

See Compliance – a key factor to a successful clinical trial.

I’m confused.  Is compliance the best interest of the patient or the best interest of the PI, or the sponsor or all of the above?  We know that the PI must monitor participants’ compliance with study requirements. Failure to monitor patients adequately can sabotage the entire study and damage the site’s reputation. 

CROS not collecting patient compliance metrics. Busy sites. Lack of tools. PIs who are generally not hands-on with the patients.   Sounds like a classic finger-pointing situation.  

We hear of the importance of site selection, but if patient compliance is not a CRO metric, then how do we measure site performance properly?  

The 4 quadrants of patient compliance

In fact, the question of HOW to measure the importance of compliance is intimately related to 4 factors – and interestingly enough is totally unrelated to the site or the PI.  The 4 factors of patient compliance are:

  1. How do you collect data?

  2. What is the indication?

  3. What does the product do?

  4. How involved is the patient in the treatment?

In order to understand why there is dissenting opinion on the topic of the importance of patient compliance – we can map life-science products into 4 quadrants:  (Patient-centric, Digital, Investigator-centric, Implanted).  The top right quadrant in green is a digital clinical trial for chronic disease, the top left is a traditional EDC operation with varying degrees of patient involvement, the bottom left is little patient involvement and EDC data collection from paper source and the bottom right is no patient involvement but with data collection from implanted devices (an interesting and extremely important use case in its own right).

The above picture tells the whole story.

Patient compliance in clinical trials is crucial in digital clinical trials and patient-centric trials using traditional EDC and patient reported outcomes.

In the end it is about the patient – not the PI, not the site operations team and their training, policies and procedures and not about the CRO.

But hey – this is something any sponsor worth their salt already knows.

Israeli Medical device innovation for high patient compliance

One of the most challenging problems in medical device clinical trials and in real-life is how to achieve high levels of patient compliance to the protocol.    
Automated patient compliance technology in medical device clinical trials is confronting CROs with an unpleasant status-quo of SDV as a low-value-add, high-cost, time-consuming activity for patient compliance assurance.  The approach that this company takes provides continuous patient monitoring without requiring patient compliance at all. 

EarlySense is an Israeli medical device that is based on a paddle placed under the patient’s mattress that continuously monitors patient movement, HR and RR trends.

The EarlySense device helps facilitate timely interventions for patients in non-ICU settings by adding a layer of care with continuous monitoring, drawing attention to those who may show early signs of deterioration and may require clinical intervention.

Since the EarlySense contact-free sensor (it looks like a small plate)  is placed under the mattress and there are no leads attached to the patient – there is no need for patient compliance.

EarlySense bedside unit

We spoke Dalia Argaman, VP Clinical & Regulatory at EarlySense to understand how to take medical devices from the design and engineering stages, navigate regulatory pathways and execute medical device clinical trials that receive FDA approval and save lives.

“I have a BSc in Chemistry and MSc in Chemical Physics and I was lucky enough to start my career immediately after completing my education. I joined Direx (a startup employing 6-7 people at that time) and became a part of developing an innovative medical device in the field of shockwave lithotripsy which is designed to break kidney stones without invasive procedure. Beforehand the standard of care was that patients with kidney stones had to undergo surgical procedure full of discomfort and further complications. I was the one who took the medical device through clinical trials  when the first prototype was created”.

It was back then that Dalia got introduced to the world of medical device clinical trials, submissions to regulatory authorities (FDA, CE, CFDA) and clinical data management.

Dalia has been on top of this innovative world through the 20 years of her career, working in several companies that develop different medical devices. Alongside success, she experienced a number of professional and personal challenges that shaped her career.

“You actually might be surprised to find out that the biggest challenge I had to overcome during my career was to try and balance family life with professional. I am a career person but I also have a family and I always had to juggle between being a mother, raising a family and moving forward on the career ladder. In addition, I think when you are a regulatory person, working with various people with different understanding, different backgrounds and fields and trying to get everyone at the same page is a huge professional challenge”, she said.

While working for Glucon (Developer of Non-Invasive Glucose Monitoring Devices in Israel) Dalia was involved in clinical trials with patients who had diabetes: that included children as well. She recalls that this experience was most memorable through her entire career as it required creating a medical device that would make patient compliance easy.

“People with diabetes are prone to numerous complications: patients have to monitor their blood glucose level constantly to avoid hyperglycaemia: the procedure is usually done by pricking the finger and drawing blood for analysis. You can only imagine how uncomfortable it is for young children or their parents who sometimes have to wake their child up several times a night to check it.  Being involved in a company that develops a device capable of making so many people’s lives easy is a subject of great pride for me”, she says.

Dalia Argaman is currently in charge of clinical regulatory affairs and quality assurance in Earlysense. The company develops contactless sensors that are placed under hospital bed mattresses and allow to monitor vital signs (heart rate, respiratory rate and other parameters) in a contactless way without making the patient feel uncomfortable.

“They can help the physicians, nurses to continuously supervise the patient and detect early signs of deteriorating in order to intervene early, thus reaching a better outcome”.   All of this is done using passive monitoring of the patient’s movement and without requiring patient compliance.

“There is currently a long delay between the time that a medical device is being developed by research and development teams, execution of medical device clinical trials, analysis of data received from clinical data management team, submission to FDA and the time that products get to the end users. The delay is connected with vigorous testing a product has to get through in order to be in compliance with standards and be approved. I think FDA understands well the importance of using automation to accelerate the process of executing clinical trials in order enable these medical devices to get to market and start saving lives sooner”.

Why paper is not an option for your medical device clinical trial

This is a piece David wrote a couple of years ago originally entitled “Why you cannot afford to use paper in your first Phase I efficacy trial for your medical device”.   David’s premise is that people do not like change.

Why you cannot afford to use paper in your first Phase I efficacy trial

In all walks of life, people do not like change.

We have heard the axiom change is good all throughout our lives, but the fact remains that people, as basic animals, are hesitant to embrace change and take on new endeavors. Human beings are creatures of habit, and are more often than not content within their comfort zones, regardless if they are losing out on valuable experiences, money, etc.

Studies have even been conducted revealing how opposed to change we creatures of habit are. People will sacrifice the opportunity to enhance their quality of life because it may require a change to their routine, or learning new habits, and humans hate that.

In the clinical research industry, paper-based data capture methods have been used effectively, and for decades. Paper is the norm, and many a successful study have been conducted using this such method. While paper is the traditional, tried-and-true method for data capture and management (especially during Phase I efficacy trials of medical devices, which typically have smaller subject counts and shorter study durations), that does not mean it is the best method available, or that it is the most cost efficient.

In fact, the last point is no longer true whatsoever.

There are small CROs and clinical study sponsors that are so used to paper data capture for small medical device clinical trials, that they oppose the change to electronic data capture (EDC). However, while some early objections were valid in opposition of EDC for Phase I, they no longer ring true.

EDC has been implemented for clinical studies, particularly in later-stage trials such as Phase III studies with thousands of patients, for a little over 15 years. By now, many of the concerns regarding the ample paper vs EDC debate at any clinical study stage are now moot.

Today we are going to touch upon why one cannot afford to use paper for Phase I efficacy trials for medical devices, and will greatly benefit from the change to cloud EDC.

Time savings on amendments

During Phase I efficacy testing, pharmaceutical companies are getting their feet wet for the first time while developing a new drug. This is the stage with the highest level of patient risk, and EDC quickly thwarts paper-based systems in this realm.

Phase I experiences the most amendments to drug administration frequency, dosing, and amendments to trial need to always be compliant with the FDA’s 21 CFR Part 11. Vendor validated EDC systems are easily augmented to comply with changes to FDA regulations, and have measures in place within the software, to monitor and ensure that study SOPs are compliant every step of the way.

Paper simply cannot do that.

Also, amendments, whether at the hands of a regulatory agency or medical device company, tack on months of extended study time and costs. According to a study done by Tufts University, a single amendment using paper-based systems increases study time by an average of 2 months and costs the study over $400,000. The study also showed that on average, each study experiences 2.3 amendments to protocol.

The time savings, and thus cost savings, on amending SOPs is enormous for studies conducted using EDC, as the software and eCRFs can be augmented in the blink of an eye. Also, if further amendments are required down the study chain, they are made just as quickly.

Real-time data monitoring of cleaner, faster study data

Using EDC instead of paper affords clinicians and data monitors real-time access to data capture. Also, cloud EDC like Clear Clinica is mobile accessible, so all members of the study team can remotely access data using mobile devices like smartphones or tablets, the very second it is entered. This is especially valuable for Phase I trials, because these have the highest risk to study subjects, as they are the first in line to test the drugs.

Even though Phase I efficacy trials do not typically involve hundreds of subjects, adjustment to treatment protocol and dosing need to be made with as little delay possible. Patient safety is a top priority for clinical studies.

For studies using patient reported outcomes (ePRO), EDC wins over paper-based systems every time. When a patient enters data into the system, risk-based monitoring protocols within the software inform study teams whether or not a patient is at risk for harming their health if they proceed at the administered dose. This allows clinicians to make adjustments to dosage, or cease the subjects participation in the study, before their health is harmed if the dosage is too high or if grossly adverse effects are experienced by the patient.

Again, paper simply cannot perform in this manner for Phase I efficacy testing.

Further, using EDC for Phase I is smarter than paper regarding cleaner, error-free data. Human error occurs. Even the brightest of clinicians and data monitors will make a mistake when entering data, or miss an incomplete form. Especially being that Phase I is the first stage for drug testing, the cleaner data is from the get-go the smoother it will be for conducting Phase II and III of the trial.

The EDC system software can be set so that eCRF values are standardized, so that when data is captured and entered into the eCRF fields error notifications are displayed when data is outside of the field parameters. The same goes for submission of eCRFs that are incomplete. EDC systems like Clear Clinica are programmed according to data parameters set by sponsor or CRO staff for each trial’s needs. Also, once the eCRF parameters are set, they can be modified if needed according to amendments, but otherwise they remain uniform, saving time during the entire study cycle’s lifetime.

You cannot afford to not use EDC

Nobody will deny that the up-front costs of implementing cloud EDC for Phase I will cost more than a paper-based system. However, not doing so because of that reason is myopic and short-sighted. The safety, risk, time, and data quality savings are well worth the initial investment, as the system is not going to be used for only one study.

Down the road, after incorporating an EDC for Phase I, and using it for II and III, the money spent is quickly offset by the costs saved on time, IT personnel expenses (EDC vendors have support staff to solve whatever issues may arise), and data assurance, amongst others. The sooner you switch to EDC, the sooner future studies will save your CRO or sponsor organization money, and mitigate Phase I patient risks.

A structured 7 step process for risk assessment of a medical device clinical trial

In this essay, I discuss a systematic methodology for evaluating risk in your medical device clinical trial. This is a methodology that has proven itself in hundreds of security and privacy compliance risk assessment projects in a wide variety of healthcare, clinical and IT scenarios. 

It is a given that the people charged with your clinical trial planning,regulatory affairs and operations are better at executing standard operating procedures then in performing risk analysis and thinking like attackers.


Risk assessment is a process that starts before you write the protocol, when you are writing the CRF (to determine what data to collect) and any time there are amendments to the study.

See the below graphic from the Transcelerate Web site to see why procedures do not protect your clinical trial and why SDV does not assure patient compliance to the protocol.  Note that “Material risk” is any threat to the success of the study from problems with study startup to problems with poor patient compliance.


Does counting compliance activities secure the deliverables of your clinical trial?

First define “secure”.

Security is about reducing the impact of unpredictable attacks on assets – in your case, attacks on the 2 most critical assets of your clinical trial – the data and the subjects.

Some examples of unpredictable attacks on your clinical trial:

There may be multiple sources of data errors at sites, ranging from mistakes, misunderstandings, sloppiness and all the way to incompetence.

There may data fraud – deliberate fabrication or falsification of data

There are patients that comply and patients that take their treatment randomly and in strange and wonderful ways.

There are patient reported outcomes that make sense and then there are the people who write War and Peace in the ePRO system and crash the SAS analysis program with special characters they used.

Will compliance activity check-boxing mitigate ANY of the above attacks?


How to mitigate unexpected attacks on your data and patients

Once we understand that check-box compliance procedures are not a good countermeasure for threats to your study deliverables (solid scientific data, patient safety, patient compliance with the clinical protocol) what are our options for mitigation?

Consider your strengths and weaknesses.

Starting with your weaknesses, it is a given that the people charged with your clinical trial planning,regulatory affairs and operations are better at executing standard operating procedures then in performing risk analysis and thinking like attackers.

There is a fundamental divide, a metaphorical valley of death of  mentality and skill sets between a regulatory-affairs and clinical operations mindset and a professional security mindset.

This essay offers a systematic approach – if you will, a  common language, a language  of people-centric threat modeling that helps clinical managers cross the chasm between thinking like a regulatory affairs person and thinking like an attacker who wants to destroy your study.

Start by thinking about how your study can be attacked.

Analyzing the impact of attacks on medtech studies requires hard work, hard data collection and hard analysis.  It’s not a sexy, fun to use, feel-good application like Apple Music.   Risk analysis  may yield results that are not career enhancing, and as  the threats  get deeper and wider  with  bigger and more complex trials – so the security valley of death deepens and gets more untraversable.

There is a joke about systems programmers – they have heard that there are real users out there, actually running applications on their systems – but they know it’s only an urban legend. Like any joke, it has a grain of truth. IT and security are primarily systems and procedures-oriented instead of  customer-safety oriented. Similarly – clinical regulatory affairs are primarily paper and process-oriented instead of attack-oriented.

Leave your paper and process comfort zone

If the essence of security is protecting the people who use a company’s products and services then the essence of security for a clinical trial is protecting patients and acquiring reliable data.

A structured 7-step process for risk analysis of your clinical trial

We propose a structured process for risk analysis and ongoing risk management.  No previous training is required and the process can become a key part of a medtech developer’s management toolkit.

The risk analysis and management process has 7 steps as described in the below schematic (“the risk management loop”). The process uses threat modeling and quantitative risk assessment methods based on providing a financial value to assets (such as EDC systems and patient eCRF records) in order to determine value at risk and prioritize security countermeasures.

The 7 step risk process provides a systematic way to manage risk while responding to changes in regulation, business environment and clinical research feature set/functionality. Let’s start with some basic definitions:


Vulnerability is a weakness, limitation or a defect in one or more of the system’s elements that can be exploited to disrupt the normal functionality of the system. The weakness or defect may be either in specific areas of the system, its layout, its users, operators, and/or in its policies and procedures.

Countermeasure is a technical, physical or procedural safeguard that mitigates one or more vulnerabilities.

Asset – data, systems, physical assets or intellectual property of value to the organization.

Threat – action(s) that exploit vulnerabilities in order to damage assets.

Asset value – the financial value of an asset that is destroyed of stolen. Assets may be digital (software source, physical (a server) or commercial (a corporate brand).

Damage to Asset – damage to a physical asset or damage to a digital asset in terms of breach of confidentiality, impacted system availability or broken integrity of systems and/or data.  Damage is estimated in financial terms.

Threat probability is the likelihood that a threat will turn into a real attack. Threat probability can be described in terms of ARO – Annual Rate of Occurrence; i.e. how many times a year that the attack is forecasted to happen.

Threat risk is the likelihood of damage that may be caused to one or more assets by the threat. 
Recommended countermeasures the possible countermeasures that reduce the threat’s risk based on the countermeasures that mitigate the threat vulnerabilities.

Actual countermeasures (aka mitigation plan) is a subset of recommended countermeasures that is assumed to be the most effective for mitigating a specific threat. Choice of specific safeguards is often a judgment call of the threat analyst.

Countermeasure cost is the financial value that is associated with the implementation of a specific countermeasure.

Countermeasure cost effectiveness is the degree of mitigation introduced by a specific countermeasure to the overall risk in the system in relation with the cost of implementing this specific countermeasure.

Attacker is a person (or group of persons) that may perform the steps of a specific threat scenario.

Attacker Types are the various classes of attackers that are differentiated according to their motivation, qualification, available attack tools and their accessibility to the attacked system’s resources.

Entry Points are points of entry made by attackers into the system, for example doors in a building or users who have a login to your EDC system.

The 7 step risk analysis loop

Risk analysis is not a one-way, one-time process you do, report and file away.   Analyzing attacks and risk in your studies is an ongoing exercise always relying on quality human intel from the field – from CRCs, subjects and site monitors.

Step 1 Set scope

The threat analyst(s) will identify reasonable threat scenarios and their probability.

Read this if you are new to risk analysis

Choose one (1) question you want to answer.  That’s it. Only one (1). For example – “what is the threat scenario for patients participating in the study and not passing inclusion/exclusion criteria”?  After you have nailed the question, nail the threat scenario – i.e. how it can happen.  After you nail the threat scenario – quantify the threat in terms of probability of occurrence and its impact and potential damage to your study.    

Read this if you are a medtech developer

In a medtech study which uses wearables, connected medical devices or mobile medical device apps (or any combination thereof), having up-to-date documentation of software functionality and architecture is required in order to correctly identify vulnerabilities and threat scenarios. The following documentation is required as part of the risk analysis process:

 1. Functional description of the system including relevant use cases

 2. Architectural diagram of the system

 3. Documentation of sub-modules

How to document the risk assessment for your medical device study

Up-to-date documentation of the study protocol and CRF is required in order to correctly identify vulnerabilities and threat scenarios. Historical records of protocol amendments is unnecessary.

The following source documentation is required as part of the risk analysis process:

Study protocol

Treatment schedule and visit flow


CRF edit checks

These documents must be detailed enough to be used as reference for the decisions regarding the applicability of various threat scenarios to the analyzed system.

Step 2 Identify assets of your study

The correct mapping of assets (EDC database, patient safety, drug accountability data, etc), their financial value and the evaluation of financial loss to the sponsor when these assets are damaged or stolen, is one of the most critical tasks in the threat analysis process. The assets value is used as the basis for calculating threat risks and countermeasures priorities.

Asset valuation is not a one-time activity

Due to the importance of asset valuation, the asset list and corresponding values should be reviewed once a year by the controller or CEO during the course of the study.

Step 3 Identify the moving parts (components) in your study

Using a systems approach to your study, map the moving parts in your study. This will include application software components (EDC, IWRS, ePRO, centralized monitoring systems etc), people functions (study monitors, site monitors, project manager, CRCs, principal investigators).

Map the “moving part” entities to assets (for example patient records) and update the threat model with the components and functions. 

Tagging different components and functions in the system help the analyst in classifying the various data and software entities and relating them to specific vulnerabilities and safeguards such as protecting PHI processed by an outsourced call center.

Step 4 Identify your study vulnerabilities

Identifying and classifying vulnerabilities requires the analyst to be intimate with the study primary and secondary endpoints, safety endpoints, protocol design, implementation and deployment details. The analyst should also be familiar with clinical operations procedures and the types of users, customers and patients that use the system or are involved with delivering services.

Step 5 Build / update the threat model

Classifying attacker types

The basic attacker types are: study user roles (site and study monitors,Pis, CRC, project managers, IT staff or cloud EDC providers) , malicious outsiders, trusted insiders and other site staff and outsourcing service providers. Additional attacker types (such as hacktivists) may be added when relevant.  Different attacker types will have different motivations and different costs for mounting an attack. Attack motivation and cost are an important part in estimating threat probability since cheap attacks by highly motivated individuals are more likely than expensive attacks by attackers with little to gain.

Identifying attack entry points

The best strategy for this step is to review attacker types and document every possible way a potential attackers could access the system. The list of entry points may be refined in the course of the risk management loop.

Step 6 Build your risk mitigation plan. Calculate residual risk

Risk assessment is not over until the fat lady sings.    You walk away from the risk assessment table with a much deeper understanding of what threats count and how much residual risk you have after deploying controls – technical controls, monitoring of deliverables, patient safety monitoring

This is the most important step of the risk analysis and management process. The outputs are:

A map of the relationships between threats and area tags, assets, attacker types, entry points and vulnerabilities

An evaluation of the total damage and risk parameters for each of the threats

Write mitigation plans

Calculate residual risk – i.e. how much risk exists after you implement your new controls.

Since threats are the most complex entities in the model, the process of identifying and constructing threat’s elements and parameters has a ‘decomposition’ flavor. During this step the analyst(s) will have to return to previous analysis steps in order to create missing entities, such as assets and vulnerabilities that are referenced by the threat that is constructed.

Step 7 Validate your findings

The accurate identification of countermeasures and their relations with vulnerabilities is the basis for validating the correctness of the risk mitigation plan. The best way in our experience of validating a risk analysis is to show it other people outside your office and ask them what they think.

Validation output

A list of countermeasures that mitigate vulnerabilities: The list should include the implementation cost and an indication if the countermeasure is already implemented.

A map of the relationships between countermeasures and vulnerabilities: This map shows which vulnerability is mitigated by which countermeasure(s).

A validated risk mitigation plan will include the following management level reports:

Threats ordered by risk

Threats ordered by the financial damage

Safeguards ordered by risk mitigation percentages

Safeguards ordered by their effectiveness (mitigation/implementation cost)

Asset value at risk before mitigation

Residual value at risk after the mitigation plan


We have presented a systematic 7 step process for identifying and analyzing threats to the assets of your clinical trial – whether its unpredictable user behavior or patients at risk.

Assessing the risk posture of any study will benefit from this proven systematic methodology  and will help you take a paper and process-oriented study team from a place of weekly and monthly reports and activity-counting to a faster-moving, and vastly more effective place of risk understanding and mitigation.

100X faster to deviation detection in medical device studies.

Automated Patient compliance deviation detection and response on the platform for a connected medical device clinical trial is 100X faster than manual monitoring. Automated compliance monitoring analytics and real-time alerts let you focus your site monitoring visits on work with the PI and site coordinators to take total ownership and have the right training and tools to meet their patient recruitment and patient compliance goals.