4 strategies to get connected medical devices faster to FDA submission

Introduction

Better designs, site-less trials, all-digital data collection and PCM (patient compliance monitoring) can all save time and money in connected medical device clinical trials.  This article will help you choose which strategies will be a good fit to help you validate your connected medical device and its intended use for submission to FDA.

What is the baseline cost? (hint don’t look at the costs of drug studies)

If you want to save, you need to know the price tag. Note that the costs of drug trials, including CRO and regulatory affairs is an order of magnitude higher than for connected medical devices.  A JAMA report from Nov 2018, looked at drug trials and concluded that a mean cost of $19M was cheap compared to the total cost of drug development – $1-2BN.

Findings:  In this study of 59 new therapeutic agents approved by the FDA from 2015 to 2016, the median estimated direct cost of pivotal efficacy trials was $19 million, with half of the trial cost estimates ranging from $12 million to $33 million. At the extremes of the distribution were 100-fold cost differences, and patient enrollment varied from fewer than 15 patients to more than 8000 patients.

By comparison, the estimated cost of medical device clinical trials to support approval by the FDA, ranges from $1 million to $10 million. A report from May 2017 surveyed the costs of medical device clinical trials and the potential of patient registries to save time and money. The report has some interesting numbers:

1.The average cost to bring a low-to-moderate concern device from concept to 510(K) approval is $31 million. 77% of that is spent on FDA-related/regulatory-affairs activities.

2.The average cost for a high-risk PMA device averages $94 million, with $75 million spent on FDA-related/regulatory-affairs activities. Average of 4.5 years from first contact with FDA to device approval.

3.Clinical trials outside the US are 30% to 50% cheaper. Less than 50% of medical device trials are now conducted in the US.

I. Better study designs

Real-world data (RWD) and real-world evidence (RWE) are being used for post-market safety surveillance and for designing studies, but they are not replacements for conducting a randomized trial with a controlled clinical protocol.  FDA recently issued guidance for use of real-world evidence for regulatory decisions.  FDA uses RWD and RWE to monitor post-market safety and adverse events and to make regulatory decisions.

RWD and RWE can be used in 4 ways improve the design of medical device clinical trials when there is a predicate device that is already being used for treating patients.

1.Use RWD/RWE to improve quality and efficiency of device evaluation at study phases (feasibility, pivotal, and post-market), allowing for rapid iteration of devices at a lower cost.

2.Explore new indications for existing devices

3.Cost efficient method to compare a new device to standard of care.

4.Establish best practices for the use of a device in sub-populations or different sub-specialties.

You will need to factor in the cost of obtaining access to the data and cost of data science.

But real-world data may not be reliable or relevant to help design the study.  As FDA notes in their guidance for Using Real-world evidence to support regulatory decision making:

RWD collected using a randomized exposure assignment within a registry can provide a sufficient number of patients for powered subgroup analyses, which could be used to expand the device’s indications for use. However, not all RWD are collected and maintained in a way that provides sufficient reliability. As such, the use of RWE for specific regulatory purposes will be evaluated based on criteria that assess their overall relevance and reliability. If a sponsor is considering using RWE to satisfy a particular FDA regulatory requirement, the sponsor should contact FDA through the pre-submission process.

II. Site-less trial model

Certain kinds of studies for chronic diseases with simple treatment protocols can use the site-less trial model.  The term site-less is actually an oxymoron, since site-less or so-called virtual trials are conducted with a central coordinating site (or a CRO like Science37). Nurses and mobile apps are using to collect data from patients at home.   You still need a PI (principal investigator).

The considerable savings accrued by eliminating site costs, need to be balanced with the costs of technology, customer support, data security and salaries and travel expenses of nurses visiting patients at homes.

III. Mostly-digital data collection

For a connected medical device, mostly-digital data collection means 3 things:

1.Collect patient reported outcome data using a mobile app or text messaging

2.Collect data from the connected medical device using a REST API

3.Enable the CRC (clinical research coordinator) to collect data from patients (IE, ICF for example) using a Web or mobile interface (so-called eSource) and skip the still-traditional paper-transcription step. In drug studies, this is currently impossible because hospital source documents are paper or they are locked away in an enterprise EMR system. For connected medical device studies in pain, cannabis and chronic diseases, most of the source data can be collected by the CRC with direct patient interviews. Blood tests will still need to be transcribed from paper. Mostly-digital means mostly-fast. Data latency for the paper source should be 24 hours and data latency for the digital feeds should be zero.

There are a number of companies like Litmus Health moving into the space of digital data collection from mobile devices, ePRO and wearables. However, unlike validating a connected medical device for a well-defined intended use, Litmus Health is focused on clinical data science for health-related quality of life.

IV. PCM (patient compliance monitoring)

Once the data is in the system, you are almost there.  Fast (low-latency) data from patients, your connected device and the CRC (which may be nurses in a site-less trial) are 3 digital sources which can be correlated in order to create patient compliance metrics.  But that is a story for another essay.

Summary

We have seen that new business models and advanced technologies can help sponsors conduct connected medical device trials cheaper and faster. It may not be a good fit for your product.  Contact us and we will help you evaluate your options.

For more information read Gail Norman’s excellent article  Drugs, Devices, and the FDA: An overview of the approval process

 

 

 

 

 

 

 

 

Putting lipstick on the pig of electronic CRF?

esource for people in clinical trials

Good online systems do not use paper paradigms. In this post – I will try and entertain you with historical perspective and quantitative tools for choosing an EDC system for your medical device study.

Decades of common wisdom in clinical trials still hold to a paper-based data processing model. One of the popular EDC systems talks about the advantages of having online forms that look exactly like paper forms.  True – familiarity is a good thing, but on the other hand a digital UX has far more possibilities for user engagement and ease-of-use than paper.   So – it is, in a way admitting failure to provide a better UX and downgrading to paper.

We recently engaged with an Israeli medical device vendor who has an innovative device for helping solve a common medical indication for men over 50.

I won’t go into details.

If you are a guy over 50, you know what I mean.

If not, it doesn’t matter.

The client CEO was interested in an eCRF (electronic CRF – case report form) system.  eCRF is better than paper, but it is, at the end of the day just a paper form in an electronic format.

I was having a lot of trouble trying to understand the CEO’s business requirements.   My attempts to steer the conversation to a discussion of how to obtain fast data for his clinical trial and reduce his time to FDA submission fell on deaf ears. A follow up conversation and demo of Flaskdata with the clinical and regulatory manager focused more on reports and how to manage queries.  Queries are a vestige from the paper CRF period, where a study monitor would visit the research site once/month, compare the paper source with the electronic data entry and raise queries or discrepancies.

In order to put this process into a historical context, let’s compare accounting systems from the late 70s, early 80s to an eCRF system.

Accounting versus eCRF

Feature Accounting circa 1970 eCRF circa 2019
Input data Paper JV – journal voucher Paper source
Data entry Data entry to a 2-sided accounting system Data entry to an eCRF
Data processing A batch job, processes punch-card data entry and produces a data entry report and data error report Site coordinators enter data to a Web app 1-3 days after the patient visit. Data entry errors or invalid data create data validation queries which are ignored until the study monitor visit a month later
Exception reporting Data error report – with non-numeric or invalid dates Queries
Management reports Trial balance

Profit and loss

Cash flow

Bean counters of CRF/items

What is profit and loss?

What does a cash flow model have to do with clinical trials?

 

 

Cost justification and TCO for medical device EDC systems

My first recommendation would be don’t buy an EDC system just because its cheap. Charging $100-300/month for a data entry application is not be a reason to give someone money.  As a client of ours once said – “I know I can use Google Forms for data entry and its free but Google Forms does not have an audit trial so  Google Forms is not an option for clinical trials”.

As a rule-of-thumb, a good EDC system for medical device studies should include audit trails and a clinical cash flow report (the flow of patients in and out, the flow of data items in and out).   The EDC should also be able to produce a clinical Profit and loss statement, showing you how well you are doing on your primary and secondary efficacy and safety end points. A well-designed and well-implemented EDC should include a robust data model for testing the primary endpoint and collecting safety data.    At the minimum, a solid design and implementation will cost at least $10,000.  Over 10 months, that’s a starting cost of $1000/month.   As Robert Heinlein said – “There is no such thing as a free lunch”.

Your decision to buy EDC should be based on an economic breakeven point.   One breakeven method is based on cost reduction in site monitoring.  Assume the EDC system costs $4000/month (weighted cost including implementation) and assuming a monthly site visit costs $800/day, then your EDC system must be able to save 5 site visit a month and assure protocol compliance. This places an upper bound on the price you can pay.

This is albeit problematic for small, 1 site studies which often use DIY implementations.  Just remember, that ignoring the implementation cost, does not make the product cheaper. In other words calculate your TCO (Total cost of ownership).

Or as one wise man one said – I’m too poor to buy a cheap car.

Israel Biomed 2019-the high-social, low stress STEM conference

Impressions from Biomed 2019 in Tel Aviv

This week was the annual 3 day Biomed/MIXiii (I have no idea what MIXiii means btw) conference in Tel Aviv.  The organizers also billed it as the “18th National Life Science and Technology Week” (which I also do not know what that means). This was a particular difficult time for a conference of medical device and pharma in Tel Aviv since it coincided with the Eurovision 2019 activities – and the traffic was tough.

There were a huge number of lectures and participants from all over the world and I suppose from that perspective, the conference is a success and tribute to the burgeoning Israeli biomed industry.  Forbes calls Biomed “The High-Paying, Low-Stress STEM Job You Probably Haven’t Considered”.  I think that this is probably a good description for the conference – high participation but low stress.

My colleagues and I come to the conference to network, schmooze, meet customers and suppliers.  It’s a good opportunity to take a few meetings, say hi to friends and hustle for new business.  Having said that, I did meet a few really interesting companies:

RCRI – is a Minneapolis MN based medical device CRO.  I met Todd Anderson and his boss Lisa Olson and pitched our approach for fast data in clinical trials to assure high levels of patient compliance to the protocol and submit faster to FDA.    Todd and Lisa get it and they were open about the CRO business model being more people-hours not speed.     They seemed genuinely interested in what we are working on but its hard to tell with Americans.

Docdok Health – is a startup founded by Yves Nordman, who is a Swiss MD living in Carmiel.  It’s a doctor-patient communications platform beginning to branch out into Post-marketing studies with RWD.    We shared demos and it seems that there is synergy between our regulatory platform and their post-marketing work.

Resbiomed – met Alex Angelov, the CEO.  Alex is leading a consortium including Flaskdata, Carl Zeiss, Collplant, PreciseBio and Pluristem for a Horizon2020 submission for an amazing project for an implant to the cornea.  Dan Peres from Pluristem got us together.   Cheer for us!

BSP Medical and ICB (Israel China Biotech investment) – my buddy Hadas Kligman literally took me by hand to visit to Yehuda Bruner and Andrew Zhang and I did my 60s elevator pitch on getting medical device companies to FDA/CFDA 6-12 months faster.   We agree to talk after the conference.

Butterfly Medical –  I met Idan Geva, the CEO last year at Biomed – we ate lunch at the same table.  I pitched him but he was uninterested – they were using EDC2Go – and he didn’t want to hear other options.     At the Minnesota pavilion talking to Todd Anderson from RCRI,   Idan shows up and looks at me and says “Heah – Hi Danny – I left a contact me request on your web site yesterday and no one got back to me. I said shame on us.  He says – he was referred to us by someone from Florida who used to use Medidata.  I asked where/who? was it Miami?  He says yeah it was Miami and checks his phone – says its someone from Precision Clinical Research that are using Flaskdata and recommended.    (Precision is one of our customer’s Miami sites).  I asked what happened to EDC2Go – he said well you know – they are end of life (I think this means the end of low-cost EDC) and we are now entering questionnaires manually on paper and it is driving us crazy.   He said – can you stick around and give us a demo at 15:00?  I said sure.  We met at 15:00 by the bar upstairs in the David Intercontinental and I demoed the system – he said “Show me the Forms designer”. I showed him.  He says “show me how CRC enters data” – I showed him.  He says “Show me how to extract data” – I showed him.  I think he actually did not believe how fast the Extract to CSV process was and asked me twice if that was the data.  In the end – the format of Mac Numbers was a bit strange for him. I showed him a quick presentation – and he saw that Serenno is a customer – and says – “Heah Tomer is a neighbor of ours in the incubator in Yokneam”.    He asked how much and I said $2K for a basic onboarding package and $1500 / month.  Or $10K and we will build the CRF (their CRF is super simple btw).  He wanted a discount, being Israeli.  I said – “lets meet with your clinical person and get her to buy-in to the solution.  If she buys in – you and I can talk business but before that, there is no point horse-trading.

Count the probabilities of this happening and you will see that it is an impossible event.

Thursday I went back to demo Todd and meet Dr Yael Hayun from Syqe Medical. Yael is one of the most impressive people I’ve met in a long time. She is an MD from Hadassah and one of the movers and shakers in LogicBio Therapeutics.    After we chatted – I told her that Syqe is lucky to have her onboard.   I did our Today is about Speed presentation and a short demo. She was suitably impressed and then mentioned they had met with a Danish EDC company called Smart Trial – which turns out is yet another low-cost eCRF provider.   I said look – eCRF is like 10% of the solution you need – in the case of Syqe, you have a digital inhaler and with cannabis, you are going to have a lot of concerns about patient compliance.

This is what we do – fast data collection from patients, investigators and digital inhalers and automated deviation detection and response.

On the way back – huge traffic from Eurovision.   Didn’t hear a single lecture but the meetings and people were outstanding.

 

Perverse incentives

The perverse incentive for the high costs of medical devices and delay to market

The CRO outsourcing model and high US hospital prices result in higher total CRO profits via higher costs to companies developing innovative medical devices.   These costs are passed down to consumers after FDA clearance.

We’ll take a look at the cost dynamics of medical device clinical trials and the clinical trial value chain.

We’ll then consider an alternative business model that changes the way medical device sponsors conduct clinical trials, reduce their costs by 70-80% and shortens time to FDA submission.

The high costs of US hospitals

By 2000, the US spent more on healthcare than any other country, whether measured per capita or a percentage of GDP.

U.S. per capita health spending was $4,631 in 2000, an increase of 6.3 percent over 1999. 4 The U.S. level was 44 percent higher than Switzerland’s, the country with the next-highest expenditure per capita; 83 percent higher than neighboring Canada; and 134 percent higher than the OECD median of $1,983. 5

It’s the prices, stupid.

In 2011, the US Affordable Care Act set a requirement for MLR (Medical Loss Ratio) that insurers must spend 80-85% of revenue on medical services.    This reduced insurer margins, and drove up hospital prices to make up for lower margin.

The CRO business model

CROS (clinical research organizations) are outsourcing businesses that provide an array of services for clinical trial management and monitoring, reporting and regulatory submission.   For medical device studies, CROS employ 2 basic outsourcing models, people sourcing and functional sourcing. In people out-sourcing, the medical device company is responsible for managing contractors. In functional outsourcing, the company may buy a set of functions, for example study monitoring and medical writing.

Neither CRO model has an explicit incentive to complete a study faster since that would reduce outsourcing revenue for the CRO. The more time a CRO spends on monitoring, site visits, SDV and study closeout, the more revenue it generates.

A medical device sponsor may elect to do it himself which shifts the CRO cost to an internal headcount cost supplemented with additional costs for consultants with risk and time delays by not having the CRO expertise and infrastructure. There is tacitly no free lunch, as we will discuss later in this article.

The result is a perverse incentive for delay and higher costs to bring innovative medical devices to market.

The CRO business model combined with higher hospital prices drive higher total profits via higher costs to customers. The higher cost of innovative medical devices is then passed down to consumers (patients) after FDA clearance.

Consumer value chains

A consumer value chain looks generically like this:

Suppliers -> Distributers -> Consumers

By the early 90’s, the PC industry led by Intel and Microsoft used a 2-tier value chain:

MSFT->Distis->Resellers->Customers.

Resellers were further segmented according the customer size and industry segment – Retail, Large accounts, SMB and VARS (value-added-resellers) selling their own products and services to a particular industry vertical.   The PC industry value-chain model left Microsoft with 50% of the SRP (suggested retail price) and delivered products to customers that were 45-50% less than SRP, leaving the channel with 0-5%.

The channel was forced to implement extremely efficient operations and systems and sell value-added services and products in order to survive.

By the new millennium, Apple introduced a 1-tier model with a user-experience designed and controlled by Apple.

The Apple 1-tier channel looks like this:

Apple->Apple Stores->Consumers

Eventually the Apple channel model broadened to include a 2-tier model similar to PC industry:

Apple->Distis->Retail->Consumers

By the mid-2000s, Amazon AWS (and generally the entire cloud service / SaaS industry) evolved the channel model to 0-tiers with a direct subscription and delivery model.

AWS->Consumers

As AWS grew and introduced spot pricing, an aggregation sub-market developed, looking extremely similar to movie and TV distribution models.

AWS->Aggregators->Consumers

AWS also became a distribution channel for other cloud products similar to content distribution (Think Netflix).

Third-party products->AWS->Consumers

Outstanding user-experience and aggregation are the hallmarks of companies like Airbnb, Netflix and Uber.    

The common thread is that AWS and Netflix deliver a digital product end-to-end, whereas Airbnb and Uber aggregate trusted suppliers inside the Airbnb and Uber brand environment and provide an outstanding and uniform user experience to all the consumers.  This is in contrast to the variegate user experience a customer got from the 90’s Microsoft channel. There are great resellers and terrible resellers.

We will return to user experience and aggregation later.

The medical device clinical trial value chain

The first published RCT (randomized clinical trial) in medicine appeared in the 1948 paper entitled Streptomycin treatment of pulmonary tuberculosis.

The clinical trial value chain for medical devices looks strange once after the historical perspective of how Intel, Microsoft, Amazon and Netflix evolved their value chains.

The medical device clinical trial value chain has 3 tiers with patients that are both suppliers and consumers.

Patients->Hospitals->CROS->Medical device companies->Patients

A dystopian user experience

Little has changed in the past 71 years regarding clinical trials.    Clinical trials and hospital operations now have a plethora of complex expensive, difficult-to-use IT with a value chain that provides a dystopian user experience for hospitals, patients and medical device companies.

HCOs (healthcare operators) rely on data collection technology procured by companies running clinical research (sponsors and CROs). This creates a number of inefficiencies:

1 – HCO staff are faced with a variety of systems on a study by study basis. This results in a large amount of time spent learning new systems, staff frustration and increased mistakes. This is passed on in costs and time to sponsors after CRO markup.

2 – The industry is trending towards the use of eSource and EMR to EDC data transfer. eSource/ePRO tools need to be integrated into the patient care process. Integration of EMR with EDC becomes logistically difficult due to the number of EDC vendors on the market (around 50 established companies).

3 – Siloed data collection in hospitals with subsequent manual data re-entry results in large monitoring budgets for Source Data Verification, and delays caused by data entry errors and related query resolution. Delays can be on the order of weeks and months.

4 – Use of multiple disconnected clinical systems in the hospital creates a threat surface of vendor risk, interface vulnerabilities and regulatory exposure.

Losing focus on patients

One of the consequences of the 3-tier medical device value chain is loss of focus on the patient user experience.  Upstream and to the left, patients are ‘subjects’ of the trial. The patient reported outcomes apps they use vary from study to study. Downstream and to the right (what FDA calls ‘post-marketing’), patients are consumers of the medical device and the real-world user experience is totally different than the UX in the study.   The real-world data of device efficacy and safety is disconnected from the clinical trial data of device efficacy and safety.

Clinical trial validation

Patient compliance is critical to clinical trial validation of medical device. Who owns patient compliance to the research protocol?  The medical device sponsor, the CRO, the hospital site or the subject?   The CRO may not collect a patient compliance metric since he outsources to the hospital. The hospital may not have the tools and the medical device company is outside the loop. My essay on determining when patient compliance is important in medical device trials goes into more detail on the problem of losing focus on the patient.

Vertical integration and aggregation

We previously made a qualitative claim that hospital site costs are high for medical device studies.  How high are they relative to consumer healthcare?

In a medical device trial recently done on the Flaskdata.io platform, the sponsor paid the hospital investigatory sites $700K for a 100 subject, 7 month multi-site study. (There were no medical imaging and blood test requirements).

In 2016, Medicare Advantage primary care spend was $83 PMPM (per member per month).      Let’s say that a premium service should cost $100 PMPM.    Let’s use that as a benchmark for the cost of processing a patient in a medical device trial.  Take this medical device Phase II medical device trial with 100 patients, running for 7 months:

That’s 100 x 7 x 100 = $70K for patients. Not $700K.

Perhaps the law of small numbers is killing us here.  The way to solve that is with aggregation and vertical integration. Let’s return to the medical device clinical trial value chain. As we can see, there are too many moving parts and a disconnect between patients in the clinical studies and consumers in the real world.

Patients->Hospitals->CROS->Medical device companies->Patients

One alternative is to integrate backwards and to the left.   This requires managing hospital site functions and to a certain degree is done in the SMO (site management organization model).

The other alternative is to integrate forward and to the right.   This is the path that Airbnb, Uber and Netflix took aggregating consumer demand with an outstanding user experience.  The aggregation gives Airbnb, Uber and Netflix buying power to the left, enabling them to choose the best and most cost-effective suppliers.

The value chain would then look like this:

Suppliers->Medical device companies->Patients

This is a model that we see increasingly with Israeli medical device vendors with limited budgets.   The Medical device company uses a cloud platform to collect digital feeds from investigators, patients and devices and automate monitoring for deviations. Focus on the patient user experience begins with design of the device and continues to post-marketing. Aggregation of patients enables purchasing power with suppliers – research sites, clinical consultants and study monitors.

Flaskdata - esource, ePro, patient compliance montoring,

Short-term versus long-term cost allocation

The reality is that using a technology platform for vertical integration is more expensive initially for implementation by the medical device company.   It should be.

Under-funding your infrastructure results in time delays and cost spikes to the medical device sponsor at the end of the study.

The current CRO methodology of study close-out at the end of a clinical trial lowers costs during the trial but creates an expensive catch-up process at the end of the study.

The catch-up process of identifying and closing discrepancies can take 2-6 months depending on the size and number of sites. The catch-up process is expensive, delaying submission to FDA and revenue since you have to deal with messy datasets.   The rule of thumb is that it costs 100X more to fix a defect after the product is manufactured than during the manufacturing process. This is true for clinical trials as well.   A real-time alert on treatment non-compliance during the study can be resolved in 5 minutes.   By waiting to the end of study it will take a day of work-flow, data clarifications and emails to the PI.

Summary

Vertical integration reduces costs and delay at study-end with continuous close. It is more expensive initially for the medical device company and it should be because it accelerates time to submission and reduces monitoring and close-out costs.

 

Patient compliance – the billion dollar question

The high failure rate of drug trials

The high failure rate of drugs in clinical trials, especially in the later stages of development, is a significant contributor to the costs and time associated with bringing new molecular entities to market. These costs, estimated to be in excess of $1.5 billion when capitalized over the ten to fifteen years required to develop a new chemical entity, are one of the principal drivers responsible for the ongoing retrenchment of the pharmaceutical industry. Therapeutic areas such as psychiatry, now deemed very high risk, have been widely downsized, if not abandoned entirely, by the pharmaceutical industry. The extent to which patient noncompliance has marred clinical research has in some cases been underestimated, and one step to improving the design of clinical trials may lie in better attempts to analyze patient compliance during drug testing and clinical development. Phil Skolnick, Opiant Pharmaceuticals The Secrets of a successful clinical trial, compliance, compliance, compliance.

Compliance, compliance, compliance

Compliance is considered to be key to success of a medical treatment plan. (1, 2, 3)

It is the “billion dollar question” in the pharma and medical device industry.

In home-use medical devices in particular and in chronic diseases in general – there is wide consensus that patient compliance is critical to the success of the clinical trial.   Our experience with Israeli innovative medical device vendors is that they understand the criticality of patient compliance. They “get it”.

However, as Skolnick et al note – patient compliance with the clinical protocol is often underestimated in drug trials.

There are 4 challenges for assuring patient compliance in medical device trials.

1. The first challenge is maintaining transparency.    An executive at IQVIA noted (in a personal conversation with me) that IQVIA does not calculate patient compliance metrics since they assume that patient compliance is the responsibility of the sites.    The sponsor relies on the CRO who does not collect the metrics who relies on the sites who do not share their data.

2. The second challenge is having common standard metrics of compliance. Site performance on patient compliance may vary but if sites do not share common metrics on their patients’ compliance, the CRO and the sponsor cannot measure the most critical success factor of the study.

3. The third challenge is timely data.   In the traditional clinical trial process, low-level data queries are resolved in the EDC but higher-level deviations often wait until study-closeout.  The ability of a study team to properly resolve thousands of patient compliance issues months (or even years) after the patient participated is limited to say the least

4. The final and fourth challenge is what happens after the clinical trial.  How do we take lessons learned from a controlled clinical trial and bring those lessons into evidence-based practice?

A general approach to measuring and sharing patient compliance metrics

A general approach to addressing these challenges should be based on standard metrics, fast data and active monitoring and reinforcement and reuse. 

1. Use standard metrics for treatment and patient reporting compliance. The metrics then become a transparent indicator of performance and a tool for improvement.

A simple metric of compliance might be a score based on patient reporting, treatment compliance and treatment violations. We may consider a threshold for each individual metric – for example a 3 strike rule like in baseball.

A more sophisticated measure of compliance might be similar to beta in capital market theory where you measure the ‘volatility’ of individual patient compliance compared to the study as a whole. (Beta is used in the capital asset pricing model, which calculates the expected return of an asset based on its beta and expected market returns or expected study returns in our case).

2. Fast data means automating for digital data collection from patients, connected medical devices and sites eliminating paper source and SDV for the core data related to treatment and safety endpoints.

3. Actively monitor and help patients sustain a desired state of compliance to the treatment protocol, both pharmacologic and non-pharmacologic. Not everything is about pill-counting. This can be done AI-based reminders using techniques of contextual bandits and decision trees.

4. Reuse clinical trial data and extract high quality training information that can be used for evidence-based practice.

Patient compliance teardown

Measures of patient compliance can be classified into 3 broad categories:

Patient reporting – i.e how well patient reports her own outcomes

1. Treatment compliance – how well the treatment conforms to the protocol in terms of dosing quantities and times of application. 2. Research suggests that professional patients may break the pill counting model

3. Patient violations – if the patient does something contrary to the protocol like taking a rescue medication before the migraine treatment

Confounding variables

Many heart failure patients are thought to be non-compliant with their treatment because of prior beliefs – believing that the study treatment would not help them. In the European COMET trial with over 3000 patients it was found that a Lack of belief in medication at the start of the study was a strong predictor of withdrawal from the trial (64% versus 6.8%; p < 0.0001). Those patients with very poor well-being and limited functional ability (classified as NYHA III–IV) at baseline significantly (p = 0.01) increased their belief in the regular cardiac medication but not in their study medication (4)

But numerous additional factors also contribute to patient non-compliance in clinical trials:  lack of home support, cognitive decline, adverse events, depression, poor attention span, multiple concomitant medications, difficulty swallowing large pills, difficult-to-use UI in medical devices and digital therapeutics and inconveniences of urinary frequency with diuretics for heart failure patients (for example).

It seems that we can identify 6 main confounding variables that influence compliance:

1. Patient beliefs – medication is useless, or this specific medication cannot help or this particular chronic condition is un-curable

2. Concerns about side effects – this holds for investigators and for patients and may account for levels of PI non-compliance.

3. Alert fatigue – patients can be overwhelmed by too many reminder message

4. Forgetfulness – old people or young persons. Shift workers.

5. Language –  the treatment instructions are in English but the patient only speaks Arabic.

6. Home support – patient lives alone or travels frequently or does not have strong support from a partner or parent for their chronic condition.

Summary

Flaskdata.io provides a HIPAA and GDPR-compliant cloud platform that unifies EDC, ePRO, eSource and connected medical devices with automated patient compliance monitoring. The latest version of Flaskdata.io provides standard compliance metrics of patient reporting and active messaging reminders to help keep patients on track.  Your users can subscribe to real-time alerts and you can share metrics with the entire team.

Contact Batya for a free demo and consult and learn how fast data, metrics and active reinforcement can help you save time and money on your next study.

References

1. Geriatr Nurs. 2010 Jul-Aug;31(4):290-8. Medication compliance is a partnership, medication compliance is not.
Gould E1, Mitty E. https://www.ncbi.nlm.nih.gov/pubmed/20682408

2. Depression Is a Risk Factor for Noncompliance With Medical Treatment: Meta-analysis of the Effects of Anxiety and Depression on Patient compliance. DiMatteo et al http://jamanetwork.com/journals/jamainternalmedicine/fullarticle/485411

3. Importance of medication compliance in cardiovascular disease and the value of once-daily treatment regimens. Frishman. https://www.ncbi.nlm.nih.gov/pubmed/17700384

4. Adherence and perception of medication in patients with chronic heart failure during a five-year randomised trial Ekman, Andersson et al. https://doi.org/10.1016/j.pec.2005.04.005

 

 

 

Invisible gorillas and detection of adverse events in medical device trials

Weekly Episode #1 - Patients and study monitors are both people.

What is easier to detect in your study – Slow-moving or fast moving deviations?

This post considers human frailty and strengths.

We recently performed a retrospective study of the efficacy of  Flaskdata.io automated study monitoring in orthopedic trials. An important consideration was the ability to monitor patients who had received an implant and were on a long term follow-up program. Conceptually, monitoring small numbers of slow-moving, high-risk events is almost impossible to do manually since we miss a lot of what goes on around us, and we have no idea that we are missing so much. See the invisible gorilla experiment for an example.

One of patients in the study had received a spinal implant and was on a 6 month follow-up program dived into a pool to swim a few laps and died by drowning despite being a strong swimmer. Apparently, the pain caused by movement of the insert resulted  in loss of control and a severe adverse event. The patient had disregarded instructions regarding strenuous physical activity and the results were disastrous. 

It seems to me that better communications with the patients in the medical device study could have improved their level of awareness of safety and risk and perhaps avoided an unnecessary and tragic event.

Subjects and study monitors are both  people.

This might be a trivial observation but I am going to say it anyhow, because there are lessons to be learned by framing patients and monitors as people instead of investigation subjects and process managers. 

People are the specialists in their personal experience, the clinical operations team are the specialists in the clinical trial protocol. Let’s not forget that subjects and study monitors are both  people.

Relating to patients in a blinded study as subjects without feelings or experience is problematic. We can relate to patients in a personal way without breaking the double blinding and improve their therapeutic experience and their safety. 

We should relate to study monitors in a personal way as well, by providing them with great tools for remote monitoring and enable them to prioritize their time on important areas such as dosing violations and sites that need more training. We can use analytics of online data from the EDC, ePRO and eSource and connected medical devices in order to enhance and better utilize clinical operations teams’ expertise in process and procedure.

A ‘patient-centered’ approach to medical device clinical trials

In conditions such as Parkinsons Disease, support group meetings and online sharing are used to stay on top of medication, side effects, falls and general feeling of the patient even though the decisions on the treatment plan need to be made by an expert neurologist / principal investigator and oversight of protocol violations and adverse events is performed by the clinical operations team. There are many medical conditions where patients can benefit by taking a more involved role in the study. One common example is carpal tunnel syndrome. 

According to the findings of an August 3rd, 2011 issue of the Journal of Bone and Joint Surgery (JBJS), patients receiving treatment for carpal tunnel syndrome (CTS) prefer to play a more collaborative role when it comes to making decisions about their medical or surgical care. 

Treatment of carpal-tunnel syndrome which is very common and also extremely dependent upon patient behavior and compliance is a great example of the effectiveness of “shared decision-making, or collaborative, model” in medicine, in which the physician and patient make the decision together and exchange medical and other information related to the patient’s health.

As the article in JBJS concludes:

“This study shows the majority of patients wanted to share decision-making with their physicians, and patients should feel comfortable asking questions and expressing their preferences regarding care. Patient-centered care emphasizes the incorporation of individual styles of decision making to provide a more patient-centered consultation,” Dr. Gong added. 

In a ‘patient-centered’ approach to medical device clinical trials, patients’ cultural traditions, personal preferences and values, family situations, social circumstances and lifestyles are considered in the decision-making process.

Automated patient compliance monitoring with tools such as Flaskdata.io are a great way to create a feedback loop of medical device clinical data collection,  risk signatures improvement, detection of critical signals and communications of information to patients. Conversely, automated real-time patient compliance monitoring is a a great way of enhancing clinical operations team expertise.

Patients and study monitors are both people. 

Millennials are the future of clinical trial data management

esource tp get smart to market

Millennials, born between 1980 and 2000 and the first native generation of the digital age, are the quickly approaching additions to the modern workforce. Regardless of whether private or public sector Millennials are soon to become the bulk of the global workforce.

At present, Millennials represent 34% of the current US workforce (up 9% from 25% in 2015), and by 2020 50% of workers will be of the Millennial generation. As the demographics of present job seekers continues to shift, companies need to adjust their culture, facilities and technology to cater to the new generation.

Regarding the clinical trial industry, Millennials are not only the next generation of data managers and monitors, but will soon make up the bulk of the study subjects as well.

Choosing the right tool and UX for millennial subjects becomes acute considering usability factors and patient compliance issues for people under 30.

(more…)

A powerful alternative to checklists for assuring patient compliance

medical device clinical trials

Danny Lieberman, founder and CEO of FlaskData.io the leading cloud provider of
clinical compliance as a service, talks about breaking out of a patient compliance
checklist mentality by starting with one question.

The 3 pillars of GCP (good clinical practice)

1. Patient safety
2. Protocol compliance
3. Data quality

(We note that setting the focus on the primary clinical and safety end-points results in formulation of GCP as an exercise in optimizing patient compliance to the protocol.)

With the understanding that clinical trial site monitors commonly use checklists for their site visits our first question is to challenge the utility of checklists:

To what extent do fixed checklists enable the study monitor and sponsor
to assess the impact that study deviations have on protocol compliance?

Take for example the activity of monitoring IC (informed consent); a best practice informed
consent monitoring checklist looka like this:

Informed consent monitoring checklist

1. Was the consent form used, and translated versions, approved by the IRB?
2. Was the ICF the most current and approved version?
3. If the consent is available in more than one language, was the participant given a chance to choose the language he/she prefers?
4. Did the participant receive full explanation of the contents of the ICF?
5. Did the participant have ample time to ask any questions and were they addressed adequately? Was the ICF signed before any study procedures? (N/A if the trial has received an exemption from IRB to consent after some study procedures)
6. If the subject is unable to read, was an independent witness present throughout the consent process?
7. Was the participant coerced?
8. Did the participant apparently understand the contents of the ICF?
9. Was IC form signed appropriately?
10. Was the environment suitable for the IC process?
(Courtesy of Global Health Trials)

You can check-off 11 items on the list but there is only 1 question that matters:

“Are there patients participating now in the study who did not sign the ICF (informed consent form)”

Why does the version or the environment matter if the patient is enrolled without informed consent? How does this checklist evaluate the impact of deviations? Does the checklist provide any quantitative measures of patient compliance?

After you ask that 1 question – (Are are any patients enrolled who did not sign ICF?) you can go on to quantify the impact (by asking how many patients are enrolled in the study without signed ICF) and then proceed to provide corrective and preventive actions.

In this article we suggest considering an alternative approach based on generating and analyzing multiple threat scenarios for the clinical study being monitored.

Since clinical trial data is highly-dimensional (typically 500-1000+ dimensions) we may reap significant benefits from this approach since with so many dimensions there tend to be many unconnected and undiscovered stovepipes of compliance and data governance.

Multiple threat scenarios enable auditors and study monitors to side-step large scale self-assessment checklists and problematic integration of data across stovepipes (large drug studies and large CROS like Quintiles, PPD and ICON typically use multiple systems from multiple vendors creating multiple unconnected stovepipes of data – one of the key reasons it takes 5-7 weeks to respond to a deviation) and focus on key assets, attacks and common vulnerabilities in key operational processes of the clinical trial like informed consent, eligibility criteria and treatment compliance (whether treatment is self-administered by the patient or administered by medical staff in a hospital).

In our experience, the sponsor is primarily interested in how cheaply the audit can be done and how much time and money they can save further down the road. For the  business unit developing the medical device or drug, using a technique of multiple threat analysis will help show the best and most cost-effective way to progress from audit to patient compliance.

Do you base your regulatory affairs policy on Google?

You can do some homework online and then hire a clinical regulatory and compliance consultant who will walk you through the various GCP requirements and help you implement as many items as possible. This seems like a reasonable approach, but the more controls you implement, the more money you spend and moreover, you do not necessarily know if your risk posture has improved since you have not examined your value at risk – i.e how much money it will cost you for rework if more patients have to be enrolled due to non-adherence to protocol.  Recall that patient protocol compliance is central to the success of your clinical trial and the defense of your claims with the FDA should rely on your experimental design, data and risk-analysis and not on the percentage SDV (source document verification) that study monitors performed.

Top-down risk-analysis

Taking a page out of the privacy and security playbook, we want to do a top-down risk analysis, and then continue with risk management and periodic protocol compliance activity review during the course of the clinical trial.

The best way to do that top down risk analysis is to build probable threat scenarios – considering what could go wrong – sites doing shoddy data entry or a hacker sniffing the hospital wired LAN for PHI and destroying the integrity of your randomized controlled trial.

Threat scenarios as an alternative to compliance check lists

When we perform a software security assessment of a medical device or healthcare system, we think in terms of “threat scenarios” or “attack scenarios”, and the result of that thinking manifests itself in planning, penetration testing, security countermeasures, and follow-up for compliance. The threat scenarios are not “one size fits all”.

The threat scenarios for clinical trials for AIDS diagnostics using medical devices that automatically scan and analyze blood samples, or an Army hospital using a networked brain scanning device to diagnose soldiers with head injuries, or an implanted cardiac device with mobile connectivity or immunotherapy treatment for cancer are all totally different.

We evaluate the medical device / investigational product from an attacker point of view, then from the management team point of view, and then recommend specific cost-effective, security countermeasures to mitigate the damage from the most likely attacks.

In our experience, building a risk control portfolio based on attack scenarios has 3 clear benefits;

1. A robust, cost-effective monitoring portfolio based on attack analysis results in robust compliance over time since you now have a formal methodology for evaluating new emerging issues such as mobile devices or changes to regulation.
2. Executives related well to the concepts of threat modeling / attack analysis. Competing, understanding the value of their assets, taking risks and protecting themselves from attackers is really, at the end of the day why executives get the big bucks.
3. Threat scenarios are a common language between IT, clinical operations teams and the business area managers. This last benefit is extremely important in your organization, since business delegates compliance to regulatory affairs and regulatory affairs delegates assessment to the site monitor teams and there is clearly a disconnect by the time you go from a business manager to a CRA.

As I wrote in a previous essay “The valley of death between IT and security“, there is a fundamental disconnect between IT operations (built on maintaining predictable business processes) and security operations (built on mitigating vulnerabilities).

The disconnect between sponsor business management and site monitors.

Business executives delegate clinical operations to VP Clinical who delegates to CROs who delegate compliance to sites on the tacit assumption that each are the experts in their own particular domain. This is a necessary but not sufficient condition.

In the current environment of rapidly evolving types of attacks (hacktivisim, nation-state attacks, credit card attacks mounted by organized crime, script kiddies, competitors and malicious insiders and more…), it is essential that business managers, sites and regulatory affairs professionals, communicate effectively regarding the types of attacks that their organization may face and what is the potential business impact on the clinical trial.

If you have any doubt about the importance of sponsors sharing data with sites, consider that leading up to 9/11, the CIA had intelligence on Al Qaeda terrorists and the FBI investigated people taking flying lessons, but no one asked the question why Arabs were learning to fly planes but not land them.

With fundamental disconnects between 3 key stakeholders of clinical data (sites, monitors and sponsors), it is no wonder that organizations are having difficult assessing GCP compliance in a timely fashion –

Sponsors, monitors and sites (and increasingly patients) need a common language to execute their mission, and I submit that building risk control portfolio de your clinical trial around most likely threat scenarios from an attacker perspective is the best way to cross that valley of death.

There seems to be a tacit assumption with pharma and medtech executives that regulatory compliance is already a common language of compliance for a clinical trial, but as we demonstrated at the beginning of this article, compliance checklists like ICF monitoring etc, are a dangerous replacement for not thinking through the most likely threats to your clinical trials.

Let me illustrate why compliance checklists are not the common language we need by taking an example from another compliance area – credit cards.

PCI DSS 2.0 has an obsessive preoccupation with anti-virus. It does not matter if you have a 16 quad-core Linux database server that is not attached the Internet with no removable device nor Windows connectivity.

PCI DSS 2.0 wants you to install an anti-virus and open the server up to the Internet for the daily anti-virus signature updates. This is an example of a compliance control policy that is not rooted in a probable threat scenario that creates additional vulnerabilities for the business.

Consider some deeper ramifications of check-list-based compliance to the protocol.

When a QSA or HIPAA auditor records an encounter with a customer, he records the planning, penetration testing, controls, and follow-up, not under a threat scenario, but under a control item (like access control). The next auditor that reviews the compliance posture of the business needs to read about the planning, testing, controls, and follow-up and then reverse-engineer the process to arrive at which threats are exploiting which vulnerabilities.

In the cyber security space, actors such as government agencies (DHS for example) and security researchers go through the same process. They all have their own methods of churning through the planning, test results, controls, and follow-up, to reverse-engineer the data in order to arrive at which threats are exploiting which vulnerabilities.

This ongoing process of “reverse-engineering” is the root cause for a series of additional problems:

1. Lack of overview of the the threats and vulnerabilities to clinical trials that really count.
2. No sufficient connection to best practice controls, no indication on which controls to follow or which have been followed.
3. No connection between controls and protocol deviation events, except circumstantial.
4. No ability to detect and warn for negative interactions between controls (for example – edit checks that generate large number of queries on every field, hobbling the ability of the sites to collect data in a timely manner).
5. No archiving or demoting of less important and solved threat scenarios (since the checklists are control based).
6. Lack of overview of compliance status of a particular site, only a series of historical observations disclosed or not disclosed. (Is Bank of America getting better at data security or worse? Is the Department of Clinical Neuropathology at King’s College Hospital getting better at GCP compliance or worse?)

7. An excess of paper documents that cannot possibly be read by the regulatory and clinical affairs manager at every encounter.

8. Regulatory and data borders are hard to define since the border definitions are networks, systems and applications not

Beyond checklists – using value at risk to assess impact of patient compliance violations

Checklists are good for ensuring a repeatable process but threats to your study are rooted in unforeseen events like patients without informed consent. Your threat scenarios should consider your study assets (your data, systems, management attention, reputation) values, vulnerabilities, threats and effective security countermeasures.

Threat analysis as a methodology for monitoring your clinical trial does not count activities like site visits and SDV. It is a systematic way to help you consider the fastest and most cost-effective way to reduce your risks of protocol non-compliance, safety and data quality.

10 ways to reduce clinical trial risk and they are all free

Are the lights on but no one home in your medical device clinical trial?

Collecting low-quality data means that your trial is likely to fail. You will not be able to prove or disprove the scientific hypothesis of your medical device clinical trial. You will have wasted your time.

You cannot outsource quality, you have to build it into the trial design

(more…)

The 2 most common mistakes in Clinical Research Data Management

medical device clinical trials

Another survey piece that David wrote about common mistakes in clinical data management and some basic controls to stay from the common issues.

Use automated monitoring to empower people

We are all human. As much as we would like to rely on technology to automate every sector of clinical trial research, we still need the human component to study, assess data, store it and make decisions based on captured research data. We cannot remove human study monitors and data managers entirely out of the equation.

(more…)