10 ways to detect people who are a threat to your clinical trial

Flaskdata.io helps Life Science CxO teams outcompete using continuous data feeds from patients, devices and investigators mixed with a slice of patient compliance automation.

One of the great things about working with Israeli medical device vendors is the level of innovation, drive and abundance of smart people.

It’s why we get up in the morning.

There are hundreds of connected medical devices and digital therapeutics (last time I checked over 300 digital therapeutics alone).

When you have an innovative device with network connectivity, security and patient privacy, availability of your product and integrity of the data you collect has got to be a priority.

Surprisingly, we get a  range of responses from people when we talk about the importance of cyber security and privacy for clinical research,

Most get it but some don’t.   The people that don’t get it, seem to assume that security and privacy of patient data is someone else’s problem in clinical trials.

The people who don’t work in security, assume that the field is very technical, yet really – it’s all about people.   Data security breaches happen because people or greedy or careless.    100% of all software vulnerabilities are bugs, and most of those are design bugs which could have been avoided or mitigated by 2 or 3 people talking about the issues during the development process.

I’ve been talking to several of my colleagues for years about writing a book on “Security anti-design patterns” – and the time has come to start. So here we go:

Security anti-design pattern #1 – The lazy employee

Lazy employees are often misdiagnosed by security and compliance consultants as being stupid.

Before you flip the bozo bit on a site coordinator as being non-technical, consider that education and technical aptitude are not reliable indicators of dangerous employees who are a threat to the clinical trial assets.

Lazy employees may be quite smart but they’d rather rely on organizational constructs instead of actually thinking and executing and occasionally getting caught making a mistake.

I realized this while engaging with a client who has a very smart VP – he’s so smart he has succeeded in maintaining a perfect record of never actually executing anything of significant worth at his company.

As a matter of fact – the issue is not smarts but believing that organizational constructs are security countermeasures in disguise.

So – how do you detect the people (even the smart ones) who are threats to PHI, intellectual property and system availability of your EDC?

1 – Their hair is better organized then their thinking

2 – They walk around the office with a coffee cup in their hand and when they don’t, their office door is closed.

3 – They never talk to peers who challenge their thinking.   Instead they send emails with a NATO distribution list to everyone on the clinical trial operations team.

4 – They are strong on turf ownership.  A good sign of turf ownership issues is when subordinates in the company have gotten into the habit of not challenging the VP coffee-cup holding persons thinking.

5 – They are big thinkers.    They use a lot of buzz words.

6 – When an engineer challenges their GCP/regulatory/procedural/organizational constructs – the automatic answer is an angry retort “That’s not your problem”.

7 – They use a lot of buzz-words like “I need a generic data structure for my device log”.

8 – When you remind them that they already have a generic data structure for their device log and they have a wealth of tools for data mining their logs – amazing free tools like Elasticsearch and R….they go back and whine a bit more about generic data structures for device logs.

9 – They seriously think that ISO 13485 is a security countermeasure.

10 – They’d rather schedule a corrective action session 3 weeks after the serious security event instead of fixing it the issue the next day and documenting the root causes and changes.

If this post pisses you off (or if you like it),  contact  me –  always interested in challenging projects with challenged people who challenge my thinking.

Competitive buzzwords in EDC companies

We recently did a presentation to a person at one of the big 4 pharma.  His job title was

Senior IT Project Manager Specialized in Health IT.

I looked at the persons LinkedIn profile before the call and I noticed that the sentence is in past tense. Specialized in Health IT implying that he was now a Senior IT manager who no longer specialized in anything.

I have a friend who worked at Pfizer in IT. He was discouraged by pharma IT mediocrity especially  when he compared it to the stellar talents in the R&D departments.

So it stands to reason that the EDC vendors are just a notch up the technology ladder from the pharma IT guys. If you do not have a unique technology value proposition, you have to resort to marketing collateral gymnastics.

To test this hypothesis – I took a look at the web sites of 4 EDC vendors:  Medidata, Medrio, Omnicomm and Oracle Life Sciences.

Medidata

Run Your Entire Study On A Unified, Intelligent Platform Built On Life Science’s Largest Database.

At Medidata, we’re leading the digital transformation of clinical science, so you can lead therapies to market faster, and smarter. Using AI and advanced analytics, our platform brings data managers, clinical operations, investigators, and patients together to accelerate the science and business of research.

MediData is making a disturbing suggestion in their marketing collateral that they leverage other companies trial data in their Life Science Database to help you lead therapies to market faster.

Medrio

Clinical trial data collection made easy. The industry’s leading early-phase EDC and eSource platform.

The only EDC vendor that actually admitted to being an EDC vendor was Medrio. You have to give them a lot of credit for honesty.

Omnicom

eClinical Solutions for Patient-Centric Clinical Trials
Effective Clinical Tools Driving Excellence in Life Science Research

Software has the power to save lives. OmniComm Systems understands that power and delivers eClinical solutions designed to help life science companies provide crucial medical treatments and therapies to patients around the globe.

OmniComm Systems fills a role in enhancing patient lives by shortening the time-to-market of essential life-saving treatments. Our eClinical suite of products includes electronic data capture (EDC) solutions, automated coding and randomization systems, risk-based monitoring (RBM) and analytics.

This is nice positioning, but it makes you wonder when OmniComm turned into a healthcare provider of crucial medical treatments and therapies to patients around the globe.

Oracle Life Science

Oracle Life Sciences—Reimagining What’s Possible

Innovation in science and medicine demands new technology, and innovation in
technology makes new things possible in science and medicine. Oracle is equipping the life sciences industry today, for the clinical trials of tomorrow.

Solutions Supporting the Entire Clinical Development Lifecycle

Oracle Health Sciences helps you get therapies to market faster and detect risks earlier. Oracle offers a complete set of clinical and safety solutions that support critical processes throughout the clinical development lifecycle—from study design and startup to conduct, close-out, and post-marketing.

SOLUTIONS
Oracle Health Sciences Clinical One cloud environment changes the way clinical research is done—accelerating all stages of the drug development lifecycle by eliminating redundancies, creating process efficiencies, and allowing the sharing of information across functions.

Unlike OmniComm and Medidata,   Oracle is firmly focused on the clinical development lifecycle; not pretending that they are a healthcare provider or leverage the patient data in their EDC databases.

Flaskdata.io

Helping life-science C-suite teams outperform their competitors.

Patient compliance is critical to the statistical power and patient retention of a study.

We help senior management teams complete studies and submission milestones faster and under budget. We do this by providing EDC, ePRO and integration of connected medical devices into a single data flow. We then automate detection and response of patient compliance deviations in clinical trials 100x faster than current manual monitoring practices.

 

 

5 ways to make your clinical trials run real fast

medical device clinical trials

This week, we had a few charming examples of risk management in clinical trials with several of our customers.   I started thinking about what we could do to get things to run real fast and avoid some of the inevitable potholes and black swans that crop up in clinical trials.

Engaged in basic science and stuck in data traffic

There is something very disturbing  about an industry that develops products using advanced basic science.

It is disturbing because the industry uses 40-year old processes and information technology.

This industry accepts a reality of delays of a year or more due to manual data processing.

This industry is called life sciences.

That’s what disturbs on a personal and strategic level.   We can and should do better.  The disconnect between basic science and modern software should disturb anyone involved with clinical research because the cost to society is enormous.      We are enamoured with Instagram, Uber and WeWork but we choose to pretend that life science research exists in a parallel untouchable universe protected by ICH GCP, FDA, MDR and a slew of other TLAs.

Alright.  I am Israeli and trained as a physicist.   Let’s look for some practical, real-world solutions. Let’s try them out and iterate.

5 ways to make your clinical research run real fast

1. Data model

Before designing your eCRF, design your data model.  If you do not know what data modelling means, then 4 weeks before the study starts is a bad time to start learning.   Hire a specialist in data modelling, preferably someone who does not work in life sciences.   Pay them $500/hour.  It’s worth every penny. The big idea is to design an abstract data model for your study for speed of access and usability by patients, site coordinators, study monitors and statisticians before designing the eCRF.

2. Discipline equals speed

Start early. Go slow and speak softly and then run fast.  There is a story about the difference between a Japanese wood sculpture artist and an Israeli artist. The  Japanese artist goes into his studio and looks at a big piece of wood. He walks around the wood and observes.   He goes home.  The next day and for the next month, he observes the wood in his studio, without touching his tools.    After a month of observation, he comes in, picks up a . hammer and chisel and chop, chop chop, produces a memorable work of art.      The Israeli goes into his studio and looks at a big piece of wood. He starts carving away and improvising all kinds of ideas from his head. He goes home.  The next day and for the next month, he chops away at wood and replaces raw material several times.   After a year, he has a work of art.

The big idea is that discipline equals speed.  It prepares you for the unexpected. See point 6 below.

A good book that presents this approach in a very practical way is Discipline equals Freedom by Jocko Willink.

3.Date and time

Date/time issues can be visualised as a triangle.

Side 1 of the triangle is the site coordinator who collects data into the EDC.

Side 2 of the triangle is the CRA who monitors CRC work and data quality and performs SDV.

Side 3 of this triangle is the subject who needs to come and visit the doctor on certain days that study coordinator scheduled for her when she started the trial.

Pay attention to your date and time fields.    This is a much neglected part of data design in clinical trials.

The challenge is that you need to get your clinical data on different timelines.     Most people ignore the fact that clinical trials have several parallel timelines.

One timeline is the study schedule.  Another timeline is adverse events.  Another timeline is patient compliance.    You get it.   If you collect high quality date times in your data model, you can facilitate generating  the different time-series.

One of the most popular pieces on this blog is an essay Jenya wrote on dates and times in clinical data management.  You can read it here.

4.Do not DIY your EDC

You can DIY a chair from Ikea but not your clinical trial.   I know that there are a lot of low-cost eCRF packages out there like Castor EDC and Smart Clinical. The notion of a researcher or clinical manager, untrained in data modelling, data analysis and user interface design using a cheap DIY tool to develop the most important part of your study should make you stop and think.  To put this in different perspective, if you are spending $5,000/month to monitor 3 sites, you should not be paying $450/month for a DIY EDC.    It’s called penny-wise and pound foolish.

5.Prioritise deviations.

While it is true that protocol deviations need to be recorded, not every protocol deviation is created equal.      I was stunned recently to hear from a quality manager at one of the big CROs that they do not prioritise their deviation management.     Biometrics, dosing, patient compliance and clinical outcomes should be at the top of list when they relate to the primary clinical endpoint or safety endpoint.    This is related to the previous points of not DIY, data modelling and observing before cutting wood.

6.Do some up-front risk assessment but don’t kid yourself.

Before you start the study, any threat analysis you do is worthless.   A risk analysis without data is worthless.  You may have some hypotheses based on previous work you or someone else did but do not kid yourself.   First collect data, then analyse threats.   I’ve written about how to do a risk assessment in clinical trials here, here, here and here.  Read my essay on invisible gorillas.

Temperature excursions and APIs to reduce study monitor work

I did a lot of local excursions the past 3 days – Jerusalem, Tel Aviv, Herzliya and Haifa.   For some reason, the conversations with 2 prospects had to do with refrigerators.   I do not know if this is Freudian or not, considering the hot weather of July in Israel.

The conversations about refrigerators had to do with storing drugs / investigational product at the proper temperatures.

Temperature excursion is a deviation

The great thing about not coming from the clinical trials space is that you are always learning new things.

Yesterday – I learned that a Temperature excursion is a deviation from given instructions. It is defined in the WHO Model Guidance as “an excursion event in which a Time Temperature Sensitive Pharmaceutical Product (TTSPP) is exposed to temperatures outside the range(s) prescribed for storage and/or transport.

Storing drugs at the proper temperature is part of GCP. Here is an SOP for Monitoring and Recording Refrigerator & Freezer Temperatures

1 Introduction All refrigerators and freezers used for the storage of Investigational Medicinal Products (IMPs) must be temperature controlled, and continuously monitored and maintained within the appropriate ranges as defined by the protocol. ICH GCP Principle 2.13 states “Systems with procedures that assure the quality of every aspect of the trial should be implemented.”

Moving on:

5 Procedure
 Current maximum/minimum thermometers must be monitored as a minimum at least once on a daily basis on all working days, and recorded legibly on the temperature monitoring log.
 The digital maximum/minimum thermometer –
□ Should be read from the outside of the refrigerator without opening the door.
□ Have an accuracy of at least +/- 1 oC.
□ Be able to record temperatures to one decimal place.
□ Be supplied with a calibration certificate.
□ Have the calibration check on an annual basis.
 Temperature logs should be kept close to the refrigerator/freezer (but not inside) to which they relate for ease of reference, and should be clearly identified as relating to that appliance.
 A separate temperature record must be kept for each fridge/freezer. (The use of whiteboards as a method of logging results is not acceptable.)
 It is good practice to record the temperature at a similar time each day e.g., first thing in the morning before the refrigerator door is opened for the first time. This will allow review of trends in results recorded; help highlight any changes in temperatures recorded and deviation in refrigerator performance.

There is a lot of manual work involved looking at refrigerators

I believe a study monitor will spend 20’/day checking logs of refrigerator temperature readings. When you add in time for data entry to the site coordinators – that’s another 20’/day and then you have to multiply by the number of sites and refrigerators.   This is only the reading temperatures and capturing data to the EDC part of the job.   Then you have to deal with queries and resolving deviations.

For something so mundane (although crucial from a medical research perspective), its a lot of work. The big problem with using study monitors to follow temperature excursions is that the site visits are every 1-3 months. With the spiralling costs of people, the site visits are getting less frequent.

This means that it is entirely plausible that patients are treated with improperly stored drugs and the deviation is undetected for 3 months.

Whenever I see a lot of manual work and late event detection, I see an opportunity.

It seems that there are a few vendors doing remote monitoring of refrigerators.  A Polish company from Krakow, called Efento has a complete solution for remote monitoring of refrigerators storing investigational product.  It looks like this:

 

null

 

What is cool (to coin a pun) about Efento is that they provide a complete solution from hardware to cloud.

The only thing missing is calling a Flask API to insert data into the eCRF for the temperature excursions.

Once’s we’ve got that, we have saved all of the study coordinators and study monitors time.

More importantly, we’ve automated an important piece of the compliance monitoring puzzle – ensuring that temperature excursions are detected and remediated immediately before its too late.

The gap between the proletariat and Medidata (or should I say Dassault)

We need a better UX before [TLA] integration

The sheer number and variety of eClinical software companies and buzzwords confuses me.
There is EDC, CTMS, IWRS, IVRS, IWRS, IRT, eSource, eCOA, ePRO and a bunch of more TLAs.
For the life of me I do not understand the difference between eCOA and ePRO and why we need 2 buzzwords for patient reporting.

Here is marketing collateral from a CRO.   As you will see – they miss the boat on all the things that are important for site coordinators and study monitors.

We adapt responsively to change in your clinical trial to minimize risk and drive quality outcomes. Clinical research is complicated and it’s easy to get off track due to inexperienced project leaders, inflexible workflows, or the failure to identify risks before they become issues. We derive expert insights from evidence-based processes and strategic services to be the driving force behind quality outcomes, including optimized data, patient safety, reduced time-to-market, and operational savings.

What CRCs and CRAs have to say about the leading eClinical solutions

I recently did an informal poll on Facebook of what problems the CRA/CRC proletariat have to deal with on the job.

I want to thank Tsvetina Dencheva for helping me grok and distill people’s complaints
into 3 central themes.

Theme no. 1 – enter data once

Enable administrators to enter data once and have their authorized user lists, sites and metrics update automatically without all kinds of double and triple work and fancy import/export footwork between different systems. Failing a way of managing things in one place –
at least have better integration between the EDC and the CTMS.

The IT guys euphemistically call this problem information silos. I’ve always thought that they used the word silos (which are used to store animal food) as way of identifying with people who farm, without actually having to get their hands dirty by shovelling silage (which is really smelly btw).

I understand the rationale for having a CTMS and an EDC about as much as I understand the difference between eCOA and ePRO.

Here is some raw data from the informal Facebook survey

If I enter specific data, it would be great if there’s an integrated route to all fields connected to the said data. An easy example is – if I enter a visit, it transfers to my time sheet.

Same goes to contact reports. Apps! All sorts of apps, ctms, verified calculators, edc, ixrs, Electronic TMF. The list goes on and on. How could I forget electronic training logs? Electronic all sorts of log.

There are a lot of things we do day to day that are repetitive and can take away from actually moving studies forward. Thinking things like scanning reg docs, auto capturing of reg doc attributes (to a point), and integration to the TMF. Or better system integration, meaning where we enter a single data point (ie CTMS) and flowing to other systems (ie new site in CTMS, create new site in TMF. Enrolment metrics from EDC to CTMS) and so on.

If only the f**ing CTMS would work properly.

Theme number 2 – single sign-on.

The level of frustration with having to login to different systems is very high. The ultimate solution is to use social login – just login to the different systems with your Google Account and let Google/Firebase authenticate your identity.

Theme number 3 – data integrity

EDC edit check development eats up a lot of time and when poorly designed generates thousands of queries. Not good.

There is a vision of an EDC that understands the data semantics from context of the study protocol.

This is a very cool and advanced notion.

One of the study monitors put it like this:

The EDC should be smart enough to identify nonsense without having to develop a bunch of edit checks each time and have to deal with queries.

The EDC should be able to calculate if a visit is in a proper time window, or if imaging is in a proper time window. Also for oncology if RECIST 1.1 is used, then the EDC should be able to calculate: Body Surface Area, correct dosing based on weight and height of a patient, RECIST 1.1 tumor response and many other things that simply can be calculated.

About flaskdata.io

We specialise in faster submission for connected medical devices. We can shorten your
time to market by 9-12 months with automated patient compliance detection and response.

Call us and we’ll show you how. No buzzwords required.

4 strategies to get connected medical devices faster to FDA submission

Introduction

Better designs, site-less trials, all-digital data collection and PCM (patient compliance monitoring) can all save time and money in connected medical device clinical trials.  This article will help you choose which strategies will be a good fit to help you validate your connected medical device and its intended use for submission to FDA.

What is the baseline cost? (hint don’t look at the costs of drug studies)

If you want to save, you need to know the price tag. Note that the costs of drug trials, including CRO and regulatory affairs is an order of magnitude higher than for connected medical devices.  A JAMA report from Nov 2018, looked at drug trials and concluded that a mean cost of $19M was cheap compared to the total cost of drug development – $1-2BN.

Findings:  In this study of 59 new therapeutic agents approved by the FDA from 2015 to 2016, the median estimated direct cost of pivotal efficacy trials was $19 million, with half of the trial cost estimates ranging from $12 million to $33 million. At the extremes of the distribution were 100-fold cost differences, and patient enrollment varied from fewer than 15 patients to more than 8000 patients.

By comparison, the estimated cost of medical device clinical trials to support approval by the FDA, ranges from $1 million to $10 million. A report from May 2017 surveyed the costs of medical device clinical trials and the potential of patient registries to save time and money. The report has some interesting numbers:

1.The average cost to bring a low-to-moderate concern device from concept to 510(K) approval is $31 million. 77% of that is spent on FDA-related/regulatory-affairs activities.

2.The average cost for a high-risk PMA device averages $94 million, with $75 million spent on FDA-related/regulatory-affairs activities. Average of 4.5 years from first contact with FDA to device approval.

3.Clinical trials outside the US are 30% to 50% cheaper. Less than 50% of medical device trials are now conducted in the US.

I. Better study designs

Real-world data (RWD) and real-world evidence (RWE) are being used for post-market safety surveillance and for designing studies, but they are not replacements for conducting a randomized trial with a controlled clinical protocol.  FDA recently issued guidance for use of real-world evidence for regulatory decisions.  FDA uses RWD and RWE to monitor post-market safety and adverse events and to make regulatory decisions.

RWD and RWE can be used in 4 ways improve the design of medical device clinical trials when there is a predicate device that is already being used for treating patients.

1.Use RWD/RWE to improve quality and efficiency of device evaluation at study phases (feasibility, pivotal, and post-market), allowing for rapid iteration of devices at a lower cost.

2.Explore new indications for existing devices

3.Cost efficient method to compare a new device to standard of care.

4.Establish best practices for the use of a device in sub-populations or different sub-specialties.

You will need to factor in the cost of obtaining access to the data and cost of data science.

But real-world data may not be reliable or relevant to help design the study.  As FDA notes in their guidance for Using Real-world evidence to support regulatory decision making:

RWD collected using a randomized exposure assignment within a registry can provide a sufficient number of patients for powered subgroup analyses, which could be used to expand the device’s indications for use. However, not all RWD are collected and maintained in a way that provides sufficient reliability. As such, the use of RWE for specific regulatory purposes will be evaluated based on criteria that assess their overall relevance and reliability. If a sponsor is considering using RWE to satisfy a particular FDA regulatory requirement, the sponsor should contact FDA through the pre-submission process.

II. Site-less trial model

Certain kinds of studies for chronic diseases with simple treatment protocols can use the site-less trial model.  The term site-less is actually an oxymoron, since site-less or so-called virtual trials are conducted with a central coordinating site (or a CRO like Science37). Nurses and mobile apps are using to collect data from patients at home.   You still need a PI (principal investigator).

The considerable savings accrued by eliminating site costs, need to be balanced with the costs of technology, customer support, data security and salaries and travel expenses of nurses visiting patients at homes.

III. Mostly-digital data collection

For a connected medical device, mostly-digital data collection means 3 things:

1.Collect patient reported outcome data using a mobile app or text messaging

2.Collect data from the connected medical device using a REST API

3.Enable the CRC (clinical research coordinator) to collect data from patients (IE, ICF for example) using a Web or mobile interface (so-called eSource) and skip the still-traditional paper-transcription step. In drug studies, this is currently impossible because hospital source documents are paper or they are locked away in an enterprise EMR system. For connected medical device studies in pain, cannabis and chronic diseases, most of the source data can be collected by the CRC with direct patient interviews. Blood tests will still need to be transcribed from paper. Mostly-digital means mostly-fast. Data latency for the paper source should be 24 hours and data latency for the digital feeds should be zero.

There are a number of companies like Litmus Health moving into the space of digital data collection from mobile devices, ePRO and wearables. However, unlike validating a connected medical device for a well-defined intended use, Litmus Health is focused on clinical data science for health-related quality of life.

IV. PCM (patient compliance monitoring)

Once the data is in the system, you are almost there.  Fast (low-latency) data from patients, your connected device and the CRC (which may be nurses in a site-less trial) are 3 digital sources which can be correlated in order to create patient compliance metrics.  But that is a story for another essay.

Summary

We have seen that new business models and advanced technologies can help sponsors conduct connected medical device trials cheaper and faster. It may not be a good fit for your product.  Contact us and we will help you evaluate your options.

For more information read Gail Norman’s excellent article  Drugs, Devices, and the FDA: An overview of the approval process

 

 

 

 

 

 

 

 

Putting lipstick on the pig of electronic CRF?

esource for people in clinical trials

Good online systems do not use paper paradigms. In this post – I will try and entertain you with historical perspective and quantitative tools for choosing an EDC system for your medical device study.

Decades of common wisdom in clinical trials still hold to a paper-based data processing model. One of the popular EDC systems talks about the advantages of having online forms that look exactly like paper forms.  True – familiarity is a good thing, but on the other hand a digital UX has far more possibilities for user engagement and ease-of-use than paper.   So – it is, in a way admitting failure to provide a better UX and downgrading to paper.

We recently engaged with an Israeli medical device vendor who has an innovative device for helping solve a common medical indication for men over 50.

I won’t go into details.

If you are a guy over 50, you know what I mean.

If not, it doesn’t matter.

The client CEO was interested in an eCRF (electronic CRF – case report form) system.  eCRF is better than paper, but it is, at the end of the day just a paper form in an electronic format.

I was having a lot of trouble trying to understand the CEO’s business requirements.   My attempts to steer the conversation to a discussion of how to obtain fast data for his clinical trial and reduce his time to FDA submission fell on deaf ears. A follow up conversation and demo of Flaskdata with the clinical and regulatory manager focused more on reports and how to manage queries.  Queries are a vestige from the paper CRF period, where a study monitor would visit the research site once/month, compare the paper source with the electronic data entry and raise queries or discrepancies.

In order to put this process into a historical context, let’s compare accounting systems from the late 70s, early 80s to an eCRF system.

Accounting versus eCRF

Feature Accounting circa 1970 eCRF circa 2019
Input data Paper JV – journal voucher Paper source
Data entry Data entry to a 2-sided accounting system Data entry to an eCRF
Data processing A batch job, processes punch-card data entry and produces a data entry report and data error report Site coordinators enter data to a Web app 1-3 days after the patient visit. Data entry errors or invalid data create data validation queries which are ignored until the study monitor visit a month later
Exception reporting Data error report – with non-numeric or invalid dates Queries
Management reports Trial balance

Profit and loss

Cash flow

Bean counters of CRF/items

What is profit and loss?

What does a cash flow model have to do with clinical trials?

 

 

Cost justification and TCO for medical device EDC systems

My first recommendation would be don’t buy an EDC system just because its cheap. Charging $100-300/month for a data entry application is not be a reason to give someone money.  As a client of ours once said – “I know I can use Google Forms for data entry and its free but Google Forms does not have an audit trial so  Google Forms is not an option for clinical trials”.

As a rule-of-thumb, a good EDC system for medical device studies should include audit trails and a clinical cash flow report (the flow of patients in and out, the flow of data items in and out).   The EDC should also be able to produce a clinical Profit and loss statement, showing you how well you are doing on your primary and secondary efficacy and safety end points. A well-designed and well-implemented EDC should include a robust data model for testing the primary endpoint and collecting safety data.    At the minimum, a solid design and implementation will cost at least $10,000.  Over 10 months, that’s a starting cost of $1000/month.   As Robert Heinlein said – “There is no such thing as a free lunch”.

Your decision to buy EDC should be based on an economic breakeven point.   One breakeven method is based on cost reduction in site monitoring.  Assume the EDC system costs $4000/month (weighted cost including implementation) and assuming a monthly site visit costs $800/day, then your EDC system must be able to save 5 site visit a month and assure protocol compliance. This places an upper bound on the price you can pay.

This is albeit problematic for small, 1 site studies which often use DIY implementations.  Just remember, that ignoring the implementation cost, does not make the product cheaper. In other words calculate your TCO (Total cost of ownership).

Or as one wise man one said – I’m too poor to buy a cheap car.

Israel Biomed 2019-the high-social, low stress STEM conference

Impressions from Biomed 2019 in Tel Aviv

This week was the annual 3 day Biomed/MIXiii (I have no idea what MIXiii means btw) conference in Tel Aviv.  The organizers also billed it as the “18th National Life Science and Technology Week” (which I also do not know what that means). This was a particular difficult time for a conference of medical device and pharma in Tel Aviv since it coincided with the Eurovision 2019 activities – and the traffic was tough.

There were a huge number of lectures and participants from all over the world and I suppose from that perspective, the conference is a success and tribute to the burgeoning Israeli biomed industry.  Forbes calls Biomed “The High-Paying, Low-Stress STEM Job You Probably Haven’t Considered”.  I think that this is probably a good description for the conference – high participation but low stress.

My colleagues and I come to the conference to network, schmooze, meet customers and suppliers.  It’s a good opportunity to take a few meetings, say hi to friends and hustle for new business.  Having said that, I did meet a few really interesting companies:

RCRI – is a Minneapolis MN based medical device CRO.  I met Todd Anderson and his boss Lisa Olson and pitched our approach for fast data in clinical trials to assure high levels of patient compliance to the protocol and submit faster to FDA.    Todd and Lisa get it and they were open about the CRO business model being more people-hours not speed.     They seemed genuinely interested in what we are working on but its hard to tell with Americans.

Docdok Health – is a startup founded by Yves Nordman, who is a Swiss MD living in Carmiel.  It’s a doctor-patient communications platform beginning to branch out into Post-marketing studies with RWD.    We shared demos and it seems that there is synergy between our regulatory platform and their post-marketing work.

Resbiomed – met Alex Angelov, the CEO.  Alex is leading a consortium including Flaskdata, Carl Zeiss, Collplant, PreciseBio and Pluristem for a Horizon2020 submission for an amazing project for an implant to the cornea.  Dan Peres from Pluristem got us together.   Cheer for us!

BSP Medical and ICB (Israel China Biotech investment) – my buddy Hadas Kligman literally took me by hand to visit to Yehuda Bruner and Andrew Zhang and I did my 60s elevator pitch on getting medical device companies to FDA/CFDA 6-12 months faster.   We agree to talk after the conference.

Butterfly Medical –  I met Idan Geva, the CEO last year at Biomed – we ate lunch at the same table.  I pitched him but he was uninterested – they were using EDC2Go – and he didn’t want to hear other options.     At the Minnesota pavilion talking to Todd Anderson from RCRI,   Idan shows up and looks at me and says “Heah – Hi Danny – I left a contact me request on your web site yesterday and no one got back to me. I said shame on us.  He says – he was referred to us by someone from Florida who used to use Medidata.  I asked where/who? was it Miami?  He says yeah it was Miami and checks his phone – says its someone from Precision Clinical Research that are using Flaskdata and recommended.    (Precision is one of our customer’s Miami sites).  I asked what happened to EDC2Go – he said well you know – they are end of life (I think this means the end of low-cost EDC) and we are now entering questionnaires manually on paper and it is driving us crazy.   He said – can you stick around and give us a demo at 15:00?  I said sure.  We met at 15:00 by the bar upstairs in the David Intercontinental and I demoed the system – he said “Show me the Forms designer”. I showed him.  He says “show me how CRC enters data” – I showed him.  He says “Show me how to extract data” – I showed him.  I think he actually did not believe how fast the Extract to CSV process was and asked me twice if that was the data.  In the end – the format of Mac Numbers was a bit strange for him. I showed him a quick presentation – and he saw that Serenno is a customer – and says – “Heah Tomer is a neighbor of ours in the incubator in Yokneam”.    He asked how much and I said $2K for a basic onboarding package and $1500 / month.  Or $10K and we will build the CRF (their CRF is super simple btw).  He wanted a discount, being Israeli.  I said – “lets meet with your clinical person and get her to buy-in to the solution.  If she buys in – you and I can talk business but before that, there is no point horse-trading.

Count the probabilities of this happening and you will see that it is an impossible event.

Thursday I went back to demo Todd and meet Dr Yael Hayun from Syqe Medical. Yael is one of the most impressive people I’ve met in a long time. She is an MD from Hadassah and one of the movers and shakers in LogicBio Therapeutics.    After we chatted – I told her that Syqe is lucky to have her onboard.   I did our Today is about Speed presentation and a short demo. She was suitably impressed and then mentioned they had met with a Danish EDC company called Smart Trial – which turns out is yet another low-cost eCRF provider.   I said look – eCRF is like 10% of the solution you need – in the case of Syqe, you have a digital inhaler and with cannabis, you are going to have a lot of concerns about patient compliance.

This is what we do – fast data collection from patients, investigators and digital inhalers and automated deviation detection and response.

On the way back – huge traffic from Eurovision.   Didn’t hear a single lecture but the meetings and people were outstanding.

 

Perverse incentives

The perverse incentive for the high costs of medical devices and delay to market

The CRO outsourcing model and high US hospital prices result in higher total CRO profits via higher costs to companies developing innovative medical devices.   These costs are passed down to consumers after FDA clearance.

We’ll take a look at the cost dynamics of medical device clinical trials and the clinical trial value chain.

We’ll then consider an alternative business model that changes the way medical device sponsors conduct clinical trials, reduce their costs by 70-80% and shortens time to FDA submission.

The high costs of US hospitals

By 2000, the US spent more on healthcare than any other country, whether measured per capita or a percentage of GDP.

U.S. per capita health spending was $4,631 in 2000, an increase of 6.3 percent over 1999. 4 The U.S. level was 44 percent higher than Switzerland’s, the country with the next-highest expenditure per capita; 83 percent higher than neighboring Canada; and 134 percent higher than the OECD median of $1,983. 5

It’s the prices, stupid.

In 2011, the US Affordable Care Act set a requirement for MLR (Medical Loss Ratio) that insurers must spend 80-85% of revenue on medical services.    This reduced insurer margins, and drove up hospital prices to make up for lower margin.

The CRO business model

CROS (clinical research organizations) are outsourcing businesses that provide an array of services for clinical trial management and monitoring, reporting and regulatory submission.   For medical device studies, CROS employ 2 basic outsourcing models, people sourcing and functional sourcing. In people out-sourcing, the medical device company is responsible for managing contractors. In functional outsourcing, the company may buy a set of functions, for example study monitoring and medical writing.

Neither CRO model has an explicit incentive to complete a study faster since that would reduce outsourcing revenue for the CRO. The more time a CRO spends on monitoring, site visits, SDV and study closeout, the more revenue it generates.

A medical device sponsor may elect to do it himself which shifts the CRO cost to an internal headcount cost supplemented with additional costs for consultants with risk and time delays by not having the CRO expertise and infrastructure. There is tacitly no free lunch, as we will discuss later in this article.

The result is a perverse incentive for delay and higher costs to bring innovative medical devices to market.

The CRO business model combined with higher hospital prices drive higher total profits via higher costs to customers. The higher cost of innovative medical devices is then passed down to consumers (patients) after FDA clearance.

Consumer value chains

A consumer value chain looks generically like this:

Suppliers -> Distributers -> Consumers

By the early 90’s, the PC industry led by Intel and Microsoft used a 2-tier value chain:

MSFT->Distis->Resellers->Customers.

Resellers were further segmented according the customer size and industry segment – Retail, Large accounts, SMB and VARS (value-added-resellers) selling their own products and services to a particular industry vertical.   The PC industry value-chain model left Microsoft with 50% of the SRP (suggested retail price) and delivered products to customers that were 45-50% less than SRP, leaving the channel with 0-5%.

The channel was forced to implement extremely efficient operations and systems and sell value-added services and products in order to survive.

By the new millennium, Apple introduced a 1-tier model with a user-experience designed and controlled by Apple.

The Apple 1-tier channel looks like this:

Apple->Apple Stores->Consumers

Eventually the Apple channel model broadened to include a 2-tier model similar to PC industry:

Apple->Distis->Retail->Consumers

By the mid-2000s, Amazon AWS (and generally the entire cloud service / SaaS industry) evolved the channel model to 0-tiers with a direct subscription and delivery model.

AWS->Consumers

As AWS grew and introduced spot pricing, an aggregation sub-market developed, looking extremely similar to movie and TV distribution models.

AWS->Aggregators->Consumers

AWS also became a distribution channel for other cloud products similar to content distribution (Think Netflix).

Third-party products->AWS->Consumers

Outstanding user-experience and aggregation are the hallmarks of companies like Airbnb, Netflix and Uber.    

The common thread is that AWS and Netflix deliver a digital product end-to-end, whereas Airbnb and Uber aggregate trusted suppliers inside the Airbnb and Uber brand environment and provide an outstanding and uniform user experience to all the consumers.  This is in contrast to the variegate user experience a customer got from the 90’s Microsoft channel. There are great resellers and terrible resellers.

We will return to user experience and aggregation later.

The medical device clinical trial value chain

The first published RCT (randomized clinical trial) in medicine appeared in the 1948 paper entitled Streptomycin treatment of pulmonary tuberculosis.

The clinical trial value chain for medical devices looks strange once after the historical perspective of how Intel, Microsoft, Amazon and Netflix evolved their value chains.

The medical device clinical trial value chain has 3 tiers with patients that are both suppliers and consumers.

Patients->Hospitals->CROS->Medical device companies->Patients

A dystopian user experience

Little has changed in the past 71 years regarding clinical trials.    Clinical trials and hospital operations now have a plethora of complex expensive, difficult-to-use IT with a value chain that provides a dystopian user experience for hospitals, patients and medical device companies.

HCOs (healthcare operators) rely on data collection technology procured by companies running clinical research (sponsors and CROs). This creates a number of inefficiencies:

1 – HCO staff are faced with a variety of systems on a study by study basis. This results in a large amount of time spent learning new systems, staff frustration and increased mistakes. This is passed on in costs and time to sponsors after CRO markup.

2 – The industry is trending towards the use of eSource and EMR to EDC data transfer. eSource/ePRO tools need to be integrated into the patient care process. Integration of EMR with EDC becomes logistically difficult due to the number of EDC vendors on the market (around 50 established companies).

3 – Siloed data collection in hospitals with subsequent manual data re-entry results in large monitoring budgets for Source Data Verification, and delays caused by data entry errors and related query resolution. Delays can be on the order of weeks and months.

4 – Use of multiple disconnected clinical systems in the hospital creates a threat surface of vendor risk, interface vulnerabilities and regulatory exposure.

Losing focus on patients

One of the consequences of the 3-tier medical device value chain is loss of focus on the patient user experience.  Upstream and to the left, patients are ‘subjects’ of the trial. The patient reported outcomes apps they use vary from study to study. Downstream and to the right (what FDA calls ‘post-marketing’), patients are consumers of the medical device and the real-world user experience is totally different than the UX in the study.   The real-world data of device efficacy and safety is disconnected from the clinical trial data of device efficacy and safety.

Clinical trial validation

Patient compliance is critical to clinical trial validation of medical device. Who owns patient compliance to the research protocol?  The medical device sponsor, the CRO, the hospital site or the subject?   The CRO may not collect a patient compliance metric since he outsources to the hospital. The hospital may not have the tools and the medical device company is outside the loop. My essay on determining when patient compliance is important in medical device trials goes into more detail on the problem of losing focus on the patient.

Vertical integration and aggregation

We previously made a qualitative claim that hospital site costs are high for medical device studies.  How high are they relative to consumer healthcare?

In a medical device trial recently done on the Flaskdata.io platform, the sponsor paid the hospital investigatory sites $700K for a 100 subject, 7 month multi-site study. (There were no medical imaging and blood test requirements).

In 2016, Medicare Advantage primary care spend was $83 PMPM (per member per month).      Let’s say that a premium service should cost $100 PMPM.    Let’s use that as a benchmark for the cost of processing a patient in a medical device trial.  Take this medical device Phase II medical device trial with 100 patients, running for 7 months:

That’s 100 x 7 x 100 = $70K for patients. Not $700K.

Perhaps the law of small numbers is killing us here.  The way to solve that is with aggregation and vertical integration. Let’s return to the medical device clinical trial value chain. As we can see, there are too many moving parts and a disconnect between patients in the clinical studies and consumers in the real world.

Patients->Hospitals->CROS->Medical device companies->Patients

One alternative is to integrate backwards and to the left.   This requires managing hospital site functions and to a certain degree is done in the SMO (site management organization model).

The other alternative is to integrate forward and to the right.   This is the path that Airbnb, Uber and Netflix took aggregating consumer demand with an outstanding user experience.  The aggregation gives Airbnb, Uber and Netflix buying power to the left, enabling them to choose the best and most cost-effective suppliers.

The value chain would then look like this:

Suppliers->Medical device companies->Patients

This is a model that we see increasingly with Israeli medical device vendors with limited budgets.   The Medical device company uses a cloud platform to collect digital feeds from investigators, patients and devices and automate monitoring for deviations. Focus on the patient user experience begins with design of the device and continues to post-marketing. Aggregation of patients enables purchasing power with suppliers – research sites, clinical consultants and study monitors.

Flaskdata - esource, ePro, patient compliance montoring,

Short-term versus long-term cost allocation

The reality is that using a technology platform for vertical integration is more expensive initially for implementation by the medical device company.   It should be.

Under-funding your infrastructure results in time delays and cost spikes to the medical device sponsor at the end of the study.

The current CRO methodology of study close-out at the end of a clinical trial lowers costs during the trial but creates an expensive catch-up process at the end of the study.

The catch-up process of identifying and closing discrepancies can take 2-6 months depending on the size and number of sites. The catch-up process is expensive, delaying submission to FDA and revenue since you have to deal with messy datasets.   The rule of thumb is that it costs 100X more to fix a defect after the product is manufactured than during the manufacturing process. This is true for clinical trials as well.   A real-time alert on treatment non-compliance during the study can be resolved in 5 minutes.   By waiting to the end of study it will take a day of work-flow, data clarifications and emails to the PI.

Summary

Vertical integration reduces costs and delay at study-end with continuous close. It is more expensive initially for the medical device company and it should be because it accelerates time to submission and reduces monitoring and close-out costs.

 

Patient compliance – the billion dollar question

The high failure rate of drug trials

The high failure rate of drugs in clinical trials, especially in the later stages of development, is a significant contributor to the costs and time associated with bringing new molecular entities to market. These costs, estimated to be in excess of $1.5 billion when capitalized over the ten to fifteen years required to develop a new chemical entity, are one of the principal drivers responsible for the ongoing retrenchment of the pharmaceutical industry. Therapeutic areas such as psychiatry, now deemed very high risk, have been widely downsized, if not abandoned entirely, by the pharmaceutical industry. The extent to which patient noncompliance has marred clinical research has in some cases been underestimated, and one step to improving the design of clinical trials may lie in better attempts to analyze patient compliance during drug testing and clinical development. Phil Skolnick, Opiant Pharmaceuticals The Secrets of a successful clinical trial, compliance, compliance, compliance.

Compliance, compliance, compliance

Compliance is considered to be key to success of a medical treatment plan. (1, 2, 3)

It is the “billion dollar question” in the pharma and medical device industry.

In home-use medical devices in particular and in chronic diseases in general – there is wide consensus that patient compliance is critical to the success of the clinical trial.   Our experience with Israeli innovative medical device vendors is that they understand the criticality of patient compliance. They “get it”.

However, as Skolnick et al note – patient compliance with the clinical protocol is often underestimated in drug trials.

There are 4 challenges for assuring patient compliance in medical device trials.

1. The first challenge is maintaining transparency.    An executive at IQVIA noted (in a personal conversation with me) that IQVIA does not calculate patient compliance metrics since they assume that patient compliance is the responsibility of the sites.    The sponsor relies on the CRO who does not collect the metrics who relies on the sites who do not share their data.

2. The second challenge is having common standard metrics of compliance. Site performance on patient compliance may vary but if sites do not share common metrics on their patients’ compliance, the CRO and the sponsor cannot measure the most critical success factor of the study.

3. The third challenge is timely data.   In the traditional clinical trial process, low-level data queries are resolved in the EDC but higher-level deviations often wait until study-closeout.  The ability of a study team to properly resolve thousands of patient compliance issues months (or even years) after the patient participated is limited to say the least

4. The final and fourth challenge is what happens after the clinical trial.  How do we take lessons learned from a controlled clinical trial and bring those lessons into evidence-based practice?

A general approach to measuring and sharing patient compliance metrics

A general approach to addressing these challenges should be based on standard metrics, fast data and active monitoring and reinforcement and reuse. 

1. Use standard metrics for treatment and patient reporting compliance. The metrics then become a transparent indicator of performance and a tool for improvement.

A simple metric of compliance might be a score based on patient reporting, treatment compliance and treatment violations. We may consider a threshold for each individual metric – for example a 3 strike rule like in baseball.

A more sophisticated measure of compliance might be similar to beta in capital market theory where you measure the ‘volatility’ of individual patient compliance compared to the study as a whole. (Beta is used in the capital asset pricing model, which calculates the expected return of an asset based on its beta and expected market returns or expected study returns in our case).

2. Fast data means automating for digital data collection from patients, connected medical devices and sites eliminating paper source and SDV for the core data related to treatment and safety endpoints.

3. Actively monitor and help patients sustain a desired state of compliance to the treatment protocol, both pharmacologic and non-pharmacologic. Not everything is about pill-counting. This can be done AI-based reminders using techniques of contextual bandits and decision trees.

4. Reuse clinical trial data and extract high quality training information that can be used for evidence-based practice.

Patient compliance teardown

Measures of patient compliance can be classified into 3 broad categories:

Patient reporting – i.e how well patient reports her own outcomes

1. Treatment compliance – how well the treatment conforms to the protocol in terms of dosing quantities and times of application. 2. Research suggests that professional patients may break the pill counting model

3. Patient violations – if the patient does something contrary to the protocol like taking a rescue medication before the migraine treatment

Confounding variables

Many heart failure patients are thought to be non-compliant with their treatment because of prior beliefs – believing that the study treatment would not help them. In the European COMET trial with over 3000 patients it was found that a Lack of belief in medication at the start of the study was a strong predictor of withdrawal from the trial (64% versus 6.8%; p < 0.0001). Those patients with very poor well-being and limited functional ability (classified as NYHA III–IV) at baseline significantly (p = 0.01) increased their belief in the regular cardiac medication but not in their study medication (4)

But numerous additional factors also contribute to patient non-compliance in clinical trials:  lack of home support, cognitive decline, adverse events, depression, poor attention span, multiple concomitant medications, difficulty swallowing large pills, difficult-to-use UI in medical devices and digital therapeutics and inconveniences of urinary frequency with diuretics for heart failure patients (for example).

It seems that we can identify 6 main confounding variables that influence compliance:

1. Patient beliefs – medication is useless, or this specific medication cannot help or this particular chronic condition is un-curable

2. Concerns about side effects – this holds for investigators and for patients and may account for levels of PI non-compliance.

3. Alert fatigue – patients can be overwhelmed by too many reminder message

4. Forgetfulness – old people or young persons. Shift workers.

5. Language –  the treatment instructions are in English but the patient only speaks Arabic.

6. Home support – patient lives alone or travels frequently or does not have strong support from a partner or parent for their chronic condition.

Summary

Flaskdata.io provides a HIPAA and GDPR-compliant cloud platform that unifies EDC, ePRO, eSource and connected medical devices with automated patient compliance monitoring. The latest version of Flaskdata.io provides standard compliance metrics of patient reporting and active messaging reminders to help keep patients on track.  Your users can subscribe to real-time alerts and you can share metrics with the entire team.

Contact Batya for a free demo and consult and learn how fast data, metrics and active reinforcement can help you save time and money on your next study.

References

1. Geriatr Nurs. 2010 Jul-Aug;31(4):290-8. Medication compliance is a partnership, medication compliance is not.
Gould E1, Mitty E. https://www.ncbi.nlm.nih.gov/pubmed/20682408

2. Depression Is a Risk Factor for Noncompliance With Medical Treatment: Meta-analysis of the Effects of Anxiety and Depression on Patient compliance. DiMatteo et al http://jamanetwork.com/journals/jamainternalmedicine/fullarticle/485411

3. Importance of medication compliance in cardiovascular disease and the value of once-daily treatment regimens. Frishman. https://www.ncbi.nlm.nih.gov/pubmed/17700384

4. Adherence and perception of medication in patients with chronic heart failure during a five-year randomised trial Ekman, Andersson et al. https://doi.org/10.1016/j.pec.2005.04.005