This week, we had a few charming examples of risk management in clinical trials with several of our customers. I started thinking about what we could do to get things to run real fast and avoid some of the inevitable potholes and black swans that crop up in clinical trials.
Engaged in basic science and stuck in data traffic
There is something very disturbing about an industry that develops products using advanced basic science.
It is disturbing because the industry uses 40-year old processes and information technology.
This industry accepts a reality of delays of a year or more due to manual data processing.
This industry is called life sciences.
That’s what disturbs on a personal and strategic level. We can and should do better. The disconnect between basic science and modern software should disturb anyone involved with clinical research because the cost to society is enormous. We are enamoured with Instagram, Uber and WeWork but we choose to pretend that life science research exists in a parallel untouchable universe protected by ICH GCP, FDA, MDR and a slew of other TLAs.
Alright. I am Israeli and trained as a physicist. Let’s look for some practical, real-world solutions. Let’s try them out and iterate.
5 ways to make your clinical research run real fast
1. Data model
Before designing your eCRF, design your data model. If you do not know what data modelling means, then 4 weeks before the study starts is a bad time to start learning. Hire a specialist in data modelling, preferably someone who does not work in life sciences. Pay them $500/hour. It’s worth every penny. The big idea is to design an abstract data model for your study for speed of access and usability by patients, site coordinators, study monitors and statisticians before designing the eCRF.
2. Discipline equals speed
Start early. Go slow and speak softly and then run fast. There is a story about the difference between a Japanese wood sculpture artist and an Israeli artist. The Japanese artist goes into his studio and looks at a big piece of wood. He walks around the wood and observes. He goes home. The next day and for the next month, he observes the wood in his studio, without touching his tools. After a month of observation, he comes in, picks up a . hammer and chisel and chop, chop chop, produces a memorable work of art. The Israeli goes into his studio and looks at a big piece of wood. He starts carving away and improvising all kinds of ideas from his head. He goes home. The next day and for the next month, he chops away at wood and replaces raw material several times. After a year, he has a work of art.
The big idea is that discipline equals speed. It prepares you for the unexpected. See point 6 below.
Side 2 of the triangle is the CRA who monitors CRC work and data quality and performs SDV.
Side 3 of this triangle is the subject who needs to come and visit the doctor on certain days that study coordinator scheduled for her when she started the trial.
Pay attention to your date and time fields. This is a much neglected part of data design in clinical trials.
The challenge is that you need to get your clinical data on different timelines. Most people ignore the fact that clinical trials have several parallel timelines.
One timeline is the study schedule. Another timeline is adverse events. Another timeline is patient compliance. You get it. If you collect high quality date times in your data model, you can facilitate generating the different time-series.
One of the most popular pieces on this blog is an essay Jenya wrote on dates and times in clinical data management. You can read it here.
4.Do not DIY your EDC
You can DIY a chair from Ikea but not your clinical trial. I know that there are a lot of low-cost eCRF packages out there like Castor EDC and Smart Clinical. The notion of a researcher or clinical manager, untrained in data modelling, data analysis and user interface design using a cheap DIY tool to develop the most important part of your study should make you stop and think. To put this in different perspective, if you are spending $5,000/month to monitor 3 sites, you should not be paying $450/month for a DIY EDC. It’s called penny-wise and pound foolish.
While it is true that protocol deviations need to be recorded, not every protocol deviation is created equal. I was stunned recently to hear from a quality manager at one of the big CROs that they do not prioritise their deviation management. Biometrics, dosing, patient compliance and clinical outcomes should be at the top of list when they relate to the primary clinical endpoint or safety endpoint. This is related to the previous points of not DIY, data modelling and observing before cutting wood.
6.Do some up-front risk assessment but don’t kid yourself.
Before you start the study, any threat analysis you do is worthless. A risk analysis without data is worthless. You may have some hypotheses based on previous work you or someone else did but do not kid yourself. First collect data, then analyse threats. I’ve written about how to do a risk assessment in clinical trials here, here, here and here. Read my essay on invisible gorillas.
The gap between the proletariat and Medidata (or should I say Dassault)
The sheer number and variety of eClinical software companies and buzzwords confuses me.
There is EDC, CTMS, IWRS, IVRS, IWRS, IRT, eSource, eCOA, ePRO and a bunch of more TLAs.
For the life of me I do not understand the difference between eCOA and ePRO and why we need 2 buzzwords for patient reporting.
Here is marketing collateral from a CRO. As you will see – they miss the boat on all the things that are important for site coordinators and study monitors.
We adapt responsively to change in your clinical trial to minimize risk and drive quality outcomes. Clinical research is complicated and it’s easy to get off track due to inexperienced project leaders, inflexible workflows, or the failure to identify risks before they become issues. We derive expert insights from evidence-based processes and strategic services to be the driving force behind quality outcomes, including optimized data, patient safety, reduced time-to-market, and operational savings.
What CRCs and CRAs have to say about the leading eClinical solutions
I recently did an informal poll on Facebook of what problems the CRA/CRC proletariat have to deal with on the job.
I want to thank Tsvetina Dencheva for helping me grok and distill people’s complaints
into 3 central themes.
Theme no. 1 – enter data once
Enable administrators to enter data once and have their authorized user lists, sites and metrics update automatically without all kinds of double and triple work and fancy import/export footwork between different systems. Failing a way of managing things in one place –
at least have better integration between the EDC and the CTMS.
The IT guys euphemistically call this problem information silos. I’ve always thought that they used the word silos (which are used to store animal food) as way of identifying with people who farm, without actually having to get their hands dirty by shovelling silage (which is really smelly btw).
I understand the rationale for having a CTMS and an EDC about as much as I understand the difference between eCOA and ePRO.
Here is some raw data from the informal Facebook survey
If I enter specific data, it would be great if there’s an integrated route to all fields connected to the said data. An easy example is – if I enter a visit, it transfers to my time sheet.
Same goes to contact reports. Apps! All sorts of apps, ctms, verified calculators, edc, ixrs, Electronic TMF. The list goes on and on. How could I forget electronic training logs? Electronic all sorts of log.
There are a lot of things we do day to day that are repetitive and can take away from actually moving studies forward. Thinking things like scanning reg docs, auto capturing of reg doc attributes (to a point), and integration to the TMF. Or better system integration, meaning where we enter a single data point (ie CTMS) and flowing to other systems (ie new site in CTMS, create new site in TMF. Enrolment metrics from EDC to CTMS) and so on.
If only the f**ing CTMS would work properly.
Theme number 2 – single sign-on.
The level of frustration with having to login to different systems is very high. The ultimate solution is to use social login – just login to the different systems with your Google Account and let Google/Firebase authenticate your identity.
Theme number 3 – data integrity
EDC edit check development eats up a lot of time and when poorly designed generates thousands of queries. Not good.
There is a vision of an EDC that understands the data semantics from context of the study protocol.
This is a very cool and advanced notion.
One of the study monitors put it like this:
The EDC should be smart enough to identify nonsense without having to develop a bunch of edit checks each time and have to deal with queries.
The EDC should be able to calculate if a visit is in a proper time window, or if imaging is in a proper time window. Also for oncology if RECIST 1.1 is used, then the EDC should be able to calculate: Body Surface Area, correct dosing based on weight and height of a patient, RECIST 1.1 tumor response and many other things that simply can be calculated.
We specialise in faster submission for connected medical devices. We can shorten your
time to market by 9-12 months with automated patient compliance detection and response.
Call us and we’ll show you how. No buzzwords required.
Good strategy bad strategy for study monitors in connected device studies
The core content of a strategy is a diagnosis of the situation at hand, creation or identification of a guiding policy for dealing with the critical difficulties and a set of coherent actions.
For starters, strategy is not pie-in-the-sky visualisation of a better world/better product/better process. Strategy is not about wanting something bad enough and boom it happens. It won’t. The essence of a good strategy is a set of coherent actions. If we perform a diagnosis of EDC operations for connected medical devices, we often see strange thingsgoing on. As a result of the diagnosis, you can formulate a set of coherent actions to fix the problem.
EDC work flow in a connected device study
1. We see sites doing SDV on device and patient-reported outcome data in the EDC. When the source data is digital, there is no point in performing source document verification.
2.Then you see valid EDC inclusion/exclusion with queries that the source was not filled out. (Right now, I see 40 queries with valid EDC data but missing questions in the paper source questionnaire). There was a site monitoring visit June 6 and the CRA raised queries related to data entered on 19-Apr-2019. These are queries 49 days after the data was captured.
I happen to know the CRA and she is great. No questions there at all.
But why fix paper source 49 days after the patient was found to be eligible for the study and is already in treatment? Something seems wrong with the process. What’s going on, is that we are fixing paper source over 6 weeks after the fact. That sucks.
You don’t need deep technology to reduce the data cycle by 98%
Let’s apply Rumelt’s strategy method to EDC for connected medical device studies. We create a guiding policy:
Reduce the data cycle to 1 day from 49 days
The set of coherent actions to execute the guiding policy is:
1.Do not SDV connected device and ePRO data. There is not point in validating electronic source against itself.
2.Improve GCP compliance and save time retroactively fixing paper documents. Just enter the Inclusion/Exclusion criteria directly into the EDC and make the EDC the electronic source according to the FDA Guidelines on electronic source
If there is a connectivity issue in the treatment room, you can use Flask Forms on your phone to enter the IE CRF directly into the EDC.
It is easy to improve the process. All you need is a good strategy.
How to improve patient compliance in your medical device study
There is considerable online discussion about real-world data in clinical trials, virtual trials, digital trials, medical IoT, wearables, AI, machine learning for finding best candidates for treatment and digital therapeutics. From the EDC vendors’ web sites – everything is perfect in a perfect world. Medidata Rave – for example:
Run Your Entire Study On A Unified, Intelligent Platform, Built On Life Science’s Largest Database.
But what happens when the EDC generates 2011 new erroneous queries?
I’m pretty sure that all the hi-fangled buzzwords don’t mean much at this point.
Why EDC queries are bad
There are 6 reasons why EDC queries are bad:
1.EDC queries push workload deeper and later into the clinical trial process. Deeper and later creates a traffic jam of work.
2.EDC queries are a vestige of paper processing and do not belong in a modern online transaction processing system. If you are not sure about this – ask yourself if AirBnB, Lufthansa and Booking require you to use queries to fix data entry mistakes.
3. EDC queries are resolved in a separate work-flow from data capture. This creates a work-flow context switch which slows down the site from fast and accurate data collection. It’s like driving and texting at the same time. But the work-flow context switch is even worse with queries – see point 4.
4.EDC queries are often resolved days or weeks after the data capture. Most people can remember what they did this morning but 3 weeks ago? How is this an effective process.
5.When EDC systems like Rave generate large numbers of erroneous automated queries – Moonshine is called for.
6.When we look at the number of automated EDC queries generated by edit checks versus discrepancy notes generated by study monitors we see that there are perhaps 500-1000X more automated queries than the high-value notes generated by a CRA. This is bad. The high-quality signals from the CRA get lost in the noise of automated EDC queries.
3 things we can do to improve study productivity
1.Turn off automated EDC queries completely
2.Resolve data issues during data entry. Don’t delay resolution more than a work-day.
3. Trace your query close rate. With a small number of high value notes from the CRA – you can do it on your fingers.
Good online systems do not use paper paradigms. In this post – I will try and entertain you with historical perspective and quantitative tools for choosing an EDC system for your medical device study.
Decades of common wisdom in clinical trials still hold to a paper-based data processing model. One of the popular EDC systems talks about the advantages of having online forms that look exactly like paper forms. True – familiarity is a good thing, but on the other hand a digital UX has far more possibilities for user engagement and ease-of-use than paper. So – it is, in a way admitting failure to provide a better UX and downgrading to paper.
We recently engaged with an Israeli medical device vendor who has an innovative device for helping solve a common medical indication for men over 50.
I won’t go into details.
If you are a guy over 50, you know what I mean.
If not, it doesn’t matter.
The client CEO was interested in an eCRF (electronic CRF – case report form) system. eCRF is better than paper, but it is, at the end of the day just a paper form in an electronic format.
I was having a lot of trouble trying to understand the CEO’s business requirements. My attempts to steer the conversation to a discussion of how to obtain fast data for his clinical trial and reduce his time to FDA submission fell on deaf ears. A follow up conversation and demo of Flaskdata with the clinical and regulatory manager focused more on reports and how to manage queries. Queries are a vestige from the paper CRF period, where a study monitor would visit the research site once/month, compare the paper source with the electronic data entry and raise queries or discrepancies.
In order to put this process into a historical context, let’s compare accounting systems from the late 70s, early 80s to an eCRF system.
Accounting versus eCRF
Accounting circa 1970
eCRF circa 2019
Paper JV – journal voucher
Data entry to a 2-sided accounting system
Data entry to an eCRF
A batch job, processes punch-card data entry and produces a data entry report and data error report
Site coordinators enter data to a Web app 1-3 days after the patient visit. Data entry errors or invalid data create data validation queries which are ignored until the study monitor visit a month later
Data error report – with non-numeric or invalid dates
Profit and loss
Bean counters of CRF/items
What is profit and loss?
What does a cash flow model have to do with clinical trials?
Cost justification and TCO for medical device EDC systems
My first recommendation would be don’t buy an EDC system just because its cheap. Charging $100-300/month for a data entry application is not be a reason to give someone money. As a client of ours once said – “I know I can use Google Forms for data entry and its free but Google Forms does not have an audit trial so Google Forms is not an option for clinical trials”.
As a rule-of-thumb, a good EDC system for medical device studies should include audit trails and a clinical cash flow report (the flow of patients in and out, the flow of data items in and out). The EDC should also be able to produce a clinical Profit and loss statement, showing you how well you are doing on your primary and secondary efficacy and safety end points. A well-designed and well-implemented EDC should include a robust data model for testing the primary endpoint and collecting safety data. At the minimum, a solid design and implementation will cost at least $10,000. Over 10 months, that’s a starting cost of $1000/month. As Robert Heinlein said – “There is no such thing as a free lunch”.
Your decision to buy EDC should be based on an economic breakeven point. One breakeven method is based on cost reduction in site monitoring. Assume the EDC system costs $4000/month (weighted cost including implementation) and assuming a monthly site visit costs $800/day, then your EDC system must be able to save 5 site visit a month and assure protocol compliance. This places an upper bound on the price you can pay.
This is albeit problematic for small, 1 site studies which often use DIY implementations. Just remember, that ignoring the implementation cost, does not make the product cheaper. In other words calculate your TCO (Total cost of ownership).
Or as one wise man one said – I’m too poor to buy a cheap car.
Israel Biomed 2019-the high-social, low stress STEM conference
This week was the annual 3 day Biomed/MIXiii (I have no idea what MIXiii means btw) conference in Tel Aviv. The organizers also billed it as the “18th National Life Science and Technology Week” (which I also do not know what that means). This was a particular difficult time for a conference of medical device and pharma in Tel Aviv since it coincided with the Eurovision 2019 activities – and the traffic was tough.
There were a huge number of lectures and participants from all over the world and I suppose from that perspective, the conference is a success and tribute to the burgeoning Israeli biomed industry. Forbes calls Biomed “The High-Paying, Low-Stress STEM Job You Probably Haven’t Considered”. I think that this is probably a good description for the conference – high participation but low stress.
My colleagues and I come to the conference to network, schmooze, meet customers and suppliers. It’s a good opportunity to take a few meetings, say hi to friends and hustle for new business. Having said that, I did meet a few really interesting companies:
RCRI – is a Minneapolis MN based medical device CRO. I met Todd Anderson and his boss Lisa Olson and pitched our approach for fast data in clinical trials to assure high levels of patient compliance to the protocol and submit faster to FDA. Todd and Lisa get it and they were open about the CRO business model being more people-hours not speed. They seemed genuinely interested in what we are working on but its hard to tell with Americans.
Docdok Health – is a startup founded by Yves Nordman, who is a Swiss MD living in Carmiel. It’s a doctor-patient communications platform beginning to branch out into Post-marketing studies with RWD. We shared demos and it seems that there is synergy between our regulatory platform and their post-marketing work.
Resbiomed – met Alex Angelov, the CEO. Alex is leading a consortium including Flaskdata, Carl Zeiss, Collplant, PreciseBio and Pluristem for a Horizon2020 submission for an amazing project for an implant to the cornea. Dan Peres from Pluristem got us together. Cheer for us!
BSP Medical and ICB (Israel China Biotech investment) – my buddy Hadas Kligman literally took me by hand to visit to Yehuda Bruner and Andrew Zhang and I did my 60s elevator pitch on getting medical device companies to FDA/CFDA 6-12 months faster. We agree to talk after the conference.
Butterfly Medical – I met Idan Geva, the CEO last year at Biomed – we ate lunch at the same table. I pitched him but he was uninterested – they were using EDC2Go – and he didn’t want to hear other options. At the Minnesota pavilion talking to Todd Anderson from RCRI, Idan shows up and looks at me and says “Heah – Hi Danny – I left a contact me request on your web site yesterday and no one got back to me. I said shame on us. He says – he was referred to us by someone from Florida who used to use Medidata. I asked where/who? was it Miami? He says yeah it was Miami and checks his phone – says its someone from Precision Clinical Research that are using Flaskdata and recommended. (Precision is one of our customer’s Miami sites). I asked what happened to EDC2Go – he said well you know – they are end of life (I think this means the end of low-cost EDC) and we are now entering questionnaires manually on paper and it is driving us crazy. He said – can you stick around and give us a demo at 15:00? I said sure. We met at 15:00 by the bar upstairs in the David Intercontinental and I demoed the system – he said “Show me the Forms designer”. I showed him. He says “show me how CRC enters data” – I showed him. He says “Show me how to extract data” – I showed him. I think he actually did not believe how fast the Extract to CSV process was and asked me twice if that was the data. In the end – the format of Mac Numbers was a bit strange for him. I showed him a quick presentation – and he saw that Serenno is a customer – and says – “Heah Tomer is a neighbor of ours in the incubator in Yokneam”. He asked how much and I said $2K for a basic onboarding package and $1500 / month. Or $10K and we will build the CRF (their CRF is super simple btw). He wanted a discount, being Israeli. I said – “lets meet with your clinical person and get her to buy-in to the solution. If she buys in – you and I can talk business but before that, there is no point horse-trading.
Count the probabilities of this happening and you will see that it is an impossible event.
Thursday I went back to demo Todd and meet Dr Yael Hayun from Syqe Medical. Yael is one of the most impressive people I’ve met in a long time. She is an MD from Hadassah and one of the movers and shakers in LogicBio Therapeutics. After we chatted – I told her that Syqe is lucky to have her onboard. I did our Today is about Speed presentation and a short demo. She was suitably impressed and then mentioned they had met with a Danish EDC company called Smart Trial – which turns out is yet another low-cost eCRF provider. I said look – eCRF is like 10% of the solution you need – in the case of Syqe, you have a digital inhaler and with cannabis, you are going to have a lot of concerns about patient compliance.
This is what we do – fast data collection from patients, investigators and digital inhalers and automated deviation detection and response.
On the way back – huge traffic from Eurovision. Didn’t hear a single lecture but the meetings and people were outstanding.
The high failure rate of drugs in clinical trials, especially in the later stages of development, is a significant contributor to the costs and time associated with bringing new molecular entities to market. These costs, estimated to be in excess of $1.5 billion when capitalized over the ten to fifteen years required to develop a new chemical entity, are one of the principal drivers responsible for the ongoing retrenchment of the pharmaceutical industry. Therapeutic areas such as psychiatry, now deemed very high risk, have been widely downsized, if not abandoned entirely, by the pharmaceutical industry. The extent to which patient noncompliance has marred clinical research has in some cases been underestimated, and one step to improving the design of clinical trials may lie in better attempts to analyze patient compliance during drug testing and clinical development. Phil Skolnick, Opiant Pharmaceuticals The Secrets of a successful clinical trial, compliance, compliance, compliance.
Compliance, compliance, compliance
Compliance is considered to be key to success of a medical treatment plan. (1, 2, 3)
It is the “billion dollar question” in the pharma and medical device industry.
In home-use medical devices in particular and in chronic diseases in general – there is wide consensus that patient compliance is critical to the success of the clinical trial. Our experience with Israeli innovative medical device vendors is that they understand the criticality of patient compliance. They “get it”.
However, as Skolnick et al note – patient compliance with the clinical protocol is often underestimated in drug trials.
There are 4 challenges for assuring patient compliance in medical device trials.
1. The first challenge is maintaining transparency. An executive at IQVIA noted (in a personal conversation with me) that IQVIA does not calculate patient compliance metrics since they assume that patient compliance is the responsibility of the sites. The sponsor relies on the CRO who does not collect the metrics who relies on the sites who do not share their data.
2. The second challenge is having common standard metrics of compliance. Site performance on patient compliance may vary but if sites do not share common metrics on their patients’ compliance, the CRO and the sponsor cannot measure the most critical success factor of the study.
3. The third challenge is timely data. In the traditional clinical trial process, low-level data queries are resolved in the EDC but higher-level deviations often wait until study-closeout. The ability of a study team to properly resolve thousands of patient compliance issues months (or even years) after the patient participated is limited to say the least
4. The final and fourth challenge is what happens after the clinical trial. How do we take lessons learned from a controlled clinical trial and bring those lessons into evidence-based practice?
A general approach to measuring and sharing patient compliance metrics
A general approach to addressing these challenges should be based on standard metrics, fast data and active monitoring and reinforcement and reuse.
1. Use standard metrics for treatment and patient reporting compliance. The metrics then become a transparent indicator of performance and a tool for improvement. A simple metric of compliance might be a score based on patient reporting, treatment compliance and treatment violations. We may consider a threshold for each individual metric – for example a 3 strike rule like in baseball. A more sophisticated measure of compliance might be similar to beta in capital market theory where you measure the ‘volatility’ of individual patient compliance compared to the study as a whole. (Beta is used in the capital asset pricing model, which calculates the expected return of an asset based on its beta and expected market returns or expected study returns in our case).
2. Fast data means automating for digital data collection from patients, connected medical devices and sites eliminating paper source and SDV for the core data related to treatment and safety endpoints.
3. Actively monitor and help patients sustain a desired state of compliance to the treatment protocol, both pharmacologic and non-pharmacologic. Not everything is about pill-counting. This can be done AI-based reminders using techniques of contextual bandits and decision trees.
4. Reuse clinical trial data and extract high quality training information that can be used for evidence-based practice.
Patient compliance teardown
Measures of patient compliance can be classified into 3 broad categories:
Patient reporting – i.e how well patient reports her own outcomes
1. Treatment compliance – how well the treatment conforms to the protocol in terms of dosing quantities and times of application. 2. Research suggests that professional patients may break the pill counting model
3. Patient violations – if the patient does something contrary to the protocol like taking a rescue medication before the migraine treatment
Many heart failure patients are thought to be non-compliant with their treatment because of prior beliefs – believing that the study treatment would not help them. In the European COMET trial with over 3000 patients it was found that a Lack of belief in medication at the start of the study was a strong predictor of withdrawal from the trial (64% versus 6.8%; p < 0.0001). Those patients with very poor well-being and limited functional ability (classified as NYHA III–IV) at baseline significantly (p = 0.01) increased their belief in the regular cardiac medication but not in their study medication (4)
But numerous additional factors also contribute to patient non-compliance in clinical trials: lack of home support, cognitive decline, adverse events, depression, poor attention span, multiple concomitant medications, difficulty swallowing large pills, difficult-to-use UI in medical devices and digital therapeutics and inconveniences of urinary frequency with diuretics for heart failure patients (for example).
It seems that we can identify 6 main confounding variables that influence compliance:
1. Patient beliefs – medication is useless, or this specific medication cannot help or this particular chronic condition is un-curable
2. Concerns about side effects – this holds for investigators and for patients and may account for levels of PI non-compliance.
3. Alert fatigue – patients can be overwhelmed by too many reminder message
4. Forgetfulness – old people or young persons. Shift workers.
5. Language – the treatment instructions are in English but the patient only speaks Arabic.
6. Home support – patient lives alone or travels frequently or does not have strong support from a partner or parent for their chronic condition.
Flaskdata.io provides a HIPAA and GDPR-compliant cloud platform that unifies EDC, ePRO, eSource and connected medical devices with automated patient compliance monitoring. The latest version of Flaskdata.io provides standard compliance metrics of patient reporting and active messaging reminders to help keep patients on track. Your users can subscribe to real-time alerts and you can share metrics with the entire team.
Contact Batya for a free demo and consult and learn how fast data, metrics and active reinforcement can help you save time and money on your next study.
Danny teeters on the edge of the precipice of privacy and security. Step on the brakes not on the gas and don’t look down. Take a 500m leap of faith into the chasm of medical device clinical trials. Validate digital therapeutics. Venture into uncharted territory of medical cannabis trials.
At some stage in my “let’s do something different and risky” life after leaving the safety of Intel culture, I stumbled into cybersecurity.
Cybersecurity and privacy for medical devices
I started helping Israeli medical device and digital Health startups with privacy and security consulting. We built and analysed medical device threat models. The threat analysis approach succeeded in helping people improve their systems and privacy compliance.
Over time, the threat analysis methodology that was developed was adopted by thousands of security analysts globally – PTA Technologies.
I did not do this on my own. I owe these opportunities to my friend and colleague Mike Zeevi from Softquest Systems.
Over time, I figured out what works and how to comply with standards – HIPAA, FDA and GDPR. This came from real-life implementations and FDA submissions. I got hands-on in compliance audits with large US healthcare organisations like BC/BS Dignity Health.
Development practices for connected medical device and digital health apps
Many startups in the digital health and medical IoT space make 3 mistakes when engineering their systems.
1) First they Google. 2) Then they Guess. 3) Then they DIY when the Guesses Fail.
Some companies add an additional step: “Contract to a Software House that Talks Big” and then DIY or switch contractors.
This is a costly and risky pattern. As Jim McCarthy says –
More people have ascended bodily into heaven than have shipped great software on time.
– Jim McCarthy, Dynamics of Software Development by Jim McCarthy, Denis Gilbert
For Israeli digital health startups, there is an additional risk. This is the risk of not having an organisational memory. Youth has energy, hip viewpoints and updated expertise on latest technology. Who knew that a similar technology failed 30 years ago before you were born?
Build versus buy for digital health platforms
Digital health startups face 2 challenges. The first is an engineering challenge. The second is a validation challenge.
AWS cloud services have changed the way we engineer connected medical devices and digital health apps.
However, you need to factor in the cost and time requirements for a slew of additional activities. You need reliable DevOps, application integration, data integration, performance, configuration management, security, privacy, compliance and risk management.
The validation challenge is about clinical trials. About 4 years ago, we saw that our medical device customers wanted cheaper and faster ways to collect, monitor and analyse clinical trial data.
Building the product yourself and building a digital clinical trial systems is neither simple nor cheap. Resorting to paper studies to save money, turns short-term savings into long-term losses in time and data quality.
The solution – full-stack digital clinical trial platform
I joined forces with Jenya and we took a strange and wonderful decision to help Israeli medical device companies run clinical trials in the cloud.
This is what Flaskdata.io – patient compliance automation for medical device studies does. We provide a full-stack 21 CFR, HIPAA, GDPR compliant platform for collecting and monitoring data from investigators, patients and devices. Organisations like Theranica Therapeutics and Weizmann Institute all trust our platform for their human research. Today, Flaskdata.io helps site coordinators and clinical trial manager assure patient compliance using real-time alerts and trends at over 300 sites globally.
We work hard to bring modern technology to our customers instead of paper and save time and money.
Platform as a Service offerings like IBM Watson digital health has an amazing set of tools. You have to build your own product, integrate, test, secure, verify and validate.
By comparison, validated Software as a Service platform like Flaskdata.io enables you to get started immediately. You can design data collection using visual UI and integrate the open Flask API for medical devices. Check out our Swagger here.
There is a free tier that enables very early stage startups to start running pilots for free. And yes, we support, English, Hebrew and Chinese.
Give us a shot – you will not be sorry.
100X faster to deviation detection in medical device studies.
Automated Patient compliance deviation detection and response on the flaskdata.io platform for a connected medical device clinical trial is 100X faster than manual monitoring.
Automated compliance monitoring analytics and real-time alerts let you focus your site monitoring visits on work with the PI and site coordinators to take total ownership and have the right training and tools to meet their patient recruitment and patient compliance goals.
This is a post David wrote a while back and it still seems relevant. If you would have asked me 2 years ago – I would have told you that in 2018, no one would be doing paper medical device clinical trials the same way that no one does paper accounting. I would have thought that logic would prevail considering the advantage of using automation and technology instead of using your Chief science officer to manually enter data into Excel.
Medical science is the foundation for innovative medical devices. Taking medical science and developing a medical device product requires translating basic science into technology. This is self-evident.
So why do so many innovative medical device vendors conduct their clinical trials using paper? Damn if I know. Using paper for medical device clinical trials is somewhere between penny-wise and pound foolish and plain dumb.
Every year, 20,000 clinical trials are performed. An electronic data capture (EDC) system is quickly becoming adopted as the modern standard for monitoring in clinical trials. EDC solves the problems that are inherent to traditional, paper-based methods of data capture. During medical device clinical studies, the accessibility to real-time data capture and storage during conduction is key to performing a study that is cost efficient, and effective in generating results.
Paper-based = slow and costly
EDC = quick and efficient
Do not forget these simple equations, as they should become your mantra.
As seen below, the number of medical device clinical trials conducted is like the global population; it only keeps increasing. The pressure is on for product developers to conduct studies in the most expedient fashion possible, and collecting data that is not only pertinent and useful, but is clean and devoid of doubt concerning its accuracy.
Thanks to technological advances (read: EDC), on-site monitoring and clumsy, paper-based data storage are going the way of the dodo bird. The use of EDC as a basis for automating patient compliance during medical device clinical trials is quickly developing as more an more medical devices become connected via mobile and home wireless networks.
Paper-based data capture systems are irrelevant for connected medical clinical trials.
90% of drug development costs are invested in clinical trial conduction. EDC systems facilitate automation of patient compliance during the duration of the medical device trial. And while not every medical device trial uses connectivity and automated patient compliance monitoring, there is an increasing understanding that the direction is digital and not paper.
The majority of the public values clinical trials for the healthcare industry, as seen below. Implementing an EDC system for medical device clinical trial monitoring has proven to reduce study costs by 59%. So, ask yourself, what are you waiting for?
As seen above, the value of clinical trials is understood by the public, and as clinical trials continue to grow in scope of variables and number of participants, they require a more efficient means of data capture in order to cut the costs involved in monitoring. EDC systems provide exactly that. Here we will touch upon why an EDC system is becoming an essential for clean and efficient risk-based monitoring in clinical trials.
Medical device monitoring data is available in real-time
Using an EDC system affords the opportunity for study monitors to receive data entered by clinicians as soon as it is collected. By using hand held devices, such as a tablet, that are logged into an EDC system, makes risk-based monitoring a breeze. No longer does one need to record data on a clipboard, and then duplicate the same data into an on-site hard drive. This means that monitors are getting their hands on information the second it is captured.
Simply put, the faster that you get data into the hands of your monitors, the greater the efficiency of the study.
Increased study efficiency through cloud notifications
Recently, for the past 20 years or so, medical device clinical trials have been substantially increasing in scale and complexity as they continue to become more valued and salient as a means of biomedical development. Often, they involve a sizable number of people responsible for entering data, and study monitors assigned the task of monitoring specific variables and patient compliance to the protocol.
An EDC system automates the appropriate delivery of fresh and high-quality data from investigational sites, patients and connected medical devices. Whoever needs to receive whatever data variables from a clinician are notified in their personal account via the cloud. Not only does cloud-based EDC keep monitors informed in real time, but the organization and delivery properties ensure that the right monitors are receiving the right data, increasing efficiency in increasingly complex studies.
When data is entered after capture, an EDC system can automate from the entry user the delivery of data to the assigned monitor. Email alerts can also be integrated into the EDC system, so that whenever data is entered for review by a monitor, they are informed even if they are not logged into the EDC system.
Reduced monitor travel costs with remote compliance monitoring
Not only does using EDC keep monitors informed of new captured data as soon as it happens, reducing subject risks, but monitors can perform their tasks from abroad, saving travel time and expenses. The features of using a cloud-based EDC system are nearly endless, but the decentralizing of on-site data monitoring is one of its greatest boons.
Monitors that work from home going to be willing to receive lower salaries, and people are generally happier when they can work from home. Your study will save time and money by an increased retention rate in monitor personnel, that are willing to work with a clinical trial sponsor, study after study.
Further, and this is a benefit from remote monitoring of your medical device clinical trial that most would not think of, consider reduced human traffic at your study site. The less people you have at your study site the better, as there is simply less for on-site study managers to focus on. This is a minor benefit of an EDC system, compared to the speed of data delivery with EDC, but a benefit nonetheless.
Also consider that remote monitoring can allow the outsourcing of monitors. If your study site is located in California, but there’s a team of specialists in India, willing to perform exceptional quality of monitoring for lower salaries, of course you are not going to fly them over to work for you; cost prohibitive. If you are using the standard on-site monitoring method that comes with paper-based systems, your resources are limited to only those that can geographically travel to your study site.
Cleaner, consistent data submission to monitors
EDC systems can use a study-specific standardized data collection form, reducing errors in collection and delivery to monitors. Consistency is key to running a smooth, hassle-free medical device clinical trial. By using standardized electronic data collection forms, your study will erase the possibility for inconsistent data submission from data managers to monitors.
Paper-based data capture systems may seem familiar and comfortable to clinicians, and making the transition to an EDC system may seem like a plunge into unknown territory, but the data is plainly cleaner when conducting a study with EDC. The deficit of errors and omissions that are caused by implementing EDC are a tremendous ROI for your study. Consider the following:
For example, in a paper-based system, data is recorded by hand, and even something as seemingly trivial as handwriting comes into play and can muddle data. Not every clinician will have the best penmanship, so this opportunity to corrupt data is entirely circumvented by using an EDC system.
A more frequent, and damaging, corruption of data that occurs when using the standard paper-based system are data errors and omissions when recording data. People make mistakes, for whatever reason. It is natural, and bound to happen. Say, for instance, that you have a subject XY-1001-9, for which the clinician is collecting data; it is very easy to write YX-1001-9, XY-101-9 or XY-1010-9 if a clinician is distracted, or maybe just operating on little sleep from the previous night.
By working with an EDC system with standardized data collection forms, the above scenarios are entirely avoided. That being said, standardized forms are not going to write themselves. During the planning stage of your study, devote time to organizing and developing the standardized form model you are going to use for each subject in your study to reduce errors and omissions. In the long run, your ROI will go through the roof.
However, even in an EDC system, mistakes can be made. No system is entirely error-proof, especially when being implemented for the very first time. When a mistake inevitably does happen, it is far less of a headache to solve and prevent from recurring using an EDC system. For starters, FDA compliance adherence measures should already be in place at the hands of the EDC software vendor. As compliance standards are modified by the FDA, they can be updated in the EDC system without a hitch. When data entry errors occur, they can be addressed by programming the software to recognize proper form entries.
Another feature of EDC systems for reducing errors and omissions is data entry recognition standards. Remember the subject number examples? If you write something down on paper, there is no real way to tell if you got it right the first time, than somebody else telling you who has noticed that you have made a mistake, and then correcting it themselves. Every field of the EDC user interface can be programmed to recognize whether the data entered was in the proper format, and whether any fields were skipped or not submitted.
An EDC system also reaps tangible data capture benefits for studies using subject-submitted data. Many subjects are not experts in clinical trial data management and entry, and unless you are conducting a study into only a single variable, patient submitted data, which saves time and costs, is an impractical approach to collecting study data. However, whilst being cloud-based, EDC allows any subject with a smartphone, tablet or computer the ability to submit data, at the very moment it is noticed and measured, mitigating subject risks and saving on study personnel expenses.
For user submitted data, the standardized data collection form with checks in place for data submission ensures that the subject will not make a mistake when submitting data. You will be able to get by and hire less clinicians for future studies, a further cost saver of EDC.
Facilitating future medical device studies
After you take the plunge (and please do, ASAP) into EDC and forego paper-based data capture, the benefits will be noticed immediately for your next clinical trial. Not only will every facet of your data capture and monitoring be smooth sailing, but think of the future studies you will be sponsoring, and how they will benefit.
Not only does EDC facilitate the aforementioned features, but after you and your study personnel (and subjects if applicable) are trained and familiarized with the use of the EDC system you have chosen, future studies will be up and running faster than you can say “outdated, paper-based data capture.”
EDC systems significantly cut the time spent during the planning and preparation phases of a medical device clinical trial. Consider how while you are planning the variables and factors to be measured, you can instantly enter them into the software, saving time and money that would otherwise be spent on designing paper forms and making copies. EDC systems are flexible, and if study personnel is trained properly by the software vendor from the get go, require little maintenance for their design.
What are you waiting for?
Hopefully you now have a better understanding of how vital an EDC system is for an efficient medical device clinical trial, and how many headaches it alleviates for monitoring clinical trials. When you are looking for a vendor, ask how they can eliminate rework and detect problematic trends in real-time. Ask them if they require expensive third-party analytics and if they limit the number of users that can use risk-based monitoring tools and make sure they have a great training program. Enjoy your streamlined future studies.