10 ways to detect people who are a threat to your clinical trial

Flaskdata.io helps Life Science CxO teams outcompete using continuous data feeds from patients, devices and investigators mixed with a slice of patient compliance automation.

One of the great things about working with Israeli medical device vendors is the level of innovation, drive and abundance of smart people.

It’s why we get up in the morning.

There are hundreds of connected medical devices and digital therapeutics (last time I checked over 300 digital therapeutics alone).

When you have an innovative device with network connectivity, security and patient privacy, availability of your product and integrity of the data you collect has got to be a priority.

Surprisingly, we get a  range of responses from people when we talk about the importance of cyber security and privacy for clinical research,

Most get it but some don’t.   The people that don’t get it, seem to assume that security and privacy of patient data is someone else’s problem in clinical trials.

The people who don’t work in security, assume that the field is very technical, yet really – it’s all about people.   Data security breaches happen because people or greedy or careless.    100% of all software vulnerabilities are bugs, and most of those are design bugs which could have been avoided or mitigated by 2 or 3 people talking about the issues during the development process.

I’ve been talking to several of my colleagues for years about writing a book on “Security anti-design patterns” – and the time has come to start. So here we go:

Security anti-design pattern #1 – The lazy employee

Lazy employees are often misdiagnosed by security and compliance consultants as being stupid.

Before you flip the bozo bit on a site coordinator as being non-technical, consider that education and technical aptitude are not reliable indicators of dangerous employees who are a threat to the clinical trial assets.

Lazy employees may be quite smart but they’d rather rely on organizational constructs instead of actually thinking and executing and occasionally getting caught making a mistake.

I realized this while engaging with a client who has a very smart VP – he’s so smart he has succeeded in maintaining a perfect record of never actually executing anything of significant worth at his company.

As a matter of fact – the issue is not smarts but believing that organizational constructs are security countermeasures in disguise.

So – how do you detect the people (even the smart ones) who are threats to PHI, intellectual property and system availability of your EDC?

1 – Their hair is better organized then their thinking

2 – They walk around the office with a coffee cup in their hand and when they don’t, their office door is closed.

3 – They never talk to peers who challenge their thinking.   Instead they send emails with a NATO distribution list to everyone on the clinical trial operations team.

4 – They are strong on turf ownership.  A good sign of turf ownership issues is when subordinates in the company have gotten into the habit of not challenging the VP coffee-cup holding persons thinking.

5 – They are big thinkers.    They use a lot of buzz words.

6 – When an engineer challenges their GCP/regulatory/procedural/organizational constructs – the automatic answer is an angry retort “That’s not your problem”.

7 – They use a lot of buzz-words like “I need a generic data structure for my device log”.

8 – When you remind them that they already have a generic data structure for their device log and they have a wealth of tools for data mining their logs – amazing free tools like Elasticsearch and R….they go back and whine a bit more about generic data structures for device logs.

9 – They seriously think that ISO 13485 is a security countermeasure.

10 – They’d rather schedule a corrective action session 3 weeks after the serious security event instead of fixing it the issue the next day and documenting the root causes and changes.

If this post pisses you off (or if you like it),  contact  me –  always interested in challenging projects with challenged people who challenge my thinking.

Competitive buzzwords in EDC companies

We recently did a presentation to a person at one of the big 4 pharma.  His job title was

Senior IT Project Manager Specialized in Health IT.

I looked at the persons LinkedIn profile before the call and I noticed that the sentence is in past tense. Specialized in Health IT implying that he was now a Senior IT manager who no longer specialized in anything.

I have a friend who worked at Pfizer in IT. He was discouraged by pharma IT mediocrity especially  when he compared it to the stellar talents in the R&D departments.

So it stands to reason that the EDC vendors are just a notch up the technology ladder from the pharma IT guys. If you do not have a unique technology value proposition, you have to resort to marketing collateral gymnastics.

To test this hypothesis – I took a look at the web sites of 4 EDC vendors:  Medidata, Medrio, Omnicomm and Oracle Life Sciences.

Medidata

Run Your Entire Study On A Unified, Intelligent Platform Built On Life Science’s Largest Database.

At Medidata, we’re leading the digital transformation of clinical science, so you can lead therapies to market faster, and smarter. Using AI and advanced analytics, our platform brings data managers, clinical operations, investigators, and patients together to accelerate the science and business of research.

MediData is making a disturbing suggestion in their marketing collateral that they leverage other companies trial data in their Life Science Database to help you lead therapies to market faster.

Medrio

Clinical trial data collection made easy. The industry’s leading early-phase EDC and eSource platform.

The only EDC vendor that actually admitted to being an EDC vendor was Medrio. You have to give them a lot of credit for honesty.

Omnicom

eClinical Solutions for Patient-Centric Clinical Trials
Effective Clinical Tools Driving Excellence in Life Science Research

Software has the power to save lives. OmniComm Systems understands that power and delivers eClinical solutions designed to help life science companies provide crucial medical treatments and therapies to patients around the globe.

OmniComm Systems fills a role in enhancing patient lives by shortening the time-to-market of essential life-saving treatments. Our eClinical suite of products includes electronic data capture (EDC) solutions, automated coding and randomization systems, risk-based monitoring (RBM) and analytics.

This is nice positioning, but it makes you wonder when OmniComm turned into a healthcare provider of crucial medical treatments and therapies to patients around the globe.

Oracle Life Science

Oracle Life Sciences—Reimagining What’s Possible

Innovation in science and medicine demands new technology, and innovation in
technology makes new things possible in science and medicine. Oracle is equipping the life sciences industry today, for the clinical trials of tomorrow.

Solutions Supporting the Entire Clinical Development Lifecycle

Oracle Health Sciences helps you get therapies to market faster and detect risks earlier. Oracle offers a complete set of clinical and safety solutions that support critical processes throughout the clinical development lifecycle—from study design and startup to conduct, close-out, and post-marketing.

SOLUTIONS
Oracle Health Sciences Clinical One cloud environment changes the way clinical research is done—accelerating all stages of the drug development lifecycle by eliminating redundancies, creating process efficiencies, and allowing the sharing of information across functions.

Unlike OmniComm and Medidata,   Oracle is firmly focused on the clinical development lifecycle; not pretending that they are a healthcare provider or leverage the patient data in their EDC databases.

Flaskdata.io

Helping life-science C-suite teams outperform their competitors.

Patient compliance is critical to the statistical power and patient retention of a study.

We help senior management teams complete studies and submission milestones faster and under budget. We do this by providing EDC, ePRO and integration of connected medical devices into a single data flow. We then automate detection and response of patient compliance deviations in clinical trials 100x faster than current manual monitoring practices.

 

 

Develop project management competencies to speed up your clinical trials

The biggest barrier to shortening clinical trial data cycle times is not recruitment.   It is not having a fancy UI for self-service eCRF forms design.   It is not software.

It is not, to paraphrase Medidata, having the ability to Run Your Entire Study On A Unified, Intelligent Platform Built On Life Science’s Largest Database.

It is incompetence in managing a construction project.

That construction project is called designing a clinical trial and the information system for collecting and monitoring data.

For a long time, I thought that this was peculiarly an Israeli problem.

However, conversations with colleagues in the US and Europe suggest that late starts, feet-dragging and time-consuming  change requests may be the norm. Collecting too many variables in the data model is the norm. Complex, long forms that make life hard for the site coordinators is the norm,  Surfeits of edit checks and thousands of queries are the norm.

Most companies spend little  money on project management training and even less money on clinical project strategy development.  Most training is on process, regulatory compliance and standard operating procedures.

Rarely, do we see medical device companies spend money on competencies that will help employees construct clinical trial projects more effectively.

There are verbal commitments that are rarely action commitments.

Yet there is a direct linkage between clinical operations team knowledge and corporate revenue growth which is dependent upon delivering innovative drugs and devices to market.

One way management teams can maximise their investments in project training and clinical project strategy development (outsourced or in-sourced) is to link clinical operations team training to study management competency models that management can qualify and measure.

But the development of a clinical team competency model has strategic and operational barriers that must be managed to make it successful.

Clinical trial project management competency model example

Clinical team Competency Setup Considerations

1. Clinical people often think that building the ‘database’ is an art, not a science, and don’t like to be measured in what they perceive is a non-core skill.

2.  Your project  competency model must include both soft and hard skills training to make it effective.

3. Clinical trial management teams must focus on the competency requirements to make it work and it must be a hands-on approach.

4. You must be able to quantitatively measure the competencies (time to design forms, edit check design, monitoring signals, data cycle time, time spent in meetings, change requests).

5. Competency clinical trial management training programs must be continuous training and educational events, not a one-time event or else the program will fail.

6. The steps of your competency program must be very specific and delineated to make sure it can be delivered and measured.

7. Your clinical operations team must agree that the competencies you are measuring truly help them deliver the study faster (They don’t have to like doing it, just agree that there are required action steps to reduce data cycle times)

8. When implementing your project competencies audits, the certification should be both written and experientially measured to get an accurate reading of the clinical operations team member capabilities.

9. All project  competency certification candidates should have the ability to retest to confirm skills growth.

10. Project competency assessments should never be used solely as a management scorecard tool to make employment decisions about clinical operations team members.

To increase your company revenues and clinical project training success, build and deliver project competency models.

5 ways to make your clinical trials run real fast

medical device clinical trials

This week, we had a few charming examples of risk management in clinical trials with several of our customers.   I started thinking about what we could do to get things to run real fast and avoid some of the inevitable potholes and black swans that crop up in clinical trials.

Engaged in basic science and stuck in data traffic

There is something very disturbing  about an industry that develops products using advanced basic science.

It is disturbing because the industry uses 40-year old processes and information technology.

This industry accepts a reality of delays of a year or more due to manual data processing.

This industry is called life sciences.

That’s what disturbs on a personal and strategic level.   We can and should do better.  The disconnect between basic science and modern software should disturb anyone involved with clinical research because the cost to society is enormous.      We are enamoured with Instagram, Uber and WeWork but we choose to pretend that life science research exists in a parallel untouchable universe protected by ICH GCP, FDA, MDR and a slew of other TLAs.

Alright.  I am Israeli and trained as a physicist.   Let’s look for some practical, real-world solutions. Let’s try them out and iterate.

5 ways to make your clinical research run real fast

1. Data model

Before designing your eCRF, design your data model.  If you do not know what data modelling means, then 4 weeks before the study starts is a bad time to start learning.   Hire a specialist in data modelling, preferably someone who does not work in life sciences.   Pay them $500/hour.  It’s worth every penny. The big idea is to design an abstract data model for your study for speed of access and usability by patients, site coordinators, study monitors and statisticians before designing the eCRF.

2. Discipline equals speed

Start early. Go slow and speak softly and then run fast.  There is a story about the difference between a Japanese wood sculpture artist and an Israeli artist. The  Japanese artist goes into his studio and looks at a big piece of wood. He walks around the wood and observes.   He goes home.  The next day and for the next month, he observes the wood in his studio, without touching his tools.    After a month of observation, he comes in, picks up a . hammer and chisel and chop, chop chop, produces a memorable work of art.      The Israeli goes into his studio and looks at a big piece of wood. He starts carving away and improvising all kinds of ideas from his head. He goes home.  The next day and for the next month, he chops away at wood and replaces raw material several times.   After a year, he has a work of art.

The big idea is that discipline equals speed.  It prepares you for the unexpected. See point 6 below.

A good book that presents this approach in a very practical way is Discipline equals Freedom by Jocko Willink.

3.Date and time

Date/time issues can be visualised as a triangle.

Side 1 of the triangle is the site coordinator who collects data into the EDC.

Side 2 of the triangle is the CRA who monitors CRC work and data quality and performs SDV.

Side 3 of this triangle is the subject who needs to come and visit the doctor on certain days that study coordinator scheduled for her when she started the trial.

Pay attention to your date and time fields.    This is a much neglected part of data design in clinical trials.

The challenge is that you need to get your clinical data on different timelines.     Most people ignore the fact that clinical trials have several parallel timelines.

One timeline is the study schedule.  Another timeline is adverse events.  Another timeline is patient compliance.    You get it.   If you collect high quality date times in your data model, you can facilitate generating  the different time-series.

One of the most popular pieces on this blog is an essay Jenya wrote on dates and times in clinical data management.  You can read it here.

4.Do not DIY your EDC

You can DIY a chair from Ikea but not your clinical trial.   I know that there are a lot of low-cost eCRF packages out there like Castor EDC and Smart Clinical. The notion of a researcher or clinical manager, untrained in data modelling, data analysis and user interface design using a cheap DIY tool to develop the most important part of your study should make you stop and think.  To put this in different perspective, if you are spending $5,000/month to monitor 3 sites, you should not be paying $450/month for a DIY EDC.    It’s called penny-wise and pound foolish.

5.Prioritise deviations.

While it is true that protocol deviations need to be recorded, not every protocol deviation is created equal.      I was stunned recently to hear from a quality manager at one of the big CROs that they do not prioritise their deviation management.     Biometrics, dosing, patient compliance and clinical outcomes should be at the top of list when they relate to the primary clinical endpoint or safety endpoint.    This is related to the previous points of not DIY, data modelling and observing before cutting wood.

6.Do some up-front risk assessment but don’t kid yourself.

Before you start the study, any threat analysis you do is worthless.   A risk analysis without data is worthless.  You may have some hypotheses based on previous work you or someone else did but do not kid yourself.   First collect data, then analyse threats.   I’ve written about how to do a risk assessment in clinical trials here, here, here and here.  Read my essay on invisible gorillas.

Temperature excursions and APIs to reduce study monitor work

I did a lot of local excursions the past 3 days – Jerusalem, Tel Aviv, Herzliya and Haifa.   For some reason, the conversations with 2 prospects had to do with refrigerators.   I do not know if this is Freudian or not, considering the hot weather of July in Israel.

The conversations about refrigerators had to do with storing drugs / investigational product at the proper temperatures.

Temperature excursion is a deviation

The great thing about not coming from the clinical trials space is that you are always learning new things.

Yesterday – I learned that a Temperature excursion is a deviation from given instructions. It is defined in the WHO Model Guidance as “an excursion event in which a Time Temperature Sensitive Pharmaceutical Product (TTSPP) is exposed to temperatures outside the range(s) prescribed for storage and/or transport.

Storing drugs at the proper temperature is part of GCP. Here is an SOP for Monitoring and Recording Refrigerator & Freezer Temperatures

1 Introduction All refrigerators and freezers used for the storage of Investigational Medicinal Products (IMPs) must be temperature controlled, and continuously monitored and maintained within the appropriate ranges as defined by the protocol. ICH GCP Principle 2.13 states “Systems with procedures that assure the quality of every aspect of the trial should be implemented.”

Moving on:

5 Procedure
 Current maximum/minimum thermometers must be monitored as a minimum at least once on a daily basis on all working days, and recorded legibly on the temperature monitoring log.
 The digital maximum/minimum thermometer –
□ Should be read from the outside of the refrigerator without opening the door.
□ Have an accuracy of at least +/- 1 oC.
□ Be able to record temperatures to one decimal place.
□ Be supplied with a calibration certificate.
□ Have the calibration check on an annual basis.
 Temperature logs should be kept close to the refrigerator/freezer (but not inside) to which they relate for ease of reference, and should be clearly identified as relating to that appliance.
 A separate temperature record must be kept for each fridge/freezer. (The use of whiteboards as a method of logging results is not acceptable.)
 It is good practice to record the temperature at a similar time each day e.g., first thing in the morning before the refrigerator door is opened for the first time. This will allow review of trends in results recorded; help highlight any changes in temperatures recorded and deviation in refrigerator performance.

There is a lot of manual work involved looking at refrigerators

I believe a study monitor will spend 20’/day checking logs of refrigerator temperature readings. When you add in time for data entry to the site coordinators – that’s another 20’/day and then you have to multiply by the number of sites and refrigerators.   This is only the reading temperatures and capturing data to the EDC part of the job.   Then you have to deal with queries and resolving deviations.

For something so mundane (although crucial from a medical research perspective), its a lot of work. The big problem with using study monitors to follow temperature excursions is that the site visits are every 1-3 months. With the spiralling costs of people, the site visits are getting less frequent.

This means that it is entirely plausible that patients are treated with improperly stored drugs and the deviation is undetected for 3 months.

Whenever I see a lot of manual work and late event detection, I see an opportunity.

It seems that there are a few vendors doing remote monitoring of refrigerators.  A Polish company from Krakow, called Efento has a complete solution for remote monitoring of refrigerators storing investigational product.  It looks like this:

 

null

 

What is cool (to coin a pun) about Efento is that they provide a complete solution from hardware to cloud.

The only thing missing is calling a Flask API to insert data into the eCRF for the temperature excursions.

Once’s we’ve got that, we have saved all of the study coordinators and study monitors time.

More importantly, we’ve automated an important piece of the compliance monitoring puzzle – ensuring that temperature excursions are detected and remediated immediately before its too late.

Doctor-Patient Communication – the key to success and the struggle to succeed.

Katherine Murphy, Chief Executive of the Patients Association London once said,

“The huge rise in complaints in relation to communication and a lack of respect are of particular concern. Patients are not receiving the compassion, dignity and respect which they deserve.”

As clinical trial technology guys, you would assume that we love code more than we love the patients and site coordinators who use our software.

I took a random sample of  home pages from 3 of our competitors – and this is what I found.   We can discuss if real-time visibility to  data is going to make the clinical operations team more effective or not – but that is a story for another post.

EMPOWER YOUR CLINICAL TRIAL EDC + ePRO and a bunch of other features to make your clinical trial successful. ( viedoc )

Oracle Health Sciences InForm. Accelerate Clinical Trial Timelines While Reducing Trial Cost and Risk.

Collect and deliver higher-quality data faster through advanced data capture and query management, real-time visibility to data, standards-based, integrated workflows, and security best practices.

Faster, smarter medical research. Castor is the end-to-end data solution, enabling researchers to easily capture and integrate data from any source on one platform. Thousands of medical device, biotech, and academic researchers around the world are using Castor EDC (Electronic Data Capture), ePRO, and eTMF to accelerate their studies.

In this article we’ll discuss the doctor-patient communications gap as a generic problem. We will briefly examine the root cause of the problem and conclude by proposing a light-weight easy-to-use Web service for sharing and private messaging with patients and physicians as a way to ameliorate the problem.

Poor patient-doctor communications as a generic problem

Doctor-Patient communication is the key to the success of a treatment plan and reduction of hospital readmission. However, doctors and nurses often fail in communicating with their patients properly.

What is the nature of poor doctor-patient communications?

Some patients say that their doctors need to polish their communication skills; although they are excellent diagnosticians.

Other patients say that their doctors know how to talk, but seem to have no time to listen.

Many patients also complain that their doctors don’t explain things in terms patients can understand. Poor communications between doctors/nurses and their patients can come at a high cost, creating misunderstandings that can  lead to malpractice suits.

In a hospital setting, we often hear that patients feel that they are not getting any useful information while the medical staff feel that they have taken the time to communicate all the data that the patients and their families need in order to understand and comply with the treatment plan.

The question is why some doctors find it hard to communicate properly and share things with their patients in a desired manner while other doctors succeed in communicating the therapeutic plan to the patient in a manner that the patient understands.

Poor physician-patient communications is rooted in cognitive and cultural gaps

Patients are the experts at their personal feelings and experiences.  Physicians are the experts in the medical science.  Cultural and language differences and preconceived notions about the doctors role only contribute to the cognitive gap between emotion and science.

Besides the cultural and cognitive gaps, high patient volume and work overload is another root contributor to poor doctor patient communications.  This generally happens in poor countries. In the third world, working over capacity is one of the biggest barriers to effective communication. Hospitals, doctors and nurses are forced to admit more and more patients and are compelled to handle more than they can manage. Under such circumstances, health professionals cannot devote enough time to their patients let alone sit down with them in a quiet corner and explain the therapeutic plan.

Sharing and private messaging with patients  and doctors helps bridge the gaps

The solutions are out there.

In this always-on age of mobile medical devices and cloud services, both healthcare professionals and the patients have immediate access to the latest solutions that can help them communicate more effectively and efficiently. There are private social networks for healthcare that have been exclusively developed for sharing and private messaging with doctors, nurses and patients, enabling doctors and patients to interact and share where and whenever they need the interaction.

Neither the patient nor the physician need to be tied down to a proprietary healthcare provider portal.

Secure Web-based sharing and private messaging services improve the ways doctors and nurses communicate with their patients. This helps them improve the quality of service and lower operational costs, and enables doctors to treat more patients in less time and with less stress.

In summary

Poor patient-doctor communications has a number of causes and it is rooted in both cultural, language and cognitive differences.   Using a neutral medium such as online sharing and private messaging with patients and doctors helps bridge the gaps we discussed.

We’d love to hear what you think – please comment!

Thanks!

Urban medical legends

Because I was trained as a solid-state physicist I am skeptical of many medical claims – including the efficacy of digital health apps.  Gina Kolata wrote this post last week.  I’ll let you decide for yourself.

You might assume that standard medical advice was supported by mounds of scientific research. But researchers recently discovered that nearly 400 routine practices were flatly contradicted by studies published in leading journals.

 

(more…)

What takes precedence? GCP or hospital network security?

patient compliance in medical clinical device trials

This is a piece I wrote a while back on my medical device security blog – Cybersecurity for medical devices.

One of the biggest challenge of using connected medical devices in clinical trials is near real-world usage of devices that are not commercially-ready.

We have a couple of customers that are performing clinical trials of medical devices in the ER and ICU. The tradeoffs between cybersecurity and patient safety are not insignificant.

What takes precedence? GCP or hospital network security?

Data quality, protocol compliance and patient safety are the 3 main pillars of GCP.

What is more important – patient safety or the health of the enterprise hospital Windows network?

What is more important – writing secure code or installing an anti-virus?

In order to answer these question, we performed a threat analysis on a medical device being studied in intensive care units.  The threat analysis used the PTA (Practical threat analysis) methodology.

Risk analysis of a medical device

Our analysis considered threats to three assets: medical device availability, the hospital enterprise network and patient confidentiality/HIPAA compliance. Following the threat analysis, a prioritized plan of security countermeasures was built and implemented including the issue of propagation of viruses and malware into the hospital network (See Section III below).

Installing anti-virus software on a medical device is less effective than implementing other security countermeasures that mitigate more severe threats – ePHI leakage, software defects and USB access.

A novel benefit of our approach is derived by providing the analytical results as a standard threat model database, which can be used by medical device vendors and customers to model changes in risk profile as technology and operating environment evolve. The threat modelling software can be downloaded here.

(more…)

Why Microsoft is evil for medical devices

Another hot day in paradise. Sunny and 34C.

Not a disaster but still a PITA

We just spent 2 days bug-fixing and regression-testing code that was broken by Microsoft’s June security update to Windows operating systems and Explorer 11.    Most of the customers of the FlaskData EDC, ePRO, eSource and automated detection and response platform use Chrome or Firefox on their desktops.   This was no solace to site coordinators in one of the sites using Flaskdata.  They came into work on Monday and the hospital-standard Explorer 11 no longer supported our application.

Microsoft published KB4503259 as a cumulative security update but it was much more.  The update included major changes to the Explorer JavaScript engine. Its because of delightful black swans like this, running a SaaS business is not for the faint of heart.

I once wrote an essay on my cybersecurity for medical device blog called The Microsoft Monoculture as a threat to national security.

Why Microsoft is evil for medical devices

I suggested that the FDA might consider banning Windows as an operating system platform for medical devices and their accompanying information management systems.

One of my readers took umbrage at the notion of legislating one monoculture (Microsoft) with another (Linux) and how the Linux geeks are hooked on the CLI just like Windows users are hooked on a GUI.

The combination of large numbers of software vulnerabilities,  user lock in created by integrating applications with Windows,  complexity of Microsoft products and their code and Microsoft predatory trade practices are diametrically different than Linux and the FOSS movement.

The biggest threats to medical devices in hospitals is old Windows versions

One of the biggest threats to medical devices in hospitals is the widespread use of USB flash disk drives and Windows notebooks to update medical device software. With the infamous auto-run feature on Microsoft USB drives – flash memory is an easy attack vector for propagating malware via Windows based medical devices into a hospital network. This is one (and not the only) reason, why I am campaigning against use of Windows in medical devices.

This  has nothing to do with the CLI or GUI of the operating system and personal preferences for a user interface.

This has everything to do with manufacturing secure embedded medical devices that must survive in most demanding, heterogeneous and mission critical environment one can imagine – a modern hospital.

I never advocated mandating Linux by law for medical devices.

It might be possible to mandate a complex set of software security requirements instead of outlawing Windows in embedded medical devices as a more politically-correct but far more costly alternative for the the FDA and the US taxpayer.

Regardless of the politics involved (and they are huge…) – if the FDA were to remove Windows from an approved list of embedded medical device operating systems – the costs to the FDA would decrease since the FDA would need less Windows expertise for audits and the threat surface they would have to cover for critical events would be smaller.

How to measure clinical response in medical device clinical trials

clinical data management

It is 19:15 and daylight savings time.   It is too hot to go out and run or bike.  Time to write.

Today we were helping a customer with hardware issues. At the end of a long day, I started thinking that even hardware issues are valuable data to the decision-making process of measuring efficacy of treatment.

We specialise in reducing time to regulatory submission in clinical trials.  Our medtech customers use the Flaskdata.io platform to collect data from patients, devices and investigators and automate and prove efficacy of their device.  As Skolnick et al note in Compliance, Compliance, Compliance – the secrets of a successful clinical trial :

Although more commonly considered to be a phase 1 phenomenon, efficacy trials also attract professional subjects, particularly when entry criteria and endpoints are “soft,” such as trials using subjective rating scales which can be “gamed.”

Not good.  How do you measure a valid clinical response? The VAS scale for pain reduction can be gamed so is it a simple measure of pain reduction sufficient? My thesis advisor in solid state physics always said that he only does simple things (simple may be a mantra that many grad students hear while they trudging on a steep uphill slope of research).

Simple is not always sufficient

When you rely on a simple mode of clinical response you get a partial picture.

The clinical response to treatment is an important indicator of the therapeutic effect of a medtech device or drug. The value and interpretation of clinical response has to be carefully considered within the intended use by the patient.

Let’s consider a simple model of clinical response that is based on cause and effect.

-You use an electroceutical device and the pain is reduced as measured by a VAS scale.

-You stimulate vagus nerve and the patient has a bowel movement as measured by a ePRO.

Is this sufficient?  Maybe maybe not.   Did the device perform as expected?   Did the patient game the treatment?

A better model of assessing clinical response

Response assessment should be combined with other indicators of the patient’s condition to contribute to the decision-making process.

We can use additional connected devices in order to measure biological response and biometrics. With additional data, we can validate patient compliance and proper operation of the device.  We can also detect and respond to patients who game the treatment.

One example that comes to mind is measuring peptide levels in saliva (which requires a lab test) or skin temperature (which can be measured directly). Another example is weight differences between medication-compliant cohort and the placebo group. Weight loss may be indication of medication adherence in certain treatments and it is simple to measure.

For home-use devices, we can consider proper operation of the device itself. The device should provide 3 kinds of introspection.  (Introspection means that the device tells us how it’s doing and how it was used)

1.The device should record whether or not it functioned properly when used. A simple success/fail code will do.
2.The device should record proper operation; whether or not it was used properly. A simple count of number of operations will be helpful.
3.The device should record timestamps of operation to enable comparison with timestamps in the device log. The best solution is for the device to transmit log records via an API to a secure service with its internal timestamps. The timestamps will tell us if the device is using NTS properly, or if the clock is drifting. The timestamps will also help analyze multiple activations (mistakes or professional patients gaming the treatment).

More data helps us detect and respond to patient and device compliance issues

Device introspection will reveal that the patient misused the device, or the device failed. The first case is an issue of patient compliance. The second case is an issue of the device not providing the treatment as designed.

Listening to the device and to the body

The model of clinical response to treatment can be enriched with additional data sources from the device itself as well as biological response and biometrics measurements.

A richer model helps us detect and respond to patient compliance deviations.   The additional data also helps us understand if the investigational product is defective.   This is an advantage that medtech has over drugs.