Invisible gorillas and detection of adverse events in medical device trials

Weekly Episode #1 - Patients and study monitors are both people.

What is easier to detect in your study – Slow-moving or fast moving deviations?

This post considers human frailty and strengths.

We recently performed a retrospective study of the efficacy of automated study monitoring in orthopedic trials. An important consideration was the ability to monitor patients who had received an implant and were on a long term follow-up program. Conceptually, monitoring small numbers of slow-moving, high-risk events is almost impossible to do manually since we miss a lot of what goes on around us, and we have no idea that we are missing so much. See the invisible gorilla experiment for an example.

One of patients in the study had received a spinal implant and was on a 6 month follow-up program dived into a pool to swim a few laps and died by drowning despite being a strong swimmer. Apparently, the pain caused by movement of the insert resulted  in loss of control and a severe adverse event. The patient had disregarded instructions regarding strenuous physical activity and the results were disastrous. 

It seems to me that better communications with the patients in the medical device study could have improved their level of awareness of safety and risk and perhaps avoided an unnecessary and tragic event.

Subjects and study monitors are both  people.

This might be a trivial observation but I am going to say it anyhow, because there are lessons to be learned by framing patients and monitors as people instead of investigation subjects and process managers. 

People are the specialists in their personal experience, the clinical operations team are the specialists in the clinical trial protocol. Let’s not forget that subjects and study monitors are both  people.

Relating to patients in a blinded study as subjects without feelings or experience is problematic. We can relate to patients in a personal way without breaking the double blinding and improve their therapeutic experience and their safety. 

We should relate to study monitors in a personal way as well, by providing them with great tools for remote monitoring and enable them to prioritize their time on important areas such as dosing violations and sites that need more training. We can use analytics of online data from the EDC, ePRO and eSource and connected medical devices in order to enhance and better utilize clinical operations teams’ expertise in process and procedure.

A ‘patient-centered’ approach to medical device clinical trials

In conditions such as Parkinsons Disease, support group meetings and online sharing are used to stay on top of medication, side effects, falls and general feeling of the patient even though the decisions on the treatment plan need to be made by an expert neurologist / principal investigator and oversight of protocol violations and adverse events is performed by the clinical operations team. There are many medical conditions where patients can benefit by taking a more involved role in the study. One common example is carpal tunnel syndrome. 

According to the findings of an August 3rd, 2011 issue of the Journal of Bone and Joint Surgery (JBJS), patients receiving treatment for carpal tunnel syndrome (CTS) prefer to play a more collaborative role when it comes to making decisions about their medical or surgical care. 

Treatment of carpal-tunnel syndrome which is very common and also extremely dependent upon patient behavior and compliance is a great example of the effectiveness of “shared decision-making, or collaborative, model” in medicine, in which the physician and patient make the decision together and exchange medical and other information related to the patient’s health.

As the article in JBJS concludes:

“This study shows the majority of patients wanted to share decision-making with their physicians, and patients should feel comfortable asking questions and expressing their preferences regarding care. Patient-centered care emphasizes the incorporation of individual styles of decision making to provide a more patient-centered consultation,” Dr. Gong added. 

In a ‘patient-centered’ approach to medical device clinical trials, patients’ cultural traditions, personal preferences and values, family situations, social circumstances and lifestyles are considered in the decision-making process.

Automated patient compliance monitoring with tools such as are a great way to create a feedback loop of medical device clinical data collection,  risk signatures improvement, detection of critical signals and communications of information to patients. Conversely, automated real-time patient compliance monitoring is a a great way of enhancing clinical operations team expertise.

Patients and study monitors are both people. 

Strong patient adherence in real life starts with strong people management


Patient adherence in real-life starts in clinical trials determining the safety, side effects and efficacy of the intervention, whether a drug or a medical device.

Like any other industry – success in clinical trials is all about the people.

The hugely successful movie – “Hidden figures” tells the story of the gifted black women mathematicians who played key roles in the NASA space program in the Mercury and Apollo space programs. It is a moving, inspiring and (sometimes hilarious) story of how NASA, a dominantly white male organization came to accept diversity during American desegregation.

By comparison, the Israeli life science industry lives in a different time and place and women are in leadership roles at all levels  of Israeli life science companies.

In this 4 part series of articles, we will tell the story of the gifted Israeli women who are the   “Hidden figures” of the Israel biomed/biotech industry.

Women comprise about 65 percent of Israel’s biotechnology workforce, and about 13 percent of top management positions in companies listed on the Tel Aviv Biomed index. In order to find out what attracts Israeli women into this globally male dominated field, I talked to a number of well-respected women, tried to learn about their story, get acquainted with their mindsets and solve the “mystery” of Israeli women invading this field.

Part 1 of the series tells the story of Hagit Nof – former Country Manager of IQVia in Israel and  currently the COO & BD of nRollmed an Israeli startup that helps clinical trial sponsors speed up their study using online patient recruitment and optimization.

(IQVia is the world’s largest provider of biopharmaceutical development and commercial outsourcing services ).

Hagit has a great story of a dream come true for a person who was not afraid to make a risky decision at the right time and was able to build a career in the biopharmaceutical industry literally from scratch.


What real-time data and Risk-based monitoring mean for your CRO

A widely neglected factor in cost-effective risk-based clinical trial monitoring is availability and accessibility of data.

RBM methods used by a central clinical trial  monitoring operation that receives stale data (any data from patients that is more than a day old is stale) are ineffective. Every day that goes by without having updated data from patients, devices and investigators reduces the relevance and efficacy of remote monitoring.

Real-time data is a sine-que-non for RBM.

Sponsors and Contract research organizations (CROs) should therefore approach real-time data and risk-based monitoring (RBM) as 2 closely related priorities for executing clinical trials. Use of modern data technologies for real-time data collection and remote risk-based monitoring will reduce non-value added rework, people and paper in clinical trials and help speed up time to statistical report.


The 3 tenets for designing a clinical data management system

This post reviews the importance of 1) proper study design, 2) good data modeling and 3) realistic estimation of project timetables. The article concludes with a discussion of eSource and attempts to dispel some of the myths including how DIY EDC study build save time (they don’t).


The trend of DIY: good for EDC vendors, less good for sponsors

The trend for small studies/IIS (investigator-initiated studies) is to use cloud EDC applications
that enable end-users to build eCRF and edit checks using a graphical user interface. This so-called DIY (do-it-yourself) approach is used by most cloud EDC vendors such as Medrio and Clincapture as a way of lowering their barriers to entry to the market.

However – what is good for vendors (lowered barriers to entry) is not necessarily good for sponsors (faster time to market of their innovative drug or medical device).


How to ensure patient compliance in patient-centric clinical trials

Patient-centered clinical trials is a growing trend. As both drug and medtech companies increasingly explore use of medical IoT for clinical trials are we discovering new opportunities or forgetting old lessons learned? Dr. Jane Bluestein talks about how to ensure patient compliance by understanding the subtlety of differences between boundaries and rules


Using AI to assure patient compliance in heart failure patients

Innovative clinical trial data management

Can AI be used to help patients with heart failure?

Each year cardiovascular disease (CVD) causes 3.9 million deaths in Europe and over 1.8 million deaths in the European Union (EU). CVD accounts for 45% of all deaths in Europe and 37% of all deaths in the EU. In 2015, almost 49 million people were living with CVD in the EU.(13)
Adherence is considered to be key to success of the treatment plan. (10, 16, 17)

In this essay, we will survey alternatives for improving adherence in heart failure and CVD in general.

It is notable that at some point, I think around 2014, the literature started using the term adherence instead of compliance for patients who consistently stick with a treatment plan.   I believe that this is related in some way to PC (politically-correctness) that is prevalent in the US and Europe – where adherence is more about what the patient does when well-informed and educated as opposed to compliance to strict guidelines from a physician.   We will use the term adherence in this article.

Patient adherence approaches

There seem to be 3 main approaches today; each has a different business model.

1 – Institutional approach that patient health is a physician-patient partnership.(12) 

2 – A more tech-oriented approach that patient health is about optimizing pill consumption.    This initially seemed strange to me since treating congestive heart failure is not just about medication but also related to diet and physical exercise not to mention support of family and care-givers. One of the pill-adherence companies is an Israeli company called Medisafe who have been around for almost 10 years and combine a mobile app, pillbox sensor and patient-engagement.   The company collaborates with pharma where there is a direct ROI story – the more pills patients take, the more money the pharma generates.  On the strength of collaboration with pharma, Medisafe raised $14.5M in 2017 – so they seem to be getting traction.

Medisafe will be focusing on three aspects of its platform: improving medication adherence, increasing patient engagement, and generating data-driven insights about patient behavior. The new personalization tool highlights all three.

 3 – A more human-oriented approach to adherence is remote coaching.  There are a number of companies doing this and Livongo is getting a lot more investor traction than Medisafe. The coaching business model is clearer and more direct than the pill model and is based on a simple notion that maintaining healthy employees is good for business, so companies are willing to pay for Livongo subscriptions for their employees to keep them healthy and at work and perhaps reduce insurance premiums.

Livongo gets $105 million, signs deal with insurer Cambia Health Solutions

April 11, 2018

Digital chronic disease management company Livongo has raised $105 million in new Series E funding, mostly from existing investors. The company also announced a partnership with nonprofit health insurance company Cambia Health Solutions. General Catalyst and Swedish investment company Kinnevik led the round, with additional participation from existing investors DFJ, Kleiner Perkins Caufield…

Why the institutional approach is losing ground to coaching and AI-based adherence

Existing strategies of health-care providers for improving patient adherence to treatment rely on data collection and tracking in order to improve physician-patient interactions so that the physician can monitor the patient for side-effects, improvement or degradation in condition.

In spite of important advances in heart failure therapy, derived from better physiopathological understanding, hospital readmissions rate continued to increase(1) which suggests that the strategy of improving patient adherence via education and physician follow-up does not work.

The data is telling us that education does not work well for protocol compliance (since patients quickly forget or do not believe in the efficacy of treatment and/or relevance of the treatment to their condition) and physician follow-up does not work because the vast majority of physicians do not have the time to follow-up on their patients – only on the most critical and high-risk and even then, physician follow-up is iffy.

Big noisy data is not a good way to train compliance models

Another barrier to the institutional approach is that big health provider electronic health records are often cited but rarely used  effectively to train AI compliance models. The data in EHR systems is poorly structured, noisy and generally unreliable considering that physicians have 7′ to talk to patients,  type on their keyboard and fill out scrips.(15)

Use of AI in a mobile agent – a simple example of how RL might improve adherence.

A mobile agent may communicate directly with patients and suggest actions using reinforcement learning that maximizes a reward of better health in order to achieve a goal of adherence. (9)

The mobile  agent reinforces its knowledge in the course of user interaction.(2,3,4,5,6,7,8)

In reinforcement learning, an agent who is both a learner and decision maker interacts with its environment, the patients. In the case of patient adherence, the interaction is a continuing task – where the agent proposes actions to the patient and evaluates a continuous reward function using discounted returns.(14)

If its that simple, why isn’t everyone doing it?

There are I think, 4 challenges to using AI-based mobile agents to improve patient adherence

1. What is the reward function? Are we maximizing pills or patient health?

2. What about usability?  Is the mobile agent really usable by older, sick people?

3. What about accessibility? Do older sick people have the equipment to use the app?

4. What is the right computational tool?  Perhaps not AI.

A very tricky and non-trivial question in AI-based adherence is “What is the reward function?”.  Is it pill-taking (the approach that Medisafe and others have taken) or is it a more analytical and measurable clinical metric such as EF – Ejection fraction.

The left ventricle is the heart’s main pumping chamber that pumps oxygenated blood through the ascending (upward) aorta to the rest of the body, so ejection fraction is usually measured only in the left ventricle (LV). An LV ejection fraction of 55 percent or higher is considered normal.

 The excellent review article Deep Reinforcement Learning Doesn’t work yet – talks to the depth of problems with RL including proper choice of reward.

Another tricky and non-trivial problem that is usually neglected is accessibility.

The mobile agent is usually implemented  in a smartphone mobile app on IOS and Android.  Perhaps there is logic in a much simpler app-less/server-side solution that communicates with patients using text messaging and a dumb phone that costs $10 in order to make the service accessible to people with limited means and connectivity.

A third constraint is usability.

Much work needs to be done on the UX (user experience) in order to enable older people to interact effectively with mobile adherence agents since the majority of CVD patients are over 60.    Having said that – this problem will go away by itself over time as the young people of today become the older people of tomorrow – confident and fluent in digital media.

And finally – perhaps deep RL is not the right computational tool for adherence.

Perhaps we should consider Model predictive control (MPC) as a simpler and easier to implement alternative to AI.  Model predictive control is an advanced method of process control that is used to control a process while satisfying a set of constraints. It has been in use in the process industries in chemical plants and oil refineries since the 1980s.  Certainly no reason not to consider it as an alternative to AI.

The social value of AI

But really the biggest concern is the question of what problem are we trying to solve.

Is patient adherence a process control-optimization problem from a patient perspective in order to reduce adverse events and improve well-being?

Or is patient adherence  a revenue-optimization problem from a pharma perspective in order to grow sales?

Are we trying to maximize  pill-taking so that the pharma can keep sales high or do we want to minimize medication and medical interventions where appropriate and rely more on alternative medicine and physical fitness strategies to keep people healthy?

This is why pill-taking is such a cool benchmark (besides being great for pharma business). Not only is it easy to get lots of samples, the goal in every interaction with a patient is to maximize pill-taking, so you never have to worry about defining your reward, and you know everyone else has the same reward function.

I believe that we need to balance dosing, physical and social activity based on risk-assessment of the patient’s clinical state and enable an AI agent to take into account the risk level of a patient and modulate reinforcement for dosing, physical or social activity accordingly.  In this respect, training models from coaching data and using evidence-based medicine for heart-failure derived from from clinical trial data may be a valuable approach.

An AI-based adherence agent should be able to free up physicians from low-risk patients and time-consuming office visits to focus on treatment of high-risk patients. AI-based adherence agents should be able to help people live well and be healthier while also reducing the cost and risk of interaction with healthcare providers and risk of adverse events due to drug interactions between concomitant medications.

But whatever mix of strategy  we use, it seems to me that the priority of AI-based adherence should be to add value to society by improving the lives of people and reducing the impact of heart disease by using clinical, evidence-based metrics such as EF.

The cost to society justifies usage of AI.

CVD (cardiovascular disease) is estimated to cost the EU economy €210 billion a year. Of the total cost of CVD in the EU, around 53% (€111 billion) is due to health care costs, 26% (€54 billion) to productivity losses and 21% (€45 billion) to the informal care of people with CVD.(13)

1-2% of health care expenditure are attributed to heart failure in Europe and  74% of heart failure patients suffer from at least 1 comorbidity: more likely to worsen the patient’s overall health status.

Combining approaches of coaching and smart mobile apps with AI moves the communications focus from a physician-patient partnership based on rules (what you must take, what you cannot eat) and reframes rules as boundaries(11), as “if . . . then” actions that are more positive, more effective and less power-oriented than rules. We believe that the reward (10) of positive outcomes empowers patients better than the threat of negative consequences by their physician. Recommendations for action may also include informative explanations, which may further motivate the patient.

Behavior-modification – the next stage of evolution in AI-based adherence?

Mobile medical apps that measure signs like heart rate and sugar generally use measurements and statistics for informational and educational purposes rather than trying to change the patient’s state. However – AI can help modify behavior and not just coach.

The notion of  reward and learning from interaction with patients data to achieve a goal of better health or stability is immensely more valuable to a patient than maximizing pill-taking or simply taking measurements and providing education.(9) | The team behind the world’s fastest patient compliance system Helping over 300 sites around the world and thousands of patients/month


1. Krumholz et al. Randomized trial of an education and support intervention to prevent readmission of patients with heart failure. J Am Coll Cardiol 2002;39:83-9.
2.  Harvest: an open platform for developing web-based biomedical data discovery
3. Reuse of Clinical Data C. Safran
4.Recycling side-effects into clinical markers for drug repositioning

5. Barriers to sharing
6. Clinical study data sharing
7. CRF standardization
8. Clinical trial data sharing
9. A Reinforcement Learning System to Encourage Physical Activity in Diabetes Patients. Hochberg et al…
Geriatr Nurs. 2010 Jul-Aug;31(4):290-8.
10. Medication adherence is a partnership, medication compliance is not.Gould E1, Mitty E.
11. Rules and boundaries in patient-centered clinical trials
12. Bluestein, D. Lieberman – Rules and boundaries in clinical trials
13. What to teach to patients with heart failure and why: the role of nurses in heart failure clinics. Eneida Rejane Rabelo et al.
14. European Cardiovascular Disease Statistics 2017
15. Reinforcement Learning, An Introduction, Sutton and Barto, MIT Press, 1999
16. Reliability of SNOMED-CT Coding by Three Physicians using Two Terminology Browsers, Chiang et al
17. Depression Is a Risk Factor for Noncompliance With Medical Treatment: Meta-analysis of the Effects of Anxiety and Depression on Patient Adherence. DiMatteo et al
18. Importance of medication adherence in cardiovascular disease and the value of once-daily treatment regimens. Frishman.
19. Comparison of Approaches for Heart Failure Case Identification From Electronic Health Record Data. Blecker, Katz, Horwitz et al.
20. Automating the integration of clinical studies into medical ontologies Mark Roantree, Jim O’Donoghue, Noel O’Kelly, Martin van Boxtel, Sebastian Kohler
21. A configurable deep network for high-dimensional clinical trial dataJim O’Donoghue, Mark Roantree, Martin Van Boxtel
22. AI shows promise as a clinical development tool
23. Srinivas Karri , Oracle
24. Artificial intelligence in healthcare: past, present and futureFei Jiang, Yong Jiang, Hui Zhi, Yi Dong, Hao Li, Sufeng Ma, Yilong Wang, Qiang Dong, Haipeng Shen, Yongjun Wang  DOI: 10.1136/svn-2017-000101 Published 21 June 2017
25. Mining Patterns of Adverse Events Using Aggregated Clinical Trial ResultsZhihui Luo, PhD, Guo-Qiang Zhang, PhD, and Rong Xu, PhD
26. Data mining for better clinical designMichelle Marlborough, Medidata in the book “Re-Engineering Clinical Trials : Best Practices for Streamlining the Development Process”
27. The Digital Revolution comes to US Healthcare Goldman Sachs, Equity Research Report, 29/6/2015 Roman et al
28. Using Artificial Intelligence to Reduce the Risk of Nonadherence in Patients on Anticoagulation Therapy Daniel L. Labovitz, Laura Shafner, Morayma Reyes Gil, Deepti Virmani, Adam Hanina (all physicians from Montefiore Medical Center, Bronx)
29. Method and apparatus for fractal multilayered medication identification, authentication and adherence monitoring Hanina, Yaron Ganor et al
30. Mobile Health Strategies for Veterans With Coronary Heart DiseaseThe purpose of this study is to determine whether text messages/messaging (TM) or a mobile application (app), compared with an educational website-control provided to all Veterans, can improve adherence to antiplatelet therapy among patients following acute coronary syndrome or percutaneous coronary intervention (ACS/PCI). Will start recruitment beginning of 2018.
31. How to Truly Solve the Patient Engagement Problem with AIWaqaas Al-Siddiq, CEO and Founder of Biotricity Inc.
32. What’s wrong with mobile apps for medication adherence?
33. Assessment of medication adherence app features, functionality, and health literacy level and the creation of a searchable Web-based adherence app resource for health care professionals and patients Heldenbrand, et al.
34. BCT – Behavior Change Technique taxonomyMichie et al 2013
35. The Potential Impact of Intelligent Systems for Mobile Health Self-Management Support: Monte Carlo Simulations of Text Message Support for Medication Adherence Piette et al. 2014 Annals of Behavioral Medicine.
36. Learning from Demonstrations for Real World Reinforcement Learning, Hester et al, “Deep Q-learning from Demonstrations (DQfD), that leverages this data to massively accelerate the learning process even from relatively small amounts of demonstration data and is able to automatically assess the necessary ratio of demonstration data while learning thanks to a prioritized replay mechanism.
37. Contextual Bandits with Linear Payoff Functions Chu, et al.
38. Contextual Linear Bandit Problem and applications Feng et al.

The pay-off for reducing cycle time in medical device clinical trials

Innovative clinical trial data management

Well – it certainly isn’t science and technology innovation or even the FDA.At a recent executive roundtable, according to Tufts University’s Center for the Study of Drug Development director and associate professor Ken Getz, it is that many pharma corporations and CROs have not yet fully embraced the newest, most efficient technologies, and this has been a huge problem in recent years.

More expensive. Riskier than ever.

“Drug development cycle times have not gotten faster, costs continue to increase, and drug development has become riskier than ever with only 11.8% of products that enter clinical testing receiving regulatory approval, about half the rate of the 1990s,” says Getz.

Which, of course, is true, as we in the clinical trial industry know that it takes about a decade for a drug to be developed and on average seven years for it to be tested before it makes it to market, and only if it gains regulatory approval. So, what would happen if clinical testing time could be shortened?


How to overcome 5 eSource implementation challenges

Jenya wrote a piece about the challenges of clinical trials operations change management for regulatory people who have to work with medical technology developers and I just had to write my own intro.

Frankly, its easier to talk about change for other people than for yourself. A lot easier.  I have written here, here and here about the gaps between the stakeholders in medical device clinical trials – security, IT, engineering, product marketing ,regulatory affairs and medical device security to name a few.

Overlook change management at your own risk

Change management is a topic usually overlooked when medtech companies implement cloud EDC, and introduce medical IoT for collecting data from patients directly and use electronic source documents for their connected device/mobile medical app or device clinical trial.

In this post, Jenya talks about how to manage change during the transition from traditional medical device clinical trial data management to cloud technologies, remote monitoring, medical IoT and electronic source data.

So enjoy.



Cost effective EDC for medical device clinical trials

clinical data management

There is a saying in American English dating back to the 1940’s – “Call me when you have a nickel in your pocket”. With limited budgets, small, innovative medical device vendors will be looking at the nickel in their pocket and thinking that they cannot afford a cloud EDC offering for clinical data management and opt for the paper default or the DIY option for monitoring. None of which are a particular good idea.


The key is not first to eSource, the key is smart to market

This post is not for the Pfizers, Novartis, Merck and GSK giants of the life science industry.

Its for the innovators, the smaller, creative life science companies that are challenged by the costs, the regulatory load and complexity of executing a clinical trial.

This post is dedicated to the startup entrepreneurs of the world.

Building an EDC system for your clinical trial requires executing a plan in order to successfully recruit patients, collect high quality data, sustain patient safety and produce your statistical report in a timely fashion. You can potentially embark on an EDC journey without a plan, without a simple, well-designed protocol, and without appropriate clinical monitoring. This will guarantee you a long trek of pain, burning cash while you resolve issues and clean data.

The pivotal question to any clinical decision maker is this: Do you want to start building an ECRF (electronic case report form) now and pay in pain and cash later, or plan now and own the process?

Simple concept, but important message.

It doesn’t matter if your business is a one-person startup or a “Big Five” bio-technology company. If you develop medical devices, medtech, biotech or drugs on a daily basis, you are faced with an increasing stable of competitors, and barriers to success that can frustrate you as a business manager or a startup entrepreneur trying to make payroll.

Being an entrepreneur like you, I’ve constantly been exposed to walls that have continuously tried to prevent me from success. In this post, you will learn how to plan and execute EDC quickly, efficiently and successfully and break through the business, clinical and regulatory barriers that stand in your way. In a world where competition erodes market share, depresses product pricing, and where large company branding and marketing tramples the innovative medtech startup, the key is not first to eCRF the key is smart to market.

So – here are 2 factors to consider to help get you faster to the finish line.