Get Your Medical Mobile App Verified By IEEE

Patients who use mobile apps to manage health conditions, including depression and chronic pain, might assume that the apps have been evaluated by regulatory agencies as safe and effective. But this is not necessarily the case.

Most of the more than 55,000 medical apps that claim to diagnose or treat disease – or those that provide clinical decision support, known as “therapeutic” apps – have never been evaluated by neutral bodies or trusted regulatory agencies to assess their technical soundness, ethical design, or clinical benefits. Apps often do not follow regional data security and privacy laws to protect people’s sensitive health information.

Medical apps differ from traditional wellness apps, which provide users with information to improve their health, such as tracking their fitness activities, monitoring their blood pressure, and analyzing their sleep patterns.

There is no reliable way to verify that therapeutic applications produce the results they claim. To ensure the credibility of these applications, the IEEE Standards Association (IEEE SA) recently launched the IEEE Global Medical Mobile App Assessment and Registry. The publicly searchable directory is designed to list apps that have been reviewed by experts on multiple criteria, including technical soundness, ethical design, compliance with data security and privacy regulations, and clinical effectiveness, which is evidence of clinical benefit to the patient.

“Patients, clinicians, payers and health systems often have difficulty distinguishing clinically meaningful therapeutic applications from those that are simply well-marketed,” says Yuri Quintana, IEEE senior member and chair of the Evaluation and Registry Program. He is chief of the clinical informatics division at Beth Israel Deaconess Medical Center, Boston. “Our goal is to establish a standardized review method using criteria developed by experts. »

Why regulation is lacking

Because the apps are intended for medical use without being part of a medical device, they fall under the Software as a Medical Device (SaMD) designation, according to the International Medical Device Regulators Forum. SaMD is supposed to be regulated by public health agencies such as the U.S. Food and Drug Administration, but the apps have expanded and gained popularity so quickly that regulators haven’t been able to keep up, Quintana says. Some companies have received approval, but most have not, he said.

Many users are unaware of the regulatory gap, he says.

“Seeing an app from a well-known company often gives the impression that it has been thoroughly vetted for safety and effectiveness, even if that is not the case,” he says.

Some companies use misleading advertising to sell their products, he adds. Marketing materials may claim that all of a company’s health apps are certified, even if only one app has been approved by a regulatory agency to treat a particular disease. The verbiage could also imply that the company has clinical evidence proving its app works, even though the app has never been independently tested.

Another concern is that updated applications are not verified, says Maria Palombini, director of the global healthcare and life sciences practice at IEEE SA.

“The original app may have received approval from a regulatory agency, but the updated version did not,” says Palombini. “There could have been significant changes from the original.”

“Not all medical applications trigger the same classification or regulatory review in all jurisdictions,” adds Quintana. “This leaves a large gray area of ​​clinically relevant but lower risk applications that have not undergone independent evaluation. The IEEE registry was created to help fill these gaps.”

“IEEE is the best organization to solve this problem because it is fundamentally a challenge around standards, trust, interoperability and conformance assessment,” he says. IEEE “is the world’s largest technical professional organization with deep expertise in developing globally recognized standards, including in the areas of healthcare, cybersecurity, AI ethics, and interoperability.”

“Through the IEEE Conformity Assessment Program, we already run rigorous assessment and registration programs,” says Palombini. “Our neutral, consensus-based, multidisciplinary approach – bringing together clinicians, regulators, developers and ethicists without commercial bias – places IEEE in a unique position to create trusted global safeguards that can extend across jurisdictions and support regulatory harmonization.

How the registry works

The assessment framework was developed by a multidisciplinary group of 35 volunteer experts from 10 countries, says Quintana. The panel includes academics, AI experts, application developers, clinicians, ethicists, mental health experts, patient advocates, regulators, researchers, technologists, and those who evaluate health care safety.

The register concerns any application used for clinical or therapeutic purposes and claiming to demonstrate a medical benefit. This includes applications designed for cardiology, diabetes, mental health, neurology, oncology, rehabilitation and respiratory diseases, Quintana says.

Initially, he says, the focus will be on apps aimed at treating mental health issues, given the large number of offerings in this area and the expertise of the registry committee.

Submission of applications is voluntary. There is no government mandate requiring a company to use the IEEE registry.

Products will be evaluated against approximately 150 consensus criteria in three main areas:

• Clinical effectiveness including therapeutic effectiveness, sustained benefits, risk management, comparison to standard care, user engagement and real clinical value.
• Technical solidity including accessibility, privacy and security, error handling, interoperability, AI governance, usability and operational quality.
• Ethical design including bias prevention, patient consent, data governance, conflict of interest transparency, responsible use of AI and large language models, and prioritization of public health benefits.

The IEEE charges a non-refundable submission fee that covers the cost of the evaluation as well as the annual subscription to the registry for the first year.

Developers must first demonstrate that they are a legally established entity before they can complete the app publisher registration form and then submit product documentation and certifications.

It’s estimated that it takes between six and eight weeks for an application to be reviewed by the IEEE, Palombini says. The assessment results will be shared privately with the app publisher, she says, and to be listed on the registry, an app must achieve more than 85% compliance in each category.

Upgraded applications must be submitted and re-evaluated, says Palombini. In the same way that users are notified when an app on their smart devices has an app, the registry will be notified when a new update to the listed apps is available, she says.

Applicants who do not pass the assessment should receive feedback explaining why. They will have the opportunity to make changes or provide additional documentation, Palombini says.

“It’s quite a methodological process, with checks and balances,” says Quintana. “We are very transparent about the process.”

Approved apps added to the registry receive an IEEE certification badge and submission ID, which the company can display on its website, app store listings, and marketing materials.

“The badge provides visible evidence that the app has met independent, consensus-based assessment of its clinical value, technical robustness, and ethical design,” says Quintana.

The registry will be accessible to the public free of charge, he said.

Patients and families looking for safe and reliable apps, as well as payers and insurers evaluating reimbursement potential, will find the registry useful, he says.

The application site is open. The public registry page does not yet list the specific number of approved applications because reviews are ongoing. Approved applications and their unique identifiers must be published once initial reviews are completed.

To learn more, you can watch a webinar recorded in March.

The evaluation framework that underpins the registry supports formal recognition of IEEE P3962 for Criteria Evaluation Framework f