MSGID: 1:317/3 64a3a096   
   PID: hpt/lnx 1.9.0-cur 2019-01-08   
   TID: hpt/lnx 1.9.0-cur 2019-01-08   
    Dangerous chatbots: AI chatbots to be approved as medical devices?   
      
      
     Date:   
         July 3, 2023   
     Source:   
         Technische Universita"t Dresden   
     Summary:   
         LLM-based generative chat tools, such as ChatGPT or Google's   
         MedPaLM have great medical potential, but there are inherent risks   
         associated with their unregulated use in healthcare. A new article   
         addresses one of the most pressing international issues of our   
         time: How to regulate Large Language Models (LLMs) in general and   
         specifically in health.   
      
      
         Facebook Twitter Pinterest LinkedIN Email   
      
   ==========================================================================   
   FULL STORY   
   ==========================================================================   
   "Large Language Models are neural network language models with remarkable   
   conversational skills. They generate human-like responses and engage   
   in interactive conversations. However, they often generate highly   
   convincing statements that are verifiably wrong or provide inappropriate   
   responses. Today there is no way to be certain about the quality,   
   evidence level, or consistency of clinical information or supporting   
   evidence for any response. These chatbots are unsafe tools when it   
   comes to medical advice and it is necessary to develop new frameworks   
   that ensure patient safety," said Prof. Stephen Gilbert, Professor for   
   Medical Device Regulatory Science at Else Kro"ner Fresenius Center for   
   Digital Health at TU Dresden.   
      
   Challenges in the regulatory approval of large language models Most   
   people research their symptoms online before seeking medical advice.   
      
   Search engines play a role in decision-making process. The forthcoming   
   integration of LLM-chatbots into search engines may increase users'   
   confidence in the answers given by a chatbot that mimics conversation. It   
   has been demonstrated that LLMs can provide profoundly dangerous   
   information when prompted with medical questions.   
      
   LLM's underlying approach has no model of medical "ground truth," which   
   is inherently dangerous. Chat interfaced LLMs have already provided   
   harmful medical responses and have already been used unethically in   
   'experiments' on patients without consent. Almost every medical LLM use   
   case requires regulatory control in the EU and US. In the US their lack   
   of explainability disqualifies them from being 'non devices'. LLMs with   
   explainability, low bias, predictability, correctness, and verifiable   
   outputs do not currently exist and they are not exempted from current   
   (or future) governance approaches.   
      
   In this paper the authors describe the limited scenarios in which LLMs   
   could find application under current frameworks, they describe how   
   developers can seek to create LLM-based tools that could be approved as   
   medical devices, and they explore the development of new frameworks that   
   preserve patient safety.   
      
   "Current LLM-chatbots do not meet key principles for AI in healthcare,   
   like bias control, explainability, systems of oversight, validation and   
   transparency. To earn their place in medical armamentarium, chatbots   
   must be designed for better accuracy, with safety and clinical efficacy   
   demonstrated and approved by regulators," concludes Prof. Gilbert.   
      
       * RELATED_TOPICS   
             o Health_&_Medicine   
                   # Today's_Healthcare # Medical_Education_and_Training #   
                   Medical_Imaging   
             o Computers_&_Math   
                   # Computer_Modeling # Internet # Mathematical_Modeling   
             o Science_&_Society   
                   # Public_Health # Privacy_Issues # Bioethics   
       * RELATED_TERMS   
             o Quality_of_life o Personalized_medicine o   
             Evidence-based_medicine o Language_acquisition o Medicine o   
             Public_health o General_fitness_training o CAT_scan   
      
   ==========================================================================   
      
    Print   
      
    Email   
      
    Share   
   ==========================================================================   
   ****** 1 ****** ***** 2 ***** **** 3 ****   
   *** 4 *** ** 5 ** Breaking this hour   
   ==========================================================================   
       * Screens_More_Versatile_Than_LED:_Fins_and_...   
      
       * GM_Pig_Heart_in_a_Human_Patient:_Update *   
       Multiple_Sclerosis_Severity * Wind_Farm_Noise_and_Road_Traffic_Noise   
       * Mavericks_and_Horizontal_Gene_Transfer *   
       Early_Reading_for_Pleasure:_Brains,_...   
      
       * New_Light_Shed_On_Evolution_of_Animals *   
       Gullies_On_Mars_from_Liquid_Meltwater?   
       * DNA_Organization_in_Real-Time *   
       How_the_Cat_Nose_Knows_What_It's_Smelling   
      
   Trending Topics this week   
   ==========================================================================   
   HEALTH_&_MEDICINE Personalized_Medicine Fitness Nervous_System   
   MIND_&_BRAIN K-12_Education Caregiving Intelligence LIVING_&_WELL Fitness   
   Staying_Healthy Nutrition   
      
      
   ==========================================================================   
      
   Strange & Offbeat   
   ==========================================================================   
   HEALTH_&_MEDICINE   
   Grocery_Store_Carts_Set_to_Help_Diagnose_Common_Heart_Rhythm_Disorder_and   
   Prevent_Stroke DNA_Can_Fold_Into_Complex_Shapes_to_Execute_New_Functions   
   Everyone's_Brain_Has_a_Pain_Fingerprint_--_New_Research_Has_Revealed_for_the   
   First_Time MIND_&_BRAIN   
   Scientists_Discover_Spiral-Shaped_Signals_That_Organize_Brain_Activity   
   Illusions_Are_in_the_Eye,_Not_the_Mind   
   Long_Missions,_Frequent_Travel_Take_a_Toll_on_Astronauts'_Brains   
   LIVING_&_WELL Amputees_Feel_Warmth_in_Their_Missing_Hand   
   Why_Do_Champagne_Bubbles_Rise_the_Way_They_Do?_Scientists'_New_Discovery_Is   
   Worthy_of_a_Toast 'Gluing'_Soft_Materials_Without_Glue Story Source:   
   Materials provided by Technische_Universita"t_Dresden. Note: Content   
   may be edited for style and length.   
      
      
   ==========================================================================   
   Journal Reference:   
      1. Stephen Gilbert, Hugh Harvey, Tom Melvin, Erik Vollebregt,   
      Paul Wicks.   
      
         Large language model AI chatbots require approval as medical   
         devices.   
      
         Nature Medicine, 2023; DOI: 10.1038/s41591-023-02412-6   
   ==========================================================================   
      
   Link to news story:   
   https://www.sciencedaily.com/releases/2023/07/230703133029.htm   
      
   --- up 1 year, 18 weeks, 10 hours, 50 minutes   
    * Origin: -=> Castle Rock BBS <=- Now Husky HPT Powered! (1:317/3)   
   SEEN-BY: 15/0 106/201 114/705 123/120 153/7715 218/700 226/30 227/114   
   SEEN-BY: 229/110 112 113 307 317 400 426 428 470 664 700 291/111 292/854   
   SEEN-BY: 298/25 305/3 317/3 320/219 396/45 5075/35   
   PATH: 317/3 229/426