this post was submitted on 22 Apr 2025
12 points (80.0% liked)

Technology

69545 readers
3517 users here now

This is a most excellent place for technology news and articles.

Our Rules

  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago
MODERATORS
L3s@lemmy.world
enu@lemmy.world
technopagan@lemmy.world
L4s@lemmy.world
L3s@hackingne.ws
L4s@hackingne.ws
12
LLM Virology Capabilities Test (www.virologytest.ai)
submitted 1 week ago* (last edited 1 week ago) by zaxvenz@lemm.ee to c/technology@lemmy.world
1 comments fedilink hide all child comments
 

We present the Virology Capabilities Test (VCT), a large language model (LLM) benchmark that measures the capability to troubleshoot complex virology laboratory protocols. VCT is difficult: expert virologists with access to the internet score an average of 22.1% on questions specifically in their sub-areas of expertise. However, the most performant LLM, OpenAI's o3, reaches 43.8% accuracy and even outperforms 94% of expert virologists when compared directly on question subsets specifically tailored to the experts' specialites.

https://archive.ph/xILJR

top 1 comments
sorted by: hot top controversial new old
[–] themurphy@lemmy.ml 1 points 1 week ago

Great results. Would an AI build for this not be better, or is it just meant as a kind of benchmark for LLMs?