490
The DOJ is using fake "AI" case citations to justify ICE detainment (europe.pub)
submitted 3 months ago* (last edited 3 months ago) by to c/fuck_ai@lemmy.world
45 comments fedilink hide all child comments
 

cross-posted from: https://infosec.pub/post/49593520

Source

https://storage.courtlistener.com/recap/gov.uscourts.miwd.120609/gov.uscourts.miwd.120609.11.0.pdf

you are viewing a single comment's thread
view the rest of the comments
[–] 7 points 3 months ago (17 children)

Wow, this is lazy. I feel like you could probably set up an AI agent thing to check every case it references to make sure it is both real and relevant. But they were so lazy not only did they not check it themselves, they didn't even have a clanker check it.

  • permalink
  • fedilink
  • source
  • hideshow 17 child comments
    •
  • [–] [S] 19 points 3 months ago* (last edited 3 months ago) (6 children)

    I don't think LLMs are great at figuring out if something is written by an LLM, or factual.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 1 point 3 months ago* (2 children)

    I wouldn't have it do that. I'm thinking more along the lines of an AI that drafts legal documents.

    • It would find a reference to a case, something an ai is good at.
    • Then it would send out a query to the database to check if that case exists, something you would need to build some tooling for.
    • If it retrieves anything it would then check if the source and the document agree
    • if not, it would run a loop to re-write that section.

    I'm not like an agent engineer or anything like that, and I'm sure there's more efficient ways of having an LLM draft legal documents. But so long as you can get it to reference primary sources there should be a lot less cleanup work to do.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -1 points 3 months ago (2 children)

    You dont need to - you just have to make sure you set up one or two to find holes. Make it 50 points per hole, and 1000 bonus if all is proved wrong. Make two compete and let them know

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 13 points 3 months ago (6 children)

    Asking a chatbot to stop lying is about as useful as asking a river to stop flowing. It won't happen unless you do it yourself.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] -1 points 3 months ago (5 children)

    I lay out how I would do it in another comment. I think it would be pretty easy to get something rough draft quality pretty easy. The difficult problem is compliance, you would probably have to use either local or legal work approved LLMs. But I'm saying they actually just used a chat bot instead of an agent that could give you better results. It's pretty easy to get a clanker not to lie, use it as a tool to rearrange text and don't let it be creative.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 6 points 3 months ago* (last edited 3 months ago) (1 child)

    The law is a hundreds of billions of dollar annual industry, just in the US. It was worth 367 billion just in 2022. An AI that flawlessly checks over a legal brief would probally be worth billions/year because that means it could also write them flawlessly. Since its so easy, you should probally whip it up and make your billions.

    Hell, you seem to have a way to solve chatbots persistent error rate that trillion dollar companies can't fix, so you might actually have a trillion dollar answer. Why stop at just law if "its so easy to make a LLM not lie?"

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 3 months ago

    It's a saturated business, there's dozens of companies already doing it. I don't like AI, I think it's destroying a lot of parts of our life. But it's here and it's only going to sink it's claws deeper. Also I'm not describing chat bots.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 5 points 3 months ago* (2 children)

    And I'm saying that lying is inherent to the technology. Generative AI uses probability to generate output. It's constantly rolling dice when determining output. Any data fed into it is used to calculate odds on what words should follow other words.

    It's like making a smoothie and then trying to consume specific vitamins. No matter how good your smoothie making machine is, that's gonna be an impossible task because blending everything together is inherent to making a smoothie. You'd be much better served just eating individual bananas to get your potassium intake.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 3 months ago* (last edited 3 months ago) (1 child)

    And I agree with you. When you let an AI be creative it will lie. But if they are reading documents they generally don't. That information is now a part of their context and it will reuse that information. It's a good fuzzy search. It is however terrible at a lot of other things.

    Edit: it's like a plagerism machine! Which in this one case is probably good

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 3 months ago

    If you feed a chatbot the sentences, "Bananas are a good source of potassium. Bananas are not a good source of protein." There is a good chance it will spit out the phrase, "Bananas are a good source of protein." because the phrase "good source of protein" is in the original data it was fed. There will always be a significant chance of spitting out this phrase.

    No matter how much you improve the AI, because it uses probabilistic generation, it will always be cobbling together sentences without any regard for what the overall sentence means. An intern will eventually learn case law, an AI agent will never be trustworthy. When you ask an AI agent a question, it will always lie to you about being certain of its answer because its answer is built upon a roll of the dice.

  • permalink
  • fedilink
  • source
  • parent