We have been trying to create a prompt that applies the law to specific facts. Are there any specific techniques?
law is subjective to the culture of the populus and an innately human experience derived system. Ai does not currently have the capacity for any meaningful level of subjective experience and therefore cannot correctly articulate the law given the require nuance of law. it is highly advisable that ai be only used to gather generalised reference material of aspects related to the situation, while always having a human understand the contextual and subjective requirements of the situation and always having a human pass judgement regarding the totality of that situation. that being said i need to clarify your question, do you mean you are trying to decern aspects of law that relate to certain facts? or manipulate a fact to bend it around the law? or create a law regarding specific facts?
Hi Liam. Thanks for taking the time to comment my topic. We are trying to issue instructions as follows:
- we offer a specfic legal provision (or various) of a specific law;
- we then share some facts
- we offer the questions
And then we want create a prompt that applies the legal provision to the facts provided so that the output answers the question. So we do not want manipulate facts and bend them around the law and we also do not wish to create laws. Hope this helps clarify
ok so from what i understand of your intent, this is a hypothetical in creation that you’re trying to use as a base point to develop a more robust method for legal aid using ai, you are trying to determine whether something is legal or not regarding a certain provision of a specific law? that more or less sum it up?
i would try and avoid using ai in this way, but you can just ask someone else so i might as well help in a way that’s ethical.
so before I help, let me just explain, I know a decent amount about ai, not about the law, so I’m giving this advice from the perspective of ai knowledge.
I would take it from the angle of concept matching, ai is good with this and can be used in this way to not answer the question but frame it in a way that is readily digestible and easily answered by the proprietor of judgement.
try making the ai create a meta prompt that allows the making of lists of concepts associated with the provision of law and the facts provided and then outputting an alignment percentage or confidence value base on association between the two.
now just to be clear using ai to determine guilty or not guilty is inherently immoral as it removes the jury of peers and judgement of the wise completely from method of applying the law. just because the ai says 75% guilty doesnt mean that being in question is 75% guilty because ai cannot take into account nuance to the level required for real application of law.
Indeed. AI is more like an intern that helps you summarize and point some logical points.
But to judge a legal case, you can’t rely on summarization that could overlook subtle information that only a real person can notice.
Again, AI is helpful. It can point some obvious cases where a certain law applies. But ultimately, it is no more than an assistant, not the lawyer nor the judge itself.
That said, you can search about RAG and embeddings, that supposedly serves the purpose of providing context (laws and examples) for the prompt to be processed, with the previously stated limitations.
2 Likes
I think that, with supervision, an LLM could do well at certain aspects of this. If we restrict it to U.S. state criminal law for the sake of argument (and sanity), what do we have to work with?
-
At the root, a statute defining an offense by its elements, and detailing the degree of offense, any defenses available, and sentencing maximums, sometimes minimums.
-
Case law: Prior decisions in the same jurisdiction or from a higher court with binding authority over “this” court (at some point, this all has to be evaluated in context of a specific court, even if arbitrary).
-
Evidence and its admissibility.
-
Burden of proof: Innocence is presumed. To establish guilt, the prosecution must prove each element of the crime beyond a reasonable doubt.
Some of this might be within reach, but only to an extent, I think. On the surface, it may seem straightforward to iterate through the elements of a crime, but on deeper examination, it quickly becomes harder than it seems.
For reference, let’s look at robbery in the Texas Penal Code:
Sec. 29.02. ROBBERY. (a) A person commits an offense if, in the course of committing theft as defined in Chapter 31 and with intent to obtain or maintain control of the property, he:
(1) intentionally, knowingly, or recklessly causes bodily injury to another; or
(2) intentionally or knowingly threatens or places another in fear of imminent bodily injury or death.
(b) An offense under this section is a felony of the second degree.
In turn, we also need to know what theft is. Bear with me here:
Sec. 31.03. THEFT. (a) A person commits an offense if he unlawfully appropriates property with intent to deprive the owner of property.
(b) Appropriation of property is unlawful if:
(1) it is without the owner’s effective consent;
The section on theft continues for a good couple of pages or more, but it’s mostly to do with the nuances of buying stolen property and is immaterial to our example.
Now, to properly apply these statutes, we have to know what some of the words in them mean, according to the same penal code.
-
“Unlawfully”: What defines whether a given appropriation of property is unlawful? This is answered by Sec. 31.03(b):
Appropriation . . . is unlawful if: . . . "
-
“Appropriates”: What does it mean to appropriate property? The answer to this can be found in the definitions at the beginning of Chapter 31 (Title 7):
“Appropriate” means:
(A) to bring about a transfer or purported transfer of title to or other nonpossessory interest in property, whether to the actor or another; or
(B) to acquire or otherwise exercise control over property other than real property.So, basically, taking something, taking over something, or putting through paperwork that says something belongs to you.
-
“Property”: What exactly is property? This does, in fact, matter, and the word has a particular meaning here:
“Property” means:
(A) real property;
(B) tangible or intangible personal property including anything severed from land; or
(C) a document, including money, that represents or embodies anything of value.
Well, now, we sure have covered a lot of ground. So far we know that robbery is something you might do in the course of committing a theft. And we know that theft is unlawfully appropriating property.
Don’t relax just yet. That was the easy part.
A person commits an offense if he unlawfully appropriates property with intent to deprive the owner of property.
What is intent?
The plot doth thicken.
In the Texas Penal Code, Title 2 (General Principles of Criminal Responsibility), Chapter 6 (Culpability Generally):
Sec. 6.03. DEFINITIONS OF CULPABLE MENTAL STATES. (a) A person acts intentionally, or with intent, with respect to the nature of his conduct or to a result of his conduct when it is his conscious objective or desire to engage in the conduct or cause the result.
(This section goes on to define what it means to do something “knowingly”, “recklessly”, or “with criminal negligence”.)
So let’s say someone is charged with theft. And let’s say we’ve already established that the defendant:
- appropriated (because he physically took it with his hands)
- property (let’s say the thing he took was some cash (currency))
- unlawfully (because it was without the owner’s consent)
Note that we’re glossing over the question of who the owner is, how we know that for sure, and how we proved that the money was taken without the owner’s consent. There is only so much time in the day, after all.
Now how do we prove that the defendant appropriated the property unlawfully with the intent to deprive the owner of it?
Here we have to establish that it was the defendant’s conscious objective or desire to do so. I’ll spare you any further drilling down into this; simply naming that objective should be enough to illustrate how prohibitively complex it can be.
So let’s say we’ve proven that a theft did, in fact, take place. Now, was it a robbery?
To have a robbery, we must have a theft, and during that theft, the thief must have intentionally, knowingly, or recklessly harmed (“caused bodily injury” to) someone. To simplify, we’ll leave it at that.
Now we have several more things to deal with:
Did the thief hurt someone? Did it happen during the theft? Does “bodily injury” or “during” have a special definition here? Does it meet the standards of reckless, knowing, or intentional action per Title 2, Chapter 6?
And so on and so forth.
Once we’re clear of this damn near fractal maze of statutory elements of the alleged crime, then we still need to examine applicable case law for binding decisions that might be relevant to the facts of our case.
We also need to take into account the evidence we (as the prosecution) are presenting and whether it’s admissible according to the rules of evidence in Texas.
(Rest assured that the case law and the admissibility of evidence are no less complex, by any means, than what we’ve covered above.)
And, to meet our burden of proof, we’ll need to handle (preferably anticipate) the “reasonable doubt” that the defense might present and beyond which we must prove each and every element of the crime.
Anyway, sounds totally doable, looking forward to seeing the prototype! (-:
1 Like