I want to create a legal gpt. Fine tuning would be necessary to teach the model of critical thinking. Retrieval would be necessary to provide legal expertise to the legal gpt.
For fine tuning purpose I hv structured the data as user-system conversation. Each conversation will consist of 1 landmark case law. I hv about 5000 such cases. Each case roughly being about 10000 words.
For retrieval purpose, I want to upload case laws and Indian statutes that are cited in the above mentioned 1500 fine tuning case laws. I hv about 25000 such case laws. Each case is about 500KB or about 25000 words.
First I ll feed the legal gpt with the 25000 case laws and Indian statutes for retrieval.
Second, I ll fine tune the model with 5000 case laws.
The fine tuning prompts will be structured as:
User: [‘Facts of the case from case law’]. Provide the legal issues in this case.
System: [‘Legal issues from the case law’].
User: provide arguments on behalf of petitioners for legal issue 1
System: [‘Petitioner argument for legal issue 1 from the case law’]
User: provide arguments on behalf of petitioners for legal issue
System: [‘petitioner arguments for legal issue 2 from the case law’]
…
…
…
User: provide arguments on behalf of respondents for legal issue 1
System: [‘Respondents argument for legal issue 1’]
…
…
User: provide the judicial principles established in this case
System: [‘Judicial principles established from the case law’]
Petitioner and respondents arguments, and judicial principles will mention and cite excerpts from landmark relevant case laws(data from RAG)
Fine tuning the model will learn of critical thinking as well how to effectively use the retrieval ability to cite statutes and landmark case laws in the petitioner arguments and respondents arguments.
Is my strategy right?
What should I use for retrieval- embeddings or retrieval assistant or plugin?