Problem on EMBEDDED TEXT ADA 002 and FINE TUNED ON THE TRAINING DATA SET

Community
1

I’m GETTING DIFFERENT RESPONSES FROM THE GPT3 CHAT COMPLETION.
I was creating itinerary , but same hotels are coming ?
why

2

First, write a proper post explaining your problem. Give examples, show what you have done, what you were expecting to get, etc.

3

built AI bot for creating automatic itineraries on user prompt constraints like location , budget and vibes , by embedding our inventory of (hotels , restaurants and activities) , fine tuning davinci model using Open AI apis. The generated json response will link straight to booking operation on frontend. I fine tuned on prefilled itineraries and it is giving me exact that … not a new itinerary which is not present in automatic prefilled itinerary

4

Still confused: are you using embeddings AND fine-tuning to address your requirements? If so, you probably need to sketch the design.

Opinion: No one attempting this should do so without a requirements document that makes it easier for engineers to help you. Getting attention from some brilliant AI practitioners in this community is directly proportional to the information you share about your implementation challenges.

5

HOW TO RESOLVE THIS ?

AttributeError: module ‘tiktoken’ has no attribute ‘get_encoding’

docs = SimpleDirectoryReader(path).load_data()

print(docs)

llmp = LLMPredictor(
llm=OpenAI(
temperature=0,
model_name=“text-davinci-003”,
max_tokens=1000,
frequency_penalty=0.1,
presence_penalty=0.0,
)
)

max_input_size = 512
num_output = 512
max_chunk_overlap = 20
sizew = 600
prompt_helper = PromptHelper(max_input_size,max_chunk_overlap,chunk_size_limit=sizew)

service_context = ServiceContext.from_defaults(
llm_predictor=llmp, prompt_helper=prompt_helper
)
vectorINdex = GPTSimpleVectorIndex.from_documents(docs, service_context=service_context)

vectorINdex.save_to_disk(“dataemd.json”)

print(vectorINdex)

loader = DirectoryLoader(‘apiserver’, glob=“/media/rachit/New Volume/Untitled Folder 3/itinerary-planner-api/apiserver/newfile.txt”)

docs = loader.load()

char_text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)

doc_texts = char_text_splitter.split_documents(docs)

openAI_embeddings = OpenAIEmbeddings(openai_api_key=os.environ[‘OPENAI_API_KEY’])

vStore = Chroma.from_documents(doc_texts, openAI_embeddings)

model = VectorDBQA.from_chain_type(llm=OpenAI(), chain_type=“stuff”, vectorstore=vStore)