API is returning empty text on random

Prompting
1

Hi Community

Have any of you come across an API call where it returned empty text for no reason even though the call was successful? I have tried running the same prompt on the playground and it worked all the time. But when calling via API, it sometimes returns the empty text for no apparent reason. It seems to be happening on the Devinci-instruct engine only.

Thanks
Sandeep

1 Like
2

Hi @Not_Wrogn, could you share the API call you’re making?

This may happen when you have a trailing space or empty line at the end, or are immediately hitting a stop sequence due to incorrect prompt design.

2 Likes
4

Hi @joey and @m-a.schenk , thanks for getting back to me.

This is the output
For the same prompt and input:
When it works, this is what I get
1)
“{“id”: “cmpl-3Uo7BvodaapzDUUZzERRnDV1xV9Gd”, “object”: “text_completion”, “created”: 1628471493, “model”: “if-davinci-v2”, “choices”: [**{“text”: " chocolate, customise, deliver**”, “index”: 0, “logprobs”: null, “finish_reason”: “stop”}]}\n"

“{“id”: “cmpl-3Uo95c0HA4LSdj5jfKuKkIbEAJ4RI”, “object”: “text_completion”, “created”: 1628471611, “model”: “if-davinci-v2”, “choices”: **[{“text”: " we, customise, chocolate, deliver**”, “index”: 0, “logprobs”: null, “finish_reason”: “stop”}]}\n"

When it does not work, this is what I get.
1)
“{“id”: “cmpl-3UoBCqMrl2aUwPOxpzR7QODk6xYcV”, “object”: “text_completion”, “created”: 1628471742, “model”: “if-davinci-v2”, “choices”: [{“text”: “”, “index”: 0, “logprobs”: null, “finish_reason”: “stop”}]}\n”

Thanks
Sandeep

1 Like
6

Hi @m-a.schenk, thanks for your help. I did and this is what I get. How to read it?
“{“id”: “cmpl-3UqYGLw0zXwcB2VqJO8VuMuJirZ5G”, “object”: “text_completion”, “created”: 1628480860, “model”: “if-davinci-v2”, “choices”: [{“text”: " “, “index”: 0, “logprobs”: {“tokens”: [” “, “\n”, “\n”, “Text”, “:”, " “, “\n”, “Key”], “token_logprobs”: [-1.6254022, -0.007504819, -0.9521135, -2.268376, -0.0083111655, -2.369574, -0.030923532, -1.3950536], “top_logprobs”: [{” “: -1.6254022}, {”\n”: -0.007504819}, {”\n": -0.9521135}, {“Text”: -2.268376}, {":": -0.0083111655}, {" “: -2.369574}, {”\n": -0.030923532}, {“Key”: -1.3950536}], “text_offset”: [51, 52, 52, 52, 52, 52, 52, 52]}, “finish_reason”: “stop”}]}\n"

Thanks
Sandeep

1 Like
8

Hi @m-a.schenk, that is really interesting. You are right. I have changed the stop sequence to something else i.e. “###” and now it is fine. Thanks

Sandeep

2 Likes
9

Actually, changing the stop sequence didn’t work. I probably need to learn more on prompt design :frowning:

Thanks
Sandeep

1 Like
11

Thanks for your suggestion M A Schenk, I am looking at handling the “empty text” exception as well as playing around with logprobs as you suggested and observing the output. Hopefully, get some permanent fix or at least I will learn something new :slight_smile:

1 Like