I am using the API for proofreading and it hits it maximum limits very quickly, which imo means it can’t be used to read through a text of say 2000 words and proofread it.
Am I doing something wrong or is this just the way it is now?
There can be different reasons for this, could you show us your code?
def proofread_text(text):
openai.api_key = OPENAI_API_KEY
response = openai.ChatCompletion.create(
model="gpt-4",
messages=[
{
"role": "system",
"content": PROMPT
},
{
"role": "user",
"content": text
}
],
temperature=1,
max_tokens=1000
)
text = response.choices[0].message.content
if re.search(r'\[([\s\S]*?)\]', text):
return re.search(r'\[([\s\S]*?)\]', text).group(1)
else:
return text
1 Like
Thanks mate!
Here’s your problem:
The max tokens parameter is for both your input and the response, try setting this to somewhere between 4-6000 and see if that solves your problem 
1 Like
Nope, that’s not the issue, the issue is the max 10.000 pr. minute:
Rate limit reached for 10KTPM-200RPM in organization org-*********** on tokens per min. Limit: 10000 / min. Please try again in 6ms. Contact us through our help center at help.openai.com if you continue to have issues.
If anything, it is the other way around. Requesting too many tokens. It’s not based on actual tokens used, but only the requested tokens.
I see the problem you are having, you really can’t even make two long calls in the same minute with the current token limits.
What you can do is completely omit the max_tokens parameter, and then that value won’t be immediately used against your rate limit remaining to block you. Then the AI will use all the remaining context length - if it wants to.
Here’s what I was preparing before you gave more information, also true.
gpt-4 is giving very unsatisfactory answers and output these days. You can ask all you want for something long, and you get about 500 tokens.
It’s the model training doing you wrong. They want it to make brief outputs.
Language like “rewrite”, “improve”, “clarify” or such give the AI opportunity to mash your text to meaninglessness. Take one of the American classics and just specify a modern style rewrite? You get 3300 tokens into 1000.
Let’s check in again with how gpt-4 does when instructed distinctly not to reduce the length or change any sentences beyond the writing within each sentence. The instruction also prohibits uses we might want. Satisfactory output costing me $0.20.
system: You are a helpful writing assistant.
Instruction: improve the quality by using more common, modern, and clear words. Do not omit sentences or alter structure of composition otherwise.
3300 tokens of Mark Twain input.
In compliance with the request of a friend of mine, who wrote me from the East, I called on good-natured, garrulous old Simon Wheeler, and inquired after my friend’s friend, Leonidas W. Smiley, as requested to do, and I hereunto append the result. I have a lurking suspicion that Leonidas W. Smiley is a myth that my friend never knew such a personage; and that he on conjectured that if I asked old Wheeler about him, it would remind him of his infamous Jim Smiley, and he would go to work and bore me to death with some exasperating reminiscence him as long and as tedious as it should be useless to me. If that was the design, it succeeded.
I found Simon Wheeler dozing comfortably by the bar-room stove of the dilapidated tavern in the decayed mining camp Angel’s, and I noticed that he was fat and bald-headed, and had an expression of winning gentleness and simplicity upon his tranquil countenance. He roused up, and gave me good day. I told him that a friend of mine had commissioned me to make some inquiries about a cherished companion of his boyhood named Leonidas W. Smiley—Rev. Leonidas W. Smiley, a young minister of the Gospel, who he had heard was at one time resident of Angel’s Camp. I added that if Mr. Wheeler could tell me anything about this Rev. Leonidas W. Smiley, I would feel under many obligations to him.
Simon Wheeler backed me into a corner and blockaded me there with his chair, and then sat down and reeled off the monotonous narrative which follows this paragraph. He never smiled he never frowned, he never changed his voice from the gentle flowing key to which he tuned his initial sentence, he never betrayed the slightest suspicion of enthusiasm; but all through the interminable narrative there ran a vein of impressive earnestness and sincerity, which showed me plainly that, so far from his imagining that there was anything ridiculous or funny about his story, he regarded it as a really important matter, and admired its two heroes as men of transcendent genius in ‘finesse.’ I let him go on in his own way, and never interrupted him once.
"Rev. Leonidas W. H’m, Reverend Le—well, there was a feller here, once by the name of Jim Smiley, in the winter of '49—or maybe it was the spring of '50—I don’t recollect exactly, somehow, though what makes me think it was one or the other is because I remember the big flume warn’t finished when he first come to the camp; but anyway, he was the curiousest man about always betting on anything that turned up you ever see, if he could get anybody to bet on the other side; and if he couldn’t he’d change sides. Any way that suited the other man would suit him any way just so’s he got a bet, he was satisfied. But still he was lucky, uncommon lucky; he most always come out winner. He was always ready and laying for a chance; there couldn’t be no solit’ry thing mentioned but that feller’d offer to bet on it, and take any side you please, as I was just telling you.
p031.jpg (27K)
If there was a horse-race, you’d find him flush or you’d find him busted at the end of it; if there was a dog-fight, he’d bet on it; if there was a cat-fight, he’d bet on it; if there was a chicken-fight, he’d bet on it; why, if there was two birds setting on a fence, he would bet you which one would fly first; or if there was a camp-meeting, he would be there reg’lar to bet on Parson Walker, which he judged to be the best exhorter about here, and so he was too, and a good man. If he even see a straddle-bug start to go anywheres, he would bet you how long it would take him to get to—to wherever he was going to, and if you took him up, he would foller that straddle-bug to Mexico but what he would find out where he was bound for and how long he was on the road. Lots of the boys here has seen that Smiley, and can tell you about him. Why, it never made no difference to him—he’d bet on any thing—the dangdest feller. Parson Walker’s wife laid very sick once, for a good while, and it seemed as if they warn’t going to save her; but one morning he come in, and Smiley up and asked him how she was, and he said she was considerable better—thank the Lord for his inf’nite mercy—and coming on so smart that with the blessing of Prov’dence she’d get well yet; and Smiley, before he thought, says, ‘Well, I’ll resk two-and-a-half she don’t anyway.’
"Thish-yer Smiley had a mare—the boys called her the fifteen-minute nag, but that was only in fun, you know, because of course she was faster than that—and he used to win money on that horse, for all she was so slow and always had the asthma, or the distemper, or the consumption, or something of that kind. They used to give her two or three hundred yards’ start, and then pass her under way; but always at the fag end of the race she get excited and desperate like, and come cavorting and straddling up, and scattering her legs around limber, sometimes in the air, and sometimes out to one side among the fences, and kicking up m-o-r-e dust and raising m-o-r-e racket with her coughing and sneezing and blowing her nose—and always fetch up at the stand just about a neck ahead, as near as you could cipher it down.
"And he had a little small bull-pup, that to look at him you’d think he warn’t worth a cent but to set around and look ornery and lay for a chance to steal something. But as soon as money was up on him he was a different dog; his under-jaw’d begin to stick out like the fo’castle of a steamboat, and his teeth would uncover and shine like the furnaces. And a dog might tackle him and bully-rag him, and bite him, and throw him over his shoulder two or three times, and Andrew Jackson—which was the name of the pup—Andrew Jackson would never let on but what he was satisfied, and hadn’t expected nothing else—and the bets being doubled and doubled on the other side all the time, till the money was all up; and then all of a sudden he would grab that other dog jest by the j’int of his hind leg and freeze to it—not chaw, you understand, but only just grip and hang on till they throwed up the sponge, if it was a year. Smiley always come out winner on that pup, till he harnessed a dog once that didn’t have no hind legs, because they’d been sawed off in a circular saw, and when the thing had gone along far enough, and the money was all up, and he come to make a snatch for his pet holt, he see in a minute how he’d been imposed on, and how the other dog had him in the door, so to speak, and he 'peared surprised, and then he looked sorter discouraged-like and didn’t try no more to win the fight, and so he got shucked out bad. He give Smiley a look, as much as to say his heart was broke, and it was his fault, for putting up a dog that hadn’t no hind legs for him to take holt of, which was his main dependence in a fight, and then he limped off a piece and laid down and died. It was a good pup, was that Andrew Jackson, and would have made a name for hisself if he’d lived, for the stuff was in him and he had genius—I know it, because he hadn’t no opportunities to speak of, and it don’t stand to reason that a dog could make such a fight as he could under them circumstances if he hadn’t no talent. It always makes me feel sorry when I think of that last fight of his’n, and the way it turned out.
"Well, thish-yer Smiley had rat-tarriers, and chicken cocks, and tomcats and all them kind of things, till you couldn’t rest, and you couldn’t fetch nothing for him to bet on but he’d match you. He ketched a frog one day, and took him home, and said he cal’lated to educate him; and so he never done nothing for three months but set in his back yard and learn that frog to jump. And you bet you he did learn him, too. He’d give him a little punch behind, and the next minute you’d see that frog whirling in the air like a doughnut—see him turn one summerset, or maybe a couple, if he got a good start, and come down flat-footed and all right, like a cat. He got him up so in the matter of ketching flies, and kep’ him in practice so constant, that he’d nail a fly every time as fur as he could see him. Smiley said all a frog wanted was education, and he could do ‘most anything—and I believe him. Why, I’ve seen him set Dan’l Webster down here on this floor—Dan’l Webster was the name of the frog—and sing out, ‘Flies, Dan’l, flies!’ and quicker’n you could wink he’d spring straight up and snake a fly off’n the counter there, and flop down on the floor ag’in as solid as a gob of mud, and fall to scratching the side of his head with his hind foot as indifferent as if he hadn’t no idea he’d been doin’ any more’n any frog might do. You never see a frog so modest and straightfor’ard as he was, for all he was so gifted. And when it come to fair and square jumping on a dead level, he could get over more ground at one straddle than any animal of his breed you ever see. Jumping on a dead level was his strong suit, you understand; and when it come to that, Smiley would ante up money on him as long as he had a red. Smiley was monstrous proud of his frog, and well he might be, for fellers that had traveled and been everywheres all said he laid over any frog that ever they see.
"Well, Smiley kep’ the beast in a little lattice box, and he used to fetch him down-town sometimes and lay for a bet. One day a feller—a stranger in the camp, he was—come acrost him with his box, and says:
"‘What might it be that you’ve got in the box?’
"And Smiley says, sorter indifferent-like, ‘It might be a parrot, or it might be a canary, maybe, but it ain’t—it’s only just a frog.’
"And the feller took it, and looked at it careful, and turned it round this way and that, and says, 'H’m—so 'tis. Well, what’s HE good for.
"‘Well,’ Smiley says, easy and careless, 'he’s good enough for one thing, I should judge—he can outjump any frog in Calaveras County.
"The feller took the box again, and took another long, particular look, and give it back to Smiley, and says, very deliberate, ‘Well,’ he says, ‘I don’t see no pints about that frog that’s any better’n any other frog.’
"‘Maybe you don’t,’ Smiley says. 'Maybe you understand frogs and maybe you don’t understand ‘em; maybe you’ve had experience, and maybe you ain’t only a amature, as it were. Anyways, I’ve got my opinion, and I’ll resk forty dollars the he can outjump any frog in Calaveras County.’
"And the feller studied a minute, and then says, kinder sad-like, 'Well, I’m only a, stranger here, and I ain’t got no frog; but if I had a frog, I’d bet you.
"And then Smiley says, ‘That’s all right—that’s all right if you’ll hold my box a minute, I’ll go and get you a frog.’ Any so the feller took the box, and put up his forty dollars along with Smiley’s, and set down to wait.
"So he set there a good while thinking and thinking to himself and then he got the frog out and prized his mouth open and took a teaspoon and filled him full of quail-shot-filled him pretty near up to his chin—and set him on the floor. Smiley he went to the swamp and slopped around in the mud for a long time, and finally he ketched a frog, and fetched him in, and give him to this feller and says:
"‘Now, if you’re ready, set him alongside of Dan’l, with his fore paws just even with Dan’l’s, and I’ll give the word.’ Then he says, ‘One-two-three—git’ and him and the feller touches up the frogs from behind, and the new frog hopped off lively but Dan’l give a heave, and hysted up his shoulders—-so-like a Frenchman, but it warn’t no use—he couldn’t budge; he was planted as solid as a church, and he couldn’t no more stir than if he was anchored out. Smiley was a good deal surprised, and he was disgusted too, but he didn’t have no idea what the matter was of course.
"The Teller took the money and started away; and when he was going out at the door, he sorter jerked his thumb over his shoulder—so—at Dan’l, and says again, very deliberate, ‘Well,’ he says, ‘I don’t see no pints about that frog that’s any better’n any other frog.’
“Smiley he stood scratching his head and looking down at Dan’l a long time, and at last he says, 'I do wonder what in the nation that frog throw’d off for—I wonder if there ain’t something the matter with him—he ‘pears to look mighty baggy, somehow.’ And he ketched Dan’l by the nap of the neck, and hefted him, and says, ‘Why blame my cats if he don’t weigh five pound!’ and turned him upside down and he belched out a double handful of shot. And then he see how it was, and he was the maddest man—he set the frog down and took out after that feller, but he never ketched him. And—”
p035.jpg (39K)
[Here Simon Wheeler heard his name called from the front yard, and got up to see what was wanted.] And turning to me as he moved away, he said: “Just set where you are, stranger, and rest easy—I ain’t going to be gone a second.”
But, by your leave, I did not think that a continuation of the history of the enterprising vagabond Jim Smiley would be likely to afford me much information concerning the Rev. Leonidas W. Smiley, and so I started away.
At the door I met the sociable Wheeler returning, and he buttonholed me and recommenced:
“Well, thish-yer Smiley had a yaller one-eyed cow that didn’t have no tail, only just a short stump like a bannanner, and—”
However, lacking both time and inclination, I did not wait to hear about the afflicted cow, but took my leave.
results at max_tokens 4000? It managed to shed over 450 tokens from the output.
Twain - modernized by AI
Following a friend’s request from the East, I visited the friendly, talkative old Simon Wheeler to ask about my friend’s acquaintance, Leonidas W. Smiley. I have a sneaking suspicion that Leonidas W. Smiley is a fabrication, a character my friend invented, thinking that if I asked Wheeler about him, it would remind him of his notorious Jim Smiley, leading him to regale me with a long, tedious story about him. If that was the plan, it worked.
I found Simon Wheeler comfortably dozing by the bar-room stove in the rundown tavern of the abandoned mining camp, Angel’s. He was a plump, bald man with a gentle, simple expression on his peaceful face. He woke up and greeted me. I told him that a friend had asked me to inquire about a childhood friend of his named Leonidas W. Smiley—Rev. Leonidas W. Smiley, a young minister who was once a resident of Angel’s Camp. I added that if Mr. Wheeler could tell me anything about this Rev. Leonidas W. Smiley, I would be very grateful.
Simon Wheeler cornered me with his chair, sat down, and began the monotonous story that follows. He never smiled, frowned, or changed his voice from the gentle tone he started with. He never showed any excitement; but throughout the long story, there was a vein of earnestness and sincerity, which made it clear that he saw his story as important and admired its two heroes as men of extraordinary cunning. I let him continue in his own way, without interrupting him once.
"Rev. Leonidas W. H’m, Reverend Le—well, there was a guy here, once named Jim Smiley, in the winter of '49—or maybe it was the spring of '50—I don’t remember exactly. But anyway, he was the most peculiar man, always betting on anything that came up, if he could find someone to bet against. If he couldn’t, he’d switch sides. Any way that suited the other person would suit him, as long as he got a bet, he was happy. But he was lucky, very lucky; he almost always won. He was always ready and waiting for a chance; there wasn’t a single thing mentioned that he wouldn’t offer to bet on, and take any side you like, as I was just telling you.
If there was a horse race, you’d find him either rich or broke at the end of it; if there was a dog fight, he’d bet on it; if there was a cat fight, he’d bet on it; if there was a chicken fight, he’d bet on it; even if there were two birds sitting on a fence, he would bet you which one would fly first. If he even saw a bug start to go anywhere, he would bet you how long it would take it to get to wherever it was going, and if you took him up on it, he would follow that bug to Mexico if he had to, just to find out where it was going and how long it took. Many of the guys here have seen that Smiley and can tell you about him. It didn’t matter to him—he’d bet on anything—the strangest guy. Parson Walker’s wife was very sick once, for a long time, and it seemed as if they weren’t going to save her; but one morning he came in, and Smiley asked him how she was, and he said she was much better—thank the Lord for his infinite mercy—and improving so quickly that with God’s help she’d recover; and Smiley, without thinking, says, ‘Well, I’ll risk two-and-a-half she doesn’t anyway.’
This Smiley had a mare—the guys called her the fifteen-minute nag, but that was just a joke, because of course she was faster than that—and he used to win money on that horse, even though she was so slow and always had asthma, or distemper, or consumption, or something like that. They used to give her a two or three hundred yards’ head start, and then pass her on the way; but always at the end of the race she would get excited and desperate, and come galloping and straddling up, and scattering her legs around, sometimes in the air, and sometimes out to one side among the fences, and kicking up more dust and making more noise with her coughing and sneezing and blowing her nose—and always end up at the stand just about a neck ahead, as close as you could figure it.
And he had a little bull-pup, that to look at him you’d think he wasn’t worth a cent but to sit around and look mean and wait for a chance to steal something. But as soon as money was bet on him he was a different dog; his lower jaw would begin to stick out like the front of a steamboat, and his teeth would uncover and shine like furnaces. And a dog might attack him and bully him, and bite him, and throw him over his shoulder two or three times, and Andrew Jackson—which was the name of the pup—Andrew Jackson would never let on but what he was satisfied, and hadn’t expected anything else—and the bets being doubled and doubled on the other side all the time, until all the money was up; and then all of a sudden he would grab that other dog just by the joint of his hind leg and hold on to it—not chew, you understand, but just grip and hang on until they gave up, even if it took a year. Smiley always came out winner on that pup, until he bet on a dog once that didn’t have any hind legs, because they’d been cut off in a circular saw, and when the thing had gone far enough, and all the money was up, and he went to grab for his favorite hold, he saw in a minute how he’d been tricked, and how the other dog had him beat, so to speak, and he looked surprised, and then he looked kind of discouraged and didn’t try anymore to win the fight, and so he lost badly. He gave Smiley a look, as if to say his heart was broken, and it was his fault, for betting on a dog that didn’t have any hind legs for him to grab, which was his main strategy in a fight, and then he limped off a bit and laid down and died. He was a good pup, that Andrew Jackson, and would have made a name for himself if he’d lived, because he had talent—I know it, because he didn’t have many opportunities, and it doesn’t make sense that a dog could put up such a fight as he did under those circumstances if he didn’t have talent. It always makes me feel sorry when I think of that last fight of his, and the way it turned out.
Well, this Smiley had rat terriers, and roosters, and tomcats and all kinds of things, until you couldn’t rest, and you couldn’t bring anything for him to bet on but he’d match you. He caught a frog one day, and took him home, and said he planned to train him; and so he did nothing for three months but sit in his backyard and teach that frog to jump. And you bet he did teach him, too. He’d give him a little nudge from behind, and the next minute you’d see that frog spinning in the air like a doughnut—see him do a somersault, or maybe two, if he got a good start, and land flat-footed and perfectly, like a cat. He trained him so well in catching flies, and kept him in practice so constantly, that he’d catch a fly every time as far as he could see it. Smiley said all a frog needed was education, and he could do almost anything—and I believe him. Why, I’ve seen him put Dan’l Webster down here on this floor—Dan’l Webster was the name of the frog—and shout out, ‘Flies, Dan’l, flies!’ and quicker than you could blink he’d jump straight up and snatch a fly off the counter there, and flop down on the floor again as solid as a lump of mud, and start scratching the side of his head with his hind foot as if he hadn’t done anything more than any frog might do. You never saw a frog so modest and straightforward as he was, despite his talent. And when it came to jumping on a flat surface, he could cover more ground in one leap than any animal of his kind you ever saw. Jumping on a flat surface was his specialty, you understand; and when it came to that, Smiley would bet money on him as long as he had a cent. Smiley was extremely proud of his frog, and he had every right to be, because guys who had traveled and been everywhere all said he was better than any frog they’d ever seen.
Well, Smiley kept the creature in a small lattice box, and he used to bring him downtown sometimes and look for a bet. One day a guy—a stranger in the camp—came across him with his box, and asked:
‘What might you have in the box?’
And Smiley said, kind of nonchalantly, ‘It might be a parrot, or it might be a canary, maybe, but it isn’t—it’s just a frog.’
And the guy took it, and looked at it carefully, and turned it this way and that, and said, 'H’m—so it is. Well, what’s HE good for?
‘Well,’ Smiley said, casually, 'he’s good enough for one thing, I should think—he can outjump any frog in Calaveras County.
The guy took the box again, and took another long, careful look, and gave it back to Smiley, and said, very slowly, ‘Well,’ he said, ‘I don’t see anything special about that frog that’s better than any other frog.’
‘Maybe you don’t,’ Smiley said. ‘Maybe you understand frogs and maybe you don’t; maybe you’ve had experience, and maybe you’re just a beginner, as it were. Anyway, I’ve got my opinion, and I’ll risk forty dollars that he can outjump any frog in Calaveras County.’
And the guy thought for a minute, and then said, kind of sadly, ‘Well, I’m just a stranger here, and I don’t have a frog; but if I had a frog, I’d bet you.’
And then Smiley said, ‘That’s all right—that’s all right if you’ll hold my box for a minute, I’ll go and get you a frog.’ So the guy took the box, and put up his forty dollars along with Smiley’s, and sat down to wait.
So he sat there for a while thinking to himself, and then he took the frog out and pried his mouth open and took a teaspoon and filled him full of birdshot—filled him almost up to his chin—and set him on the floor. Smiley went to the swamp and sloshed around in the mud for a long time, and finally he caught a frog, and brought him in, and gave him to this guy and said:
‘Now, if you’re ready, set him next to Dan’l, with his front paws just even with Dan’l’s, and I’ll give the word.’ Then he said, ‘One-two-three—go’ and he and the guy nudged the frogs from behind, and the new frog hopped off energetically but Dan’l gave a heave, and hunched up his shoulders—like a Frenchman, but it was no use—he couldn’t move; he was as solid as a rock, and he couldn’t budge any more than if he was anchored. Smiley was very surprised, and he was disgusted too, but he had no idea what the problem was, of course.
The guy took the money and started to leave; and as he was going out the door, he sort of jerked his thumb over his shoulder—at Dan’l, and said again, very slowly, ‘Well,’ he said, ‘I don’t see anything special about that frog that’s better than any other frog.’
Smiley stood scratching his head and looking down at Dan’l for a long time, and finally he said, ‘I do wonder why that frog jumped off—I wonder if there’s something wrong with him—he looks kind of bloated, somehow.’ And he grabbed Dan’l by the back of the neck, and lifted him, and said, ‘I’ll be damned if he doesn’t weigh five pounds!’ and turned him upside down and he belched out a double handful of shot. And then he saw how it was, and he was the angriest man—he put the frog down and took off after that guy, but he never caught him. And—"
[Here Simon Wheeler heard his name called from the front yard, and got up to see what was wanted.] And as he walked away, he said: “Just stay where you are, stranger, and rest easy—I won’t be gone a second.”
But, with your permission, I didn’t think that continuing the story of the adventurous scoundrel Jim Smiley would give me much information about the Rev. Leonidas W. Smiley, so I left.
At the door I ran into the friendly Wheeler coming back, and he cornered me and started again:
“Well, this Smiley had a yellow, one-eyed cow that didn’t have a tail, just a short stump like a banana, and—”
However, not having the time or the inclination, I didn’t wait to hear about the unfortunate cow, but took my leave.
( gpt-4-0613, 1 request - 3,346 prompt + 2,833 completion = 6,179 tokens )
Input paragraph from body:
"‘Now, if you’re ready, set him alongside of Dan’l, with his fore paws just even with Dan’l’s, and I’ll give the word.’ Then he says, ‘One-two-three—git’ and him and the feller touches up the frogs from behind, and the new frog hopped off lively but Dan’l give a heave, and hysted up his shoulders—-so-like a Frenchman, but it warn’t no use—he couldn’t budge; he was planted as solid as a church, and he couldn’t no more stir than if he was anchored out. Smiley was a good deal surprised, and he was disgusted too, but he didn’t have no idea what the matter was of course.
Output
‘Now, if you’re ready, set him next to Dan’l, with his front paws just even with Dan’l’s, and I’ll give the word.’ Then he said, ‘One-two-three—go’ and he and the guy nudged the frogs from behind, and the new frog hopped off energetically but Dan’l gave a heave, and hunched up his shoulders—like a Frenchman, but it was no use—he couldn’t move; he was as solid as a rock, and he couldn’t budge any more than if he was anchored. Smiley was very surprised, and he was disgusted too, but he had no idea what the problem was, of course.
So: you have a prompt technique you can use on the AI also - if you don’t actually want it to rewrite anything.
2 Likes
Unfotunately, GPT4 is also simply much worse at writing than Claude2, so even if GPT has an API, I really can’t use it for that purpose. Not good enough. Claude2 >>> GPT4 when it comes to writing and language.
I might be able to use GPT4 for simple classification tasks.
Then again, Claude2 has also begun giving shorter answers.
2000 words? No way my bot would not answer with that many words and I don’t care how many tokens I give it.
You must chunk your input or forget it. You have to map the additional words to the user question and they better be relevant.
Someone will build a gigantic chunking machine one day inside of the API
Until then small bytes of relevant information will be successful
Too much and you will destroy the bot
Building keyword mapping works you don’t need vector analysis just map the content using keywords and limit your content to a handful of relevant data
1 Like
The above is quite impenetrable, and gives nothing to advise the thread topic.
I only use small bytes anyway, never big bytes.
1 Like
This is exactly on point. You’ll have to break your content up into chunks and proofread each chunk. If there are things that you want it to look out for, like tone, simplifying words, succinctness, then make sure you describe the type of proofreading you want done.
10000 tokens/minute? oof, OpenAI is getting stingy with their rate limits. Mine started at 20000/minute and recently was bumped up. If you use the API consistently, they will slowly raise the limit.
I ran into the same issue at 20000/min in that I could not send my requests one at a time without exceeding the rate limit. I had to put in an 8 second delay between calls to stay under the limit. At 10k/min, you might need a delay that ensures no more than 1.5 calls/minute. The API call return time is mostly a function of how many output tokens the LLM is generating.
In terms of writing quality, it is obviously highly dependent on the instructions/prompts provided. For writing a new prompt, I like to practice a lot using ChatGPT+ and custom instructions, and I’ll run through 100-150 iterations before I start testing with the API. I know it doesn’t behave the same way, but GPT-4 calls are too costly and slow for certain use cases, particularly where there is a large amount of input tokens.
1 Like
I got it to work by just lowering tokens to 300.
It doesnt go by actual used tokens but by requested tokens.
The problem is that chunking removes context, which makes GPT much worse and turns it into more of a Word spell check or Google Translate.