Articles summarizer using API

[anton.mislawsky]

Hi, I write aggregator of news feed on some themes. Let say finance markets. I parse 20 news website and want to have from 100 articles ten at the end of a day. I use python and API ChatGPT. ChatGPT Web version works great for this purpose, but

how I can pass 10-20 article and get one summary using API?

[jochenschultz]

Hi Anton,

welcome to the developer community.

'OpenAI' object has no attribute 'Completion'

Here is a basic example: from openai import OpenAI client = OpenAI() PROMPT = "give me a random poem" def call_openai_api(): response = client.chat.completions.create( model="gpt-4", messages=[{"role": "user", "content": PROMPT}], temperature=1, max_tokens=300, top_p=1, frequency_penalty=0, presence_penalty=0 ) print(response.choices[0].message.content.strip()) if __name__ == "__main__": call_openai_api() save it as …

You need to change the prompt in that example to something like:

TLTR:

[ARTICLE1]

[ARTICLE2]

[ARTICLE3]

...

Replace the [ARTICLEX] blocks with the actual content of the articles.

And you might want to change the max_token value to something bigger than 300 except you want a very short summary

P.S. TLTR = Too long to read…

[platypus]

Hi @anton.mislawsky and welcome the community!

One potential bottleneck here is how big the articles are, and especially if there is quite a significant amount of superfluous HTML content. So some kind of hierarchical handling may be needed.

Something you may want to try that handles most of this for you, is using LangChain (open source LLM wrapper library), pass it your API key, and use MapReduce summarizer. This will give you exactly what you want.

[AdamSobieski]

Interesting topics. In addition to multi-document summarization, one might find interesting the related interdisciplinary topics of: multiple-text comprehension, synthesis writing, and related strategies.