Summarizing News From RSS Feeds

Making news summaries with LLMs based on RSS feeds sounds like a reasonable step toward having a healthy information diet and curating your own news. I have built a static site on S3 that displays summaries created with Groq API LLMs. I’ll walk you through the steps to build your own.

Design

I created the backend using AWS Step Functions, which are triggered every hour by an EventBridge rule. The first step is parsing all of the RSS feeds and storing this data into DynamoDB.

I used MapReduce to generate news summaries. In the map step, I rank each story to determine if the given news is relevant and extract the main theme and tags. The reduce step groups enriched articles from the map step into clusters with a title, short summary, and article references.

The frontend calls the API gateway to fetch the clusters from DynamoDB. The site supports Google login and per-user filtering based on tags.

UX

I optimized the site to have as little distraction as possible. A bare minimum of text with traffic-light colors indicates positive, negative, or mixed news. Small tweaks — like remembering the scroll position in the feed when returning to it and relying on the browser cache instead of hitting the backend — make the site more pleasant to use.

Deployment

I deploy the entire infrastructure with Lambdas, DynamoDB, API Gateway, and Step Functions using Terraform. This makes tuning parameters like DynamoDB retention or RSS feed scraping intervals much faster — not to mention that Codex can write Terraform configs as well as it writes code.

LLMs

I used llama-3.1–8b-instant for the map step andllama-3.3–70b-versatilefor the reduce step.

Infra Cost

I am running on the free Groq tier, so I had to build retries for when API calls in the reduce step fail. There is a $0.20 USD/month cost associated with Step Functions because I exceed the free quota. DynamoDB retention ensures the site renders quickly and I do not pay for storage.

Lessons Learned

Understand what you want to know about. Generic news summaries are very plain. If you know what specific information you are looking for, the resulting summaries are much more useful.

RSS feed quality matters even when generating summaries using LLMs, because many sites tend to produce low-value content, which affects the summaries you get. Prune your RSS sources aggressively.

Most of the time went into building the frontend and making it easy to use. Iterating on prompts with the Groq API was quick once I was able to diagnose what was off in the first version.

Carefully consider whether you should invest in ranking news summaries based on user interest. I find that not building ranking and simply ordering by time is good enough — I could not justify the added complexity.

The critical LLM step is the reduce step, because I need the Groq API to return the result in JSON format together with article references for each cluster. To my surprise, it works out of the box even without JSON Object Mode, which means I do validation in the Lambda function instead.

What’s Next

The site is a useful snapshot of what is currently happening. It makes no attempt to be engaging, grow the user base, or keep you browsing longer. It is matter-of-fact and concise. I may not use it most days, but when I do need to catch up, it gets me up to speed quickly. I think not having to rely on any single platform for news is empowering. Summarizing news is the future. We just need to figure out the UX and the filters to use.