In addition to %scraped_article% and %scraped_article_title%, I have added a %scraped_article_headings%
This allows you to use the headings of an article only to send to the AI model (instead of the entire article) to save tokens and avoid hitting token limits.
Instead of %scraped_article%
We can use %scraped_article_headings%
%scraped_article_headings% are taken from an Article H1…H4
Additionally, I added extra messages when the scraped_article macros are being used in the task log.
Each time a macro is called its logged and shows the current index and the max number available.
That way when you run out, the out of … error is more understandable.
We have reached %scraped_article_headings% 10/10
SCM moves onto the next article
But because there is only 10 headings, trying to read (11/10) is not possible so it errors out.
The solution is the lower the AI article count so that we stay under the limit.
Using scraped article headings means we can build prompts like this.
We can save on a lot of token usage by simply switching to headings instead of including the entire article text.
A new AI template will be added to SCM to take advantage of this!
Most likely called
