Hi Tim, first of all thank you for amazing product and your constant effort at improving it and keeping up to date.
I make a lot of AI articles based on paragraphs and headings from top articles from serp. Openrouter has batch models that require special format and do not return results immediately, but they are half the cost on same quality models. Like Opus, GPT Sol, Astra, etc. So it would be a great addition to SCM to make an option that supports this. It would save so much cost, and would enable to use best models instead of medium ones.
2. Simple solution
Add flag into page pipeline task for this option so that SCM collects serp as usual, extracts content from urls, sends prompt with variables, but I guess in batch mode it saves all promps into one package, after going through all keywords then sends that package. Results download can be directly via openrouter web account, so no return immediately, as batch usually executes within 1hr-1day timeframe. Is it possible to add this feature?
I saw that you made a mode that does similar thing, collecting all data into one csv file, but it didn’t mention any paragraphs and headings, i mean content from those serp urls. Without it just serp data is not enough.
Though I don’t know the limits on this my usual requests are about 10k tokens per keyword and response is also 10-15k tokens. So if taking 100 keywords this would be a really big request.
The main issue is that results are not returned immediately, median batch finished in 7 minutes, 90% finished within an hour, and 99% within about 10 hours
Apart from time, the bigger issue is how SCM works.
The page pipeline creates content using sequential prompts. Ie one prompts results go into the next one, a) chain prompts b) agent turns.
The batch requires each turn and prompt get sent in, and wait like 7 - 10hrs. A 10 step turn of research and write would take 1 hour.
It only works if we start 10 page pipelines all at once, and each pipeline sends in a prompt and it waits for a batch of turns to accumulate, it gets sent in, you want 7-10hrs and the next step happens.
Batch mode not built for agents and multi step prompt chains.
Where it works is one large prompt, but you have hundreds of them. You send them all up and wait, nothing needs to run sequentially.
The final reason also is that chain prompt ai templates are getting deprecated.
Instead we give agents a skill file, bunch of tools, and say, now write and research this.
Then it can review and self asses what it wrote, models as agents writing content like a human can.
Nothing there can be bulk ran easily. Tool calls can’t be made in batch calls. Ie search google for ‘dog training’ isn’t something that can be batched.