How to signup to Groq AI for free unlimited llama3 70B (GPT4 competitor) calls

questions faq
1

As an alternative to running LLMs on your desktop you can signup to Groq AI for free calls.

If you don’t have a beefy PC + GPU combo, Groq AI is what you what to run llama3 70B.

Advantages

  • Right now it is free for everyone, no billing details even required.
  • Groq AI inference times are even quicker than GPT4
  • Access to llama3 70B (the largest model)
  • Recent info, knowledge cutoff date is Dec 2022 for 70B and March 2023 for 8B vs SCM AI Jan 2022

Disadvantage

  • Low rate limits. You wont be able to run more than 1 AI thread
  • Rate limit wait times can be long. SCM is able to wait between calls to maximize throughput though

Get Groq

Create a free account

image

Get your API key

Click the create api key button

image
image1282×631 28.6 KB

Paste key into SCM

API logins
Select default ai = groq ai
Paste in API key

image
image1282×722 39.3 KB

Test

image
image432×618 16.9 KB

1 Like
2

Hey @Tim,

I use Groq to generate short articles, and I see that I have just 6k tokens per minute.

Wouldn’t it be a good solution to add a field so that we can add a time for a pause between an item and another?

I can make 1 item, but when Groq goes to the next one, it rarely happens or makes an error, which could be avoided if you wait 1 minute before going to the next key word on the list, and I would say writing 1 item every 70 seconds would be ideal.

or what acha gives idea?

1 Like
3

Groq uses its own rate limit based on tokens, SCM automatically follows that limit set.

In detail, groq sends back a message when you make to many calls to wait xx number of seconds.

So you can use groq to its full token limit without any worries, SCM will wait as required.