How to specify pagination

When specifying paginations with sequential numbers, it would be very useful if all we had to do was specify the base URL and the last number of the sequential number to cover all the pages in between.
Right now, for example, if a listing has 100 pages, 100 URLs must be entered.
Please let me know if this feature is currently available and I may have missed it.

Can you give me example of the paging urls?

Normally you can just set page count 1,2 3 etc.

Yes you would need to create those 100 urls.

Although you could use the keyword combiner to do it.

I am also planning out feature to add proper paging element as discussed here:

image

Maybe we can do both url and dynamic paging all at the same time

For example, the following URL.

What is ‘the keyword combiner’ and is it a SCM feature?

So page through this correct?

And grab all articles like this

image

–

Keyword combiner I mean this:

But its not ideal, you still need to type in numbers 1 - 100

That is a useful feature. Excellent!
You’re right, this still requires manual input.
Last time I used AI to output sequentially numbered URLs to save me the hassle.

I think everyone will use the automatic sequential number generation function of Pagination, so please consider adding the function.

Yes will work on it now.

Maybe have 2,

  1. Quickly generate urls with number range
  2. Paging element clicker

Quick url pager generate tool added

Modal pops up

You just enter url and the special %pager% macro

%pager% replaced with actual page count

Click generate

image

Feature already live on windows setup.

1 Like

Wow. Awesome!
I am a Mac user and I am looking forward to it.

Mac image uploading now

Will be ready in about 15 mins

1 Like

I used it immediately and it worked.
Thank you!
One question: why should I bother inserting the current target URL?
I can’t think of a use case where I wouldn’t use the currently specified page.

It’s just a shortcut.

Use which one you need.

1 Like

Added pagination action:

Eg table with pager

Pick the ‘next’ button

Add action ‘next page’

Specify how many times to click the page next button

When running…

image

Seems useful.
What should I enter for the target urls when specifying the next page element?

update
All I had to do was enter the current URL.

Needs to be the css selector for the next button so it knows to click on it.

Yes you can also use page url instead.

However for some tables there is no url, so the next button must be used.

1 Like

This is the birth of automation :partying_face:

Would love to see expanded UI section instead of dropdown and more “automation” functionality (lite RPA).

You want to be able to run your own javascript functions on the page right?

While opencv is more for image processing?

When I see the changelog and you add click() function, it trigger me to research what possible :grin:

OpenCV for OCR and image/pdf/visual processing.

It may or may not be possible to inject the jsautogui script into all pages.

JSAutoGUI specifically, uses nodejs, which is technically not available to sandboxed browser windows that SCM uses to scrape web pages.

However you always have access to document and 99% cases jquery for automation purposes.

So you can do things to the dom and click on it, but you can’t move the mouse or type the keyboard.

Its possible to allow you to click on action dropdown and select ‘javascript’ then you can write your own code.

There is theoretical problem with giving browsers access to nodejs, as it means they have access to your file system, ie give it possibility to do malicious things like create 1000s of files or start deleting files on your hard drive.

Oh ok Jquery is the keyword, I hate JS :sweat_smile:, I’m more a python student, well the high level overview is scrape and automate.

Maybe similar automation to lite RTILA, using 80/20 rule. The click() is already exciting feature and open new possibilities.

Yep full automation is limited to whatever you can do using jquery or normal javascript document.

There are some higher level libraries like selenium etc tools used normally for automated ui testing.