SCM Freezes during Google Maps Scraper

I ran it and it skips fine no problems.

I will keep running it and see if it slows down over time.

Also try the new SCM update.

Also what is your CPU usage like when it runs this slow?

Could it also be power plan related?

The problem is intermittent .. meaning it doesn’t happen all the time .. and I haven’t been able to find a pattern as to why the slowdown is happening.

I have noticed CPU is running 100% alot .. about 50% SCM .. and 50% System running 3 threads .. is that normal?

What is interesting is that it was running for weeks fine on the same PC .. and then just dropped like a rock from 70k requests/day to 19-20k .. that is too huge to overlook or blame on something ancillary. 1m between skips .. and 10m gaps. I havent installed any other programs on this computer in the meantime so Something is wrecked somewhere… but Antivirus says PC is clean.

I will try the SCM update and let you know.

Reach out to me via email, share your proxies with me and I will see if there is any overhead from that as well.

50% seems a lot, how many other tasks are you running in the background?

But if it was CPU bound, the UI should also be freeze and be unresponsive.

Does UI become unresponsive when the slow down happens?

Could it also be thermal throttling? I noticed you are running on a low power N150 device.

Maybe it has to run less threads?

I’ve never noticed the UI to become unresponsive .. I put SCM on an N150 box because I thought that running 5-10 threads wouldn’t require alot of power. but it seems that the SYSTEM is taking up as much or more CPU than SCM .. no idea why. the system does stop taking up CPU when SCM task ends though .. so its likely related.

Could it be thermal throttling? Its weird that suddenly it takes 1 min to process each log message.

Is it still doing weird things after the update?

I just checked the temps … even running 100% CPU .. temps are about 72’C .. not seeing any throttling. These n150 are good about not getting too hot .. they are very low power … for a few browsing threads I thought it would be perfect for SCM.

What was strange was that the dupe SKIP threads were exactly 1m apart .. that seems to be more a programming issue than a CPU temp throttle .. as it would be more random.

We need to keep in mind though .. that this N150 was running 5/10 threads running 70k requests/day using proxies for weeks without breaking a sweat … now it peaks at 20-30k tops.

SEE SCREENSHOT
26.08.2025_16.23.11_REC .

So .. something changed. Not sure what .. not blaming SCM .. but aside from Win11 update, SCM was the only thing I updated (or is even running on this box) between before/after. so it’s most likely proxies, Win11 update or SCM update. I will test proxies next.

Did you try running it without proxies?

I think whatever you did in the last/previous patch might have fixed it. I tried with/without proxies and have not noticed a slowdown yet. Let’s see how next 1-2 days go.

What did you do to fix it ?

Made only 1 change to task logging. But nothing else.

Not sure that it was cause of your problems.

CORRECTED: THIS ISSUE IS NOT FIXED YET. See my latest post.

Strange.

The exactly 1 min delays is weird though, as before hand it was technically only a 1 sec delay before printing messages.

The delay has been removed though.

Hopefully it keeps going without slowing again

Slowdown Problem resurfaced … it worked fine until it didn’t. 10 min gaps are back. I am trying to see if I can spot patterns as to why it started slowing down again. My only guess as of now is it seems to happen after its been running a while on a long job.

Can you open cpu-z while its running slow to verify the cpu speed?

I believe it runs slow even with proxies turned off?

9/4/2025, 4:27:15 PM	37 of 97 done - New Directions for Women Intensive Outpatient Program M438+JF Newport Beach, California
9/4/2025, 4:27:15 PM	37 of 97 looking for contact at https://www.newdirectionsforwomen.org/ (200) ...
9/4/2025, 4:27:06 PM	37 of 97 Found website meta at https://www.newdirectionsforwomen.org{"metaTitle":"Rehab for Women in California | New Directions for Women","metaDescription":"For more than 40 years, our drug and alcohol rehab center has been helping women. Our addiction treatment center in California can help you or your loved ones.","metaKeywords":"","metaImage":"https://www.newdirectionsforwomen.org/wp-content/uploads/2023/04/ndfw-social-new.jpg"}
9/4/2025, 4:27:06 PM	37 of 97 looking for website meta at https://www.newdirectionsforwomen.org (200) ...
9/4/2025, 4:16:31 PM	39 of 97 Getting next proxy: dc.decodo.com:10000 ...
9/4/2025, 4:16:31 PM	39 of 97 opening https://www.google.com/maps/place/Asana+Recovery:+Drug+%26+Alcohol+Rehab+In+California/data=!4m7!3m6!1s0x80dd20767b996981:0x9dfa2ecc79d3d33d!8m2!3d33.6374975!4d-117.9273348!16s%2Fg%2F11c1n8lkx1!19sChIJgWmZe3Yg3YARPdPTecwu-p0?authuser=0&hl=en&rclk=1
9/4/2025, 4:16:31 PM	35 of 97 done - Ayako Physical Therapy - Ayako Shono, MPT, Newport Beach, CA J47H+2P Newport Beach, California
9/4/2025, 4:16:30 PM	35 of 97 looking for contact at https://www.ayakopt.com/ (200) ...
9/4/2025, 4:16:26 PM	38 of 97 Getting next proxy: dc.decodo.com:10000 ...
9/4/2025, 4:16:26 PM	38 of 97 opening https://www.google.com/maps/place/Miramar+Health+-+Dahlia/data=!4m7!3m6!1s0x80dce12e075fd8a7:0x5980ad31099dc147!8m2!3d33.6008732!4d-117.8756084!16s%2Fg%2F11fm54_d2t!19sChIJp9hfBy7h3IARR8GdCTGtgFk?authuser=0&hl=en&rclk=1

These kind of 10m gaps are tough to spot as they happen. I usually notice them afterwards in the log. I downloaded CPU-Z .. not sure what I should look for. Is there a tool with a LOG mode so we can line up the log times?

Or does SCM have a debug mode .. when it senses timer > debug timeout between task items that it logs what it is waiting on to find out what the holdup is?

CPU-Z we are looking to see if there is any drop in CPU hz.

Also n150 is 4 core cpu, you shouldn’t be using any more than 4 threads global in SCM.

How many threads you running across all tasks?

There is no sensing for log timeouts cause normally there isn’t anything there to wait for.

All the code does is push a text string into and array, and that array is read on an interval like 50ms, and if there is a change it writes it out to a html table row.

There isn’t any IO operations there that would cause slowdowns.

You need to lower threads to one in Google maps scraper, since it looks like its running multiple operations at the same time.

It’s been a bear trying to hunt this down … but I’ve been graphing CPU usage (it only allows upto last 4 min) and I was finally lucky enough to stumble upon the CPU graph just after a 11 min gap/pause. Shockingly .. the CPU graph showed CPU near 0% during the gap time .. not 100% as we previously thought. We were under the impression the N150 couldn’t handle the threads and got bogged down. We now know this is opposite of true and not the case .. the CPU wasn’t being used at all during the gaps. I had a hunch, because before, N150 was doing just fine and after one of the proxy updates, it just started having slowdown/GAP problems.

Perhaps SCM was waiting for something to continue during the gap? Here is the log just before the gap between 3:53 and 4:04pm. Isn’t it also funny how the gaps seem to max out at about 10 min? They can be shorter… but I’ve never seen a gap over 10-15 min.

9/23/2025, 4:05:00 PM	99 of 115 looking for website meta at https://twintigerstattoo.com (200) ...
9/23/2025, 4:04:54 PM	102 of 115 Getting next proxy: dc.decodo.com:10000 ...
9/23/2025, 4:04:53 PM	102 of 115 opening https://www.google.com/maps/place/Alex+Hunter+at+North+Main+Tattoo+Studio/data=!4m7!3m6!1s0x8824b3246ef8cefd:0xe81f0922a99db562!8m2!3d42.4283115!4d-83.3390078!16s%2Fg%2F11lg811mc6!19sChIJ_c74biSzJIgRYrWdqSIJH-g?authuser=0&hl=en&rclk=1
9/23/2025, 4:04:53 PM	98 of 115 done - IamClubCartel Tattoos FPFR+C5 Southfield, Michigan
9/23/2025, 4:04:47 PM	101 of 115 Getting next proxy: dc.decodo.com:10000 ...
9/23/2025, 4:04:47 PM	101 of 115 opening https://www.google.com/maps/place/Tattoos+by+Trent+Thompson/data=!4m7!3m6!1s0x8824b4b8a3500001:0xe9642303096f81e!8m2!3d42.3700155!4d-83.3154338!16s%2Fg%2F11d_796dcv!19sChIJAQBQo7i0JIgRHviWMDBClg4?authuser=0&hl=en&rclk=1
9/23/2025, 4:04:47 PM	97 of 115 done - Toxic Cupcake Tattooz FVP3+RG Royal Oak, Michigan
9/23/2025, 4:04:43 PM	100 of 115 Found website meta at https://removery.com{"metaTitle":"Home","metaDescription":"Removery is the worldwide laser tattoo removal leader providing safe, effective tattoo removal and fading solutions tailored to your needs.","metaKeywords":"","metaImage":"https://removery.com/wp-content/uploads/2023/10/MicrosoftTeams-image-35.jpg"}
9/23/2025, 4:04:42 PM	100 of 115 looking for website meta at https://removery.com (200) ...
9/23/2025, 3:53:59 PM	100 of 115 Getting next proxy: dc.decodo.com:10000 ...
9/23/2025, 3:53:58 PM	100 of 115 opening https://www.google.com/maps/place/Removery+Tattoo+Removal+%26+Fading/data=!4m7!3m6!1s0x8824ddc4569b809d:0xdfaa3c6038ac2d8d!8m2!3d42.5171925!4d-83.1888963!16s%2Fg%2F11c58n_v8z!19sChIJnYCbVsTdJIgRjS2sOGA8qt8?authuser=0&hl=en&rclk=1
9/23/2025, 3:53:58 PM	95 of 115 done - Art By Enea Tattoo Studio 7PXG+Q2 Dearborn, Michigan
9/23/2025, 3:53:58 PM	95 of 115 looking for contact at https://www.artbyenea.com/ (200) ...
9/23/2025, 3:52:21 PM	95 of 115 Found website meta at https://www.artbyenea.com{"metaTitle":"artbyenea","metaDescription":"","metaKeywords":"","metaImage":"http://static1.squarespace.com/static/6435c4021cb1ca1acd7877b1/t/646e982223abc52d1f9312ae/1684969506597/%26+Permanent+Makeup.png?format=1500w"}
9/23/2025, 3:52:16 PM	99 of 115 Getting next proxy: dc.decodo.com:10000 ...
9/23/2025, 3:52:16 PM	95 of 115 looking for website meta at https://www.artbyenea.com (200) ...
9/23/2025, 3:52:15 PM	99 of 115 opening https://www.google.com/maps/place/Twin+Tigers+Tattoo/data=!4m7!3m6!1s0x8824ad95e7212c15:0x7f920f592af6aeff!8m2!3d42.3792705!4d-83.4614986!16s%2Fg%2F11sfb1gqz7!19sChIJFSwh55WtJIgR_672KlkPkn8?authuser=0&hl=en&rclk=1
9/23/2025, 3:52:14 PM	94 of 115 done - Legit ink tattoo studio 8M6C+62 Inkster, Michigan
9/23/2025, 3:52:14 PM	94 of 115 found mapdetailscom@gmail.com
9/23/2025, 3:52:13 PM	94 of 115 looking for contact at https://usacommetix.top/ (200) ...
9/23/2025, 3:52:11 PM	98 of 115 Getting next proxy: dc.decodo.com:10000 ...
9/23/2025, 3:52:11 PM	98 of 115 opening https://www.google.com/maps/place/IamClubCartel+Tattoos/data=!4m7!3m6!1s0x8824c97cf71e317b:0x74497ab09bc0d9e0!8m2!3d42.4735264!4d-83.2595911!16s%2Fg%2F11fkyz7k2g!19sChIJezEe93zJJIgR4NnAm7B6SXQ?authuser=0&hl=en&rclk=1
9/23/2025, 3:52:10 PM	96 of 115 done - Scalp Authority 24/7 Scalp Micropigmentation Michigan Emergency GJGR+HQ Farmington Hills, Michigan
9/23/2025, 3:52:09 PM	94 of 115 Found website meta at https://usacommetix.top{"metaTitle":"BusinessesUSA.com - America's Premier Business Directory","metaDescription":"Discover trusted American businesses from coast to coast. Find verified local companies, read authentic reviews, and connect with quality service providers across all 50 states.","metaKeywords":"local business directory, business finder, service providers, company reviews, business listings, local services","metaImage":"https://BusinessesUsa.com/og-image.png"}
9/23/2025, 3:52:09 PM	94 of 115 looking for website meta at https://usacommetix.top (200) ...
9/23/2025, 3:49:12 PM	97 of 115 Getting next proxy: dc.decodo.com:10000 ...
9/23/2025, 3:49:11 PM	97 of 115 opening https://www.google.com/maps/place/Toxic+Cupcake+Tattooz/data=!4m7!3m6!1s0x8824cf9e7f8ee45b:0xc0cf264124a83c69!8m2!3d42.487031!4d-83.14618!16s%2Fg%2F11y97t5690!19sChIJW-SOf57PJIgRaTyoJEEmz8A?authuser=0&hl=en&rclk=1
9/23/2025, 3:49:11 PM	93 of 115 done - Bramble & Thorne GVM3+8R Clawson, Michigan
9/23/2025, 3:49:11 PM	93 of 115 found mapdetailscom@gmail.com
9/23/2025, 3:49:11 PM	93 of 115 looking for contact at https://usacommerx.top/ (200) ...
9/23/2025, 3:49:08 PM	93 of 115 Found website meta at https://usacommerx.top{"metaTitle":"BusinessesUSA.com - America's Premier Business Directory","metaDescription":"Discover trusted American businesses from coast to coast. Find verified local companies, read authentic reviews, and connect with quality service providers across all 50 states.","metaKeywords":"local business directory, business finder, service providers, company reviews, business listings, local services","metaImage":"https://BusinessesUsa.com/og-image.png"}
9/23/2025, 3:49:08 PM	93 of 115 looking for website meta at https://usacommerx.top (200) ...
9/23/2025, 3:46:59 PM	96 of 115 Getting next proxy: dc.decodo.com:10000 ...

One pattern I have noticed … the gap always starts with “Getting next proxy …” … Couple that with the fact this problem started after one of your updates to separate the proxies … that might be a good place to look.

PS: In my power settings, I have both Sleep/Hibernate set to NEVER. Also, I have SCM maxxed out at 4 tasks maximum (1 per CPU) .. but as we found out today, this is not a CPU at 100% problem .. its a CPU near 0% problem.

You need to reduce your google maps threads down to 1.

In the log are multiple threads starting and running at the same time.

Ie

97 of xxx
98 of xxx

If you think it’s proxies based, you should turn off the proxies and run it without them to see if the same gap appears.

The 10min wait times are strange, it could be something like proxies rotating and scm is waiting a set amount of time.

The only 2 things that we see are 1. Multiple threads, 2. Proxies

So turn off proxies and reduce threads to isolate the possible causes.

Once you got task threads to 1 run it again and see if the wait times exist.

It’s a bit hard to track exactly where a task is stopping and restarting because multiple tasks are starting in the task log.

I think if we run one task one thread, we can see the exactly how each task is progressing and at what stage it stops.

From task log, look at 97 of 115

You can see it tries to open a page to google via proxy and ends after 5mins. There isn’t apparently any action in between such as getting site page or meta.

ok .. proxies are not the issue .. i disabled proxies and saw similar gaps. and for the first time i saw a very large gap longer than 15 min.

9/25/2025, 4:19:15 PM	51 of 120 opening https://www.google.com/maps/place/Taqueria+Alamo/data=!4m7!3m6!1s0x880e33f0b0048469:0xdf7eb01b6639730f!8m2!3d41.829165!4d-87.7708381!16s%2Fg%2F1thvhj78!19sChIJaYQEsPAzDogRD3M5Zhuwft8?authuser=0&hl=en&rclk=1
9/25/2025, 4:19:14 PM	47 of 120 done - Alexanders Restaurant & Pancake House R6C5+FC Stickney, Illinois, USA
9/25/2025, 4:19:14 PM	47 of 120 found alexanderspancake@gmail.com
9/25/2025, 4:19:14 PM	47 of 120 looking for contact at https://alexandersstickney.com/ (200) ...
9/25/2025, 4:19:13 PM	48 of 120 place retry 2 of 2 in 10s 
9/25/2025, 4:19:13 PM	48 of 120 place warning: place name missing - https://www.google.com/maps/place/Taqueria+Casa+Grande/data=!4m7!3m6!1s0x880e3386867e7e4f:0xfcacb1fb5485f980!8m2!3d41.8511011!4d-87.7719339!16s%2Fg%2F1wtbvgnz!19sChIJT35-hoYzDogRgPmFVPuxrPw?authuser=0&hl=en&rclk=1
9/25/2025, 4:19:07 PM	47 of 120 Found website meta at https://alexandersstickney.com{"metaTitle":"","metaDescription":"","metaKeywords":"","metaImage":""}
9/25/2025, 4:19:06 PM	47 of 120 looking for website meta at https://alexandersstickney.com (200) ...
9/25/2025, 1:48:57 PM	50 of 120 Getting next proxy: dc.decodo.com:10000 ...
9/25/2025, 1:48:57 PM	50 of 120 opening https://www.google.com/maps/place/Taqueria+La+Guadalupana/data=!4m7!3m6!1s0x880e339bcb862159:0xc73ec61dc47e05c6!8m2!3d41.8506624!4d-87.762471!16s%2Fg%2F1tdxfwgp!19sChIJWSGGy5szDogRxgV-xB3GPsc?authuser=0&hl=en&rclk=1
9/25/2025, 1:48:57 PM	46 of 120 done - Freddy's Pizza V65F+69 Cicero, Illinois, USA
9/25/2025, 1:48:57 PM	46 of 120 looking for contact at https://www.freddyspizza.com/ (200) ...
9/25/2025, 1:48:56 PM	48 of 120 place retry 1 of 2 in 10s 
9/25/2025, 1:48:56 PM	48 of 120 place warning: Cannot read properties of undefined (reading 'index')
9/25/2025, 1:48:53 PM	46 of 120 Found website meta at https://www.freddyspizza.com{"metaTitle":"","metaDescription":"","metaKeywords":"Pizza, Freddys Pizza, Freddys Pizzeria, Gelato, Italian Ice, Imported and Domestic Foods, Italian Deli, Pasta, Catering, Cicero Pizza, Homemade Bread","metaImage":""}
9/25/2025, 1:48:52 PM	46 of 120 looking for website meta at https://www.freddyspizza.com (200) ...
9/25/2025, 1:48:52 PM	49 of 120 Getting next proxy: dc.decodo.com:10000 ...
9/25/2025, 1:48:52 PM	49 of 120 opening https://www.google.com/maps/place/La+Xhula+Mexican+Restaurant/data=!4m7!3m6!1s0x880e3369fa2f9025:0x2dec7548213bba82!8m2!3d41.8620506!4d-87.7497879!16s%2Fg%2F11mckntz2q!19sChIJJZAv-mkzDogRgro7IUh17C0?authuser=0&hl=en&rclk=1
9/25/2025, 1:48:52 PM	45 of 120 done - Masters Pancake House Q7W5+P3 Chicago, Illinois, USA
9/25/2025, 1:48:51 PM	45 of 120 looking for contact at https://masterspancakehouse.com/ (200) ...

Now my theory is leaning more along the line of passkeys .. which I still keep getting requests for when connected via RDP. If a passkey is requested .. and there is no response to the popup dialog .. how long does it wait for before resuming?

Interesting, you think it might be passkey related?

SCM is meant to hide the passkey popup, but not sure if that actually disables it.

Assuming that it pops up, scm won’t handle it and in theory the task is meant to timeout if the browser doesn’t advance. But there should be a log message if task ends with timeout.

Will need to dig into the code and test if this is true though.

When you click test Google maps, do you have passkey appear in the test browser.

Also can you lower Google maps threads to 1?