Skip to main content

How long checking sources takes

The pace of source checks is set in advance. This is not a delay in the app. A hundred requests in a row mean a limit on the platform's side, and the limit falls on the person's address and account. Below are all the numbers that add up to the check time.

Why the check is not instant​

There are four reasons, and they add up:

  1. A pause is kept between two requests to the same platform.
  2. The number of requests to one platform within ten minutes has an upper limit.
  3. The app runs no more than four checks at once.
  4. After a "too often" refusal, the platform is not asked for a whole hour.

There is no "check now" button in the "ClipSave" window on purpose. The pace is set by the app, not by the person.

Request pace for one platform​

WhatValue
Pause between two requests to one platform4 seconds
Pause spread25%, that is, from 3 to 5 seconds
Request counting window10 minutes
Requests to one platform per windowno more than 60
Delay after a "too often" refusal1 hour

The pause is counted from the end of the previous request, not from its start. The spread is there so that requests do not arrive exactly every four seconds.

Both limits work together. The pause limits the pace at a given moment, the window limits it on average.

How long checking a hundred links takes​

A list of a hundred links from one platform is checked in about 13 minutes. The first 60 links are checked in about 4 minutes. Then the window limit kicks in, and the pace drops to six links per minute.

Links are checked one at a time, from top to bottom. The pause and the window are counted separately for each platform. So a mixed list from two platforms is checked much faster. The same hundred links fit into about 4 minutes.

Link checking runs on its own, as a queue. The window stays usable during the check. The counter next to it shows the progress: "Link 3 of 12".

How long a source check round takes​

A round is one pass over all enabled groups of the project.

WhatValue
Checks at once for the whole app4
What takes up one place in the queueA "platform and proxy" pair
Wait time for a group in quarantine5 minutes

A place in the queue is taken not by a group, but by a "platform and proxy" pair. Two groups with different proxies on the same platform are checked at the same time. Two groups with the same proxy are checked one after the other. Five hundred groups of one platform with one proxy are checked strictly one after another.

The app counts a lower estimate of a round with a simple formula: the number of sources is divided by 4 and multiplied by 4 seconds. A hundred sources give about 100 seconds. In practice a round is always longer than the estimate. The estimate includes neither retries, nor platform quarantine, nor the second request for post details.

The second request is triggered by five selection conditions: three engagement shares, language and location. It makes the check several times longer. The numbers are on the Filters page.

A group in quarantine gets no more than 5 minutes. If it takes longer, the group skips the whole round and gets the label "Skipped: the platform is still limiting the requests".

The queue is circular. The app moves a checked group to the end of the queue. In one round the app contacts each group once.

A "too often" refusal and a one-hour quarantine​

The platform refuses frequent requests. The app then shows the line "The platform limited the requests". After that there will be no requests to the platform for an hour.

The hour is not chosen at random. Platforms count the pace in tens of minutes. A retry after five minutes gets the same refusal and takes up a place in the queue.

The quarantine survives an app restart. Turning the program off and on is not a way out. For the "ClipSave" window, the delay is written to a file next to the app. For source groups, it is written to the project database. The longest wait for a group is one day.

On the group tile this shows as the label "Retry at 14:30". When the exact time is unknown, it says "Retry later".

Set a proxy for the group if the quarantine happens often. The requests will then come from a different address, and the group will get its own place in the queue.

How the periodicity is set​

Both settings are in the "Export" section: "Check sources no more often than once every" and "Create no more per day than". Their limits and defaults are described on the Sources page.

The main thing about the period: it is the smallest pause between the starts of rounds, not a schedule. Two rounds never run at the same time. A round that lasts longer than the period pushes back the start of the next one.

A group has its own participation share - "How often to check, %". At 100% the group takes part in every round. At 25%, in every fourth one.

What happens after an app restart​

Checks run only while the app is open. After a new launch, the export has to be turned on again with the "Start" button.

WhatSurvives a restart
Platform quarantineyes
List of downloaded videosyes
Round counter and time of the previous roundno
Reason for an overnight failureyes, shown with the prefix "In the previous session: "

This gives the first round after a start two features.

  1. It starts right away, and the period is not kept. The time counter was reset with the restart.
  2. It covers all groups, whatever share is set in the "How often to check, %" field. The share is counted from the number of rounds, and there have been none yet.

This is by design. The person pressed "Start" and should get videos now, not in half an hour.

Seven reasons a round does not start​

The "Start" button is pressed, but there are no checks. The app responds with one of seven lines.

ReasonWhen it clears
"Sources are being checked."When the current round ends
"Periodic source checking is off: the period is not set."Never. Set the "Check sources no more often than once every" field to a value above zero
"The license has expired. Source checking will continue after the key is activated."After the key is activated
"Credits have run out, and video creation on the device is off. Source checking will continue after the balance is topped up."After the balance is topped up
"The "Create no more per day than" setting has been triggered. Source checking will continue after midnight."At local midnight
"The daily video limit is used up. Source checking will continue after it resets."When the plan limit resets
"The period between source checks has not passed yet."When the period has passed

The last two lines look alike but mean different things. "Create no more per day than" is your setting, and it resets at local midnight. The daily video limit belongs to the plan, and its reset time is set by the server. Both limits are visible in the same window.

In periodic mode, the daily video limit only pauses the work. This differs from a regular run, where it stops everything until the button is pressed.

Reasons why the run will not happen at all are covered on the Sources page.

Warning about a period that is too short​

Before the start, the app compares the period with the round estimate. When the period is shorter, it shows the warning "Checking sources will take longer than the period you set".

The text names all three numbers: "The "Check sources no more often than once every" field is set to 5 min. Checking 300 sources take at least 5 min. A new check starts only after the previous one ends, so checks will run less often than you set.".

This does not prevent the start. Increase the period or reduce the number of sources if you want an even pace.

What stopping does to downloaded files​

Periodic checking can be stopped only with the "Stop" button in the "Export" section. You cannot change the export type on the fly.

This is what happens to the files when you stop.

  • Videos that were downloaded but did not become copies yet are deleted from the disk.
  • The "already taken" mark on these videos stays in place. The list of downloaded videos is a blacklist.
  • Such a video will not be downloaded a second time. The next check will skip it.

The "Delete downloaded videos after creating copies" switch has nothing to do with this. It decides the fate of files from which copies have already been created.

Only a person can remove the mark. Open "Downloaded videos" by right-clicking the group tile. Find the rows you need with the search and press "Delete selected". The next check will take these videos again.

The search in the window looks up both the link and the video ID.

FAQ​

I set the period to 30 minutes, but videos appear every two hours. Why?​

The period is the smallest pause between the starts of rounds, not a schedule. A round of three hundred sources lasts longer than half an hour. A new one starts only after the previous one ends. Look at the warning before the start: the app calculates the round estimate in advance and gives it as a number.

Why is link checking in the download window so slow?​

It is set up this way on purpose. A pause of about four seconds is kept between requests to one platform. Within ten minutes, the app makes no more than 60 requests to one platform. A hundred links are checked in about 13 minutes. Faster means a limit on the platform's side, and it will fall on your address and your account.

I restarted the app, but the platform still limits the requests​

That is expected. The one-hour delay is written for the group on the disk and survives a restart. Wait an hour: changing the group settings, including the proxy, does not lift this delay. It makes sense to set a proxy for the future.

The first check after the start went over all groups, although 25% is set. Is this an error?​

No. The first round after a start always covers all groups. The share is counted from the number of rounds passed, and after a restart there are zero. From the second round on, the share works as set.

I stopped the export, and the files are not in the folder​

That is expected. The app deletes videos that were downloaded but did not become copies yet. Their mark stays, so they will not be downloaded again. Delete the entries in the "Downloaded videos" window if you want to get these videos again.