Ошибка ответа vk api

Перейти к содержанию

Main Aim = To collect all the posts on a news outlet (Meduza) that mentions the word coronavirus (коронавирус).

Main Question = Why does the VK API method wall.search in my case stop collecting posts before all the available community pages’ posts are collected?

The API call appears to be working okay as it is returning values from current day based on the query string ‘коронавирус’ (coronavirus) however, in this instance always stopping on 17th of January 2021, which is the 977th post collected.

I checked manually on the community VKontakte page and their earliest post about Coronavirus is 9th February 2020, so it should keep collecting posts till then.

What I’ve tried =

• Changing the time.sleep() to 180 seconds just in case.

• Using a different VKontakte page: Rossiyskaya Gazeta. The problem was still there as this time it stopped scraping on May 31st, 2021 which was the 1076th post.

• Using different versions of the API.

import vk_api
from datetime import datetime
import pandas as pd 
import requests
import json 
import time
import csv 
from pandas import DataFrame
from time import sleep


token = " " #won't share but it works
version = 5.103
data_list = []
query_string = 'коронавирус' #coronavirus
owner_id = -76982440
owners = 1 
start_date = 1643673601  
finish_date = 1577836801
url = "https://api.vk.com/method/wall.search"
data = requests.get(url).json()


offset = 0 

for i in range(20):
    r = requests.get('https://api.vk.com/method/wall.search.json', params={
        'access_token':token,
        'v':version,
        'owner_id': owner_id,
        'domain': owner_id,
        'query': query_string,
        'owners_only':owners,
        'count':100,
        'start_date': start_date,
        'finish_date': finish_date,
        'offset':offset
        })
    data_list.append(r.json()['response']['items'])
    try:
        start_from = r.json()['response']['next_from']
    except KeyError:
        pass
    offset+=100
    time.sleep(3)

print("Process Complete")

Main Aim = To collect all the posts on a news outlet (Meduza) that mentions the word coronavirus (коронавирус).

Main Question = Why does the VK API method wall.search in my case stop collecting posts before all the available community pages’ posts are collected?

The API call appears to be working okay as it is returning values from current day based on the query string ‘коронавирус’ (coronavirus) however, in this instance always stopping on 17th of January 2021, which is the 977th post collected.

I checked manually on the community VKontakte page and their earliest post about Coronavirus is 9th February 2020, so it should keep collecting posts till then.

What I’ve tried =

• Changing the time.sleep() to 180 seconds just in case.

• Using a different VKontakte page: Rossiyskaya Gazeta. The problem was still there as this time it stopped scraping on May 31st, 2021 which was the 1076th post.

• Using different versions of the API.

import vk_api
from datetime import datetime
import pandas as pd 
import requests
import json 
import time
import csv 
from pandas import DataFrame
from time import sleep


token = " " #won't share but it works
version = 5.103
data_list = []
query_string = 'коронавирус' #coronavirus
owner_id = -76982440
owners = 1 
start_date = 1643673601  
finish_date = 1577836801
url = "https://api.vk.com/method/wall.search"
data = requests.get(url).json()


offset = 0 

for i in range(20):
    r = requests.get('https://api.vk.com/method/wall.search.json', params={
        'access_token':token,
        'v':version,
        'owner_id': owner_id,
        'domain': owner_id,
        'query': query_string,
        'owners_only':owners,
        'count':100,
        'start_date': start_date,
        'finish_date': finish_date,
        'offset':offset
        })
    data_list.append(r.json()['response']['items'])
    try:
        start_from = r.json()['response']['next_from']
    except KeyError:
        pass
    offset+=100
    time.sleep(3)

print("Process Complete")