2024/spider/aijuke_spider.git - Gitblit

2024 / spider/aijuke_spider

使用scrapy采集社交媒体的公开信息

first | « prev | next »

2024-11-09

dy_v_user_profile_spider修改2

HEAD master

diff | tree

2024-11-09

dy_v_user_profile_spider修改

diff | tree

2024-09-20

feat:新增小红书根据时间停止采集条件

diff | tree

2024-09-20

refactor:去除根据点赞数的过滤

diff | tree

2024-09-19

refactor:增加小红书对注销用户的处理

diff | tree

2024-09-18

refactor:修改小红书xpath

diff | tree

2024-09-10

refactor:修改回调接口

diff | tree

2024-09-10

feat:增加回调拓展

diff | tree

2024-09-10

refactor:优化项目结构

diff | tree

2024-09-10

feat:增加发布配置(IDE)

diff | tree

2024-09-10

feat:运行完成保存日志，新增BaseSpider，TaskLog类，新增TaskLogExtension拓展，爬虫新增task_type属性，继承...

diff | tree

2024-09-09

fix:修复关键词分割bug

diff | tree

2024-09-09

实现快手大V信息完善

diff | tree

2024-09-09

refactor:当运行平台不是快手时，不使用代理

diff | tree

2024-09-09

refactor:记录执行日志时，添加执行状态

diff | tree

2024-09-09

refactor:记录执行日志时，添加结束时间

diff | tree

2024-09-09

fix:修复抖音私密账号的处理逻辑

diff | tree

2024-09-09

refactor:移除无用模板代码

diff | tree

2024-09-09

refactor:增加小红书数据管道的稳定性，增加更多容错

diff | tree

2024-09-09

feat:增加小红书作品的过滤，不采集点赞数低于80的作品

diff | tree

2024-09-07

fix:修复用户主页采集状态异常的问题

diff | tree

2024-09-06

移除无用代码

diff | tree

2024-09-06

实现抖音大V用户完善

diff | tree

2024-09-06

refactor:requirements.txt增加scrapyd-client依赖

diff | tree

2024-09-06

refactor:取消对租户的cookie限制

diff | tree

2024-09-06

refactor:V类新增字段

diff | tree

2024-09-05

refactor:优化导包

diff | tree

2024-09-05

fix:修复数据漏采问题

diff | tree

2024-09-05

fix:添加线程锁

diff | tree

2024-09-05

fix:修复评论采集数据不准的问题

diff | tree

2024-09-04

fix:修复评论采集数据不准的问题

diff | tree

2024-09-04

refactor:取消对租户cookie的限制

diff | tree

2024-09-04

refactor:添加小红书评论采集限制

diff | tree

2024-09-03

fix:修复close_spider函数错误调用的问题

diff | tree

2024-09-03

feat:新增爬虫结束后统计功能

diff | tree

2024-09-03

feat:新增JobExecLog类

diff | tree

2024-09-02

fix:修复类型不一致造成的查询条件错误问题

diff | tree

2024-09-02

refactor:限制用户只能使用自己的cookie

diff | tree

2024-09-02

refactor:Cookie类新增tenant_id字段

diff | tree

2024-08-31

refactor:更改数据源

diff | tree

2024-08-31

refactor:修改快手大v采集问题，优化小红书抖音导包

diff | tree

2024-08-30

refactor:修改查询条件

diff | tree

2024-08-30

refactor:增加unique_no，task_name,praise

diff | tree

2024-08-30

fix:修复大v采集问题

diff | tree

2024-08-30

refactor:增加抖音和快手的点赞数获取

diff | tree

2024-08-30

fix:修复导包错误

diff | tree

2024-08-30

refactor:删除无用文件

diff | tree

2024-08-30

fix:修复导入js文件路径错误问题

diff | tree

2024-08-29

refactor:修改字段，增加稳定性

diff | tree

2024-08-29

feat:实现小红书数据管道

diff | tree

first | « prev | next »