F5-Bigip利用irule防止爬虫的一法。
爬虫的请求:
1GET /cms/rest.htm?method=ddky.cms.search.recommend.h5.o2o&pageNo=1&pageSize=6&shopId=201790&ordertypeId=0&suite=1&searchType=o2o&searchPanel=1&wd=%E6%B4%9B%E4%B8%81%E6%96%B0&lat=22.520712193695&lng=113.9233553732&city=%E6%B7%B1%E5%9C%B3%E5%B8%82&type=90&unique=05685D2A5DAB8ABBD2E5E5B26E0C960F&versionName=5.7.5&plat=H5&platform=H5&t=2020-12-15%2014%3A19%3A11&v=1.0&sign=A6BD136BC1B6F91E5C7DD5A0DA03DD79&callback=jsonp1
里面的t值是时间,t=2020-12-15%2014%3A19%3A11
但是有个问题,这个值一直不变了,那我们就利用这一点。如果T值跟当前时间对比,是3分钟前的,那就封!
F5的irule,直接return的是白名单:
1when HTTP_REQUEST {
2
3 set t [URI::decode [URI::query [HTTP::uri] t]]
4 set before [clock scan "180 seconds ago" ]
5
6 if { [IP::addr [IP::client_addr] equals 124.206.168.0/255.255.255.224]} {
7 return}
8 if { [IP::addr [IP::client_addr] equals 61.135.14.96/255.255.255.240]} {
9 return}
10 if { [IP::addr [IP::client_addr] equals 114.251.7.112/255.255.255.240]} {
11 return}
12 if { [string tolower [HTTP::uri]] contains "/cms/"} {
13
14 if {$before > [clock scan $t]} {
15 drop
16 }
17 }
18}