Extra Form
PHP PHP 7.3
CMS WordPress

안녕하세요.

 

오늘 낮에 잠깐 운영 중인 홈페이지를 들어가보니, 접속이 안되더군요 ;;

 

그래서 확인해보니 Semrushbot 이란 놈이 엄청나게 크롤을 해되고 있었습니다 

 

결국 해당 봇을 차단했고, 이리저리 검색해서 악성 봇 몇개를 차단했습니다만 .. 

 

제가 참고했던 게시글이 꽤나 오래 전 게시글들 혹시 최근에 차단하고 있는 봇 리스트가 있다면 알고 싶습니다.

 

감사합니다!

 

  • profile

    AhrefsBot Amazonbot Arachni BLEXBot DotBot Exabot Eyeotabot MegaIndex MJ12bot PetalBot SemrushBot WordPress bbot brands-bot dataforseo-bot

     

    기본적으로 이 정도 차단하고, 그 밖에 또 눈에 띄는 놈이 있으면 그때그때 차단 목록에 추가하고 있습니다. 그 중에서도 SemrushBot은 여기저기 많이 출몰하는 악질이죠.

     

    봇들이 난장판을 부리는 사이트라면 bad bot blocker처럼 어마어마하게 복잡한 설정을 사용하기보다는, 역발상으로 Googlebot, Mediapartners-Google (애드센스), bingbot, Yeti (네이버)만 빼고 모든 봇을 일괄 차단하는 초강수를 두는 편이 차라리 나을 수도 있어요. 단, 대형포털 로봇이 과부하를 일으키는 경우도 적지 않다는 것이 함정...

  • profile ?
    안녕하세요 기진곰님 !

    알려주신 봇은 전부 차단 리스트에 추가했습니다! 답변 정말 감사합니다.

    우선 테스트를 한번 해보고 그래도 과부하가 발생한다면 알려주신 방법을 시도해보도록 하겠습니다.

    정말 감사합니다!
  • profile profile
    Googlebot, Mediapartners-Google (애드센스), bingbot, Yeti (네이버)만 빼고 모든 봇을 일괄 차단하는 초강수는 어떻게 해야 하는건가요?
  • profile
    아파치에서 차단
    <RequireAll>
    Require all granted
    Require not env bad_bot
    </RequireAll>


    # SetEnvIfNoCase User-Agent ^$ bad_bot
    SetEnvIfNoCase User-Agent "^MJ12bot" bad_bot
    SetEnvIfNoCase User-Agent "^MJ12bot/v1.4.5" bad_bot


    #악성봇...
    SetEnvIfNoCase User-Agent "SemrushBot" bad_bot #181203
    SetEnvIfNoCase User-Agent "SemrushBot-SA" bad_bot #181203
    SetEnvIfNoCase User-Agent "DomainCrawler" bad_bot #181210
    SetEnvIfNoCase User-Agent "MegaIndex.ru" bad_bot #181215
    SetEnvIfNoCase User-Agent "AlphaBot" bad_bot #181219

    #기타 귀찮은 것들
    SetEnvIfNoCase User-Agent "ltx71" bad_bot
    SetEnvIfNoCase User-Agent "CCBot" bad_bot
    SetEnvIfNoCase User-Agent "Sogou" bad_bot
    SetEnvIfNoCase User-Agent "DotBot" bad_bot
    SetEnvIfNoCase User-Agent "PiplBot" bad_bot
    SetEnvIfNoCase User-Agent "MJ12bot" bad_bot
    SetEnvIfNoCase User-Agent "AhrefsBot" bad_bot
    SetEnvIfNoCase User-Agent "MauiBot" bad_bot
    SetEnvIfNoCase User-Agent "AhrefsBot" bad_bot
    SetEnvIfNoCase User-Agent "ezooms" bad_bot
    SetEnvIfNoCase User-Agent "sistrix" bad_bot
    SetEnvIfNoCase User-Agent "Yandex" bad_bot


    # Block Bad Bots & Scrapers
    SetEnvIfNoCase User-Agent "serpstatbot" bad_bot
    SetEnvIfNoCase User-Agent "DataForSeoBot" bad_bot
    SetEnvIfNoCase User-Agent "BLEXBot" bad_bot
    SetEnvIfNoCase User-Agent "Daumoa" bad_bot
    SetEnvIfNoCase User-Agent "petalbot" bad_bot
    SetEnvIfNoCase User-Agent "^Daum" bad_bot
    SetEnvIfNoCase User-Agent "Aboundex" bad_bot
    SetEnvIfNoCase User-Agent "80legs" bad_bot
    SetEnvIfNoCase User-Agent "360Spider" bad_bot
    SetEnvIfNoCase User-Agent "^Java" bad_bot
    SetEnvIfNoCase User-Agent "^Cogentbot" bad_bot
    SetEnvIfNoCase User-Agent "^Alexibot" bad_bot
    SetEnvIfNoCase User-Agent "^asterias" bad_bot
    SetEnvIfNoCase User-Agent "^attach" bad_bot
    SetEnvIfNoCase User-Agent "^BackDoorBot" bad_bot
    SetEnvIfNoCase User-Agent "^BackWeb" bad_bot
    SetEnvIfNoCase User-Agent "Bandit" bad_bot
    SetEnvIfNoCase User-Agent "^BatchFTP" bad_bot
    SetEnvIfNoCase User-Agent "^Bigfoot" bad_bot
    SetEnvIfNoCase User-Agent "^Black.Hole" bad_bot
    SetEnvIfNoCase User-Agent "^BlackWidow" bad_bot
    SetEnvIfNoCase User-Agent "^BlowFish" bad_bot
    SetEnvIfNoCase User-Agent "^BotALot" bad_bot
    SetEnvIfNoCase User-Agent "Buddy" bad_bot
    SetEnvIfNoCase User-Agent "^BuiltBotTough" bad_bot
    SetEnvIfNoCase User-Agent "^Bullseye" bad_bot
    SetEnvIfNoCase User-Agent "^BunnySlippers" bad_bot
    SetEnvIfNoCase User-Agent "^Cegbfeieh" bad_bot
    SetEnvIfNoCase User-Agent "^CheeseBot" bad_bot
    SetEnvIfNoCase User-Agent "^CherryPicker" bad_bot
    SetEnvIfNoCase User-Agent "^ChinaClaw" bad_bot
    SetEnvIfNoCase User-Agent "Collector" bad_bot
    SetEnvIfNoCase User-Agent "Copier" bad_bot
    SetEnvIfNoCase User-Agent "^CopyRightCheck" bad_bot
    SetEnvIfNoCase User-Agent "^cosmos" bad_bot
    SetEnvIfNoCase User-Agent "^Crescent" bad_bot
    SetEnvIfNoCase User-Agent "^Custo" bad_bot
    SetEnvIfNoCase User-Agent "^AIBOT" bad_bot
    SetEnvIfNoCase User-Agent "^DISCo" bad_bot
    SetEnvIfNoCase User-Agent "^DIIbot" bad_bot
    SetEnvIfNoCase User-Agent "^DittoSpyder" bad_bot
    SetEnvIfNoCase User-Agent "^Download\ Demon" bad_bot
    SetEnvIfNoCase User-Agent "^Download\ Devil" bad_bot
    SetEnvIfNoCase User-Agent "^Download\ Wonder" bad_bot
    SetEnvIfNoCase User-Agent "^dragonfly" bad_bot
    SetEnvIfNoCase User-Agent "^Drip" bad_bot
    SetEnvIfNoCase User-Agent "^eCatch" bad_bot
    SetEnvIfNoCase User-Agent "^EasyDL" bad_bot
    SetEnvIfNoCase User-Agent "^ebingbong" bad_bot
    SetEnvIfNoCase User-Agent "^EirGrabber" bad_bot
    SetEnvIfNoCase User-Agent "^EmailCollector" bad_bot
    SetEnvIfNoCase User-Agent "^EmailSiphon" bad_bot
    SetEnvIfNoCase User-Agent "^EmailWolf" bad_bot
    SetEnvIfNoCase User-Agent "^EroCrawler" bad_bot
    SetEnvIfNoCase User-Agent "^Exabot" bad_bot
    SetEnvIfNoCase User-Agent "^Express\ WebPictures" bad_bot
    SetEnvIfNoCase User-Agent "Extractor" bad_bot
    SetEnvIfNoCase User-Agent "^EyeNetIE" bad_bot
    SetEnvIfNoCase User-Agent "^Foobot" bad_bot
    SetEnvIfNoCase User-Agent "^flunky" bad_bot
    SetEnvIfNoCase User-Agent "^FrontPage" bad_bot
    SetEnvIfNoCase User-Agent "^Go-Ahead-Got-It" bad_bot
    SetEnvIfNoCase User-Agent "^gotit" bad_bot
    SetEnvIfNoCase User-Agent "^GrabNet" bad_bot
    SetEnvIfNoCase User-Agent "^Grafula" bad_bot
    SetEnvIfNoCase User-Agent "^Harvest" bad_bot
    SetEnvIfNoCase User-Agent "^hloader" bad_bot
    SetEnvIfNoCase User-Agent "^HMView" bad_bot
    SetEnvIfNoCase User-Agent "^HTTrack" bad_bot
    SetEnvIfNoCase User-Agent "^humanlinks" bad_bot
    SetEnvIfNoCase User-Agent "^IlseBot" bad_bot
    SetEnvIfNoCase User-Agent "^Image\ Stripper" bad_bot
    SetEnvIfNoCase User-Agent "^Image\ Sucker" bad_bot
    SetEnvIfNoCase User-Agent "Indy\ Library" bad_bot
    SetEnvIfNoCase User-Agent "^InfoNaviRobot" bad_bot
    SetEnvIfNoCase User-Agent "^InfoTekies" bad_bot
    SetEnvIfNoCase User-Agent "^Intelliseek" bad_bot
    SetEnvIfNoCase User-Agent "^InterGET" bad_bot
    SetEnvIfNoCase User-Agent "^Internet\ Ninja" bad_bot
    SetEnvIfNoCase User-Agent "^Iria" bad_bot
    SetEnvIfNoCase User-Agent "^Jakarta" bad_bot
    SetEnvIfNoCase User-Agent "^JennyBot" bad_bot
    SetEnvIfNoCase User-Agent "^JetCar" bad_bot
    SetEnvIfNoCase User-Agent "^JOC" bad_bot
    SetEnvIfNoCase User-Agent "^JustView" bad_bot
    SetEnvIfNoCase User-Agent "^Jyxobot" bad_bot
    SetEnvIfNoCase User-Agent "^Kenjin.Spider" bad_bot
    SetEnvIfNoCase User-Agent "^Keyword.Density" bad_bot
    SetEnvIfNoCase User-Agent "^larbin" bad_bot
    SetEnvIfNoCase User-Agent "^LexiBot" bad_bot
    SetEnvIfNoCase User-Agent "^lftp" bad_bot
    SetEnvIfNoCase User-Agent "^libWeb/clsHTTP" bad_bot
    SetEnvIfNoCase User-Agent "^likse" bad_bot
    SetEnvIfNoCase User-Agent "^LinkextractorPro" bad_bot
    SetEnvIfNoCase User-Agent "^LinkScan/8.1a.Unix" bad_bot
    SetEnvIfNoCase User-Agent "^LNSpiderguy" bad_bot
    SetEnvIfNoCase User-Agent "^LinkWalker" bad_bot
    SetEnvIfNoCase User-Agent "^lwp-trivial" bad_bot
    SetEnvIfNoCase User-Agent "^LWP::Simple" bad_bot
    SetEnvIfNoCase User-Agent "^Magnet" bad_bot
    SetEnvIfNoCase User-Agent "^Mag-Net" bad_bot
    SetEnvIfNoCase User-Agent "^MarkWatch" bad_bot
    SetEnvIfNoCase User-Agent "^Mass\ Downloader" bad_bot
    SetEnvIfNoCase User-Agent "^Mata.Hari" bad_bot
    SetEnvIfNoCase User-Agent "^Memo" bad_bot
    SetEnvIfNoCase User-Agent "^Microsoft.URL" bad_bot
    SetEnvIfNoCase User-Agent "^Microsoft\ URL\ Control" bad_bot
    SetEnvIfNoCase User-Agent "^MIDown\ tool" bad_bot
    SetEnvIfNoCase User-Agent "^MIIxpc" bad_bot
    SetEnvIfNoCase User-Agent "^Mirror" bad_bot
    SetEnvIfNoCase User-Agent "^Missigua\ Locator" bad_bot
    SetEnvIfNoCase User-Agent "^Mister\ PiX" bad_bot
    SetEnvIfNoCase User-Agent "^moget" bad_bot
    SetEnvIfNoCase User-Agent "^Mozilla/3.Mozilla/2.01" bad_bot
    SetEnvIfNoCase User-Agent "^Mozilla.*NEWT" bad_bot
    SetEnvIfNoCase User-Agent "^NAMEPROTECT" bad_bot
    SetEnvIfNoCase User-Agent "^Navroad" bad_bot
    SetEnvIfNoCase User-Agent "^NearSite" bad_bot
    SetEnvIfNoCase User-Agent "^NetAnts" bad_bot
    SetEnvIfNoCase User-Agent "^Netcraft" bad_bot
    SetEnvIfNoCase User-Agent "^NetMechanic" bad_bot
    SetEnvIfNoCase User-Agent "^NetSpider" bad_bot
    SetEnvIfNoCase User-Agent "^Net\ Vampire" bad_bot
    SetEnvIfNoCase User-Agent "^NetZIP" bad_bot
    SetEnvIfNoCase User-Agent "^NextGenSearchBot" bad_bot
    SetEnvIfNoCase User-Agent "^NG" bad_bot
    SetEnvIfNoCase User-Agent "^NICErsPRO" bad_bot
    SetEnvIfNoCase User-Agent "^niki-bot" bad_bot
    SetEnvIfNoCase User-Agent "^NimbleCrawler" bad_bot
    SetEnvIfNoCase User-Agent "^Ninja" bad_bot
    SetEnvIfNoCase User-Agent "^NPbot" bad_bot
    SetEnvIfNoCase User-Agent "^Octopus" bad_bot
    SetEnvIfNoCase User-Agent "^Offline\ Explorer" bad_bot
    SetEnvIfNoCase User-Agent "^Offline\ Navigator" bad_bot
    SetEnvIfNoCase User-Agent "^Openfind" bad_bot
    SetEnvIfNoCase User-Agent "^OutfoxBot" bad_bot
    SetEnvIfNoCase User-Agent "^PageGrabber" bad_bot
    SetEnvIfNoCase User-Agent "^Papa\ Foto" bad_bot
    SetEnvIfNoCase User-Agent "^pavuk" bad_bot
    SetEnvIfNoCase User-Agent "^pcBrowser" bad_bot
    SetEnvIfNoCase User-Agent "^PHP\ version\ tracker" bad_bot
    SetEnvIfNoCase User-Agent "^Pockey" bad_bot
    SetEnvIfNoCase User-Agent "^ProPowerBot/2.14" bad_bot
    SetEnvIfNoCase User-Agent "^ProWebWalker" bad_bot
    SetEnvIfNoCase User-Agent "^psbot" bad_bot
    SetEnvIfNoCase User-Agent "^Pump" bad_bot
    SetEnvIfNoCase User-Agent "^QueryN.Metasearch" bad_bot
    SetEnvIfNoCase User-Agent "^RealDownload" bad_bot
    SetEnvIfNoCase User-Agent "Reaper" bad_bot
    SetEnvIfNoCase User-Agent "Recorder" bad_bot
    SetEnvIfNoCase User-Agent "^ReGet" bad_bot
    SetEnvIfNoCase User-Agent "^RepoMonkey" bad_bot
    SetEnvIfNoCase User-Agent "^RMA" bad_bot
    SetEnvIfNoCase User-Agent "Siphon" bad_bot
    SetEnvIfNoCase User-Agent "^SiteSnagger" bad_bot
    SetEnvIfNoCase User-Agent "^SlySearch" bad_bot
    SetEnvIfNoCase User-Agent "^SmartDownload" bad_bot
    SetEnvIfNoCase User-Agent "^Snake" bad_bot
    SetEnvIfNoCase User-Agent "^Snapbot" bad_bot
    SetEnvIfNoCase User-Agent "^Snoopy" bad_bot
    SetEnvIfNoCase User-Agent "^sogou" bad_bot
    SetEnvIfNoCase User-Agent "^SpaceBison" bad_bot

    Order Allow,Deny
    Allow from all
    # Cyveillance
    deny from 88.214.26.0/24
    deny from 85.25.176.0/20
    deny from 85.25.192.0/20
    deny from 85.25.208.0/22
    deny from 217.113.194.0/24
    deny from 61.249.92.0/22
    deny from 109.237.96.0/22
    deny from 133.18.224.0/23
    deny from 38.100.19.8/29
    deny from 17.0.0.0/8
    deny from 114.119.128.0/18
    deny from 57.128.80.0/21
    deny from 193.150.70.0/24
    deny from 38.100.21.0/24
    deny from 38.100.41.64/26
    deny from 38.105.71.0/25
    deny from 38.105.83.0/27
    deny from 38.112.21.140/30
    deny from 38.118.42.32/29
    deny from 65.213.208.128/27
    deny from 65.222.176.96/27
    deny from 65.222.185.72/29
    deny from 211.249.0.0/16
    Deny from env=bad_bot
    </Directory>
    <Directory /usr/local/apache/htdocs>

    Order Allow,Deny
    deny from 88.214.26.0/24
    Deny from 57.128.80.0/21
    deny from 85.25.176.0/20
    deny from 85.25.192.0/20
    deny from 85.25.208.0/22
    deny from 217.113.194.0/24

    Allow from all



    nginxnginx-badbot-blocker차단
    ap $http_user_agent $bad_bot {
    default 0;
    ~*^Lynx 0; # Let Lynx go through
    libwww-perl 1;
    ~*(?i)(80legs|360Spider|Aboundex|AhrefsBot|Daumoa|DataForSeoBot|DaumBot|applebot|BLEXBot|serpstatbot|MediaMathbot|Abonti|Acunetix|^AIBOT|^Alexibot|Alligator|AllSubmitter|Apexoo|^asterias|^attach|^BackDoorBot|^BackStreet|^BackWeb|Badass|Bandit|petalbot|Baid|Baiduspider|^BatchFTP|^Bigfoot|^Black.Hole|^BlackWidow|BlackWidow|^BlowFish|Blow|^BotALot|Buddy|^BuiltBotTough|^Bullseye|^BunnySlippers|BBBike|^Cegbfeieh|^CheeseBot|^CherryPicker|^ChinaClaw|^Cogentbot|CPython|Collector|cognitiveseo|Copier|^CopyRightCheck|^cosmos|^Crescent|CSHttp|^Custo|^Demon|^Devil|^DISCo|^DIIbot|discobot|^DittoSpyder|Download.Demon|Download.Devil|Download.Wonder|^dragonfly|^Drip|^eCatch|^EasyDL|^ebingbong|^EirGrabber|^EmailCollector|^EmailSiphon|^EmailWolf|^EroCrawler|^Exabot|^Express|Extractor|^EyeNetIE|FHscan|^FHscan|^flunky|^Foobot|^FrontPage|GalaxyBot|^gotit|Grabber|^GrabNet|^Grafula|^Harvest|^HEADMasterSEO|^hloader|^HMView|^HTTrack|httrack|HTTrack|htmlparser|^humanlinks|^IlseBot|Image.Stripper|Image.Sucker|imagefetch|^InfoNaviRobot|^InfoTekies|^Intelliseek|^InterGET|^Iria|^Jakarta|^JennyBot|^JetCar|JikeSpider|^JOC|^JustView|^Jyxobot|^Kenjin.Spider|^Keyword.Density|libwww|^larbin|LeechFTP|LeechGet|^LexiBot|^lftp|^libWeb|^likse|^LinkextractorPro|^LinkScan|^LNSpiderguy|^LinkWalker|msnbot|MSIECrawler|MJ12bot|MegaIndex|^Magnet|^Mag-Net|^MarkWatch|Mass.Downloader|masscan|^Mata.Hari|^Memo|^MIIxpc|^NAMEPROTECT|^Navroad|^NearSite|^NetAnts|^Netcraft|^NetMechanic|^NetSpider|^NetZIP|^NextGenSearchBot|^NICErsPRO|^niki-bot|^NimbleCrawler|^Nimbostratus-Bot|^Ninja|^Nmap|nmap|^NPbot|Offline.Explorer|Offline.Navigator|OpenLinkProfiler|^Octopus|^Openfind|^OutfoxBot|Pixray|probethenet|proximic|^PageGrabber|^pavuk|^pcBrowser|^Pockey|^ProPowerBot|^ProWebWalker|^psbot|^Pump|python-requests|^QueryN.Metasearch|^RealDownload|Reaper|^Reaper|^Ripper|Ripper|Recorder|^ReGet|^RepoMonkey|^RMA|scanbot|SEOkicks-Robot|seoscanners|^Stripper|^Sucker|Siphon|Siteimprove|^SiteSnagger|SiteSucker|^SlySearch|^SmartDownload|^Snake|^Snapbot|^Snoopy|Sosospider|^sogou|spbot|^SpaceBison|^spanner|^SpankBot|Spinn4r|^Sqworm|Sqworm|Stripper|Sucker|^SuperBot|SuperHTTP|^SuperHTTP|^Surfbot|^suzuran|^Szukacz|^tAkeOut|^Teleport|^Telesoft|^TurnitinBot|^The.Intraformant|^TheNomad|^TightTwatBot|^Titan|^True_Robot|^turingos|^TurnitinBot|^URLy.Warning|^Vacuum|^VCI|VidibleScraper|^VoidEYE|^WebAuto|^WebBandit|^WebCopier|^WebEnhancer|^WebFetch|^Web.Image.Collector|^WebLeacher|^WebmasterWorldForumBot|WebPix|^WebReaper|^WebSauger|Website.eXtractor|^Webster|WebShag|^WebStripper|WebSucker|^WebWhacker|^WebZIP|Whack|Whacker|^Widow|Widow|WinHTTrack|^WISENutbot|WWWOFFLE|^WWWOFFLE|^WWW-Collector-E|^Xaldon|^Xenu|^Zade|^Zeus|ZmEu|^Zyborg|SemrushBot|^WebFuck|^MJ12bot|^majestic12|^WallpapersHD) 1;
    }

    ## Add here all referrers that are to blocked.
    map $http_referer $bad_referer {
    default 0;
    ~(?i)(adcash|advair|allegra|ambien|amoxicillin|adult|anal|asshole|babes|baccarat|betting|bithack|blackjack|cash|casino|celeb|cheap|cialis|craps|credit|click|cunt|deal|debt|drug|diamond|effexor|equity|faxo|finance|fisting|forsale|gambling|gaysex|girl|hardcore|hold-em|holdem|iconsurf|ilovevitaly|insurance|interest|internetsupervision|jewelry|keno|levitra|lipitor|loan|loans|love|makemoneyonline|make-money-online|meds|money|mortgage|myftpupload|nudit|omaha|organic|paxil|pharmacy|pharmacies|phentermine|pheromone|pills|piss|poker|porn|poweroversoftware|refinance|replica|rimming|roulette|screentoolkit|seoexperimenty|sex|snuff|scout|seventwentyfour|slot|slots|syntryx|teen|texas|t0phackteam|tournament|tramadol|tramidol|valtrex|vvakhrin-ws1|viagra|vicodin|webcam|xanax|xnxx|xxxrus|zanax|zippo|zoloft) 1;
    }

    ## Add here all bad referer domains to be blocked - broken up into sections
    ## Alphabetical A - E (incl numbers)
    map $http_referer $bad_urls1 {
    default 0;
    ~(?i)(^http://(www\.)?38ha(-|.).*$|^http://(www\.)?4free(-|.).*$|^http://(www\.)?4hs8(-|.).*$|^http://(www\.)?4t(-|.).*$|^http://(www\.)?4u(-|.).*$|^http://(www\.)?6q(-|.).*$|^http://(www\.)?7makemoneyonline(-|.).*$|^http://(www\.)?8gold(-|.).*$|^http://(www\.)?911(-|.).*$|^http://(www\.)?adcash(-|.).*$|^http://(www\.)?.*(-|.)?adult(-|.).*$|^http://(www\.)?.*(-|.)?acunetix-referrer(-|.).*$|^http://(www\.)?abalone(-|.).*$|^http://(www\.)?adminshop(-|.).*$|^http://(www\.)?adultactioncam(-|.).*$|^http://(www\.)?aizzo(-|.).*$|^http://(www\.)?alphacarolinas(-|.).*$|^http://(www\.)?amateur(-|.).*$|^http://(www\.)?amateurxpass(-|.).*$|^http://(www\.)?.*(-|.)?anal(-|.).*$|^http://(www\.)?ansar-u-deen(-|.).*$|^http://(www\.)?atelebanon(-|.).*$|^http://(www\.)?beastiality(-|.).*$|^http://(www\.)?bestiality(-|.).*$|^http://(www\.)?belize(-|.).*$|^http://(www\.)?best-deals(-|.).*$|^http://(www\.)?bithack(-|.).*$|^http://(www\.)?blogincome(-|.).*$|^http://(www\.)?bontril(-|.).*$|^http://(www\.)?bruce-holdeman(-|.).*$|^http://(www\.)?.*(-|.)?blow.?job(-|.).*$|^http://(www\.)?buttons-for-website(-|.).*$|^http://(www\.)?ca-america(-|.).*$|^http://(www\.)?chatt-net(-|.).*$|^http://(www\.)?cenokos(-|.).*$|^http://(www\.)?cenoval(-|.).*$|^http://(www\.)?cityadspix(-|.).*$|^http://(www\.)?commerce(-|.).*$|^http://(www\.)?condo(-|.).*$|^http://(www\.)?conjuratia(-|.).*$|^http://(www\.)?consolidate(-|.).*$|^http://(www\.)?coswap(-|.).*$|^http://(www\.)?crescentarian(-|.).*$|^http://(www\.)?crepesuzette(-|.).*$|^http://(www\.)?darodar(-|.).*$|^http://(www\.)?dating(-|.).*$|^http://(www\.)?devaddict(-|.).*$|^http://(www\.)?discount(-|.).*$|^http://(www\.)?doobu(-|.).*$|^http://(www\.)?domainsatcost(-|.).*$|^http://(www\.)?econom.co(-|.).*$|^http://(www\.)?edakgfvwql(-|.).*$|^http://(www\.)?.*(-|.)?sex(-|.).*$|^http://(www\.)?e-site(-|.).*$|^http://(www\.)?egygift(-|.).*$|^http://(www\.)?empathica(-|.).*$|^http://(www\.)?empirepoker(-|.).*$|^http://(www\.)?e-poker-2005(-|.).*$|^http://(www\.)?escal8(-|.).*$|^http://(www\.)?eurip(-|.).*$|^http://(www\.)?exitq(-|.).*$|^http://(www\.)?eyemagination(-|.).*$) 1;
    }
    ## F - I
    map $http_referer $bad_urls2 {
    default 0;
    ~(?i)(^http://(www\.)?fastcrawl(-|.).*$|^http://(www\.)?fearcrow(-|.).*$|^http://(www\.)?ferretsoft(-|.).*$|^http://(www\.)?fick(-|.).*$|^http://(www\.)?finance(-|.).*$|^http://(www\.)?flafeber(-|.).*$|^http://(www\.)?fidelityfunding(-|.).*$|^http://(www\.)?freakycheats(-|.).*$|^http://(www\.)?freeality(-|.).*$|^http://(www\.)?fuck(-|.).*$|^http://(www\.)?future-2000(-|.).*$|^http://(www\.)?.*(-|.)?gay(-|.).*$|^http://(www\.)?gobongo.info(-|.).*$|^http://(www\.)?gabriola(-|.).*$|^http://(www\.)?gallerylisting(-|.).*$|^http://(www\.)?gb.com(-|.).*$|^http://(www\.)?ghostvisitor(-|.).*$|^http://(www\.)?globusy(-|.).*$|^http://(www\.)?golf-e-course(-|.).*$|^http://(www\.)?gospelcom(-|.).*$|^http://(www\.)?gradfinder(-|.).*$|^http://(www\.)?hasfun(-|.).*$|^http://(www\.)?herbal(-|.).*$|^http://(www\.)?hermosa(-|.).*$|^http://(www\.)?highprofitclub(-|.).*$|^http://(www\.)?hilton(-|.).*$|^http://(www\.)?teaminspection(-|.).*$|^http://(www\.)?hotel(-|.).*$|^http://(www\.)?houseofseven(-|.).*$|^http://(www\.)?hurricane(-|.).*$|^http://(www\.)?.*(-|.)?incest(-|.).*$|^http://(www\.)?iaea(-|.).*$|^http://(www\.)?ilovevitality(-|.).*$|^http://(www\.)?ime(-|.).*$|^http://(www\.)?info(-|.).*$|^http://(www\.)?ingyensms(-|.).*$|^http://(www\.)?inkjet-toner(-|.).*$|^http://(www\.)?isacommie(-|.).*$|^http://(www\.)?istarthere(-|.).*$|^http://(www\.)?it.tt(-|.).*$|^http://(www\.)?italiancharms(-|.).*$|^http://(www\.)?iwantu(-|.).*$|^http://(www\.)?ilovevitality(-|.).*$|^http://(www\.)?iskalko.ru(-|.).*$) 1;
    }
    ## J - P
    map $http_referer $bad_urls3 {
    default 0;
    ~(?i)(^http://(www\.)?jfcadvocacy(-|.).*$|^http://(www\.)?jmhic(-|.).*$|^http://(www\.)?juris(-|.).*$|^http://(www\.)?kylos(-|.).*$|^http://(www\.)?laser-eye(-|.).*$|^http://(www\.)?leathertree(-|.).*$|^http://(www\.)?lillystar(-|.).*$|^http://(www\.)?linkerdome(-|.).*$|^http://(www\.)?livenet(-|.).*$|^http://(www\.)?low-limit(-|.).*$|^http://(www\.)?lowest-price(-|.).*$|^http://(www\.)?luxup.ru(-|.).*$|^http://(www\.)?macsurfer(-|.).*$|^http://(www\.)?mall.uk(-|.).*$|^http://(www\.)?maloylawn(-|.).*$|^http://(www\.)?marketing(-|.).*$|^http://(www\.)?.*(-|.)?mature(-|.).*$|^http://(www\.)?mcdortaklar(-|.).*$|^http://(www\.)?mediavisor(-|.).*$|^http://(www\.)?medications(-|.).*$|^http://(www\.)?mirror.sytes(-|.).*$|^http://(www\.)?mp3(-|.).*$|^http://(www\.)?(-|.)musicbox1(-|.).*$|^http://(www\.)?myftpupload(-|.).*$|^http://(www\.)?naked(-|.).*$|^http://(www\.)?netdisaster(-|.).*$|^http://(www\.)?netfirms(-|.).*$|^http://(www\.)?newtruths(-|.).*$|^http://(www\.)?no-limit(-|.).*$|^http://(www\.)?nude(-|.).*$|^http://(www\.)?nudeceleb(-|.).*$|^http://(www\.)?nutzu(-|.).*$|^http://(www\.)?odge(-|.).*$|^http://(www\.)?oiline(-|.).*$|^http://(www\.)?onlinegamingassoc(-|.).*$|^http://(www\.)?outpersonals(-|.).*$|^http://(www\.)?o-o-6-o-o.ru(-|.).*$|^http://(www\.)?o-o-8-o-o.ru(-|.).*$|^http://(www\.)?pagetwo(-|.).*$|^http://(www\.)?paris(-|.).*$|^http://(www\.)?passions(-|.).*$|^http://(www\.)?peblog(-|.).*$|^http://(www\.)?peng(-|.).*$|^http://(www\.)?perfume-cologne(-|.).*$|^http://(www\.)?personal(-|.).*$|^http://(www\.)?php-soft(-|.).*$|^http://(www\.)?pisoc(-|.).*$|^http://(www\.)?pisx(-|.).*$|^http://(www\.)?popwow(-|.).*$|^http://(www\.)?porn(-|.).*$|^http://(www\.)?prescriptions(-|.).*$|^http://(www\.)?priceg(-|.).*$|^http://(www\.)?.*(-|.)?pus*y(-|.).*$|^http://(www\.)?printdirectforless(-|.).*$|^http://(www\.)?ps2cool(-|.).*$|^http://(www\.)?psnarones(-|.).*$|^http://(www\.)?psxtreme(-|.).*$) 1;
    }
    ## Q - Z
    map $http_referer $bad_urls4 {
    default 0;
    ~(?i)(^http://(www\.)?quality-traffic(-|.).*$|^http://(www\.)?registrarprice(-|.).*$|^http://(www\.)?reliableresults(-|.).*$|^http://(www\.)?rimpim(-|.).*$|^http://(www\.)?ro7kalbe(-|.).*$|^http://(www\.)?rohkalby(-|.).*$|^http://(www\.)?ronnieazza(-|.).*$|^http://(www\.)?rulo.biz(-|.).*$|^http://(www\.)?responsinator(-|.).*$|^http://(www\.)?s5(-|.).*$|^http://(www\.)?samiuls(-|.).*$|^http://(www\.)?savefrom(-|.).*$|^http://(www\.)?savetubevideo.com(-|.).*$|^http://(www\.)?screentoolkit.com(-|.).*$|^http://(www\.)?searchedu(-|.).*$|^http://(www\.)?semalt.com(-|.).*$|^http://(www\.)?seoexperimenty(-|.).*$|^http://(www\.)?seventwentyfour(-|.).*$|^http://(www\.)?seventwentyfour.*$|^http://(www\.)?sex(-|.).*$|^http://(www\.)?sexsearch(-|.).*$|^http://(www\.)?sexsq(-|.).*$|^http://(www\.)?shoesdiscount(-|.).*$|^http://(www\.)?site-4u(-|.).*$|^http://(www\.)?site5(-|.).*$|^http://(www\.)?slatersdvds(-|.).*$|^http://(www\.)?slftsdybbg.ru(-|.).*$|^http://(www\.)?sml338(-|.).*$|^http://(www\.)?sms(-|.).*$|^http://(www\.)?smsportali(-|.).*$|^http://(www\.)?socialseet.ru(-|.).*$|^http://(www\.)?software(-|.).*$|^http://(www\.)?sortthemesitesby(-|.).*$|^http://(www\.)?spears(-|.).*$|^http://(www\.)?spoodles(-|.).*$|^http://(www\.)?sportsparent(-|.).*$|^http://(www\.)?srecorder(-|.).*$|^http://(www\.)?stmaryonline(-|.).*$|^http://(www\.)?superiends.org(-|.).*$|^http://(www\.)?strip(-|.).*$|^http://(www\.)?suttonjames(-|.).*$|^http://(www\.)?talk.uk-yankee(-|.).*$|^http://(www\.)?tecrep-inc(-|.).*$|^http://(www\.)?teen(-|.).*$|^http://(www\.)?terashells(-|.).*$|^http://(www\.)?thatwhichis(-|.).*$|^http://(www\.)?thorcarlson(-|.).*$|^http://(www\.)?.*(-|.)?tits(-|.).*$|^http://(www\.)?.*(-|.)?titten(-|.).*$|^http://(www\.)?tmsathai(-|.).*$|^http://(www\.)?traffixer(-|.).*$|^http://(www\.)?tranny(-|.).*$|^http://(www\.)?valeof(-|.).*$|^http://(www\.)?video(-|.).*$|^http://(www\.)?vinhas(-|.).*$|^http://(www\.)?vixen1(-|.).*$|^http://(www\.)?vpshs(-|.).*$|^http://(www\.)?vrajitor(-|.).*$|^http://(www\.)?vodkoved.ru(-|.).*$|^http://(www\.)?w3md(-|.).*$|^http://(www\.)?websocial.me(-|.).*$|^http://(www\.)?webdevsquare(-|.).*$|^http://(www\.)?whois(-|.).*$|^http://(www\.)?withdrawal(-|.).*$|^http://(www\.)?worldemail(-|.).*$|^http://(www\.)?wslp24(-|.).*$|^http://(www\.)?ws-op(-|.).*$|^http://(www\.)?xnxx(-|.).*$|^http://(www\.)?xopy(-|.).*$|^http://(www\.)?xxx(-|.).*$|^http://(www\.)?yelucie(-|.).*$|^http://(www\.)?youradulthosting(-|.).*$|^http://(www\.)?ykecwqlixx.ru(-|.).*$|^http://(www\.)?yougetsignal.com(-|.).*$|^http://(www\.)?(-|.)zindagi(-|.).*$) 1;
    }
    ## Domains Linked to Yontoo Browser Malware and a Few Other New Ones
    ## Have split this into it's own section to keep lines shorter NOTE: changes to instructions
    ## adding if ($bad_urls5) and if ($bad_urls6) to your site(s) config.
    map $http_referer $bad_urls5 {
    default 0;
    ~(?i)(^http://(www\.)?101raccoon.ru(-|.).*$|^http://(www\.)?28n2gl3wfyb0.ru(-|.).*$|^http://(www\.)?627ad6438b58439cad1fc8cf6d67a92e.com(-|.).*$|^http://(www\.)?6ab9743d0152486387559b4abaa02ada.com(-|.).*$|^http://(www\.)?a342ae9750004b14b55f7310eff0ab65.com(-|.).*$|^http://(www\.)?aa08daf7e13b6345e09e92f771507fa5f4.com(-|.).*$|^http://(www\.)?aa14ab57a3339c4064bd9ae6fad7495b5f.com(-|.).*$|^http://(www\.)?aa625d84f1587749c1ab011d6f269f7d64.com(-|.).*$|^http://(www\.)?aa81bf391151884adfa3dd677e41f94be1.com(-|.).*$|^http://(www\.)?aa8780bb28a1de4eb5bff33c28a218a930.com(-|.).*$|^http://(www\.)?aa8b68101d388c446389283820863176e7.com(-|.).*$|^http://(www\.)?aa9bd78f328a6a41279d0fad0a88df1901.com(-|.).*$|^http://(www\.)?aa9d046aab36af4ff182f097f840430d51.com(-|.).*$|^http://(www\.)?aaa38852e886ac4af1a3cff9b47cab6272.com(-|.).*$|^http://(www\.)?aab94f698f36684c5a852a2ef272e031bb.com(-|.).*$|^http://(www\.)?aac500b7a15b2646968f6bd8c6305869d7.com(-|.).*$|^http://(www\.)?aac52006ec82a24e08b665f4db2b5013f7.com(-|.).*$|^http://(www\.)?aad1f4acb0a373420d9b0c4202d38d94fa.com(-|.).*$|^http://(www\.)?asrv-a.akamoihd.net(-|.).*$|^http://(www\.)?asrvrep-a.akamaihd.net(-|.).*$|^http://(www\.)?bestpriceninja.com(-|.).*$|^http://(www\.)?bronzeaid-a.akamaihd.net(-|.).*$|^http://(www\.)?browsepulse-a.akamaihd.net(-|.).*$|^http://(www\.)?cashkitten-a.akamaihd.net(-|.).*$|^http://(www\.)?coolbar.pro(-|.).*$) 1;
    }
    map $http_referer $bad_urls6 {
    default 0;
    ~(?i)(^http://(www\.)?davebestdeals.com(-|.).*$|^http://(www\.)?discovertreasure-a.akamaihd.net(-|.).*$|^http://(www\.)?discovertreasurenow.com(-|.).*$|^http://(www\.)?foxydeal.com(-|.).*$|^http://(www\.)?gameonasia.com(-|.).*$|^http://(www\.)?gameplexcity.com(-|.).*$|^http://(www\.)?gamerextra.com(-|.).*$|^http://(www\.)?gamerscorps.com(-|.).*$|^http://(www\.)?gamewrath.com(-|.).*$|^http://(www\.)?generousdeal-a.akamaihd.net(-|.).*$|^http://(www\.)?girlgamerdaily.com(-|.).*$|^http://(www\.)?hdapp1008-a.akamaihd.net(-|.).*$|^http://(www\.)?highstairs-a.akamaihd.net(-|.).*$|^http://(www\.)?hotshoppymac.com(-|.).*$|^http://(www\.)?matchpal-a.akamaihd.net(-|.).*$|^http://(www\.)?mecash.ru(-|.).*$|^http://(www\.)?monarchfind-a.akamaihd.net(-|.).*$|^http://(www\.)?myshopmatemac.com(-|.).*$|^http://(www\.)?nottyu.xyz(-|.).*$|^http://(www\.)?onlinemegax.com(-|.).*$|^http://(www\.)?outrageousdeal-a.akamaihd.net(-|.).*$|^http://(www\.)?pijoto.net(-|.).*$|^http://(www\.)?recordpage-a.akamaihd.net(-|.).*$|^http://(www\.)?resultshub-a.akamaihd.net(-|.).*$|^http://(www\.)?rvzr-a.akamaihd.net(-|.).*$|^http://(www\.)?savingsslider-a.akamaihd.net(-|.).*$|^http://(www\.)?searchinterneat-a.akamaihd.net(-|.).*$|^http://(www\.)?searchwebknow-a.akamaihd.net(-|.).*$|^http://(www\.)?seeresultshub-a.akamaihd.net(-|.).*$|^http://(www\.)?shoppytoolmac.com(-|.).*$|^http://(www\.)?skytraf.xyz(-|.).*$|^http://(www\.)?splendorsearch-a.akamaihd.net(-|.).*$|^http://(www\.)?strongsignal-a.akamaihd.net(-|.).*$|^http://(www\.)?surfbuyermac.com(-|.).*$|^http://(www\.)?treasuretrack-a.akamaihd.net(-|.).*$|^http://(www\.)?webshoppermac.com(-|.).*$|^http://(www\.)?pospr.waw.pl(-|.).*$|^http://(www\.)?abclauncher.com(-|.).*$|^http://(www\.)?alert-fjg.xyz(-|.).*$|^http://(www\.)?analytics-ads.xyz(-|.).*$|^http://(www\.)?bamo.xsl.pt(-|.).*$|^http://(www\.)?compliance-olga.top(-|.).*$|^http://(www\.)?digital-video-processing.com(-|.).*$|^http://(www\.)?eu-cookie-law.info(-|.).*$|^http://(www\.)?findpik.com(-|.).*$|^http://(www\.)?forum20.smailik.org(-|.).*$|^http://(www\.)?free-share-buttons.top(-|.).*$|^http://(www\.)?free-social-buttons2.xyz(-|.).*$|^http://(www\.)?free-social-buttons3.xyz(-|.).*$|^http://(www\.)?free-social-buttons4.xyz(-|.).*$|^http://(www\.)?free-social-buttons5.xyz(-|.).*$|^http://(www\.)?front.to(-|.).*$|^http://(www\.)?infokonkurs.ru(-|.).*$|^http://(www\.)?mapquestz.us(-|.).*$|^http://(www\.)?quick-offer.com(-|.).*$|^http://(www\.)?rank-checker.online(-|.).*$|^http://(www\.)?rankchecker.online(-|.).*$|^http://(www\.)?rapidokbrain.com(-|.).*$|^http://(www\.)?real-time-analytics.com(-|.).*$|^http://(www\.)?sharebutton.net(-|.).*$|^http://(www\.)?sharebutton.org(-|.).*$|^http://(www\.)?shemale-sex.net(-|.).*$|^http://(www\.)?site-speed-check.site(-|.).*$|^http://(www\.)?site-speed-checker.site(-|.).*$|^http://(www\.)?trafficmania.com(-|.).*$|^http://(www\.)?website-speed-up.site(-|.).*$|^http://(www\.)?website-speed-up.top(-|.).*$|^http://(www\.)?xn--80aagddcgkbcqbad7amllnejg6dya.xn--p1ai(-|.).*$|^http://(www\.)?xn--80aikhbrhr.net(-|.).*$|^http://(www\.)?pila.pl(-|.).*$|^http://(www\.)?dytohqka.su(-|.).*$|^http://(www\.)?fqvjhqciw.net.ru(-|.).*$|^http://(www\.)?wycjrqzy.ua(-|.).*$|^http://(www\.)?0ca29773681c7e82.com(-|.).*$|^http://(www\.)?intervsem.ru(-|.).*$|^http://(www\.)?candy-glam-hp.com(-|.).*$|^http://(www\.)?thecoolimages.net(-|.).*$|^http://(www\.)?rebuildermedical.com(-|.).*$|^http://(www\.)?gaygalls.net(-|.).*$|^http://(www\.)?keywordteam.net(-|.).*$|^http://(www\.)?netfacet.net(-|.).*$|^http://(www\.)?pattersonsweb.com(-|.).*$|^http://(www\.)?trapit.com.gg(-|.).*$) 1;
    }
    ## Add here all hosts that should be spared any referrer checking.
    ## Whitelist all your own IPs in this section, each IP followed by a 0;
    geo $bad_referer {
    127.0.0.1 0;
    111.111.111.111 0;
    }

    # Geo directive to deny certain ip addresses
    geo $validate_client {
    default 0;

    # Cyveillance
    38.100.19.8/29 1;
    38.100.21.0/24 1;
    38.100.41.64/26 1;
    38.105.71.0/25 1;
    38.105.83.0/27 1;
    38.112.21.140/30 1;
    38.118.42.32/29 1;
    65.213.208.128/27 1;
    65.222.176.96/27 1;
    65.222.185.72/29 1;
    85.25.176.0/20 1;
    85.25.192.0/20 1;
    85.25.208.0/22 1;
    }