TZUTC: -0500   
   MSGID: 54360.sync@1:103/705 2cc2d743   
   REPLY: 1:154/110@fidonet.org 685bc18b   
   PID: Synchronet 3.21a-Linux master/5a0ede2e8 Jun 21 2025 GCC 13.3.0   
   TID: SBBSecho 3.28-Linux master/a43ca30ea Jun 18 2025 GCC 12.2.0   
   BBSID: EOTLBBS   
   CHRS: CP437 2   
   FORMAT: flowed   
   NOTE: FSEditor.js v1.105   
     Re: Web server and BOT's   
     By: Richard Vonzel to All on Wed Jun 25 2025 04:25:24   
      
    > I turned on the Synchronet Web server, works great, hsad my website running.   
    > Had my BBS's files available for access thru it. Yesterday I guess a BOT got   
    > on my site and jst downloaded all the file I had available. Here's the log   
    > entry for this,   
      
   I have found gptbot ignores a global disallow rule in robots.txt   
      
   User-agent: *   
   Disallow: /   
      
   But you could try adding   
      
   User-agent: gptbot   
   Disallow :/   
      
   It could take a few days but you'll see when they start to pickup your   
   robots.txt file.   
      
   Another option would be to just block the ip ranges altogether.   
      
   Another option would be to block the published subnets. I have these blocked   
   using ipsets   
      
   4.227.36.0/25   
   20.171.207.0/24   
   52.230.152.0/24   
   172.182.204.0/24   
   20.171.206.0/24   
   172.182.214.0/24   
   20.125.66.80/28   
   172.182.215.0/24   
      
   That seems to be the easiest way and just don't deal with them at all.   
      
   ---   
    þ Synchronet þ End Of The Line BBS - endofthelinebbs.com   
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)   
   SEEN-BY: 10/0 1 102/401 103/1 17 705 105/81 106/201 124/5016 128/187   
   SEEN-BY: 129/14 153/7715 154/110 218/0 1 215 601 610 700 840 860 880   
   SEEN-BY: 226/30 227/114 229/110 114 206 317 400 426 428 470 550 700   
   SEEN-BY: 229/705 266/512 280/464 291/111 301/1 320/219 322/757 342/200   
   SEEN-BY: 396/45 460/58 633/280 712/848 902/26 5075/35   
   PATH: 103/705 218/700 229/426