Training

• Home
• Training Classes

"Winners make choices,
losers make excuses."
Decide to be a Winner!!!!

± Getting Started

± Phase 1
Planning

± Website Tools

± Phase 3
Conceptual Design

± Helpful Information

± Phase 4
Physical Design

± Phase 5
Testing

± Phase 6
Implement and Market Website

± Other Web Development Items

± Multimedia

± Useful Utilities

± Programming

± Advanced Programming

± Microsoft Office Products

± Computer Maintenance

± Other

---

NOTE: This is a collection of information and links collected over the years that might provide useful information. A Safer Company LLC does not guarantee, endorse or approve any of these links or their scripts. Use these links to other websites at your own risk.

IP Addresses

Determine location of IP address

IP Information

Every visitor to your web site has an IP address.

IP Address= 13.58.200.16

ref=
agent=
Your hostname: 13.58.200.16host: ec2-13-58-200-16.us-east-2.compute.amazonaws.com

<?php
$ip=@$REMOTE_ADDR; //register_global:on in php.ini - Note: changing register global setting may be a security problem in a live hosting environment
$ip=$_SERVER['REMOTE_ADDR']; //register_global:off in php.ini
echo "<b>IP Address= $ip</b>";
?>

There are many reasons that you may want to block someone from visiting your website.

• IP address's constantly showing up in your server logs
• Another site is leeching all your bandwidth
• Ban certain users who complete and submit your forms
• Secure folders or poges on website

Blocking spambot, spammers, and scrapers from your website is easy once you have determined that a particular IP address is worthy of banishment. Usually we invoke the magical powers of htaccess to block unwanted IP Addresses or we can use PHP if the server is not apache.

Identifying bad bots

So you've noticed a certain user-agent keeps showing up in your logs, but you're not sure what it is, or if you want to ban it? There's a few ways to find out:

• Google it: Try a search like this.
• Check the User Agent Database which contains a list of User-Agents (Spiders, Robots, Crawler, Browser)
• Head over to Webmaster World and search again or start a new thread.

Once you've determined that the bot is something you want to block, the next step is to add it to your .htaccess file.

Blocking bots with .htaccess

Placed the code at the bottom of your .htaccess file. If you don't already have a file called .htaccess in your site's root directory, you can create a new one.

#get rid of the bad bot
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^BadBot
RewriteRule ^(.*)$ http://go.away/

The above lines tell your webserver to check for any bot whose user-agent string starts with "BadBot". When it sees a bot that matches, it redirects them to a non-existent site called "go.away".

Block more than one bot

#get rid of bad bots
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^BadBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^EvilScraper [OR]
RewriteCond %{HTTP_USER_AGENT} ^FakeUser
RewriteRule ^(.*)$ http://go.away/

The code above blocks 3 different bots.
Note the "[OR]" option after the first two bot names: this lets the server know there's more in the list.

Blocking Bandwidth Leeches

Hotlinking your images can eat up all your bandwidth so you may want to block the site.

RewriteEngine on
RewriteCond %{HTTP_REFERER} ^http://.*somebadforum\.com [NC]
RewriteRule .* - [F]

This code will return a 403 Forbidden error to anyone trying to hotlink your images on somebadforum.com. The end result: users on that site will see a broken image, and your bandwidth is no longer being stolen.

Here's the code for blocking more than one site:

RewriteEngine on
RewriteCond %{HTTP_REFERER} ^http://.*somebadforum\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://.*example\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://.*lastexample\.com [NC]
RewriteRule .* - [F]

Blocking Hotlinking

The solution was to write an .htaccess file to block hotlinks and save it to the images directory.

RewriteEngine On
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http(s)?://(.*\.)?asafercompany.com [NC]
RewriteCond %{HTTP_REFERER} !^http(s)?://(.*\.)?paypal.com [NC]
RewriteRule \.(jpeg|jpg|gif|png)$ http://example.org/bad.jpg [NC,R,L]

This code filters out image requests based on the site that sent the request (aka the "referer"). It only affects images with the following extensions: jpeg, jpg, gif, or png.

1. Turn the rewrite engine on, which allows us to redirect requests.
2. The second line allows viewing images from blank referers; this is important because some browsers won't send referers, even if the image is linked on your own website.
3. The next three lines allow www.asafercompany.com and two other sites, to link to my images.
4. The final line redirects anyone else to "bad.jpg" on example.org.

Keep in mind, if you're going to redirect someone to a different image, that image must not be on your server, or you will create an infinite loop!

Best

Simply block the hotlinked request by changing the last line to the following:

RewriteRule \.(jpeg|jpg|gif|png)$ - [F]

Instead of being redirected, the user will just see a broken image.

You can also use this code to block things besides images (MP3s or Zip files, for example). Just add the file extension into the last line, separated by a pipe character, like so:

RewriteRule \.(jpeg|jpg|gif|png|mp3|zip)$ - [F]

Banning IP Addresses

If you have certain IP addresses that are constantly compleleting your forms, then you may want to ban them. This can be a certain person or a bot.

Block certain IP address from accessing your website using the .htaccess file. If you do not already have a file named .htaccess, then create it and save it to your root directory.

order allow,deny
deny from 192.168.0.1
allow from all

• Block multiple IP addresses:

order allow,deny
deny from 192.168.0.2
deny from 192.168.0.3
deny from 192.168.0.4
allow from all

• Block a range of IP addresses

order allow,deny
deny from 192.168.
deny from 129.0.0
allow from all

The above code will block any IP address starting with "192.168." or "129.0.0." from accessing your site.

• Block All Traffic from the ISP
  HostnameLookups must be enabled on your server

deny from hotlinker.com

Block Everyone / Giving Access to Certain IP Addresses

Question how do you set it up to only allow access from a certain IP range. Basically I could force only my clients to allow access to the private backend of site.

This prevents any ip address from hotlinker.com from accessing your site.

Deny everyone access, then allow certain hosts/IP addresses

ErrorDocument 403 /GoAway.html
<Limit GET POST>
order deny,allow
deny from all

# Allow all IP's starting with 21.22.23.
allow from 21.22.23.

# Allow access from Google.com
allow from .google.com
</Limit>

For referrers, use this:
Block traffic from competitors site / case insensitive

RewriteEngine on
# Options +FollowSymlinks
RewriteCond %{HTTP_REFERER} competitorsite\.com [NC,OR]
RewriteRule .* - [F]

Uncomment the line with "Options +FollowSymlinks" above (remove the #) if your server is not configured with FollowSymLinks in its <directory> section of the httpd.conf, and you get a 500 Internal Server error when using the code above as is.

Using PHP code to block IP Addresses

The code must be on the pages that you want to block.

 

<?php
$deny = array("111.111.111", "222.222.222", "333.333.333");
if (in_array ($_SERVER['REMOTE_ADDR'], $deny))
{
header("location: http://www.google.com/");
exit();
}
?>

Notes:

• .htaccess only works in apache web servers.
• Users who run through a proxy server will have a different ip number each time they connect. Suggestion: use the range for the IP address: '203.156.187'. This will block all users with the ip address of 203.156.187.000 - 203.156.187.255.
• Using a range IP address might lock out legitimate visitors to your site.
• .htaccess is a very powerful tool and it is easy to create errors.
  • mistakes and typos in your .htaccess file cause the server top give an Error 500 page instead of showing your site
• Make sure you backup your .htaccess file before making any changes.
• If you'd like to learn more about writing .htaccess files, I recommend checking out the Definitive Guide to Mod_Rewrite. This book covers everything you need to know about Apache's .htaccess rewrite system.
• htaccess Tricks

top of page

Page last updated: May 31, 2012 10:45 AM