Web Filter Troubleshooting

SYMPTOM:   The Web Filter is installed but it does not appear to be filtering or blocking traffic.

Confirm whether or not the Web Filter is receiving/capturing traffic. Use the 'Realtime Traffic log' to verify whether or not the Web Filter is logging traffic: SYSTEM > Diagnostics > View Log File: Realtime Traffic Log (shadow.log). If the log file is empty there are a few things to check:

• Connectivity – Verify the Listening interface of the Web Filter is plugged in to the network device (hub, switch, network tap). Additionally, ensure the Web Filter is plugged into the correct port on the switch or network tap if the Web Filter is in Invisible Mode. See next tip.
• Port Mirroring – If the Web Filter is connected to a switch, the switch must be capable of Port Mirroring.
  • Ensure the Port Mirroring function can mirror outbound traffic. Some switches can only mirror inbound traffic (traffic sourced from the internet destined for the internal network. This will not work.)
  • Ensure the correct port/network device is set as the "source" in the switches mirroring configuration. When all else fails, try tracing cables. We have come across situations where everything checks out only to find out the port assumed to be the right port to set as the "source" was indeed the wrong port.
  • If you do not know how if your switch supports port mirroring or know how to configure the port mirroring function on your switch, contact your switch vendor's Technical Support group for guidance.
• Listening vs. Blocking Interface – Verify the correct NIC is configured as the listening interface (SYSTEM > Mode > Operation Mode).
• VLAN Detection – Depending on the switch/network, packets sent to the Web Filter may be VLAN tagged/encapsulated. By default the Web Filter can't read packets that are VLAN tagged and as a result, won't be able to analyze the information in the packet (URLs, application traffic, etc…). Enable VLAN Detection to see if that helps (SYSTEM > Control > Filter: VLAN Detection). NOTE: Many networks Utilize VLANs. Just because a network utilizes VLANs does not mean the packets will be VLAN tagged when the Web Filter receives them. So, unless this actually solves the problem, it should be left disabled.
• Range to Detect – The Web Filter can be configured to filter/ignore specific source & destination network IP Ranges. This is achieved by configuring the Range to Detect (Policy > Global Group > Range to Detect). By default the Range to Detect is empty so misconfiguration here is probably not going to be the problem if blank. If the Range to Detect, has been configured, make sure it is set to capture traffic from the networks needing filtering. NOTE: If the Web Filter is installed in a Proxy Environment and monitoring traffic destined to a Proxy Server, the Range to Detect must not be configured to IGNORE traffic destined to the Proxy Server IP Address or the network in which the Proxy server sits.

Back to Index of Topics

SYMPTOM:   The Web Filter is capturing/logging traffic but block pages are still not being presented when expected or a 'Page Cannot be Displayed' error page shows in the browser.

How the Web Filter blocks – When the Web Filter blocks a web page access attempt, it sends a redirect/block packet in the form of a "spoofed" packet back to the end-user workstation/browser. That is, the Web Filter sends the requesting workstation a packet masquerading as the Web Servers response. The redirect packet points the web browser on the end-users workstation to the Web Filter's block page(you can also specify an alternate block page but for the sake of this scenario, it is assumed that the default on-box block page is used). The block page on the Web Filter is accessed on port 81. While that is happening the Web Filter also sends a TCP Reset packet to the Web Server terminating the conversation it was preparing to have with the requesting workstation. If sites are being blocked or if sites are blocked but block pages aren't being displayed, investigate some of the options/settings below:

• Block Page Delivery Method – If the browser is not being redirected to the block page, try adjusting the Block Page Delivery Method (BPDM) by navigating SYSTEM > Mode > Operation Mode. The default setting is 'Send Block Page to Specified Host MAC Address – Default Gateway'. This tells the Web Filter to deliver the blocks by way of the Web Filter's default gateway.
       Because the block is actually a spoofed packet, some firewalls, routers, or Layer 3 switches may drop the packet. Try setting the BPDM to 'Send Block Page via ARP Table'. The Web Filter will use its internal ARP table to figure out where to send the block. This setting typically helps in "flat" networks and usually solves most problems with respect to getting the block page to appear.
       Additionally, there is an option to use an 'Alternate IP'. This will have the Web Filter send the block/redirect to the specified device IP Address and assumes the device will forward the packets to the desired destination/workstation needing to be blocked.
  
• Port 81 Access to the Web Filter/Block Page – If the request is blocked and no block page is displayed but the address bar of the web browser is pointing to the Web Filter, Port 81 access from workstation to Web Filter might not be possible. To confirm this as a problem, try to directly access the block page from a workstation - http://WebFilter_IP :81/cgi/block.cgi
  If the block page does not come up, either port 81 is blocked or sufficient network routing is not in place.
• Block Page Route Table – Depending on the network configuration in which the Web Filter is installed, it may be necessary for the Web Filter to use a Route other than through its Default Gateway to communicate or send Block/Redirects to another network. Routing statements can be added to the Web Filter via SYSTEM > Network > Block Page Route Table.
• URL Library is empty due to a Library Restore – By default, the Web Filter comes pre-loaded with a URL Library (that needs updating). However if the Web Filter recently had a library backup restored using the BACKUP/RESTORE feature, a FULL URL Library will need to be executed as the Library restoration process results in the core Library being erased.
• Review the deployment – If there are still issues, inspect the network topology as it relates to where the Listening and Blocking interfaces of the Web Filter are connected (for example, don't place the listening interface of the Web Filter on the inside of a Firewall and the block/mgmt interface on the outside of a firewall).

Back to Index of Topics

SYMPTOM:  The Web Filter does not appear to be able to download the library or patches.

This can be verified via the status information on the HOME page/tab of the Web Filter as well as by viewing the Library Update Log (Library > Updates > Library > Library Update Log: View Log) or the Patch Update Log (System > Software Update > Software Update Log). You may find it necessary change the Log Level from 1 to 2 in order to see more verbose information regarding errors. This can be changed via Library > Updates > Configuration. Changing the log level in the middle of a Library/Patch download attempt will have any impact. You will need to initiate a new download after the log level has been changed.

• Activation Code / Update Account – Ensure the Web Filter's Update account has been activated. An Activation Code (usually provided via email) should have been provided with instructions on the Activation process. If not, contact Technical Support.
       The hostname displayed on the LAN Settings page of the Web Filter must match with the hostname entered when entering the details on the activation page.
       Wait about 2-5 minutes after activating before attempting a Library or Patch Update.
• HTTPS / Port 443 access – Ensure the Web Filter is allowed to access the internet over port 443/HTTPS.
• DNS Setting / Resolution – Ensure the Web Filter is using proper DNS settings (SYSTEM > Network > LAN Settings). The Web Filter must be able to resolve secureupdate.8e6.com and patch.8e6.net . You can test DNS by accessing the System Commands page (SYSTEM > Diagnostics > System Command) and attempting to ping the hostnames secureupdate.8e6.com or patch.8e6.net . If the ping results are unsuccessful, check Web Filter DNS settings and general DNS communication channels within the network.
• Configure Proxy Settings – If the Web Filter is installed in a Proxy environment and must traverse a proxy in order to access the internet, try configuring the Proxy Settings (Library > Updates > Configuration).

Back to Index of Topics

SYMPTOM:   The Web Filter is installed but HTTPS/SSL based traffic is being negatively impacted.

The Web Filter has a configuration option called HTTPS Filtering Level (SYSTEM > Control> Filter). This setting controls how the Web Filter analyzes HTTPS/SSL based traffic and when a block will be issued.

NOTE: There are four basic levels to choose from. If Medium (default) or High is selected, it is imperative that the Web Filter be able to access the internet via HTTPS / Port 443.

One quick way to determine whether or not the Web Filter is causing the problem is to temporarily turn the HTTPS Filtering Level to NONE and confirm whether or not the problem persists.

Another option is to leave the HTTPS Filtering level in place and configure a Real Time Probe to monitor the traffic suspected of being blocked. The probe will identify the request that is being blocked and why. In this case look for a Block Method of HTTPS.

If a valid website is being blocked as a result of the HTTPS Filtering Level, there are two options available to allow access to or "Whitelist" the site:

• Add the URL and/or corresponding IP Address(es) of the destination or target server to a category (custom or built-in) and set the category to ALWAYS ALLOW for the specified filtering rule or profile.
• Using the Range to Detect feature, configure the IP Address of the site as part of the Destination Exclude. This option will cause the Web Filter to completely ignore traffic destined to the IP Addresses configured so there will be no further logging of attempts to the site once done.

What follows is a breakdown of the different HTTPS Filtering Levels and when the Web Filter will block traffic:

None: The Web Filter will not analyze HTTPS/SSL based content.

Low: The Web Filter will take the IP Address of the destination/target server (from the destination IP of the request packet) and compare it against the URL Library to see if the IP Address is categorized.

When will the Web Filter block a request?
If the IP Address of the requested site is categorized and that category is configured to be blocked, access to the site will be denied.

Medium: When HTTPS/SSL traffic is detected, the Web Filter will attempt to connect to the same site requested by the source or end-user workstation in order to retrieve the certificate for further processing. The Web Filter then takes the site/hostname found in the certificate and performs a category look-up on it.

When will the Web Filter block a request?
Blocking will occur if the URL/hostname returned by the Certificate check is listed in a blocked category. Blocking will also occur if the Web Filter is unable to actually retrieve a certificate from the destination server.

Medium (+ Forward lookup to validate qualified DNS): Same as Medium except the Web Filter will also perform a Forward DNS Lookup on the site captured in the certificate details.

When will the Web Filter block a request?
The Web Filter will block HTTPS/SSL traffic if it does not get a response when it attempts to perform a DNS Lookup on the site/hostname obtained from the certificate of the destination/target server. Sites that use Self-Signed Certificates will usually end up blocked.

High: The Web Filter will take the destination IP Address of the captured HTTPS Request, perform a Reverse DNS Lookup and compare the results to the site name/hostname provided in the certificate.

When will the Web Filter block a request?
Blocking will occur if there is a mismatch between the site name/hostname provided in the certificate and the result of the Reverse DNS Lookup on the destination IP Address of the captured HTTPS Request.

High (+ Library Lookup to overturn DNS Decision): If this option is selected, a category lookup on the destination IP Address of the HTTPS request packet is performed in addition to the other checks. The categorization of the IP Addresses, if it exists, and policies surrounding the category will overrule the decision the Web Filter comes to based on the DNS check performed in the HIGH setting.

Back to Index of Topics

SYMPTOM:   A profile has been applied to a user/group/IP Address that should not result in any sites being blocked yet sites continue to be blocked.

Minimum Filtering Level(MFL) – The Minimum Filtering Level is an umbrella filtering profile/policy that sits on top of any IP Group or LDAP based filtering profile (except for the Domain Default Rule).
     By default, the 'Pornography/Adult Content' and 'Child Pornography' categories are blocked as part of the Minimum Filtering Level. But, other category groups or categories may be configured here and could be the reason for the blocking.
     The MFL is accessed via Policy > Global Group> Minimum Filtering Level.

NOTE: The Minimum Filtering Level also impacts Exception URLs and Override Accounts. To modify the impact the MFL has on Override Accounts and/or Exception URLS, review the settings under the 'Min. Filter Bypass' tab while on the MFL configuration page.

Also please note that if a library category has been assigned an Always Allow setting, if that library category is blocked by the Minimum Filtering Level, the user will not be able to access that library category.

Back to Index of Topics

SYMPTOM:   The Web Filter appears to be responsible for blocking some internal network traffic or in some cases access to legitimate websites. When blocked from accessing websites the default block page presents the blocked URL as: pattern://{IP Address}

How can this be confirmed and is there a way to "Whitelist" sites/traffic from the Pattern based detection Engine?

• Real-Time Probe – If you suspect the Web Filter is interfering with internal traffic:
  • Launch a Real Time Probe and specify who (User field) or which workstation(using the IP Address field) you would like to monitor.
  • As the traffic is displayed, look for situations where the 'Filter Action' is Block and the 'By Method' is Pattern. If you notice requests that match, pay special attention to the IP Address shown in the 'URL in Libs' column. In most situations, that will show the IP Address of the destination or target server to which access is being denied (unless the Web Filter is monitoring traffic destined to a Proxy Server. The IP of the Proxy will only usually display in that case). If that IP Address represents an internal server, the Range to Detect setting should be adjusted so requests to internal servers are not monitored by the Web Filter. See next bullet point: Range to Detect.
• Range to Detect (Policy > Global Group > Range to Detect) – The Range to Detect can be configured so the Web Filter does not monitor traffic generated from within the network AND destined to the internal network if this is determined to be the issue. Explore the Destination Exclude portion of the configuration and enter the IP or network ranges the Web Filter should not monitor.
• Pattern Detection Whitelist (Library > Pattern Detection Whitelist) – Enter the IP Addresses of the servers to which access is being denied. This information can be obtained fright from the default block page or by identifying them within the results of a real time probe.

Back to Index of Topics

SYMPTOM:   After adding a site to a category or deleting a site from a category, the Web Filter does not appear to have applied the changes.

Reload Library button – If changes are made to the Library be sure to click the 'Reload Library' button as the final step. There will be situations where multiple changes may need to be made to the Library in one exercise. The Reload Library button does not need to be depressed after every addition or deletion. Just one Reload after all changes have been made. NOTE: It could take anywhere from 2-5 minutes for the Library to reload and the changes to take effect.

Back to Index of Topics