The growth and increased popularity of the World Wide Web has created a corresponding growth in network traffic. With this growth have come delays, slower response times, and security concerns.
The network traffic problems are partly due to the repeated retrieving of objects from remote Web servers on the Internet. Novell BorderManager Proxy Services can help improve performance by locally caching frequently requested Internet information. In general, Proxy Services stores copies of frequently requested Web information closer to the user, thereby reducing the number of times the same information is accessed over an Internet connection, the download time, and the load on the remote server.
This section contains the following information:
There are four types of caching:
With passive caching (also called basic or on-demand caching), the client (browser) sends a request directly to a proxy server, which is an HTTP server that usually runs on a firewall server. The proxy server locates the object in its cache and returns the object to the client. If the object is not in the cache, the proxy retrieves a copy from the origin Web server on the Internet, stores it in the cache on the proxy server, and returns a copy of the object to the client. The object is cached for a preset period of time or until the cache is full. If the cache disk space is low, older objects are removed from the cache. Subsequent browser requests for the cached object are made to the proxy server at local intranet speeds. This reduces Internet traffic and the request load on the source Web server, thereby reducing the delays in returning information to the client.
To the client, the proxy server has the same basic functionality as the Web server (with a subtle difference in submitting requests). To the Web server, the proxy server has the same basic functionality as the client. The proxy builds its cache based on the Web sites that users visit. When an object is retrieved from the Web and put in a cache, a Time-To-Live (TTL) value is associated with the object. Before the TTL expires, requests are filled from the cache for that object. When the TTL expires, the Web server is contacted for a newer version, the update is stored in the cache, and a new TTL is calculated.
Active caching is an add-on to passive caching to improve performance. With active caching, the proxy automatically sends a request to the origin server to retrieve an object. The server updates objects that are more frequently accessed or requested, have longer TTLs, and are actively cached during periods of low server load.
Negative caching occurs when a proxy attempts to resolve a request for a URL that does not exist or cannot be located or accessed. In this case, the proxy caches the negative result so that future requests for that URL are resolved quickly. The proxy continues to check in the background and refreshes the cache when the pages become available. Negative caching occurs for HTTP error conditions such as 403 (forbidden request) and 404 (URL not found).
Hierarchical caching allows information to be retrieved from the nearby or closest proxy servers instead of from the originating Web server. HTTP and FTP acceleration (reverse proxy cache acceleration) also allows static information to be cached by and retrieved from the border proxy servers instead of the origin Web servers to reduce the Web server load. The proxy cache uses cache aging information that Web servers provide to browsers to determine how long pages should be cached.
Access control is used by the Proxy Services software applications to forward and filter connections for such services as HTTP, Gopher, and FTP. The host running Proxy Services is known as the gateway. In general, Proxy Services allows services only for which there are proxies. For example, if a gateway has proxies for FTP, then only FTP can be requested; requests for all other services are ignored.
With gateways, you can hide the names and addresses of internal systems—the gateway is the only hostname known outside the system. Also, traffic can be logged before it reaches the internal hosts. Proxy Services improves security by hiding private network domain names and addresses and sending all requests through a single gateway. For more information about gateways, refer to Section 4.0, NAT Overview and Planning.
Proxy Services is based on both the first-generation CERN proxy technology and the newer, second-generation Harvest/Squid hierarchical proxy cache technology. The Harvest/Squid technology enhances standard CERN proxy cache services with negative URL caching and negative domain Name System (DNS) caching, and introduces hierarchical caching through the Internet Cache Protocol (ICP).
The Harvest project, an Internet Resource Discovery Project contract performed by the University of Colorado, introduced ICP hierarchical caching to improve Internet Web performance and scalability. The project was transferred to the National Laboratory for Applied Network Research (NLANR) in early 1996 as the basis for the Squid project. The goal of the Squid project is to facilitate the evolution of an efficient national architecture for handling highly popular information.
Novell BorderManager Proxy Services supports the following protocols and applications:
HTTP (0.9, 1.0, and 1.1), including HTTPS support and Secure Sockets Layer (SSL)
FTP
Domain Name System (DNS)
Gopher
Simple Mail Transfer Protocol/Post Office Protocol 3 (SMTP/POP3)
Network News Transfer Protocol (NNTP)
RealAudio and RealVideo*
Real Time Streaming Protocol (RTSP)
SOCKS 4 and 5
Generic TCP/UDP
HTTP Transparent proxy
Telnet Transparent proxy
The passive mode (PASV) is supported for FTP to allow the firewall administrator to deny incoming connections above port 1023, if necessary. Otherwise, normal (PORT) FTP mode is used. Proxy Services also supports the HTTP protocol over the Internetwork Packet Exchange™ (IPX™) software. Novell IPX/IP clients, as well as other clients, can directly access the proxy server using the gateway client transparent proxy feature.
Novell BorderManager Proxy Services combines an Internet proxy, a Web caching facility, and the NDS™ or Novell eDirectory® software to provide World Wide Web access from within a firewall. Proxy Services has the following benefits:
Reduces WAN traffic to the Internet and on the primary Web server by providing local LAN access to cached information. Proxy Services also reduces the load on Web Internet servers and increases Internet and intranet performance.
Uses a single protocol on the LAN (for HTTP proxy only). Users do not need to have separate clients; HTTP is used to communicate with a proxy server. The proxy server uses the appropriate protocol; FTP, Gopher, and so on for HTTP requests to access documents from the network.
Improves intranet security by hiding the local network from the Internet. Private network domain names and addresses are hidden and all requests are sent through a single gateway. This applies to forward proxy only. Reverse proxy is used to hide the origin host from the client or local network.
Enhances intranet security with access control and content filtering.
Distributes LAN client requests across multiple proxy servers, for example, FTP requests on one server and HTTP requests on another server.
Reduces the disk space requirements for retrieved information on client workstations and reduces the load on Web Internet servers.
Enables document access even when the Internet or intranet Web server is down or inaccessible, if the document is already cached by the proxy server and Time-To-Live is not expired.
Undeletes and serves if the origin server is down.
Provides a single point administration based on eDirectory.
Logs and filters client transactions.
These benefits apply to both Internet and intranet Web sites. Because Proxy Services supports open Internet standards, it can be used with Novell intranet and Internet products, as well as with other vendors’ browsers and Web servers.
Proxy Services includes the following features:
Support for HTTP (0.9, 1.0, 1.1), FTP, Gopher, DNS, and SSL clients
Hierarchical caching based on the Internet Cache Protocol (ICP) and other protocols
HTTP and FTP server accelerator (reverse proxy)
Application proxies, including SMTP proxy, NNTP proxy, DNS proxy, SOCKS, HTTP Transparent proxy, Telnet Transparent proxy, and RealAudio and RTSP proxies
SOCKS client support
Batch downloading of URLs
Content filtering for Java*
Simple Network Management Protocol (SNMP) Management Information Base (MIB)
Access control lists based on eDirectory user identity, IP addresses, domains, and URLs
Windows-based management console and configuration
SurfControl (third-party site-blocking software)
Event logging in Text and Relational Database Management System (RDBMS) formats