Modem-Help Forum Index Main Site
Chips, Chipsets
Help [ Old Help ]
Dedicated help on Modems
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Conteg :: Content Negotiation PHP Class :: v0.13.6 released

Post new topic   Reply to topic    Modem-Help Forum Index -> Site Info + Diary
View previous topic :: View next topic  
Author Message
Alex Kemp
Site Admin

Joined: 30 Jun 2004
Posts: 6097
Location: Nottingham, England

PostPosted: Wed Nov 07, 2007 10:01 pm    Post subject: Conteg :: Content Negotiation PHP Class :: v0.13.6 released Reply with quote

About Conteg

Conteg is a single PHP Class that allows Webmasters and Hostmasters to improve the responsiveness of a PHP site. It does this by auto-enabling Content Negotiation.

All webservers auto-enable Content Negotiation by default for static files, but do not do so for dynamic content. That single fact is the reason that most dynamic sites suffer far greater load and bandwidth consumption than is necessary. That is also one of the reasons that dynamic sites are notoriously slow to load, which in it's turn is why many dynamic-site visitors then vote with their `Back' button to go elsewhere.

Conteg Version 0.13.6

This corrects a long-overdue anomaly in Conteg. The comments at the top of the Class state:
All program defaults may be over-ridden

In fact, until v0.13.6 content-negotiation on the `If-Modified-Since' + `If-Unmodified-Since' Request headers was enforced, as was production of `Last-Modified' and `Expires' Response headers. The defaults for Last-Modified, Expires and all other Class defaults remain the same, but 2 new parameters have been added to the setup() parameter array that allow either of Last-Modified & Expires to be switched off. That means that the statement above is, finally, completely accurate:- 'All program defaults may be over-ridden'.

In common with all other setup parameters, if use of `Last-Modified' is switched OFF, the value of `If-Modified-Since' or `If-Unmodified-Since' Request headers will still be available within the Class. This is the relevant part of the Changelog:
 *  Change Log:
 *    0.13.6:  Added `use_last_modified' to Constructor parameter, for    02 Nov 07
 *            +those rare occasions where a 304 must NOT be sent, plus
 *            +`use_expires' to be consistent.
...and these are the two new parameters:
   'use_expires'        => TRUE,
   'use_last_modified'  => TRUE  // TRUE = negotiate If-Modified-Since + If-Unmodified-Since

2009-04-16 Edited by AK
Edit Reason: Fixed URLs

Alex Kemp

Last edited by Alex Kemp on Thu Apr 16, 2009 4:28 am; edited 1 time in total
Back to top
View user's profile Send private message Visit poster's website
Alex Kemp
Site Admin

Joined: 30 Jun 2004
Posts: 6097
Location: Nottingham, England

PostPosted: Wed Nov 07, 2007 11:51 pm    Post subject: Reply with quote

There is a curious bi-polar connection between Conteg v0.13.5 and v0.13.6...

This new version was prompted by increasing numbers of visitors to this site--and specifically to the Download pages--reporting that that the `Download' button for a file was still greyed-out even though they had logged-in.

It took me a little while to write the Admin utilities necessary to trawl through the site logs to gather the necessary intelligence. When done, I realised that these folks were receiving `304 Not Modified' responses (because the content date had not changed) even though other aspects of the page had changed (most notably, because the person had logged-in!) (on this site when not logged-in, the Submit-button to download a file is disabled--greyed-out; when the person logs in the button is in a normal state, and they can download the file).

I was thoroughly perplexed that such browsers would receive a 304. The site coding was specifically designed--and checked--to send such headers that a logged-in page would NOT be cached and would be re-loaded fresh.

My thinking was flawed.

Conteg v0.13.5 details a flaw within Microsoft browsers, starting with v4 Internet Explorer, and continuing through versions 5 and 6, where a Vary header will prevent the browser caching the content. In other words, no 304s. In other words, the above problem will never occur (ignoring proxy caches for the moment). MSIE 7 is now out, and a fix has been issued with Windows XP SP2 for earlier versions. This problem--and all the complaints directed at me--have been becoming more and more frequent.

Another problem is HTTP/1.0 proxies and caches. They will ignore all the HTTP/1.1 headers, and just send a If-Modified-Since header. All my careful HTTP/1.1 programming is to no avail. What is worse is that all these so-called HTTP/1.1 browsers are effectively acting as HTTP/1.0 browsers (see below).

The relevant RFC 2616 says:
HTTP/1.0 clients and caches will ignore entity tags. Generally, last-modified values received or used by these systems will support transparent and efficient caching, and so HTTP/1.1 origin servers should provide Last-Modified values. In those rare cases where the use of a Last-Modified value as a validator by an HTTP/1.0 system could result in a serious problem, then HTTP/1.1 origin servers should not provide one.

It perhaps took longer than it should, but it gradually dawned on me - the site should NOT be sending a Last-Modified value when the user was logged-in. It was at that point that I discovered that Conteg enforced sending 304s. Whoops.

Some example coding for Conteg:

The following examples are drawn from this site, and should help in showing how to program with Conteg. The various variables are exactly what their names suggest.

From very early within a common include file:
 * 2007-11-05 bugfix: `other_var' now obtained from fRetOtherStr()
 *+           `use_last_modified' added (same reason as next line, re: HTTP/1.0 proxies)
 * 2007-04-21 bugfix: macro decision changed from $_isMember to ( $_signupStatus > _SIGNUP_INACTIVE )
 *+           (page view needs to change when logging in or out)
   $_Encode            = new Conteg( array(
      'cache_control'       => array(
         'macro'   => (( $_isSecurePage or ( $_signupStatus > _SIGNUP_INACTIVE ))
            ? 'cache-none'
            : 'cache-all'
      'cpu_number'         => 1,
      'other_var'          => fRetOtherStr(),
      'noprint'            => TRUE,
      'use_apache_notes'   => TRUE,
      'use_etag'            => TRUE,
      'use_last_modified'   => (( $_username ) ? FALSE : TRUE )
/* fRetOtherStr()
 *      return string suitable for use within Conteg `other_var' parameter
 * parameter: $extra_var - to allow for page-specific possible changes
 * return string
 * 2007-11-05 added; the string is used within Class Conteg to create ETag for Response header.
   function fRetOtherStr(
      $extra_var   = ''
   ) {
      global $_bandwidth, $_benefit, $_isAdmin, $_signupStatus;

      return "$_signupStatus$_bandwidth$_benefit".
         (( $_isAdmin ) ? '1' : '0' ).

Then, from the last lines of the Download section page:
 * 2007-11-05 bugfix, foll lines (re: `other_var'):
 *+           Download Section page view not changing on logon for those behind HTTP/1.0 proxy caches,
 *+           thus `Last-Modified' no longer sent when logged in;
 *+           as belt+braces fix, ETag made more precise for sake of HTTP/1.1 proxies in all situations.
   $tmp      = ( isset( $Submit->dis ) and $Submit->dis == TRUE ) ? '1' : '0';
   if( !empty( $Error )) {
      $tmp   .= ( !empty( $Error['error'])) ? $Error['error'] : 0;
      $tmp   .= ( !empty( $Error['text'])) ? $Error['text'] : '';
   $param   = array(
      'charset'   => $Charset,
      'modified'   => strtotime( $CDATE ),
      'other_var'   => fRetOtherStr( $tmp )
   $_Encode->setup( $param );

Some extra intelligence on real-life Request/Response headers:

As part of the bug-chasing effort involved in fixing this problem, I created a log of responses from logged-in browsers where the Submit button was disabled and got a 304:
 * 2007-10-31 investigate 304s sent to logged-in users
   if( $_username and
      ( isset( $Submit->dis ) and $Submit->dis ) and
      ( $_Encode->_is304 )
   ) {
      global $_SERVER;
      $log   = _L_DIRECTORY .'304.log';
      $tmp   = "`$_username` on Download page received a 304:\n\$_SERVER variables:\n";
      foreach( $_SERVER as $key => $val ) {
         $tmp   .= "$key=$val\n";
      $req   = apache_request_headers();
      $tmp   .= "Request headers:\n";
      foreach( $req as $key => $val ) {
         $tmp   .= "   $key=$val\n";
      $tmp   .= "Response headers:\n";
      $req   = apache_response_headers();
      foreach( $req as $key => $val ) {
         $tmp   .= "   $key=$val\n";
      error_log( $tmp, 3, $log );

There were 6 people in about 12 hours in the log. Only 2 of these folk's browsers sent a Referer (the button is always dimmed if no Referer). Only one person had a browser that sent HTTP/1.1 headers (Opera 9.01). All others were MSIE 6.0 on winXP.

Here are some of the responses.

MSIE 6.0 on winXP:
Request headers:
Accept=image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/x-icq, application/, application/, application/msword, */*
Accept-Encoding=gzip, deflate
If-Modified-Since=Fri, 12 Oct 2007 23:00:00 GMT; length=32084
User-Agent=Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727)
Cookie=(both session and permanent cookies)
Opera 9.01 on winXP:
Request headers:
User-Agent=Opera/9.01 (Windows NT 5.1; U; ru)
Accept=text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1
Accept-Charset=iso-8859-1, utf-8, utf-16, *;q=0.1
Accept-Encoding=deflate, gzip, x-gzip, identity, *;q=0
If-Modified-Since=Fri, 12 Oct 2007 23:00:00 GMT
Cookie=(both session and permanent cookies)
Connection=Keep-Alive, TE
TE=deflate, gzip, chunked, identity, trailers
Response headers:
Set-Cookie=modem-help_bb2_sid=ebe1b44519cc16d709b3bbc5f63563ea; path=/;
Last-Modified=Fri, 12 Oct 2007 23:00:00 GMT
Expires=Tue, 23 Feb 1999 18:30:00 GMT
Cache-Control=must-revalidate, no-cache, no-store, private, s-maxage=0, pre-check=0, post-check=0, max-age=0
Status=304 Not Modified
Keep-Alive=timeout=15, max=99
Content-Type=text/html; charset=iso-8859-1

The last one indicates the importance of changing the (weak) Etag if the page has a significant difference. Conteg is already correctly coded to NOT send a 304 if If-Modified-Since and If-None-Match give different results. Here they both matched.

Hopefully, I have finally cracked this problem (time will show).
Alex Kemp
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    Modem-Help Forum Index -> Site Info + Diary All times are GMT
Page 1 of 1

Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Powered by phpBB © 2001, 2007 phpBB Group