Caching Problem

From: Peter (BOUGHTONP)22 Oct 2010 20:21
To: Manthorp 3 of 15

Yes, the same Expires +5min header is received for messages.

 

Since it's usually following a different URL to get to messages it's less noticable, (though I did just get it after posting this reply) - whereas the thread list is the same URL each time the Go button is pressed.

EDITED: 22 Oct 2010 20:23 by BOUGHTONP
From: Manthorp22 Oct 2010 20:32
To: Peter (BOUGHTONP) 4 of 15
I am suitably deflated.
From: Matt22 Oct 2010 22:47
To: Peter (BOUGHTONP) 5 of 15
Does it do the same from work or is it just your T-Mobile connection?

The thread list and message pane both send Last-Modified headers, but these are disabled if Beehive detects what it believes is a proxy server, which is does using a list of known HTTP request headers that proxy servers send. When a proxy server is detected it sends a past date as the expires header, specifically Beehive's DOB (or release date of 0.1 if you prefer), and the current date and time including seconds as the last-modified header (amongst others).

I've checked the server as well, and it's correctly reporting 21:47 UTC (as of writing) as it's time.

Presumably the T-Mobile proxy server isn't detectable by Beehive, or it doesn't it reveal itself. It would be interesting to find out if the T-Mobile proxy server does send anything that can be identified, in order to prevent caching. Remind me tomorrow and I'll set up a script to reply with request headers and we'll see if anything looks likely.
From: Matt22 Oct 2010 22:53
To: Peter (BOUGHTONP) 6 of 15
From: Peter (BOUGHTONP)22 Oct 2010 23:00
To: Matt 7 of 15
Just T-Mobile - both work and wifi at parents are fine.
From: Peter (BOUGHTONP)22 Oct 2010 23:10
To: Matt 8 of 15
code:
Array
(
    [USER] => www-data
    [HOME] => /var/www
    [FCGI_ROLE] => RESPONDER
    [QUERY_STRING] => 
    [REQUEST_METHOD] => GET
    [CONTENT_TYPE] => 
    [CONTENT_LENGTH] => 
    [SCRIPT_NAME] => /headers.php
    [REQUEST_URI] => /headers.php
    [DOCUMENT_URI] => /headers.php
    [DOCUMENT_ROOT] => /usr/local/nginx/html
    [SERVER_PROTOCOL] => HTTP/1.1
    [GATEWAY_INTERFACE] => CGI/1.1
    [SERVER_SOFTWARE] => nginx/0.7.65
    [REMOTE_ADDR] => 178.110.238.153
    [REMOTE_PORT] => 44225
    [SERVER_ADDR] => 178.79.137.101
    [SERVER_PORT] => 80
    [SERVER_NAME] => www.tehforum.co.uk
    [REDIRECT_STATUS] => 200
    [SCRIPT_FILENAME] => /home/tehforum/httpdocs/headers.php
    [HTTP_REFERER] => http://www.tehforum.co.uk/forum/messages.php?webtag=DEFAULT&msg=37795.5
    [HTTP_ACCEPT] => application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
    [HTTP_USER_AGENT] => Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.63 Safari/534.3
    [HTTP_ACCEPT_ENCODING] => gzip,deflate,sdch
    [HTTP_ACCEPT_LANGUAGE] => en-GB,en-US;q=0.8,en;q=0.6
    [HTTP_ACCEPT_CHARSET] => ISO-8859-1,utf-8;q=0.7,*;q=0.3
    [HTTP_COOKIE] => __utmz=47895714.1272732323.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); bh_auto_logon=Y; bh_remember_username=boughtonp; bh_remember_password=++++++; bh_remember_passhash=snip; bh_DEFAULT_style=default; bh_forum_style=default; user_logon=boughtonp; user_passhash=snip; sess_hash=snip; thread_mode_DEFAULT=1; __utma=47895714.682241493.1285301682.1287702298.1287772993.96; __utmc=47895714; __utmb=47895714.65.10.1287772993; forum_style=default
    [HTTP_HOST] => www.tehforum.co.uk
    [HTTP_CACHE_CONTROL] => max-age=43200
    [HTTP_CONNECTION] => keep-alive
    [PHP_SELF] => /headers.php
    [REQUEST_TIME] => 1287784851
)
Array
(
)


And force-refreshing:

code:
Array
(
    [USER] => www-data
    [HOME] => /var/www
    [FCGI_ROLE] => RESPONDER
    [QUERY_STRING] => 
    [REQUEST_METHOD] => GET
    [CONTENT_TYPE] => 
    [CONTENT_LENGTH] => 
    [SCRIPT_NAME] => /headers.php
    [REQUEST_URI] => /headers.php
    [DOCUMENT_URI] => /headers.php
    [DOCUMENT_ROOT] => /usr/local/nginx/html
    [SERVER_PROTOCOL] => HTTP/1.1
    [GATEWAY_INTERFACE] => CGI/1.1
    [SERVER_SOFTWARE] => nginx/0.7.65
    [REMOTE_ADDR] => 178.110.238.153
    [REMOTE_PORT] => 56880
    [SERVER_ADDR] => 178.79.137.101
    [SERVER_PORT] => 80
    [SERVER_NAME] => www.tehforum.co.uk
    [REDIRECT_STATUS] => 200
    [SCRIPT_FILENAME] => /home/tehforum/httpdocs/headers.php
    [HTTP_REFERER] => http://www.tehforum.co.uk/forum/messages.php?webtag=DEFAULT&msg=37795.5
    [HTTP_PRAGMA] => no-cache
    [HTTP_ACCEPT] => application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
    [HTTP_USER_AGENT] => Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.63 Safari/534.3
    [HTTP_ACCEPT_ENCODING] => gzip,deflate,sdch
    [HTTP_ACCEPT_LANGUAGE] => en-GB,en-US;q=0.8,en;q=0.6
    [HTTP_ACCEPT_CHARSET] => ISO-8859-1,utf-8;q=0.7,*;q=0.3
    [HTTP_COOKIE] => __utmz=47895714.1272732323.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); bh_auto_logon=Y; bh_remember_username=boughtonp; bh_remember_password=snip; bh_remember_passhash=snip; bh_DEFAULT_style=default; bh_forum_style=default; user_logon=boughtonp; user_passhash=snip; sess_hash=snip; thread_mode_DEFAULT=1; __utma=47895714.682241493.1285301682.1287702298.1287772993.96; __utmc=47895714; __utmb=47895714.67.10.1287772993; forum_style=default
    [HTTP_HOST] => www.tehforum.co.uk
    [HTTP_CACHE_CONTROL] => no-cache, max-age=43200
    [HTTP_CONNECTION] => keep-alive
    [PHP_SELF] => /headers.php
    [REQUEST_TIME] => 1287785010
)
Array
(
)




Hmmm... was assuming the remote_port change was a difference between regular loading and refreshing, but seems it's just sometimes changing on its own, independent of that. Dunno if that's meaningful.
EDITED: 22 Oct 2010 23:13 by BOUGHTONP
From: Matt22 Oct 2010 23:37
To: Peter (BOUGHTONP) 9 of 15
The remote port is the client port (i.e. your PC's port) so that will change quite often.

The only difference between your header data and my own, bar the obvious session hash etc., is the HTTP_CACHE_CONTROL header. Using Firefox 3.6.11, I'm always sending max-age=0 but you seem to be sending max-age=43200, which equates to 12 hours.

I'm not sure if max-age greater than 0 is a reliable way to detect a caching proxy server or not, but if it is, in your case it seems to imply the proxy server will cache the page for 12 hours. It would be nice to find out if that really is the case.

Would also be interesting to see what other user's (hint) get, especially those we know who are subscribed to ISPs that use proxy servers, which off the top of my head includes Virgin (well, old school ntlworld / Telewest customers - don't know if Virgin still use them) and AOL. Presumably a lot of mobile operators also use caching proxy servers too.
From: Mouse23 Oct 2010 00:02
To: Matt 10 of 15
I think I get this happening on O2 mobile broadband on android. But not on Virgin home broadband and not on Be or Plus at work.
From: Matt24 Oct 2010 14:03
To: Mouse 11 of 15
Can you try accessing the script above and copy and paste here what you receive back?

For the record (from O2 mobile) I get the same HTTP_CACHE_CONTROL header as Pete, but with an additional no-cache parameter before it.
From: Mouse24 Oct 2010 22:20
To: Matt 12 of 15
Erm, sorry.. I'm not sure how I do that.
From: Peter (BOUGHTONP)24 Oct 2010 22:43
To: Mouse 13 of 15
1. Visit Teh on your mobile.
2. Click the link above. (Or this one here.)
3. Copy output, Paste into a reply.

Which bit are you not sure about? :?
EDITED: 24 Oct 2010 22:46 by BOUGHTONP
From: Mouse24 Oct 2010 23:00
To: Peter (BOUGHTONP) 14 of 15
Ah I see sorry, hang on.
From: Mouse24 Oct 2010 23:04
To: ALL15 of 15
php code:
Array
(
    [USER] => www-data
    [HOME] => /var/www
    [FCGI_ROLE] => RESPONDER
    [QUERY_STRING] =>
    [REQUEST_METHOD] => GET
    [CONTENT_TYPE] =>
    [CONTENT_LENGTH] =>
    [SCRIPT_NAME] => /headers.php
    [REQUEST_URI] => /headers.php
    [DOCUMENT_URI] => /headers.php
    [DOCUMENT_ROOT] => /home/tehforum/httpdocs
    [SERVER_PROTOCOL] => HTTP/1.1
    [GATEWAY_INTERFACE] => CGI/1.1
    [SERVER_SOFTWARE] => nginx/0.7.65
    [REMOTE_ADDR] => 94.171.153.25
    [REMOTE_PORT] => 57430
    [SERVER_ADDR] => 178.79.137.101
    [SERVER_PORT] => 80
    [SERVER_NAME] => www.tehforum.co.uk
    [HTTP_ACCEPT_ENCODING] => gzip
    [HTTP_ACCEPT_LANGUAGE] => en-GB, en-US
    [HTTP_CLIENT_IP] =>
    [HTTP_COMING_FROM] =>
    [HTTP_FORWARDED] =>
    [HTTP_FORWARDED_FOR] =>
    [HTTP_HOST] => www.tehforum.co.uk
    [HTTP_IF_NONE_MATCH] =>
    [HTTP_IF_MODIFIED_SINCE] =>
    [HTTP_REFERER] =>
    [HTTP_SCHEME] => http
    [HTTP_USER_AGENT] => Mozilla/5.0 (Linux; U; Android 2.2.1; en-gb; Nexus One Build/FRG83) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1
    [HTTP_VIA] =>
    [HTTP_X_COMING_FROM] =>
    [HTTP_X_FORWARDED] =>
    [HTTP_X_FORWARDED_FOR] =>
    [REDIRECT_STATUS] => 200
    [SCRIPT_FILENAME] => /home/tehforum/httpdocs/headers.php
    [HTTP_ACCEPT] => application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/ png,*/*;q=0.5
    [HTTP_ACCEPT_CHARSET] => utf-8, iso-8859-1, utf-16, *;q=0.7
    [PHP_SELF] => /headers.php
    [REQUEST_TIME] => 1287957681
)
Array
(
)
EDITED: 25 Oct 2010 11:10 by MATT