FLASH - PREVENTING DATA CACHING DURING LOADVARIABLES


Overview of the Problem

The LoadVars, loadVariables, loadMovie, and getURL function in Flash 4 and beyond enables a Flash movie to request (pull) data from a server.   More explanation about this subject can be found here and here.  

Once you master the concept, the process is quite straightforward in most cases.  Problems might occur however, when dealing with data that changes periodically (non-static data).  The problem is that sometime the data is being retrieved from the cache, thus the  Flash movie will receive outdated data.  As an example, below is a Flash movie that executes a CGI script to request the current time (my  time-zone might not match yours, so just ignore the hours).  

Here's the code that is attached to the button:

on (press) 
{
  loadVariables(
    "http://www.permadi.com/time.pl", "");
}
  

Click the Update Time button.  

Wait until the data is displayed, then after 5 seconds or so, click the Update Time button again.  

Does the /:secs variable change value?  On my computer, with Internet Explorer 5, it doesn't, with Netscape 4.7, it does.  The failure in IE is because the data is not being re-loaded, but retrieved from the cache instead.  

You might ask: "It works on my computer, why should I bother?"  If you're the only person who is going to use the program, then you need not to worry so much.  However, the fact that it fails on one computer is an indication that it might fail on others as well.  Caution: when testing from Flash environment using Test Movie, the movie will not load from cache; test from a browser to see the problem.  

The Purpose of Caching

Caching is designed to speed up data loading.  When a browser request data from a web server, it checks whether that data is already loaded previously and whether it is still on the cache.  If yes, then the browser might load from the cache instead of requesting the data again from the server (this avoids a re-downloading, which could be slow on a modem connection).  

The caching behavior is somewhat unpredictable.  It depends on the browser, the user's cache-setting, and to some extent, the way the script is written, or even the server.  For example, on Internet Explorer 5, user can set how large the cache is from Tools » Internet Options » Temporary Internet Files

For all practical purposes, it is impossible to predict whether a particular piece of data is being cached or not.  Yes, we can use "pragma no-cache," and set the server to prevent caching, and so on.  But these aren't reliable either, some browsers might still ignore these directives.

What is needed is a way to "fool" the caching mechanism to think that the data being requested is new.  In Flash, this is usually done by sending a dummy variable along with the data request.  The dummy variable must always be unique to stop the caching mechanism from taking action.  For example, the three loadVariables requests below are interpreted as 3 different requests.

loadVariables("http://www.permadi.com/time.pl?1", "");
loadVariables("http://www.permadi.com/time.pl?2", "");
loadVariables("http://www.permadi.com/time.pl?3", "");

Below, we will explore several alternatives of getting the dummy variable.  But before continuing, I should to tell you that I do not have the resource to confirm which method below works best.  Consider these as food for thoughts, and not as definite answers.

TEST EXPERIMENT

Below is a Flash movie that calls a cgi script.  The cgi script will display the server's time.  

  • The Variable and the Send Using dialog box allows the user to experiment with different method of "passing a unique number" to the server.  
  • The Command window shows the code to be executed.  
  • The Update Time button will execute the code when clicked.

To do the test, select the desired Variable and Send Using parameter; and do not change anything else.  Make two data requests with the same Variable and Send Using settings twice, subsequently.  The value of /:secs must change every time a data request is made.  If it's not, then the data was pulled from the cache.

 

Here's the explanation for the other items:

  • The GET String window will display the query string passed to the CGI script.  In technical technical CGI lingo, this is the QUERY_STRING.
  • The POST String window will display the input string passed to the CGI script.  In technical CGI lingo, this is the stdin.
  • The myVar shows the value of a variable called myVar.  For now, just leave it alone, and do not change the value.  This variable is the one and only variable in the same timeline as the Update Time button (the code which calls the loadVariables is embeded in the Update Time button).  

Here's my result on Internet Explorer 5.5 (Windows 98)

 

SEND USING

  NONE GET  POST
no variable not updated not updated updated
random() variable updated updated updated
getTimer() variable updated updated updated

[shade indicates odd results]

Side note: on  Netscape 4.7 (Windows 98), my test results is: updated in every case - translation=very odd.  

OBSERVATIONS


With reference to the result table above.  Note the curious success when using random() and getTimer() with neither GET nor POST (blue shaded).  If you look the the command generated in the getTimer() case, it looks like the following:

loadVariables("http://whatever/time.pl?dummyVar=" add getTimer(),"");

Which, when executed will produce something like:
loadVariables("http://whatever/time.pl?dummyVar=2020","");or:
loadVariables("http://whatever/time.pl?dummyVar=3021","");

Why does this works, even without GET or POST?  It's because the unique number is already being appended to the URL (thus, they are considered as different requests).  I.e.: sending a request with ?something appended at the end of the URL is implicitly considered as a GET request.

To convince ourselves, let's look at the result of the GET String window.  This window shows what the cgi-script's QUERY_STRING receives when the data request was made.  The result on the NONE column shows that dummyVar is indeed being sent even without the explicit GET.

GET String / QUERY_STRING table

  NONE GET 
(Flash automatically sends/appends all variables in the current timeline to the URL)
POST
(Flash automatically sends all  variables in the current timeline)
no variable [blank] myVar=test [blank]
random() variable dummyVar=2323 dummyVar=1823?myVar=test dummyVar=2323
getTimer() variable dummyVar=323 dummyVar=842?myVar=test dummyVar=2323

So, it can also be concluded that the piece of code below:

loadVariables("http://whatever/time.pl?dummyVar=" add getTimer(),"");

is the same as:

dummyVar=getTimer();
loadVariables("http://whatever/time.pl","GET");

That is because the GET method in FLASH automatically appends a ? and URL encode all variables that in the same timeline to the end of the URL.  

A side effect of that feature can be seen in the result that shows dummyVar=842?myVar=test.  In the test movie above, myVar is being passed because this variable is in the same timeline.  So, sure enough, Flash automatically appends a ? at the end of the URL followed by all variables in the current timeline (in this case: myVar), on top of the first ? already added manually in the code by
"http://whatever/time.pl?dummyVar=" add getTimer().

So that code will first translate to:

http://whatever/time.pl?dummyVar=842 

And then Flash translates the GET to:

http://whatever/time.pl?dummyVar=842?myVar=test

If you need to do form processing, this could be problematic because the second ? is not a proper separator for URL/cgi encoding.  My suggestion is not to manually append the URL, but let Flash do it for you like below.

dummyVar=getTimer();
loadVariables("http://whatever/time.pl","GET");

Which will produce proper encoding like:
http://whatever/time.pl?dummyVar=842&myVar=test

Now let's look at the table below, which shows the value of POST String (this is the value of the stdin - standard input that the script received by the POST method):

POST String / STDIN table

  NONE GET  POST
no variable [blank] [blank] myVar=test
random() variable [blank] [blank] myVar=test
getTimer() variable [blank] [blank] myVar=test

Here, notice that the dummyVar is not actually being sent via the POST method at all in the following code:

loadVariables("http://whatever/time.pl?dummyVar=" add getTimer(),"GET");
loadVariables("http://whatever/time.pl?dummyVar=" add getTimer(),"POST");

However, if you look at the 3rd column of the QUERY_STRING table, you can see that the dummyVar is sent via the QUERY_STRING instead.

This shows two things:

1) With the POST parameter, both GET and POST methods are used (you get myVar, which is sent by POST, and dummyVar, which is sent by GET).  Try running the movie with random() and POST method to see.  

2) Technically, the following code:

loadVariables("http://whatever/time.pl?dummyVar=" add getTimer(),"POST");

is not the same as:

dummyVar=getTimer();
loadVariables("http://whatever/time.pl","POST");

ABOUT USING A RANDOM NUMBER

This method is quite reliable, given a large range of randomness.  However, there's no guarantee that your program won't get the same number more than once.  How likely?  Not very likely, but still possible.

ABOUT USING USING THE TIMER

The getTimer() in Flash  returns the number of milliseconds since the Flash movie is started.  

In reality, this technique should work every time.  However, since a "number" has a maximum value, it will wrap around (reset to 0) after it reaches the maximum value.  In the event that the data request occurs after the "time" variable wraps to the same value as a previous request, this method might fail.  How likely is this?  Again, not likely.  A more probable problem is this kind of scenario:

  • An user opens the web page containing the Flash movie.  
  • The loadVariables() is called on getTimer()=500. 
  • User needs to walk the dog, so he/she closes the browser.
  • A while later, that user comes back and reopen the web page again.
  • Just by a coincidence, the loadVariables() happens to be called on getTimer()=500.  (Remember that getTimer() gets reset every time the Flash movie is restarted.)  

Okay, that scenario is quite ridiculous, but... if the loadVariables is being called often (say every 10 second), the chance of it happening is increased.  Then again, one failure data won't be much noticeable for a movie that updates every 10 seconds, unless you get a chain-of-failures because of the fixed time interval.  

ABOUT USING USING DATE AND TIME

Since Flash now has functions to acquire the date and time of the local machine, you could potentially use those values as the randomizer.  For example:

var time=new Date().getTime();
oadVariables(
    "http://whatever/time.pl?time=" add (time), 
    "", "POST");

The getTime() function returns date and time, not just time, so this could have been a perfect solution because time is always unique no matter what.

The main problem with it is dead batteries.  Granted, it's unlikely that an user will go to a Flash site on an exact same time cycle a computer with a dead battery.  It's a tempting solution.

HOW ABOUT JUST USING A COUNTER?

One question that might occur is then: if you're going to use a number at all, why not just use a counter, won't this avoid having the same number occurrences? 

loadVariables(
    "http://whatever/time.pl?counter=" add (counter++), 
    "", "POST");

This counter method will work fine during the first session.  However, once the same user exits the movie, and then comes back at a later time, the counter value will get reset to its initial value, thus you'd get the same number again.  For example:

  • An user opens the web page containing the Flash movie.  
  • counter is reset to 0.
  • The loadVariables is called (counter is 0 at this time) 
  • User exits the browser.
  • User re-open the browser and the Flash movie.
  • counter is reset to 0 again.
  • The loadVariables is called (counter is 0 again at this time, so caching is not prevented).

This kind of situation can be seen from the test example above by the following test: 

Set to no variable, and GET, then change the value of myVar to 1.  Click the Update Time.   Write down the /:hours, /:mins, /:secs as result1 (in my case, the result is 19, 42, 39).

Still with the same setting, (no variable, and GET), change the value of myVar to 2.  Click the Update Time.  Write down the /:hours, /:mins, /:secs as result2 (in my case, the result is 19, 45, 53).

Still with the same setting, (no variable, and GET), change the value of myVar to 1 again.  Click the Update Time.  Write down the result as result3

Assuming your browser's cache is enabled, then sure enough, the value of result3 is the exact same value as result1.  Why?  Because when the data request is made, myVar has the same value.

Possible workaround: use cookie to store the counter, or even write a script to save user name and its counter value.  This is simply too much of a hassle, not reliable (user can delete the cookie).  It's just not worth it.

NON-FLASH METHOD

  • Set the server to prevent caching, this method vary between different servers, and likely to be not accessible for many webmasters.
  • Have the CGI script output the following HTTP headers:
    <META HTTP-EQUIV="Expires" CONTENT="0">
    <META HTTP-EQUIV="Pragma" CONTENT="no-cache">
    <META HTTP-EQUIV="Cache-Control" CONTENT="no-cache">
These are fine to add, but should not be relied upon because some browsers might ignore them.

FOOTNOTE

Avoid using the Counter method, it's simply not worth using.  The Non-Flash method are ok to use as supplements, but do not rely on them.

The other methods are fine.  If I have to pick one, I'd probably go with the Random number method because of it's simplicity.  I would also use POST; just because the POST method can carry more (longer) data, whereas the GET can only carry limited data (usually the limit is 256 characters).  Now, if you want to be a bit fancy, you can also use a combination of "random" and "timer" such as:

dummyVar=(getTimer()+random(100000))
// followed by the data request
loadVariables("http://whatever/time.pl","POST");

or, consider this:

dummyVar1=getTimer();
dummyVar2=random(100000);
// followed by the data request
loadVariables("http://whatever/time.pl","POST");

(The two variables are astronomically unlikely to end up with the same pair of values.)


<<
INDEX>>

(C) 2002, F. Permadi