Providing a Real Time Web Interface Using AJAX
The responsiveness of the web application will be far better if it can send data to and retrieve data from a server asynchronously (in the background) without interfering with the display and behavior of the existing page. AJAX is a method that makes it possible for a browser to send a request to a server without having to reload the entire page. All examples we have shown so far have reloaded the entire page from the server when you click the HTML form's submit button. AJAX can be used as a base for designing single-page applications or designing a hybrid client/server solution such as what is used by our LSP tutorials.
AJAX is an acronym for Asynchronous JavaScript and XML. However, in most cases, XML is not used for encoding the transported data. In most cases, data is sent to the server using standard percent (URL) encoding, and the response can be anything, including HTML, text, images, and JSON.
Have you thought about how our LSP/Lua tutorials work and what happens when you click the Run button in the examples? All examples are managed by JavaScript code in the browser (the client) and are sent to the server using an AJAX call when you click the examples's run button. In addition, each example is loaded from the server using AJAX when the page initially loads. When the server sends the example code to the browser, JavaScript code in the client injects the server's response data into the example's JavaScript powered editor.
Server Side Scripting
The traditional method of generating dynamic web pages is to generate everything on the server by, for example, using a server side scripting language such as LSP. In the LSP and HTML Forms tutorial, we showed you how to dynamically generate HTML on the server side.
Combining Server Side and Client Side
Over the years, it has become increasingly popular to combine server side generated content with client side generated content. More and more web applications rely on using Javascript as a method for dynamically changing the user interface on the fly. The bulk of the user interface is still typically generated by the server including the JavaScript code. When the browser has loaded a server generated HTML page, the embedded JavaScript code inside the HTML response data is executed by the browser. This Javascript code can, for example, connect back to the server and fetch data in real time, which is then instantly updated in the client without having to refresh the page.
AJAX is HTTP commands sent from the browser to the server using the browser's XMLHttpPRequest object. The XMLHttpPRequest object, which is accessible from JavaScript, is an API/object that enables JavaScript code to send HTTP commands directly to a server. Asynchronous means that the functions in the XMLHttpPRequest object do not block. Instead a JavaScript callback function is provided which is then called by the XMLHttpPRequest object when the server returns a response. Using callbacks (aka events) is common in JavaScript since the typical browser JavaScript environment is single threaded. A blocking call would cause the browser's user interface to freeze while waiting.
The XMLHttpPRequest API is weird and cumbersome, but luckily many JavaScript libraries make it very easy to use this object. We will be using the very popular jQuery JavaScript library instead of directly using the XMLHttpPRequest. Although you can find many libraries that help you fast track your client side JavaScript development, we particularly like jQuery since it is small and makes it easy to write compact JavaScript code without having to learn the details of the browser's Document Object Model. JQuery is your perfect companion when it comes to writing your client side JavaScript code. The JQuery JavaScript library is also embedded in the Mako Server's resurce file (mako.zip). You can load this library from the server by navigating to http://::ffff:18.221.26.250/jquery.js.
jQuery Introduction
Although providing a comprehensive tutorial on JQuery is beyond the scope of this article, we will give you a quick introduction to the $()
function since many get very confused by this JQuery library function. In JavaScript, $
is a valid character for variable names.
The JQuery $()
function is a Jack of all trades in JQuery and the function accepts a wide variety of arguments.
For example, the following code will cause the browser to popup a "hello" message when the page has loaded.
In the above example, we pass in an anonymous function to the JQuery $()
function at line 5.
Let's go ahead and create a basic JQuery example that installs a keyboard event handler. Every time the event triggers, the keyboard key
number is converted to text and the text is then appended to the HTML in the browser.
Click the above Run button and start entering text in the input field.
We install an "on page load" event callback function on line 5 by providing an anonymous function to the jQuery $()
function.
The "on page load" callback function will be called when the page has loaded and the Document Object Model (DOM) is ready.
We start modifying the DOM on line 6 so it is important that the DOM is ready at that point.
Line 14 to 16 is our static HTML. The message "Please enable JavaScript" should not be visible to users that have
JavaScript enabled since the JavaScript code on line 5 erases the content within the <h1> element.
The code construction $("#out").empty();
means look up the HTML element in the DOM tree whose ID is "out" and empty this object.
The construction on line 7 means look up the HTML element in the DOM tree whose ID is "in" and install a keyboard event handler within the realm of the <input id="in" type="text" /> element. When a key is pressed in this input element, the keyboard event triggers, which in turn looks up the "out" element, converts the keyboard event to a character, and appends the character to the "out" element. The "out" element is the H1 tag on line 14.
AJAX Example
As you have probably noticed, we have so far not used any AJAX. We will soon add AJAX to our example. It is not possible to use AJAX if you do not have an understanding of how to manipulate the browser's DOM since you cannot display the result from your AJAX response unless you know how to update the HTML by using JavaScript. The example below shows how to do that.
We use the JavaScript function String.fromCharCode()
in example 7.2 on line 8 above.
This function converts the key code from the keyboard event to a string containing one character. Let’s go ahead and modify line 7 to 9 in
example 7.2 to the following:
$("#in").keypress(function(ev) { $.getJSON(window.location,{key:ev.which}, function(rsp) { $("#out").append(rsp.char); });
jQuery provides a number of functions that simplifies the use of the browser’s raw XMLHttpPRequest API. One of these functions
is $.getJSON(), which sends$ URL encoded data to the server and expects JSON
as the response data from the server. The first argument to this function is window.location which is the URL to the page itself.
This means we will be sending the AJAX request back to the same LSP page on the server.
The second argument is a JavaScript object containing the key-value pairs we want to send to the server.
A JavaScript object is similar to a table in Lua. We only need one key/value pair in this example and that is the keyboard event number.
This number will be converted to a string containing one character by Lua code on the server side and the response will be encoded as JSON.
The last argument to $.getJSON
is the "on server response" callback function. The "rsp" argument is the decoded
JSON response data from the server. The JavaScript object "rsp" (from the decoded JSON) contains one property
"char" which is the keyboard event number converted to a string by the server side Lua code.
The complete AJAX transformation modification to example 7.2 is shown below.
Line two in the above example is strictly not needed in our example. It’s just an extra security measure that makes sure only AJAX requests can trigger the Lua code on line 3 to 14. jQuery adds the HTTP header "x-requested-with" and checking for this header is a convenient method for filtering out all requests that are not AJAX requests.
We check if our client side AJAX code (line 24) sent the server the key/value pair "char" on line 4. It would be a program error if this was not set. Line 5 to 13 deals with the AJAX request and the JSON response. The keyboard event is converted to a string character on line 8 if it is within the range of the standard printable ASCII table. Line 10 triggers if the keyboard key is a non printable character. The code on this line converts the number to a string number. Non printable characters are generated from keyboard events such as the keyboard "Enter" button which generates the key number 13.
Line 12 is for debugging purposes. The response string is printed to the server’s trace buffer. You will see this being printed if the server is running in a console-- i.e. not running as a background service.
Note: you cannot see the console output since the server is running as a background service, however, you can still see the server's output by opening the TraceLogger.
Line 13 is where the magic happens. The method response:json
takes a Lua table as argument, converts this to JSON, sends the JSON data to the client, and stops the script (the LSP page)
– in other words, no code is executed on the server after line 13. Lua functions do not require parentheses
if only one argument is supplied to the function. Line 13 can be changed to response:json{char=resp}
. We dynamically create a
Lua table and pass this table into method response:json()
. This table includes one key/value pair, where ‘char
’ is the key
and the data in ‘resp
’ is the value.
Note that the server side LSP page returns the HTML starting on line 17 and ending on line 36 if the request is not AJAX and if the client did not
send the "char
" key/value pair. If you think about this, you will realize that the LSP code in example 7.3 can send both HTML
and JSON responses to a client.
AJAX Limitations
AJAX is great in solutions where the client is the one responsible for generating all the events, but how do you use AJAX if the server is the one generating events? The server has no way of telling the client that there is new data since an HTTP request is only open when a client sends a request to the server. One way to circumvent this problem is to use AJAX and constantly poll the server for new data. However, this is not an ideal solution since you get a delay in the event notification on the client due to the poll delay. You also create unnecessary network traffic, and the server side code must be designed such that it can buffer the event data until the client sends the poll request. Our online article The scalability problem with using HTTPS explains this problem in detail.
Using WebSockets would fix the polling problem since the client and server maintain a persistent connection where the server can, at any time, send an event to the client. The next tutorial provides an introduction to WebSockets.
AJAX over WebSockets
The following example is a copy of the example included with our AJAX over WebSockets Tutorial.
Since WebSockets lets us multiplex data on the same connection, we can easily implement an AJAX library on top of WebSockets and still use the same WebSocket connection for bi-directional real-time data transfer.
The original AJAX over WebSockets example has been modified to run on the Lua example engine, and all example code has been combined into one file. You may want to read the next tutorial, WebSockets Introduction, prior to studying the following example. We also recommend reading our AJAX over WebSockets Tutorial which explains how this example works. The JavaScript code in the example below uses the new Promise API.