Back Button and Bookmarking

Dynamic web applications that avoid page refreshes are very powerful, but this normally means that the Back and Forward buttons stop working in the browser. In addition, it can be hard to give the user an URL that can be bookmarked.

However, Dojo provides a solution to these issues, allowing a web application to capture the Back and Forward button clicks, and set a unique URL in the browser's location field. The solution is to use dojo.undo.browser.

dojo.undo.browser.setInitialState(state);

var state = {

	back: function() { alert("Back was clicked!"); },

	forward: function() { alert("Forward was clicked!"); }

}; 

dojo.undo.browser.addToHistory(state);

Dojo Data

What is dojo.data?

Simple example

movies.csv ==> CsvStore ==> movies.html

What sorts of datastores are available?

As of January 2007, we have five simple datastores, which are included in dojo as example datastore implementations. The four datastores are:

The dojo.data APIs

• dojo.data.core.Read -- 10 basic read-only datastore access methods
  
  
• dojo.data.core.Write -- 9 basic methods for creating data items and updating attribute values
• dojo.data.core.Identity -- 2 basic methods for datastores that can uniquely identify items

• dojo.data.core.Notification -- update notifications
• dojo.data.core.Schema or Model
• dojo.data.core.Attribution -- create/modify timestamps, author, etc.
• dojo.data.core.Versioning -- for access to old versions of items
• dojo.data.core.Derivation -- derived attributes and calculated values

Status as of December 2006

Terminology

• data source -- the place that the raw data comes from. In the movies.csv example above, the movies.csv file is the data source. The data source could be a file, a database server, a web service, or something else completely.
• datastore -- a JavaScript object that reads data from a data source and makes that data available as data items via the dojo.data APIs
• dojo.data APIs -- the standard set of methods that datastore implements. The dojo.data module includes a set of APIs (Read, Write, etc.), and a datastore may implement one or more of the APIs.
• internal data representation -- the private data structures that a datastore uses to cache data in local memory (XML DOM nodes, anonymous JSON objects, arrays of arrays, etc.)
• item -- a data item that has attributes with attribute values. In the movies.csv example above, Alien is an item with three attributes, Title:'Alien', Year:1979, and Producer:'Ridley Scott'.
• attribute -- one of the fields or properties of an item. In the movies.csv example, Producer is an attribute.
• value -- the contents of an attribute for a given item. In the movies.csv example, 'Ridley Scott' is a string value.
• reference -- a value in an item that points to another item
• identity -- some sort of identifier that can be used to uniquely identify an item within the context of a single datastore.
• query -- some sort of specification or request that asks a datastore for some subset of the items it knows about. A query will often be a string, but in some datastores it might be a number or a date or a complex object structure.

Source code organization

API documenation

http://dojotoolkit.org/api/#dojo.data

You can also look at the API definition files themselves to see the documentation about individual API methods:

• dojo.data.core.Read
  
  
• dojo.data.core.Write
• dojo.data.core.Identity

Examples

In the dojo.data unit-tests, there's an example page that allows you to read data from any of a variety of different data sources and then display data in a few different widgets:

• http://archive.dojotoolkit.org/nightly/tests/data/test_ManyStores.html

We also have a number of dojo.data unit-test pages, which may serve as simple examples of how to use the APIs:

• http://archive.dojotoolkit.org/nightly/tests/data/

Dependency diagram

The widget code itself can be independent of the dojo.data APIs, and know nothing about how data access is done. The widget binding code depends on both dojo.data APIs and on the widget itself, but the dojo.data APIs are independent of both the widget itself and the widget binding. The widget and the binding are both independent of any particular datastore implementation, which allows a single widget binding to be used with a variety of datastores implementations.

Once one person has written one binding to display dojo.data items in a FilteringTable, then data items from any datastore can be displayed in a FilteringTable. If Dojo someday has 15 datastore implementations, and has 20 data display widgets, then dojo authors will only need to write 20 bindings in order to connect all the widgets to all the datastores. Without a standard dojo.data API, we would need a different bit of intermediary code for each possible connection between a widget and a datastore -- with 15 datastores and 20 widgets, that would require 300 different pieces of intermediary data translation code.

                                 +-----------+

    Widgets           Bindings   | dojo.data |    Datastores      Data Sources

                                 |    APIs   |

+------------------+             |           |

| Trees            |             |           |

| (TreeV3)         |-- binding --|           |

+------------------+             |           |   +------------+   +-----------+

                                 |           |---| CsvStore   |---| .csv file |

+------------------+             |           |   +------------+   +-----------+

| Tables & Grids   |             |           |

| (FilteringTable) |-- binding --|           |   +------------+   +-------------------+

+------------------+             |           |---| YahooStore |---| Yahoo web service |

                                 |           |   +------------+   +-------------------+

+------------------+             |           |

| Charts & Graphs  |             |           |   +------------+   +------------+

| (Chart)          |-- binding --|           |---| OpmlStore  |---| .opml file |

+------------------+             |           |   +------------+   +------------+

                                 |           |

+------------------+             |           |   +------------+   +------------+

| Other widgets    |             |           |---| RdfStore   |---| RDF server |

| (ComboBox)       |-- binding --|           |   +------------+   +------------+

| (SlideShow)      |-- binding --|           |

| (etc.)           |-- binding --|           |   +------------+   +--------------------+

+------------------+             |           |---| Other      |---| other file formats |

                                 +-----------+   | stores     |   | and web services   |

                                                 +------------+   +--------------------+

Goals

We are designing the data-access APIs with a wide variety of use cases in mind, and we hope that the APIs will work well with many different kinds of data: RDF, XML, SQL, CSV, OMPL, etc. Here are some of the features we've been designing the APIs to support:

• sync vs. async -- Some data sources can be read synchronously and will always have data available immediately (for example, if the data is coming from an HTML table on the web page itself). Some datastores may provide data asynchronously (for example, as the result of a search submitted to Yahoo). Some datastores may provide results incrementally -- the first 100 items in one batch, then a pause, then another 100 items, etc.
• read vs. write -- Some datastores may be read-only, and some may be read-write.
• hierarchical vs. tabular -- Some sources have hierarchical data and some sources have tabular data.
• strongly typed vs. loosely typed -- Some datastores may provide "strongly typed" data and some sources may have free-form data.
• literals vs. references -- Some datastores may only support literal values (for example, a CsvStore), while other datastores support both literals and item references.
• single-valued vs. multi-valued attributes -- Some datastores may store only single values for an attribute {name:"robert"}, and some datastores may allow multiple values for each attribute {name:["robert","bob"]}.
• simple identifiers vs. complex identifiers -- Most datastores may use simple serial numbers or key strings as unique identifiers, but relational databases might require compound keys, and other datastores may require XPath expressions, or may not support the notion of identifiers at all.
• derived attributes -- Some items may have derived attributes as well as stored attribute values (price = base-price + tax).
  
  
• lazy reference building -- Simple data sources will have only literal attribute values, but some data sources will have items with reference values that point to other data items. When the data is read from some serialized format (read from a file or retrieved from a server), all the references will be in the form of foreign keys or unique ids. At some point the datastore needs to notice the id-reference and replace it with a pointer to the referenced object. The datastore could to that reference building step in bulk for all the items immediately after the data is loaded, but the API is designed so as not to force the datastore to be implemented that way.
  
  
• lazy loading: reference faulting -- For a datastore that connects to large database, it won't be possible to load the whole data set into memory at once. For example, imagine a genealogy data set, with data records for hundreds of thousands of people, all interconnected. You might want to load only a few dozen records at first, and then incrementally load more records as the user navigates through the graph. When the user clicks on something in the UI to see who Mark's mother is, the UI code calls get(mark, mother) on the datastore API. The datastore might be able to return a result based on what's already loaded in memory, or it might need to send a request to the server to get more info.
  
  
• lazy loading: partial objects -- For some large data sets, you may want to start by loading only part of each data item. For example, in an e-mail client, you might have a list view that shows just the subject line for a bunch of messages -- when you select one of the messages, then the body of the message appears in the detail pane below. You don't want to load all the message body in bulk -- you want to get them one at a time, as needed.
  
  

Widget bindings

We hope that the dojo.data APIs will support:

• Dojo widget bindings -- Dojo includes a number of data display widgets (Trees, Charts, Data Tables, etc.), and we want to be able to "bind" any of those widgets to any datastore instance that conforms to the data-access APIs.
• third party widget bindings -- We also want the authors of proprietary third-party widgets to be able to bind those widgets to any datastore implementation.
• read/write bindings, with update notifications -- The bound widget updates when the data model updates, and the data model is updated when the user makes a change via the widget.
• easy creation of bindings -- support for creating bindings programmatically in JavaScript code or declaratively in HTML
• different granularities for bindings -- simple attribute bindings that bind one attribute of one data item to one property of one UI control (for example, an input field), as well as full data-set bindings that bind an entire set of data items to a widget like a chart or a grid

Data sources

We hope to eventually have different datastore implementations that read data from a wide variety of data sources. Here's a list of some of the kinds of data sources that we've had in mind while designing the APIs:

• text files
  • CSV, XML, JSON, OMPL, MBOX, ...
• data islands in web pages
  • XML data islands, HTML data, HTML
    and
    • lists
    conventional relational database servers

    • MySQL, Oracle, ...
    emerging semi-structured web-based databases

    • the Ning store, Dabble DB, Apache Jackrabbit, OSAF Cosmo, ...
    structured wiki content

    • JotSpot data storage, Semantic MediaWiki, Wikipedia infoboxes, ...
      
      
    web services

    • del.icio.us (via JSON API), Yahoo...
    RDF databases and application servers

    • IBM's open source Boca RDF store, Rhizome, OpenLink Virtuoso, ...

    Datastore data representations

    We hope that the dojo.data APIs do not impose unnecessary limitations on the ways that datastore implementations can represent data in memory. In particular, we've designed the APIs with a few different data representations in mind:

    • simple anonymous JSON objects: {name:"sales", headcount:38}
    • JavaScript objects that are instances of data classes: Dept.js, Employee.js...
    • XML DOM nodes

    Design notes

    Question: Why aren't the dojo.data APIs more object-oriented, with accessor methods available on the individual data items?

    For example, using the existing dojo.data APIs, a line of code might look like this:

    var value = datastore.get(kermit, color);

    Why weren't the APIs designed so that line of code instead looked like this:
    
    

    var value = kermitItem.get(color);

    Answer: Performance -- both memory use and execution speed.

    If we had used an object-oriented API for the item accessor methods, that would have required every datastore implementation to have a different JavaScript object for every data item. By putting the accessor methods on the datastore object rather than on a data item object, we made it possible for different datastore implementations to use different data structures.

    For example, our OpmlStore reads data from an XML text file using XMLHttpRequest via a call to dojo.io.bind. The XML text file is automatically parsed into XML DOM nodes, and the OpmlStore uses those XML DOM nodes themselves as its native data item representation, which avoids the unnecessary step of copying all the data from the DOM nodes into JavaScript objects.

    For more background on different approaches we considered, see the dojo.data design page on the dojo development wiki.
    
    

    History

    The dojo.data module was first envisioned back in 2005 or earlier, before Dojo release 0.1 had shipped.

    • The first experimental dojo.data code was written in January 2006, but it was geared only toward semi-structured data, and it had a single data model implementation rather than defining an API which could be implemented differently by different datastore authors.
    • In July 2006 IBM contributed a significant body of experimental data model code, including both code geared toward data from XML data sources and code geared toward data from structured data sources like relational databases.
    • Over the course of the next few months, August to October, Dojo held a series of dojo.data design meetings to try to design a unified set of APIs that would work well for a wide variety of use cases. In November 2006, for the Dojo 0.4.1 release, we checked in the initial dojo.data Read and Write APIs, as well as a handful of example datastore implementations, and a handful of test pages. The old design work is archived on the dojo wiki, on the dojo.data page.
      
      

    Future plans

    In 2007 we hope to:
    
    
    • extend the dojo.data APIs to handle features like attribution, versioning, and update notification
    • write more datastore implementations
    • write bindings for a number of dojo data display widgets
    • write more unit tests
    • improve the documentation

    How can people volunteer to help?

    If you'd like to help with the dojo.data module, here are some areas where we could especially use some help:
    
    
    • Dojo Book entry: add to this page, or improve what's already here
    • API method documentation: improve the documentation in the API definition files
    • unit tests: write more unit tests to help ensure that different datastore implementations are truly interoperable
    • datastore implementations: write new datastore classes that get data from different data sources (different web services, file formats, databases, application server, server frameworks, etc.)

    Language Utilities

    dojo.lang.* contains wrappers for common idioms found in the JavaScript language. It is not a replacement for language constructs but it does provide value added functions that go beyond what the standard JavaScript routines can do such as masking browser incompatibilities, structuring object oriented techniques and speeding up implementation of common procedures.

    The sections below show some of the idioms that deserve special mention. See the reference guide for the full list.

    All about functions

    Hitch

    Often you need to pass a function pointer as an argument (like above), but the function needs to operate within the context of an object. For example, if I have a class Foo that defines bar:

    bar: function() {
    
    	return this.i++;
    
    }

    and then I instantiate an instance of Foo:

    var fooInstance = new Foo();

    and I want a pointer to fooInstance.bar, I would use:

    dojo.lang.hitch(fooInstance, "bar");
    
    

    Curry

    Curry takes hitch one step farther, and embeds parameters into the function. If I declare an object with a function like this:

    var foo = {
    
    	bar: function(arg1, arg2){
    
    		alert(arg1 + " is a " + arg2);
    
    	}
    
    };
    
    

    And then I declare a curried function:

    	var tmp = dojo.lang.curry(foo, “bar�, "Bill");

    Then when I call tmp:

       tmp("baker")

    It will throw up the alert "Bill is a baker".
    
    

    
    
    

    Array Iterators

    Dojo provides various functions to Iteration over arrays, rather than the for() loop of traditional procedural languages. A callback function is specified and called for each element of an array.

    For example, this will display three alerts, one for each of the specified words:

    var colors = ["red", "white", "blue"];
    
    dojo.lang.forEach(colors, alert);

    This will return an array of the squares of the first five positive integers:

    var integers = [1,2,3,4,5];
    
    var squares = dojo.lang.map(integers, function(x){ return x*x; });

    There are also functions:

    • dojo.lang.filter - filter out matching items from an array into a new array
    • dojo.lang.some - tests if given function returns true for any element in the array
    • dojo.lang.every - tests if function returns true for every array element

    The iteration functions work on arrays, comma separated lists, or other array-like variables.

    Miscellaneous

    Here are a few more areas of dojo's dojo.lang module that deserve special attention.

    dojo.byId()

    Much like the prototype $() operator, dojo.byId represents the short form document.getElementById(). This will map a dom node id to a node, but if passed a node, it will return that same node. More importantly, however, is the fact that document.getElementById() is not available when using Dojo and dojo.byId() is the only way to grab nodes.

    
    
    Others
    dojo.lang.is*
    dojo.lang.setTimeout
    dojo.lang.delayThese
    dojo.lang.curry
    dojo.lang.assert

    Storage

    
    
    

    Introduction

    Imagine if web applications could store megabytes of data on the client-side, in the browser, both persistently and securely. No server needed.
    
    
    
    Imagine if web applications could work offline with the click of a button. Want to access your web based word processor when you are not on the network, with your private files stored privately, right on your own machine and not on some server? Now you can.
    
    
    
    Even better, imagine if all of this worked across the existing web; 95% of the existing browsers on the web could start using these features right now, with no software installs or funky new browsers.
    
    
    
    What could you build if you had these tools? How about a truly collaborative, web-based word processor with client-side storage for your private documents, as well as offline access? Maybe an Ajax RSS aggregator with client-side caching of the feeds you read and offline access? An offline, web-based book reader using data from the Internet Archive's Open Library would be cool.
    
    
    
    These are the goals of Dojo Storage
    
    

    What is Dojo Storage?

    Dojo Storage is a unified API to provide JavaScript applications with storage. It is a generic front-end to be able to provide all JavaScript applications a consistent API for their storage needs, whether this JavaScript is in a browser, a Firefox plugin, an ActiveX control, using Windows Scripting Host, etc. Further, the storage backends can use whatever mechanism is appropriate; dojo.storage automagically detects its environment and available storage options and selects the most appropriate one.
    
    
    
    The dojo.storage architecture is simple; a JavaScript application interacts with the Dojo Storage Manager, which selects the best available Storage Provider and makes it available. Storage Providers implement a generic interface, which makes the underlying storage system look like a simple hash table that can be saved and loaded from. A storage provider can optionally be persistent, and can provide metadata about their capabilities (isPersistent, hasSettingsUI, getMaximumSize, etc.). Under the covers, storage providers can use a wide array of possible ways to store data, from hidden Flash to native browser capabilities.
    
    
    
    Right now the dojo.storage system includes three storage providers, a Flash Storage Provider that uses a hidden Flash applet; a WHAT WG Storage Provider that uses native client-side storage abilities in Firefox 2; and a File Storage Provider that uses the native file system if a web app is loaded from the local file system. Creating other kinds of storage providers is a great way for the community to contribute.
    
    
    
    Please note that the WHAT WG storage provider is available in the 0.4.1 Dojo release, while the FileStorageProvider will be in the upcomming 0.4.2 Dojo release and is available on the Dojo trunk in the Subversion repository. The FlashStorageProvider has been available since Dojo 0.3.
    
    
    
    The Flash Storage Provider uses features available since Flash 6, including Flash's SharedObject's. By default, a web page can store up to 100K without user permission; after 100K the user is prompted for each order of magnitude increase. For example, if you store 101K, the user is prompted on whether to give you permission. Afterwards, the user will not be prompted until the next order of magnitude, which is at 1 megabyte.
    
    
    
    The WHAT WG storage provider uses native client-side storage features inside of Firefox 2. Firefox 2's API for client-side storage is based on the WHAT Working Group's specification. This API opens up at least 5 megabytes of client-side storage, instantly, in a highly-performant way. Dojo Storage will use this API before the Flash one, and fallback if the browser does not support the WHAT WG API.
    
    
    
    The File Storage Provider will use the local file system if a web app that uses Dojo Storage is loaded from a file:// or chrome:// URL; this will happen transparently, without the developer needing to invoke this storage provider. The File Storage Provider uses XPCOM on Firefox to achieve this, while it uses ActiveX on Internet Explorer. Code exists for Safari and Opera support, but is not finished yet. Please note that the FileStorageProvider has been removed from the Dojo Storage system, and is available as a manual patch -- this was to make Dojo Storage smaller, and because very few people need this provider. See ticket 2285 for the FileStorageProvider code.
    
    
    
    Dojo Storage, in combination with Flash and the WHAT WG provider, instantly works across about 95% of the existing web, covering a wide swath of the machines out there. It works with IE 6, Firefox, and Safari.

    Usage & Example

    The example usage we will describe is based on the Moxie demo application, which is a show case for Dojo.Storage; feel free to use the source code in Moxie as you please, since it is under a BSD license.
    
    
    
    At the bottom of the Moxie JS file, we first subscribe to find out when the storage system is ready for us to work with; we can not load or save values until the underlying storage provider is ready. In this case we want to call Moxie.initialize when storage is ready; if by the off chance it is already ready when we get to this code block, then we want to wait until the page is fully loaded before working with it:
    
    

    if(dojo.storage.manager.isInitialized() == false){
    
      dojo.event.connect(dojo.storage.manager,
    
                         "loaded", Moxie,
    
                         Moxie.initialize);
    
    }else{
    
      dojo.event.connect(dojo, "loaded",
    
                         Moxie,
    
                         Moxie.initialize);
    
    }

    It's very important that your code is right for listening to when storage is ready to be used; this is a common source of bugs. Your event listener must be at the top level and executed while the page is loading. Dojo.Storage writes thing out into the page during page load that won't work afterwards (i.e. it uses document.writes). The above code should be top-level and not be inside of a nested function; otherwise it won't execute on page load.
    
    
    
    When we are all loaded up, we can start to play with the storage system. For example, to load a value that was previously saved with some key, we just do the following:
    
    

    var results = dojo.storage.get(key);

    To save some value:
    
    

    try{
    
      dojo.storage.put(key, value,
    
                       saveHandler);
    
    }catch(exp){
    
      alert(exp);
    
    }

    'Value' can be a string or even a complicated JS object; we internally JSON everything before storing it as a flat string, and turn it back into an object on later retrieval. Dojo.storage actually does some nifty autodetection if it is working with strings, to avoid the JSON performance hit of evaling() and bypass this, which gives much better performance for storing large strings into storage, like XML files or digital books.
    
    
    
    Notice the 'saveHandler' above; a storage system can optionally ask the user if they want to accept or deny your storage request, so you must be ready for a save request to fail.
    
    
    
    'saveHandler' is a callback function that receives two arguments. The first is the 'status', which can be one of three values:
    
    

    • dojo.storage.SUCCESS - Saving was successful
    • dojo.storage.FAILED - User denied storage request
    • dojo.storage.PENDING - The user is being prompted with some UI on whether to approve this storage request

    The second argument is the 'keyName' that is being saved; since saving is asynchronous, it is sometimes useful to know which key is being worked with on the callback.
    
    
    
    The Flash Storage Provider pops up an underlying Flash storage dialog to the user after 100K, and every order of magnitude increase after that:
    
    
    
    
    
    The WHAT WG storage provider, used by default when Dojo Storage is loaded into Firefox 2, automatically gives you up to 5 megabytes of storage space, with no prompting like the Flash storage provider.
    
    
    
    Here's an example saveHandler:
    
    

    var saveHandler = function(status, keyName){
    
      if(status == dojo.storage.PENDING){
    
         // ...  
    
      }else if(status == dojo.storage.FAILED){
    
         // ...
    
      }else if(status == dojo.storage.SUCCESS){
    
         // ...
    
      }
    
    }
    
    
    
    try{
    
      dojo.storage.put(key, value,
    
                       saveHandler);
    
    }catch(exp){
    
      alert(exp);
    
    }

    There is more to the code, but those are the important bits in terms of understanding dojo.storage. It's pretty straightforward to use.
    
    

    Demos & References

    • Moxie - Web based rich text editor that integrates the Dojo Editor with Dojo Storage; also shows offline access
    • Dojo Storage Tester - Low level tester that can interact with information in Dojo Storage; also has quick lines to demo saving large amounts of XML into the store (an example Atom feed), as well as saving large amounts of text (a full ebook which is Goethe's Faust).
    • Blog Post on Moxie
    • Blog Post on Dojo Storage and it's architecture
    • Presentation on Dojo Storage at the Ajax Experience conference in San Francisco, 2006
    • Autogenerated API for Dojo Storage
    • Open bugs for Dojo Storage (all Dojo Storage tickets listed under "BradNeuberg" heading)
    • Blogs talking about Dojo Storage [uses Technorati]
    • The maintainer of Dojo Storage's weblog: Coding in Paradise

    API Reference

    See the automatically generated API reference

    Requires

    • dojo.lang
    • dojo.flash
    • dojo.event

    Provides

    dojo.storage

    A local durable cache

    
    
    

    dojo.storage.manager

    Manager class that autodetects available storage and instantiates best one; can be used for lower level control

    Contributors

    Brad Neuberg, bkn3@columbia.edu - Module maintainer and creator
    
    Alex Russel, alex@dojotoolkit.org - Created early version of Dojo Storage
    
    JB Boisseau, jb.boisseau@eutech-ssii.com - Created WHAT WG storage provider for Dojo Storage

    FAQ

    Help! Dojo.Storage Isn't Working!

    Some possibilities:

    • You must have the following SWF files in the same directory as your dojo.js file: flash6_gateway.swf, storage_dialog.swf, Storage_version6.swf, Storage_version8.swf
    • Make sure your web server is returning the correct MIME type for SWF files: application/x-shockwave-flash swf
    • You are on an unsupported browser. The following are supported: IE 6+, Firefox, Safari.
    • You have a really old version of Flash, or no Flash, and don't have permission on your computer to upgrade software (perhaps you aren't the administrator account and don't have permission to install Flash?)
    • You're being served from an https domain, where there is a known bug right now that needs to be fixed (I can't fix it because I don't have an SSL/HTTPS environment setup; any volunteers?)
    • Your event listener for the Dojo Storage Manager is incorrect. See below.

    It's very important that your code is right for listening to when storage is ready to be used. Your event listener must be at the top level and executed while the page is loading. Dojo.Storage writes thing out into the page during page load that won't work afterwards (i.e. it uses document.writes). Study the source code for the Moxie example storage application carefully before asking questions; compare your code and see what is different.

    How Do I Write a Storage Provider?

    See the FlashStorageProvider for an example. Basic steps:

    • Find the file that corresponds to your render environment; for example, if your provider will run in the standard web browser environment, then you will want to put your storage provider into src/storage/browser.js. If a file does not exist for your render environment yet, such as a Konfabulator storage provider, then you would create a new file corresponding to your environment, such as src/storage/konfabulator.js or src/storage/xpcom.js. All the providers for a given render environment must be in the same source file. If you have added a new file for a new environment, make sure to add the existence of this environment to src/storage/__package__.js so it is loaded for your new environment.

    • Subclass dojo.storage:
      
      

      dojo.inherits(dojo.storage.browser.FlashStorageProvider, dojo.storage);

    • Implement the dojo.storage methods, such as put() and get(). You MUST also implement the isAvailable() method in a solid way, since this method will be called by the Storage Manager to see if your provider is available in the given environment.
    • Tell the Storage Manager about our existence at the bottom of your source file; remember to order these registrations in the order you want storage providers to be chosen. For example, if you have three storage providers, such as a What WG provider, a Flash provider, and a proprietary Internet Explorer storage provider, and you want these to be preferred in this order, you would have the following at the bottom of your source file:

      // register the existence of our storage providers
      
      // choose What WG provider first
      
      dojo.storage.manager.register("dojo.storage.browser.WhatWGStorageProvider",
      
      				new dojo.storage.browser.WhatWGStorageProvider());
      
      // if not available, use Flash
      
      dojo.storage.manager.register("dojo.storage.browser.FlashStorageProvider",
      
      				new dojo.storage.browser.FlashStorageProvider());
      
      // if still not available, prefer IE method
      
      dojo.storage.manager.register("dojo.storage.browser.InternetExplorerStorageProvider",
      
      				new dojo.storage.browser.InternetExplorerStorageProvider());

    • Initialize the Storage Manager:
      
      

      // now that we are loaded and registered tell the storage manager to initialize
      
      // itself
      
      dojo.storage.manager.initialize();

    How is Dojo Storage Related to AMASS?

    AMASS, or the Ajax Massive Storage system, is an older, proof-of-concept version of Dojo.Storage written by the same author as Dojo Storage, Brad Neuberg. Dojo.Storage replaces AMASS. It is more full featured, better tested, and is supported across more browsers than AMASS. AMASS will no longer be supported.

    I Heard Dojo Storage Can Be Used For Offline Access. How?

    For offline, Julien Couvreur discovered the necessary HTTP headers. The core part of it is to use HTTP caching; your site must have the following kinds of HTTP headers on:

    • Etag
    • Last-Modified
    • Expires
    • Cache-Control

    Etags and Last-Modified are on by default on Apache 2.0; the rest have to be turned on in httpd.conf with mod_expires:
    
    

    LoadModule expires_module modules/mod_expires.so
    
    
    
    
    
      ExpiresActive On
    
      ExpiresDefault "access plus 1 month"
    
    

    The page is now in the browser cache after the first access. In IE and Firefox, the user has to go to File > Work Offline to work offline, and then can simply just navigate to the app's URL. In Safari, this is not necessary, and you can just go to the URL. I like to provide a link that can be dragged to the toolbar.
    
    
    
    The offline and dojo.storage work together, because whether you are offline or online you can access the same persistent storage, saving data while offline then syncing when online. Expect a dojo.offline and dojo.sync in the future that will provide abstractions for common operations like this. I'm looking for financial sponsors on this if you are interested so that I can finish and open source it (email me).
    
    

    The Dojo.Storage presentation goes into offline access as well:

    http://codinginparadise.org/weblog/2006/05/new-slides-from-ajax-experience-talk.html
    
    
    
    You should note that offline support is alpha, since there are issues with cache clearing.
    
    

    Who created and supports Dojo Storage?

    Dojo Storage was created by Brad Neuberg and is supported by him. Please refer all questions to the Dojo mailing lists before emailing Brad directly, as the email load is getting large. Brad might charge to answer your question, since he does Dojo Storage for free and makes money through open source consulting.

    How Can I Help With Dojo Storage?

    There are two big ways you can help Dojo Storage. The first is by creating new storage providers; a nice one would be for Internet Explorer's proprietary storage mechanism.
    
    
    
    The other big way to help is with squashing bugs, especially with the https bug that affects Dojo Storage, since I don't have a testing environment for it. See a list of all Dojo Storage bugs.
    
    
    
    Email Brad Neuberg if you are available to help with either of these.
    
    

    Why Did You Use Flash?

    Why Flash? Flash now has a greater installed base than Internet Explorer; Flash 6+ has a 97.1% penetration across the installed base of PCs, while IE 5, 6, and 7 have 64.7%. Flash is probably one of the most installed pieces of software on the planet.
    
    
    
    The Flash Storage Provider uses Flash as a hidden runtime to extend the browser, because of its ubiquity and cross-platform/cross-browser qualities. When browsers get native persistence support, your dojo.storage applications will continue to run since you write them against the generic dojo.storage APIs.

    What Is The Security of Dojo Storage?

    The Flash Storage Provider has exactly the same security characteristics as cookie based storage. Data is siloed by domain, so other domains can't access this data. The data is stored on the local disk, usually in the user's home directory.

    When working with Dojo Storage and it's Flash Storage Provider you should take exactly the same care you would take with cookies -- i.e., don't leave yourself open to Cross Site Scripting Attacks (XSS).
    
    

    What Is The Maximum Amount of Storage I Can Do?

    Using the Flash Storage Provider, this breaks down into three questions:

    1. How much total data can I store? If the user has given you permission, there should be no limit. I have successfully stored up to 10 megabytes, and just stopped there.
    2. How much data can I send over in one put()? For Internet Explorer on Windows and Firefox on all platforms, I can send over about 700K or 1 megabyte in a single put() and the performance is fine; however, for Safari on Mac, you can really only send over about 300 to 500K in one put() with acceptable performance. There will be a difference between saving a string object, which is much faster, than serializing a large JavaScript tree, which might get very slow if you are working with a set of JavaScript objects that are hundreds of K.
    3. What are the total number of records I can have? What if I have hundreds of thousands of keys? I have never tested this, so am not sure where it will fall over.

    For the WHAT WG provider, there is actually currently no upper-limit on the amount of storage a site can store! The WHAT WG spec recommends 5 megabytes, but Firefox implements no upper-limit. This is obviously a bug and will be fixed by Firefox.
    
    

    What About A Query Layer For Dojo Storage?

    In 2005 sent out a call for someone to layer over the very cool Trim Path Java Script SQL layer over AMASS (an earlier version of Dojo Storage), and someone took up the call and created a prototype linking them together. You should check it out; its very cool. No one has done this with Dojo Storage, but it shouldn't be hard. This might be very useful for offline Ajax apps.

    What About Syncing?

    This is a hard problem that is difficult to find general solutions for. I am looking for financial sponsorship to create and open source this. Email me if you are interested.

    I Don't Care About Dojo Storage or Dojo Flash - How Can I Turn Flash Off In My App?

    If you are using a 'kitchen sink' build of Dojo, you will get Dojo Storage and Dojo Flash included by default. This will cause the Dojo Flash material to be written into the page. If you don't want this to happen, you can disable all of Dojo Flash with the following djConfig variable:
    
    
    
    var djConfig = { disableFlashStorage: true; }
    
    

    How Does The Underlying Dojo Flash System Work?

    I'll briefly describe dojo.flash here, which is the layer that seperates out JS and Flash communication and which is used by the Flash Storage Provider. It might be useful for folks who are encontering any issues with the storage system due to Flash.
    
    
    
    Cross-browser, fast, reliable JS+Flash communication is really hard and ugly, so I encapsulated this portion out into it's own layer. The great thing is you don't have to know any of this externally; dojo.flash and dojo.storage work together to figure out your Flash capabilities, and use the appropriate mechanism internally. Zero hassle.
    
    
    
    Dojo.flash provides several major services:
    
    

    • dojo.flash.Info - Is Flash available + what version of Flash?
    • dojo.flash.Embed - Embeds Flash into page for Flash+JS communication
    • dojo.flash.Communicator - Provides uniform, fast, reliable, JS + Flash communication
    • dojo.flash.Install - Uniform installation and upgrading of Flash

    dojo.flash.Communicator was the real doozy to create; it was a pain in the butt, to put it lightly. This area of the system provides a method abstraction between Flash + JS. For example, JavaScript can call sayHello(), which is some Flash method, while Flash can execute DojoExternalInterface.call("dojo.storage.save", resultsHandler) to run some JS method.
    
    
    
    The heart is something called DojoExternalInterface, which is a backport of Flash 8's ExternalInterface to Flash 6. I didn't want to create this backport, but the complexity of handling all the internal tricks I was doing to make this stuff work required that I wrap this magic in some kind of maintainable API. DojoExternalInterface makes it possible to register Flash methods that can be called from JavaScript:
    
    

    DojoExternalInterface.initialize();
    
    DojoExternalInterface.addCallback("put",
    
                                     this, put);
    
    DojoExternalInterface.addCallback("get",
    
                                     this, get);
    
    DojoExternalInterface.addCallback("remove",
    
                                     this,
    
                                     remove);
    
    DojoExternalInterface.loaded();

    There are three ways to do Flash+JS communication:
    
    

    1. Using LiveConnect/ActiveX + fscommands - Flash 6
       
       Pro: Extremely fast, can send very large data, mature
       
       Con: Only works on IE and Firefox
    2. Using ExternalInterface - Flash 8
       
       Pro: Easy to use, Works on Safari
       
       Con: Unbelievably slow, performance degrades O(n^2), serious serialization bugs
    3. getURL/LocalConnection/New Flash object for each call - Flash 7
       
       Pro: Very cross platform
       
       Cons: Destroys history, serious data size limitations and performance issues

    Only methods 1 and 2 are acceptable for the Flash Storage Providers needs. It turns out that we use method 1 for IE and Firefox, and method 2, ExternalInterface, for Safari. I found workarounds to fix ExternalInterface's serious bugs, so that it works on Safari and is fast and serializes correctly, but these workarounds only work on Safari.
    
    
    
    The Flash 8 communication support for Safari took 3 months to figure out; pain in the behind. I won't go into detail on the method and workarounds, but here is a bit of info on the workarounds:
    
    

    • We chunk method call data into many different small calls through ExternalInterface,
      
      which makes performance linear rather than O(n^2), which is what the standard ExternalInterface's performance is.
    • I used a debugger to find hidden JS serialization methods used by the Flash plugin, which internally uses XML serialization and made all sorts of mistakes in terms of serializing characters - for example, it doesn't escape characters and also uses eval() so its slow as dirt. I found a way to bypass this internal serialization, do it all manually, and make the method call myself using an undocumented JS function.
    • I now double encode and decode all XML characters on both sides:
      
      & --> && This is very important for persisting XML, and I did lots of testing around this to catch every single problem character (including nulls, for example)

    With these workarounds, performance and reliability on Safari are great. Unfortunately, these workarounds only work on Safari, so we can't use ExternalInterface cross browser.
    
    
    
    Check out this testing page, which provides a lower level interface to dojo.storage; there are quick links on the left to fully save an entire book into the storage system, in this case Faust by Goethe at 250k courtesy of Project Gutenberg; there is also a quick link to save an example RSS XML feed into the storage, in this case an atom feed from my weblog. Try this on Safari; in the past it took minutes and froze the machine to save the book, now it takes seconds.
    
    
    
    I won't go into the Flash 6 communication necessary for IE and Firefox support; there were lots of fun different things that had to be figured out, like how to center the Flash dialog even if you are across many different kinds of HTML doctypes, browsers, and platforms, or isolating timing issues like the fact that the fscommand infrustructure on some versions of IE comes up after your Flash applet is already running.
    
    
    
    

    Can I Force a Certain Storage Provider to Be Used? Can I Disable Certain Storage Providers?

    Several djConfig variables exist to help you control what storage provider is used. The first is called forceStorageProvider, and will cause the Dojo Storage system to automatically use the given provider, even if the platform doesn't support it or a better one is available:
    
    
    
    

    djConfig = {forceStorageProvider: "dojo.storage.browser.WhatWGStorageProvider"};

    Two other djConfig flags exist: disableWhatWGStorage, which can be true or false and which will disable WHAT WG storage and fall back to using a different one; and disableFlashStorage, which can be true or false and which will disable using the Flash Storage Provider. The last flag is useful if you want to make sure that Flash is never used, for example if you don't want to have Flash touch your codebase.
    
    
    
    Note that the forceStorageProvider and disableWhatWGStorage/disableFlashStorage flags are mutually exclusive; you should choose to use EITHER forceStorageProvider or disableWhatWGStorage/disableFlashStorage. Things won't work correctly if you try to mix the 'force' and 'disable' flags.
    
    
    
    

    I'm Having Wierdness with the FileStorageProvider on Internet Explorer 7!

    
    
    On Internet Explorer 7, XMLHTTPRequest does not work on the local file system; this breaks Dojo's package loading, which uses XHR. To use the FileStorageProvider on IE 7 with your code, make sure to use a release build of dojo.js (i.e. the one that packs everything into a single dojo.js file), with preferably the 'storage', 'moxie', or 'kitchen_sink' profiles to make sure everything got packed into there and won't be loaded dynamically at runtime by XHR.
    
    
    
    

    When loading from file:// URLs, the user is prompted for permission to run secure code; what's up?

    
    
    On Firefox, the first time you load a page that uses Dojo Storage (and therefore the FileStorageProvider), you will be prompted on whether to give this page permission to access the local file system. If the user approves, you will not be asked again. On IE 6 and 7, you will also be prompted, but will be prompted each time the page is loaded. After you have given permission the page will work normally.
    
    
    
    

    Can I Use Dojo Storage With Other Non-Dojo, JavaScript Libraries?

    
    
    Dojo Storage can be used with other JavaScript libraries, such as TIBCO or Prototype. This FAQ question will provide pointers to sites where others have integrated Dojo Storage with their own libraries.
    
    Luke Birdeau has integrated Dojo Storage with the TIBCO GI library. See details on the TIBCO forum.
    
    
    
    

    About

    Author:	Brad Neuberg
    Version:	0.5
    Copyright:	Dojo Foundation, 2006
    Date:	2006/11/18

DOM functions

Dojo provides three modules with basic functions on the document tree:

• dojo.dom - basic functions such as moving / copying / deleting children. These functions are meant to apply equally to html or svg or vml nodes.
• dojo.style - functions relating to style attributes, such as calculating the size of elements, setting their opacity, etc.
• dojo.html - this module contains functions that only apply to HTML nodes, such as appending/removing classes

All of the functions above are written to either mask browser incompatibilites, or to provide functionality that wasn't provided within the standard javascript API.

All of the functions can take either a node or an id as their first argument.

Sizing

One particular set of functions to note are the sizing functions.

An HTML block element has the following format:

		+-------------------------+

		|  margin                 |

		| +---------------------+ |

		| |  border             | |

		| | +-----------------+ | |

		| | |  padding        | | |

		| | | +-------------+ | | |

		| | | |   content   | | | |

		| | | +-------------+ | | |

		| | +-|-------------|-+ | |

		| +-|-|-------------|-|-+ |

		+-|-|-|-------------|-|-|-+

		| | | |             | | | |

		| | | |<- content ->| | | |

		| |<------ inner ------>| |

		|<-------- outer -------->|

		+-------------------------+

There are three sizes associated with this element:

• outer - size of the outermost box, including margin, border, padding, and content
• inner - size of the second box, including border, padding, and content
• content - size of the innermost box

Depending on browser, and mode, asking for the width/height of an element will return different results; sometimes it will give you the content size and sometimes it will give you the inner size. Therefore, it's important to always use dojo's functions for getting/setting the size;

• dojo.style.getOuterWidth / dojo.style.getOuterHeight
• dojo.style.getInnerWidth / dojo.style.getInnerHeight

Classes

An HTML element can have multiple classes, such as:

Dojo provides functions to handle this class string as an ordered set, so you don't need to do string manipulations to add/remove items from the class list:

• dojo.html.addClass(node, className)
• dojo.html.prependClass(node, className)
• dojo.html.removeClass(node, className)
• dojo.html.replaceClass(node, className, oldClassName)

Drag and Drop (DnD)

When Microsoft and Netscape introduced Dynamic HTML (different versions of course), the drag-and-drop demos elicited much applause. You could put a shopping cart on the left and catalog items on the right, and the customer could drag items right to the shopping cart. Neato! It looked just like a client-server application.

Because of the incompabilities between DHMTL, writing a cross-platform drag-and-drop application was difficult. Dojo makes it easy by layering an easy-to-use API over the top.

To drag and drop, you need three things:

• Something to drag, called a DragSource. In Dojo, any HTML element can be one.
• Some place to drop, called a DropTarget. This too can be any HTML element.
• Something to do when the item is dropped

<script type="text/javascript' src="/path/to/dojo.js"></script>
<script type='text/javascript">
   dojo.require("dojo.dnd.*");
   dojo.require("dojo.event.*");

   function initKtichen() {
      // "pan" matches the id for the <div> tag with id="pan" below
      // The [] around "dest" are required, for reasons we'll see later
      new dojo.dnd.HtmlDropTarget(dojo.byId("pan"), ["dest"]);
      new dojo.dnd.HtmlDropTarget(dojo.byId("pot"), ["dest"]);
      new dojo.dnd.HtmlDropTarget(dojo.byId("oven"), ["dest"]);

      // "dest" matches the "dest" in the DropTarget's above.
      new dojo.dnd.HtmlDragSource(dojo.byId("egg"), "dest");      
      new dojo.dnd.HtmlDragSource(dojo.byId("leg"), "dest");
   }

   dojo.addOnLoad("initKitchen");
</script>
</head>

<body>
   <H1>Food</H1>
   <div id="egg">Egg</div>
   <div id="leg">ChickenLeg</div>

   <H1>Destinations</H1>
   // This is the pan, the first drop target
   <div id="pan" style="border:3px soid black;width:200px">
      Frying Pan
   </div>
   <div id="pot" style="border:3px solid black;width:200px">
      Pot of Boiling Water
   </div>
   <div id="oven" style="border:3px soid black;width:200px">
      Oven
   </div>
   <br />

    function initKitchen(){

        dojo.declare("dojo.book.dnd.DestDropTarget",dojo.dnd.HtmlDropTarget,{

            onDragOver: function(e) {

                // domNode is the drop target we're over

                this.domNode.style.borderColor = "red";

                dojo.dnd.HtmlDropTarget.prototype.onDragOver.apply(this, arguments);

            },

            onDragOut: function(e) {

                // this.domNode is the drop target we're leaving

                this.domNode.style.borderColor = "black";

                dojo.dnd.HtmlDropTarget.prototype.onDragOut.apply(this, arguments);

            }

        });

        new dojo.book.dnd.DestDropTarget(dojo.byId("pan"), ["dest"]);

        new dojo.book.dnd.DestDropTarget(dojo.byId("pot"), ["dest"]);

        new dojo.book.dnd.DestDropTarget(dojo.byId("oven"), ["dest"]);



        new dojo.dnd.HtmlDragSource(dojo.byId("egg"), "dest");

        new dojo.dnd.HtmlDragSource(dojo.byId("leg"), "dest");

    }

   function initKitchen(){
        dojo.declare(dojo.book.dnd.DestDropTarget",dojo.dnd.HtmlDropTarget,{
            onDrop: function(e) {
                alert('Ready to cook!');
                // Call the superclass method to do the actual dropping
                dojo.dnd.HtmlDropTarget.prototype.onDrop.apply(this, arguments);
            }
        });
        new dojo.book.dnd.DestDropTarget(dojo.byId("pan"), ["dest");
        new dojo.book.dnd.DestDropTarget(dojo.byId("pot"), ["dest"]);
        new dojo.book.dnd.DestDropTarget(dojo.byId("oven", ["dest"]);

        new dojo.dnd.HtmlDragSource(dojo.byId("egg"), "dest");
        new dojo.dnd.HtmlDragSource(dojo.byId("leg"), "dest");
    }

    function initKitchen(){

        // A pan can be a drop target for any drag source of type topOfStoveOnly or

        // allCookingOptions.

        new dojo.dnd.HtmlDropTarget(dojo.byId("pan"), ["allCookingOptions","topOfStoveOnly"]);

        new dojo.dnd.HtmlDropTarget(dojo.byId("pot"), ["allCookingOptions","topOfStoveOnly"]);

        new dojo.dnd.HtmlDropTarget(dojo.byId("oven"), ["allCookingOptions"]);



        // An egg can only be dropped on a target of type topOfStoveOnly

        new dojo.dnd.HtmlDragSource(dojo.byId("egg"), "topOfStoveOnly");

        new dojo.dnd.HtmlDragSource(dojo.byId("leg"), "allCookingOptions");

    }

  function initKitchen(){
     ... // Construct drop sources

     // Keep the egg from leaving the boundaries of the object kitchenDiv
     var eggSource = new dojo.dnd.HtmlDragSource(dojo.byId("egg"), "dest");
     eggSource.constrainTo("kitchenDiv");

     // Do the same with the chicken leg
     legSource = new dojo.dnd.HtmlDragSource(dojo.byId("leg"), "dest");
     legSource.constrainTo("kitchenDiv");
  }

...
<H1>Destinations </H1>

<div id="kitchenDiv">
   <div id="pan" ...

<html><head>
<script type="text/javascript" src="/path/to/dojo.js"></script>
<script type="text/javascript">
dojo.require("dojo.dnd.*");

   dojo.require("dojo.dnd.event.*");

   function initList() {
      // Loop through all li elements of list, and make them drop targets
      var dl = dojo.byId("listToRearrange");
      var lis = dl.getElementsByTagName("li");
      for (var i=0; i<lis.length; i++)
         new dojo.dnd.HtmlDragSource(lis[i], "dest");

      new dojo.dnd.HtmlDropTarget(dl,"dest");
   }

   dojo.addOnLoad(initList);
</script>
</head>
<body>
<H1>Jimi Hendrix Albums</H1>
<p>Arrange in order of your own preference.</p>
<ul id="listToRearrange">
   <li>Electric Ladyland</li>
   <li>Are You Experienced?</li>
   <li>Axis, Bold as Love</li>
</ul>
</body>
</html>