Login Register

LazyLoadTree

[moved, I'm guessing this is a critique/request of the Dijit tree, since there is no DojoX tree]
A few quick suggestions on the tree control.

1) Remove the stub syntax, discover nodes when the expand is clicked. I have 100 nodes with 100 nodes under each and have to load 10,000 instead of 100 with this syntax. It isn't too efficient for a lazy load to load this deep into the tree.
2) Don't require each node to be named uniquely throughout the entire forest. Like a folder structure on a file system, the name is unique at the node member level it exists, not the entire forest. I currently have to have each node named with the entire path in order to allow the tree to build.
3) Make is easier to pass the lazyload request node path from the root to the current selected node to a web server so it can filter to the path for the next set of nodes. I am building my json file dynamically in a J2EE Struts site and had to hack things up a bit.

Probably a faster easier ways to do these so please comment :)

Otherwise this is a fantastic control!

I've done a bit of work on

I've done a bit of work on my own lazy load tree and plan to create some dijit tutorials when I get back from my vacation in about 3 weeks ;) But maybe I can help you out a bit in the mean-time:

1) I've implemented a lazy load tree that loads the children on node expansion. I simply extended the current dijit.Tree like so:

dojo.provide("my.dijit.Tree");
dojo.require("dijit.Tree");

dojo.declare("my.dijit.Tree", dijit.Tree, {
        getItemChildren: function(/*dojo.data.Item*/ parentItem, /*function(items)*/ onComplete){
      console.log("my.dijit.Tree");
                // summary
                //           User overridable function that return array of child items of given parent item,
                //            or if parentItem==null then return top items in tree
    console.debug("my.dijit.Tree.getItemChildren: parent: " + (parentItem == null ? "(null)" : parentItem.path));
                var store = this.store;
                if(parentItem == null){
                        // get top level nodes
                        store.fetch({ query: this.query, onComplete: onComplete});
                }else{

      // Check to see if the parent item is loaded.  If it's not,
      // loading it will load stubs for the children
      if (!store.isItemLoaded(parentItem)) {
        function onParent(item) {
          onComplete(item.children);
        }
        store.loadItem({item: parentItem, onItem: onParent});
      } else {
        onComplete(parentItem.children);
      }
                }
        }
});

I may be biased, but I like my lazy load tree better than the current example ;) The store code is pretty trivial when you see loadItemChildren. One thing I like about this implementation is that you can decouple loading the tree structure from loading the data for a node, i.e. loading a node does not imply loading the children for that node. You might use this technique when clicking the label for a tree means "show me the data associated for this node" and expanding means "show the the structure below this node." I typically like this decoupling when I model my data into a hierarchical structure.

2) The name doesn't have to be unique. There is an identifier field you can set and *that* has to be unique, but, by default, (I think) the dojo.data.ItemFileReadStore uses name as the identifier. Try adding a field called identifier to your store's data, something like this:

{
  identifier: 'id',
  label: 'name',
  items: [
    {name:'1', id:1, children:[{name: '3', id:2}]},
    {name:'2', id:3, children:[{name: '3', id:4}]}
  ]
}

3) Sorry, I'm not sure I follow the problem here, an example might help.

Thanks for the quick

Thanks for the quick response!

1) I tried out the code and it works well. However, I actually don't have much information on the node itself, just lots of nodes. My concern is the definition of the node having to know all the nodes and all the nodes children at once for this level and one level down. When one level down has may nodes per node that can get expensive. Also, it currently calls my service for each node currently expanded in order to get a list of its children one level deep. Please correct me if i misunderstood as this framework is only 2 days old on me :)

2) Thanks for the clarification, that fixed this problem. sweet!

3) My issue here was simply that I updated the code directly for the LasyLoadJSIStore.js in order to get parms sent to a url instead of a folder structure. It wasn't too hard to find or change but a sample of a clean way to do this would be nice. What I did currently does not pass all the parent nodes correctly but I'll fix that next once I incorporate #2 above. Here is what I did:

//sample
                var itemName = this.getValue(item, "name");
                var parent   = this.getValue(item, "parent");
                var dataUrl  = "/mypath/mktxml.do?open=1"; //will not pick up the path?
                if (parent){
                    dataUrl += ("&parent=" + parent);
                }
                //For this store, all child input data is loaded from a url that ends with data.json
                //dataUrl += itemName + "/data.json";
                dataUrl += "&node=" + itemName;
                //alert(dataUrl);

Response to lazy-loading tree example

1) I tried out the code and it works well. However, I actually don't have much information on the node itself, just lots of nodes. My concern is the definition of the node having to know all the nodes and all the nodes children at once for this level and one level down. When one level down has may nodes per node that can get expensive. Also, it currently calls my service for each node currently expanded in order to get a list of its children one level deep. Please correct me if i misunderstood as this framework is only 2 days old on me :)

When you say you tried out the code, are you talking about the code I posted above? If so, this part:

the definition of the node having to know all the nodes and all the nodes children at once for this level and one level down.

Isn't true and that was the point of using that implementation, so you wouldn't have to have the children one-level down as you put it. You only get the children when the parent expands. Maybe just seeing the getItemChildren isn't enough, maybe you do need to see the store code as well ;) I'll try and get my tutorial up tonight and I'll post a link back for you.

2) Thanks for the clarification, that fixed this problem. sweet!

Excellent, glad to be of help!

3) My issue here was simply that I updated the code directly for the LasyLoadJSIStore.js in order to get parms sent to a url instead of a folder structure.

Yeah, I'll agree with you there, the example is really not very intuitive. I slogged through it, but for a first introduction to lazy-loading trees, it's not the best. As I said earlier, I'll see if I can't get my tutorial posted, I have data lazy-loading from a servlet right now, but the code is really rough and I need to take time to properly document the tutorial. And, with all that going on, I'm going on vacation next week for 2.5 weeks, so time is tight right now ;) I'll see what I can do and post a link here when I'm finished.

thansk

I'll look for that later tonight if you can get to it. I really do appreciate it (on vacation and all!).

Of interest, i thought it was working but in fact the overload you provided isn't getting called. Which would explain why i couldn't pull the children out of the geography json files so i assumed that had delayed loading the details only. I have tried overloading mayHaveChildren and not able to get that working either. I must be missing something fundamental (its late and I have been coding a while)... here is my div tag:

<div dojoType="my.dijit.Tree" id=tree store="continentStore" query="{type:'continent'}" labelAttr="name" typeAttr="type"></div>

I put your code at the top in the script section of the demo_lazyload.html where this appears:

dojo.require("dojo.parser");
dojo.require("dojox.data.demos.stores.LazyLoadJSIStore");

dojo.provide("my.dijit.Tree");
dojo.require("dijit.Tree");

dojo.declare("my.dijit.Tree", dijit.Tree, {
               getItemChildren: function(/*dojo.data.Item*/ parentItem, /*function(items)*/ onComplete){
...etc

I tried putting in alerts or console.debugs but neither is getting called. Tried IE6 and Firefox2. Strange...

getItemChildren

I have gone in and changed it to add onClick to verify an event is overloaded. This event is firing correctly. Next step i went through the object model using firebug in Firefox and could not find the getItemChildren in the tree interface or anywhere for that matter (as loaded on the lazyload demo page). I assume this would explain why it never gets called. I see iterators for children (getChildren) but nothing else close. I do see the docs show this as overridable but cannot seem to get anything but the onClick event to fire. My build date inside the downloaded gz is 8/20/07. I did a search through the dojo directory structure and didn't find this defined anywhere except in my code. I think I am officially stuck.

btw: i also moved the code to a .js module and referenced it that way as well, same behavior.

Any ideas? do i have the wrong release perhaps?

Thanks so much for all the support so far.

Try this out:

Try this out: http://samurai.webasap.net/cching/DojoLazyTree.zip

It's an example of loading on expand. To try it out, unzip the file and put the current 0.9 dojo trunk in the dojo-0.9 directory, so that you have this directory structure:

DojoLazyTree
\-- dojo-0.9
     \-- dijit
     \-- dojo
     \-- dojox
     \-- sample
     \-- util

NOTE: the 'sample' directory contains the extended Tree and TreeStore code.

It still borrows code heavily from LazyLoadJSIStore, but I've modified the loadItem method to load the children in. This code is still really rough, but it does show how to load the tree structure on expand. I plan to enhance the TreeStore so that it differentiates between loading the structure and loading the data for the nodes. Eventually, I want an expand to load the structure and a click on the label to load the data for a node.

To understand how it works, take a look at the data directory. It contains a set of .json files that contain the children for every node in the tree. The uniqueness comes from the path from the root to a node and this is reflected by using the attribute called 'path' as the store identifier. If you look at TreeStore.loadItem, you'll see that we use the path attribute as the url for the json file that contains that node's children. I'm still using the type:'stub' attribute to indicate that the node's children haven't yet been loaded.

I will be enhancing this code and putting together a blog/tutorial post. I started doing it last night, but it ended up being bigger than I thought it would be ;-P

Let me know if you have any problems or questions.

update

I have it setup and working with it now. Currently i had to rename A.json to A.stub.json and it expands to:

A
-A.stub
---------A
---------B
---------C
B
C

If I don't rename it it will not read the nodes. I am digging to understand and to see if I can fix it (seems like it should be simple) but if you know a simple place this might be missing something feel free to chime in :).

I'll post again once I find what is missing on my side unless I see you post first. Thanks again! This is healthy leap forward from the demo.

I didn't have a problem like

I didn't have a problem like that, so I'm not quite sure what to think. I am realizing that I never do remove the stub child, so that makes me wonder how that child isn't showing up in the tree. Let me know if you figure that out, I'll continue to look at it. Thanks for the feedback, I appreciate it, it should help me get something that is solid and right ;)

EDIT: Also, if you could console.log() the dataUrl in loadItem, that might shed some light on things.

logged results

the dataUrl was data/A.stub.json

I am currently looking for a way to get the parent node of the current child stub and then replace the current child with the new array. It is currently putting all the new entries under stub which draws on my version.

i also put a console debug under the load event to see when the call to the web service would be invoked. currently when it draws, it calls all the node.json files again. could be a byproduct of my version but still digging.

thanks

Browser?

Could this be a brower issue? What browser/version are you using? I tested with Firefox 2.0.0.7.

It shouldn't need to reload the .json files. There is a line in loadItem that deletes the type node which is the indicator that something was a stub and that it would need to be reloaded.

You say something about a web service, did you move this into a server? This was designed to just work off the local filesystem, did you try just loading it in with a URL like file:///home/cching/DojoLazyTree/LazyTree.html?

Still running from the

Still running from the filesystem but tracking how it does callbacks to make sure it does not do extra calls to what will become the web service. I am using Firefox 2.0.0.7 as well.

What i am thinking is happening is that you delete the type indicator but the object is still in the _arrayOfAllItems store and the children are populating under it instead of on the original object. what my store looks like from Firebug:
_arrayOfAllItems
-0 this is A (contains one child)
-1 this is A.stub (contains 3 children when you expand the node)
-2 this is B
-3 this is B.stub
-4 etc..
-5

Are you using dojo trunk?

I guess I just have to ask at this point, are you using the current dojo 0.9 trunk? I see the same thing, the stubs are still in the _arrayOfAllItems data member, but that shouldn't be a problem as the extended store empties the node's children array, you can see that in loadItem where I do item.children = [];

I think it's ok if the stub items are in the store as long as they aren't associated with any parent. Ideally, of course, I want to clean that up, but that means starting from scratch writing my own store and that just isn't something I've had time to do yet. I intend to do it, just haven't yet ;) With your own store implementation, you could simply remove the stubs. With the current underlying implementation, where the items are stored in an array, I'm not sure removing an item is all that good performance-wise. I'd probably try changing from an array to a hash using the store specified identifier data member as the key.

If you aren't using the current trunk (I have tested with an update in the last couple of hours), could you do that and see if you still have the same problem?

fixed

http://archive.dojotoolkit.org/nightly/

I assume the path above is what you meant by get the latest :)

I downloaded and that fixed it! the current release of 0.9.0 (august 07) release didn't work but this [nightly of 9/25 version] works perfectly once I removed all my changes. I also now have a root node for my forest too. I am cooking and performance will be smokin fast compared with the original version.

Thank you very much for your help in this!! Even on vacation you solved my problem.

Ah, ok, good! I've been

Ah, ok, good! I've been using the trunk straight out of the subversion repository, but I'm glad to hear that nightly build worked for you :D

One thing I should note, the store is kind of messed up in the current implementation. We're sort of subverting where the data is held. ItemFileReadStore wants all the data items in _arrayOfAllItems, but when we loadItem in our extended store, we don't put the child items in _arrayOfAllItems. And, note, you're only seeing the top-level nodes and their children stubs in _arrayOfAllItems. The reason for this latter behavior is that ItemFileReadStore uses the simpleFetch mixin to do the initial fetch and it loads those stubs into _arrayOfAllItems whereas our loadItem doesn't do that. The original LazyLoadJSIStore doesn't have this problem, it updates the item in the store when it is loaded. You could update the TreeStore to do what LazyLoadJSIStore does to keep it consistent and to make sure you can access the data uniformly if you update it (even though this is supposed to be a read store :P ).

Anyway, glad you got it working. I'll post a link to my blog/tutorial when I get it finished. Note, I'm not on vacation just yet, that begins on Monday, so don't expect anything for about 3 weeks still. I do have ideas for three dojo tutorials so far, so I plan on being busy when I get back!

problem with IE 6

Hi,

I tried sample project DojoLazyTree .Its working fine with Firefox ver 2.0.0.7 ....But doesnt show up tree in IE ver 6.0 sp 1...What might be the problem. ?

IE6

Yes I have noticed this as well. I have been watching the following in hopes it will be fixed soon.

http://trac.dojotoolkit.org/ticket/4378

I have a project deliverable due at the end of this week. I'll have to switch to another UI framework for my lazy trees if the issue persists as I have to deliver on IE6 :(

Everything else has been fantastic though. Nice fast tools otherwise.

Found my fix

Turns out my json file was the issue. When I used a structure from lazy programming practices like:

{my:'stuff', value2:'more',}

the trailing comma works in Firefox but IE does not read the json object correctly. Fixing this corrected my problem. No trailing comma's, needs to be cleanly formatted from top to bottom for IE.
{my:'stuff', value2:'more'}

Hope this helps someone.

Thanks,
Tim

dojo.data limitation

Hi, I'm the Tree owner now, so let me address your original point about having to load 10,000 rows from the database rather than just 100:

1) Remove the stub syntax, discover nodes when the expand is clicked. I have 100 nodes with 100 nodes under each and have to load 10,000 instead of 100 with this syntax. It isn't too efficient for a lazy load to load this deep into the tree.

This is actually an issue with dojo.data, in that an item needs to contain (at the least) a stub for each of it's children, for example:

{id: 1, label: "my tree node", children: [ _stub1, _stub2, _stub3 ] }

I added methods to tree that you can override in order to avoid this problem. They are mayHaveChildren and getItemChildren. If you change your data store to look more relational, like:

{id: 1, label: "my tree node",  parent: null}
{id: 2, label: "my child node #1",  parent: 1}
{id: 3, label: "my child node #2",  parent: 1}

and then override those functions, you should get better performance. getItemChildren would look something like:

getItemChildren: function(/*dojo.data.Item*/ parentItem, /*function(items)*/ onComplete){
  ...
  store.fetch({ query: { parent: store.getIdentity(parentItem) }, onComplete: onComplete});
  ...
}

See the i18n demo for an example.

But I guess DnD and store updates won't be handled correctly, without further tweaking.

cching added a suggested override to getItemChildren(), but it's dangerous in that it assumes that even when store.isItemLoaded(item)==false, you can still access all the attributes of that item, except for the children attribute. Reading the "spec" of dojo.data.api.Read I don't see any reason that we can assume that.

(Although it's not written explicit in the summary of dijit.Tree.getItemChildren(), the items getItemChildren() returns must be loaded, at least enough to access the id and the label.)

dojo.data limitations

Nice to see the title of your post. From my point of view dojo.data API is the main obstacle to create a really nice tree widget. It is too abstract for the tree widget. It provides loosely coupled items that are loaded and tested individually. This API seems to be good for relational data. In that case the item attributes can be used efficiently to express realtionships between the items and no additional semantics is required. However the tree widget requires data providers that clearly express a strong parent-children relation and allow to load children of the node collectively. It is impossible with dojo.data.Read.
But data loading is a relatively simple task. I wonder how to express moving of a given node to the destination node using dojo.data.Write in a simple and efficient way. That is a challenge!
Hopefully that problem was solved in the past. Dojo 0.4.x TreeControllerV3 uses a simple interface to provide and manipulate hierarchical data. I hope earlier or later something similar in logical terms will be created. Thank you very much for isolating the data store related functions of dijit.Tree. Overriding them allows to enjoy the tree widget even slightly breaking dojo.data API.

I just wanna toss my 2 cents in on the original post as well...

in regards to unique names... I have a tree that has many children with the same name/label as many others throughout the tree... the thing is you have to have a unique identifier so... what I did is when I have my datasource outputting JSON for the tree... I build the identifier attribute based on parent nodes... so...
root id = a
child 1 = b, id = a:b
grandchild 1 = d, id = a:b:d
bhild 2 = c, id = a:c

etc etc...
This isnt something that needs changing... you just need to leverage the system correctly.

-Karl

Please post the sample again

HI
I am trying to do make something similar, in which i have a tree with say 10,000 nodes and i want to load the 1st level first and then as the user clicks on a node the children for the node should be loaded.

i am generating JSON for the tree from a servlet. using Maven JSON-lib

can u please post your sample once again also help me build this tree, i am a dojo newbie so dont have much insights into dojo, am using v0.9 and IE 6.

dino