Login Register

CSV export for Grid data

Hi,
I think it has been discussed before, but I was not able to find a definite answer.

Is there a way to export the contents in a dojo grid to a CSV file (or even a MS Excel spreadsheet). I believe conversion from the Grid format to the CSV format should not be difficult, and having a link on top-right of every grid which says "download this is CSV" should not be technically very challenging. Now the question is, has anyone done that yet? If not, is there a plan to implement it down the road?

thanks
Nilesh

I agree with you. I also

I agree with you. I also need such a functionality. It's very useful for professional applications and should be, IMHO, a part of the Grid object. (Don't forget column titles).
TIA
JL

Exporting grid data requires trip to server

It's not possible to save a file directly from the browser to the user's computer. Saving the grid's content as a file requires sending a request to the server and receiving the file as a response with http response headers: content-disposition attachment. So, 'save to excel' is not possible with Dojo alone - you need a server component too. This server component can then format the data any way you want.

CSV file opens nicely in excel, but you can try also the following: return a simple html-page with file extension xls, containing the data in a html table. See: How to Export Data to Microsoft Excel

need to understand the data store system...

Hey Nilesh, JL:

What you want to do sounds like a good idea as an add-on of some sort.

So it's worth noting that there isn't one "right" format for incoming data to the grid. Instead, the grid uses a series of adapters and conventions to allow it to display data from many different kinds of data stores. For instance, here's a demo that pulls data from a CSV file and from a JSON structure for display:

http://download.dojotoolkit.org/release-1.1.0b3/dojo-release-1.1.0b3/doj...

What that's showing you is that what we see in the grid is a *view* onto these data files:

http://download.dojotoolkit.org/release-1.1.0b3/dojo-release-1.1.0b3/doj...
http://download.dojotoolkit.org/release-1.1.0b3/dojo-release-1.1.0b3/dij...

The grid itself *doesn't even know what data it could have*. It asks the backing stores for data in a lazy way, which is what allows the Dojo grid to handle such incredibly large volumes of data where other grids would just fall over.

The example I linked to above uses format-specific data providers and *they* are what do the adaptation between a potentially hierarchical (or even relation-free) data set and something that can be displayed in a grid.

That's the long way of saying that what you want to do is probably possible, but perhaps not in the way that you're thinking of it being done. Instead of focuing on the grid giving you a CSV export, perhaps we should focus on how to inspect a grid for it's column configuration and then query the grid's data store for a CSV conversion of whatever sort of query it may currently be displaying the results of in the grid.

There are also questions about what to do WRT complex cell formatting and computed/formatted values in cells which I'd love your thoughts on.

--
Project Lead, The Dojo Toolkit
President, The Dojo Foundation

Thanks for your comments. I

Thanks for your comments. I understand what you are saying. Let me re-phrase my needs:

- There should be a way that data displayed by the grid is converted to CSV format. This needs to be independent of store used for constructing the grid. The conversion should obey the query supplied when constructing the dojox.grid.data.DojoData store.

- There should be an option to display a link on top-right corner of the grid to show "save as CSV". I understand that there has to be some server-end program that initiates the file download. One inefficient way to do that is: the grid constructs the string with CSV data, sends it to server via xhrPost along with a filename, and a generic PHP file on server simply reads that data and constructs the output file with Content-type set to text/csv and Content-disposition set to the filename. (There is security issue here that server should not listen to anyone else except the xhrPost by dojo grid).

I know the Grid development team has a lot of things they are working on. This is just my wishlist which might be useful to other using the grid. Please let me know if there is a easy way to do this now. If not, are these features planned for future.

Possible solutions

You may run into security issues if you try to save the file from Javascript without any user interaction - strictly speaking, a web page should never be allowed direct access to the client filesystem, whether read or write. That's why, for example, you cannot programmatically set the filename to be uploaded in an file upload box. There are various ways around this, I think, but in general it may not be a good idea to be seen trying to circumvent client security. (However, if this for a purely intranet application running on a trusted server, and your clients are using IE on Windows, you could look at trying to use Microsoft's FileSystemObject object - it has OpenTextFile and CreateTextFile methods which let you read/write/append text files.)

That said there are (at least) two ways of handling it where the user gets the choice of whether to save the file or not - either as a standard file download from the server-side, or, as you suggest, by automating Excel. The server-side download is not really a Dojo solution, except for programming the button to make an appropriate request. Automating Excel is obviously dependent on the client running on a PC with Excel installed, may also be dependent on using IE as the browser (don't have the facilities to test that), and may be affected by security settings. I am using code similar to the following in my onClick handler:

if (resultsGrid.model && resultsGrid.model.count > 0) {
	var xlApp = new ActiveXObject("Excel.Application");
	xlApp.SheetsInNewWorkbook = 1;
	var wb = xlApp.Workbooks.Add();
	var cellDefs = resultsView.cells[0];
        var k = cellDefs.length;
        // setting column names - omit if you do not need column headers in the export
	for (var j = 0; j < k; j++) {
	        wb.ActiveSheet.Cells(1, j+1).Value = cellDefs[j].name;
	}
	for (var i = 0; i < resultsGrid.model.count; i++) {
		for (var j = 0; j < k; j++) {
			wb.ActiveSheet.Cells(i+2, j+1).Value = resultsGrid.model.getDatum(i, cellDefs[j].field);
		}
	}
	xlApp.Visible = true;
} else {
	window.alert("Please execute a query before trying to export to Excel");
}

This assumes a Grid with jsId resultsGrid and the layout using a single view defined in the resultsView object. It populates a new spreadsheet, and makes it visible to the user, leaving it up to them to save the file and close Excel. It would also be possible to automate saving (or at least popping up the "Save as" dialog) and even closing Excel (though you would need to watch out for pre-existing open spreadsheets).

Simon

Hi Simon, Thanks for your

Hi Simon,

Thanks for your response. I see your point. Also, our users come from all over the internet (it is not the safe intranet) and use Windows, Mac, Linux. By Excel, I meant, one of MS Excel, OpenOffice, etc..

Anyways, please see my response to alex as to what I was looking for.

thanks
Nilesh

Hi Nilesh I should perhaps

Hi Nilesh

I should perhaps have mentioned that I am using a model that is fully populated (based on loading my JSON data into a dojox.grid.data.Objects model object). As Alex points out, it is not always the case that the model contains all the data - for example, using a store-based implementation, you would probably need to make use of the store's fetch() method to get an item count (and possibly its getValue() or getValues() methods to actually get the data). Also, my data is all string or numeric - again as Alex points out, if you are applying formatting to transform the data for display, there would be additional work needed for the export.

For the generic solution you are looking for, I think you are more less constrained to the server-side download, but that would I suspect be difficult to implement as a Dojo functionality, since Dojo makes no assumptions about the backend.

Simon

Generic Grid layout/structure

I have another question on related note. How can I display arbitrary data using a Grid. I want a piece of code where I can plug in a URL which is location of a CSV file, and Dojo Grid displays it (like embedded spreadsheet in the browser). The problem is that I don't know the number of rows and columns when creating the grid (that information is present in the CSV file). So I dont know what to set the value of "structure" attribute for such a generic grid.

Basically, if I can get this to working, I will partly solve the "download to CSV task". The backend server will only produce CSV formatted-data. The frontend JSP file will take a CSV url as input, will display data using Grid, and will provide link to download to filesystem.

thanks
Nilesh

Dynamic construction of grid structure

It is possible to construct the grid layout dynamically - I am doing it to allow the user to choose which columns they want to display. My case is possibly a little simpler than yours - I have a master layout containing all the columns, then construct the dynamic layout using dojo.filter:

var cols = getSelectedColumns();
var filteredView = {
	cells: [ dojo.filter(masterView.cells[0], function (item) { return dojo.indexOf(cols, item.field) > -1; })]
		};
var filteredLayout = [ filteredView ];
myGrid.setStructure(filteredLayout);

where masterView is my view definition containing all the columns in a single row and getSelectedColumns() is a function that returns the columns selected by the user for display.

For your purposes, you would need to write your own xhrGet or xhrPost function with the csv file as url, with handleAs set to text, and in the load handler process the string into a multi-dimensional array (there might be existing functionality to do this - you'd have to have a look through Dojo's utility, string and array handling functions - if there isn't it might be a good enhancement to request). I assume you would probably have the column headers in the first line of the csv file - if so, you should be able to construct the cells[[]] array for the layout from the first row of your multidimensional array, using something like dojo.foreach. You would construct a dojox.grid.data.Table model from the remaining actual data rows, set the structure and the model to the grid, and then refresh the grid.

(Edit) After looking at Alex's examples above, I realise that the dojox.data.CsvStore store does much of the work for you - it does require that the first line of the store contain the column headers, and then loads the rest of the file into the store based on that. Its _attributes property contains those headers and should allow you to construct the structure. Alternatively, look at its _getArrayOfArraysFromCsvFileContents function to see how to generate the multidimensional array yourself.

Thanks. One last wish:

Thanks. One last wish: copy-paste data from grid. From a simple HTML table, the user can easily copy-paste couple of interesting rows worth of data to spreadsheet. This however is not possible for grid (copy to clipboard is not available). Copy-paste to personal word processor is a very useful thing to have, especially when browsing through tons of statistical data.

I know, not all data-formats supported by Grid are exportable to simple row-column based table format. But in my opinion this will be a great functionality to have even if its not always available.

My method of allowing users to transfer data from grid to file:

var gridDataAsCSV = function() {
        var exportRows = [];
        var exportCells = [];
       
        grid.structure[1].cells[0].forEach(function(curCell){
                        exportCells.push(curCell.name);
                });
        exportRows.push(exportCells.join(','));
        exportCells = [];
       
        grid.model.data.forEach(function(curRw){
                grid.structure[1].cells[0].forEach(function(curCell){
                        exportCells.push(curRw[curCell.field]);
                });
                exportRows.push(exportCells.join(','));
                exportCells = [];
        });
       
        return exportRows.join('\n');
    }

Granted, this is from the model itself, so you loose formatting, sort order, etc. From here you could open a window and write the return value to the document so users can "save as". Or, what I do is set the value of a hidden input and submit a form and the server returns a csv file.