What W3C standard file stream operations are on the horizon?

I would really like to embrace W3C standard technologies rather than plugins, but as a rich internet application developer I often work with various non standard proprietary files formats.

Are there any W3C standards on the horizon to get excited about? For instance, when will we be able to read and write to the local file system, as well as navigate arbitrary bytearrays (obviously under the necessary security restrictions)?

What time frame would be realistic to expect cross browser capability with this functionality?

Posted Apr 11, 2010, 7:49 PM UTC

010
dwdwdw2•

As for byte arrays, you can do that already with Canvas, and XmlHttpRequest (if you're willing to accept a few hacks ;)).

The coolest W3 standard I've seen of late is the Oracle-sponsored WebSimpleDB, which I really hope makes it to wide acceptance.

As for writing to the filesystem, why do you need to? There is some stuff on the horizon for reading locally stored files, but IIRC it's an extension of HTML forms, and not some general use tool for stashing data.

2
DemonmongerOP•

Lets say the user is viewing a dataset and they perform filtering and manipulation on the client side. I want to allow the user to download that modified dataset to csv without having to do a round trip to the server to create the file based off the modified data and then have the user download it back. Just seems like a waste of bandwidth and server resources.

1
dwdwdw2•

That's a good reason. :)

Seems right now the only way to accomplish this is with a drop of hidden Flash.

Edit: in a supported browser, you might be able to fake it with data:// URLs.

1
hofo•

Exactly the issue I'm working on now. The solution I'm coding currently is to store the data in the localStorage space and then have JS code that creates a CSV string in a div that will only show when the user wants to export the data. It will require them to cut and paste into Excel but the requirements I'm working with say no web server and no external libraries that can't be loaded from a content-delivery network (like Jquery).

1
hofo•

The WebSimpleDB looks a lot like the WebStorage stuff in HTML5 (http://dev.w3.org/html5/webstorage/). What's the diff? WebKit supports all of this now and as soon as they get over the "I don't want SQL in my Javascript" I suspect Firefox will as well. FF does do the other two parts now I think, localStorage and sessionStorage.

1
dwdwdw2•

I assume by WebStorage you mean the SQL support provided by e.g. Gears & co. WebStorage is the API behind "window.clientStorage", etc.

The major problem with "SQL in my JavaScript" is right now, and apparently despite exhaustive efforts over a few years, that statement is actually "SQLite in my JavaScript". As I understand it, this is a major obstacle to standardization, since the effort involved in creating even a reference second implementation is very high.

If you simply throw in the towel and say "ANSI SQL", well that has problems too, since all existing implementations use SQLite, which doesn't feel like what most people associate with "ANSI SQL", e.g. types vary per field in a single result column, etc.

If you throw in the towel and say SQLite is the standard, then that too has problems. Do you expect Oracle and Microsoft to implement e.g. SQLite's PRAGMAs?

In the meantime, once you look at the WebSimpleDB API and realize they've basically exposed to best bits of SQL, including joins, to JavaScript using a very general, future proof API that isn't based on a specific implementation, without actually requiring an SQL engine underneath, this looks a hell of a lot more appealing.

(I've coded against WebStorage, and have nothing against it. I just think WebSimpleDB is a much better approach, especially given all the extras it supports (indices, transactions, complex queries, ...).

1
hofo•

Well I certainly don't expect anything using SQL in my browser to compete against a full Oracle/MSSQL/MySQL server. Any data set that's big enough to really need an index doesn't belong client-side IMHO.

The applications I've used it on are simple conference attendee information apps. There might be 500 rows in the largest table. And these are really flat tables too.

1
neilk•

Behold: FileAPI.

I have no idea when or if you'll see this in Internet Explorer. It's still bleeding edge.

The HTML5 file input has some useful properties today -- that looks like it should be standard in the most recent browsers soon. As usual IE is the laggard.

2
MindStalker•

Btw, what have you found that good/easy for Client->Server streaming. Comet works well for Server->Client, but going the other way involves a huge amount of hacks from what I've found.

1
pytechd•

For your local file stuff, direct file access? No, but you probably do not need that.

For caching data locally, look into localStorage. Couple that with native JSON parsing in modern browsers (IE8 does not count as modern) and you can store arbitrary stuff.

For prompting users to save arbitrary streams to their local system, check out http://www.nihilogic.dk/labs/canvas2image/ which does it with images.

We use both of these in our app to stream & cache reporting data for our reporting tools. The actual display manipulations are all client side, and users can easily download TXT, CSV, XML, and HTM exports all client-side. PDF, and XLS exports require a round trip.

Add in caching (HTML5 offline application + manifest) and you could do this all offline, too.

1