Reference Document
Web site: <http://www.sics.se/dive/manual/fileinput.html>

Anneli Avatare - <anneli@sics.se>
Emmanuel Frécon - <emmanuel@sics.se>
Olof Hagsand - <olof@sics.se>
The Swedish Institute of Computer Science Stockholm September, 1995

1. Introduction

This document describes the C-function file input interface in Dive. This interface is intended for use by applications that link the Dive libraries. The Dive file input interface essentially provides two services:

1. read general purpose files, including general MIME documents.
2. read and parse Dive entity descriptions.

When reading general purpose files, the file contents may be directed to an external viewer or to a prealloced memory area. When general purpose Mime documents are read, the Mime type of the document is interpreted to spawn the correct viewer.

The parser recognizes two formats: the Dive file format [2] and VRML 1.0 [3]. The file extension, the Mime type and the encoding are used to determine which format to parse and if uncompression on the fly is needed.

2. Retreiving Files

All files given to the file input interface are identified by URLs (Universal Resource Locators), such as: <http://www.sics.se/dive/data/sics.vr>, where "http" defines a protocol, "www.sics.se" a host and "dive/data/sics.vr" a file.

Currently, the following protocols are recognized by DIVE:

• http [1] - Dive implements a simple http retrieval protocol, with the following restrictions:(1) retry queries are not correctly treated; (2) older http versions (pre http 1.0) are not supported and may lead to error. An example of a URL using the HTTP protocol is <http://www.sics.se/dive/data/sics.vr>.
• ftp - In the case of FTP, the "ftp" UNIX program is forked and a temporary file is created. An example of a URL using the FTP protocol is: <ftp://ftp.sics.se/pub/dive/README>.
• dive - By default, when no protocol is given, the Dive internal protocol is used. A special URL scheme is dedicated to this protocol: "dive". It works in the following way: if the URL starts with a "~" or a "/", this location will be tested first. Then, the filename will be extracted from the directory name, if needed and Dive will make use of the DIVEPATH variable and this file name. the environment variable "DIVEPATH" defines a list of directories where the file is searched for. The extension is optional and the file interface will do its best to find a matching file with the set of extensions Dive recognizes. Examples of URL using the Dive internal protocol are: "vworld", "vworld.vr", "/foo/vworld.vr" or "dive://foo".
• file - The "file" scheme can be used to access files with a full path, when the "dive" protocol is to be by-passed. An example or a URL using the local "file" scheme is "file://home/emmanuel/dive/data/vworld.vr"

The file input interface provides two functionalities: general file input and parsing.

The simplest file interface returns the file contents given an URL. A more elaborate service interprets the Mime type and spawns an external viewer for the file contents, including bitmaps, sounds and movies.

3.1 Interpreting Mime documents

Unless as specified below, the "metamail" tool is used to spawn external viewers for Mime documents. Metamail needs to know the Mime type of a document and the algorithm to determine Mime types and encoding dependends on the protocol of the URL.

The "mime.types" file described below contains a list of file extension and Mime type pairs. For example, if the "mime.types" file contains a "image/gif gif" entry, the "vworld.gif" file will be given the Mime type image/gif. The file should be in a directory pointed to by the "DIVEPATH" environment variable.

The algorithm for determining Mime types is as follows:

• FTP and dive internal protocols: The extension is examined to determine if the encoding is "x-compress" (.Z extensions) or "x-gzip" (.gz extensions), if so the document is uncompressed. The "mime.types" file is then used to associate Mime types by file extensions. Finally, metamail is used to spawn an external viewer for the document.
• HTTP: the Mime type is returned by the remote http server. However, if the type is "text/plain", it is possible that the client knows a Mime type which is unknown from the http server. The algorithm for Ftp and dive described above is then used.

Uncompression is accomplished by spawning a process running the "gzip" command with the correct arguments. Dive assures that metamail and applications that want to execute their own treatment on documents will always receive uncompressed data.

As Mime understanding rely on metamail, a good setup of this program will assure best results.

Dive applications can request the Mime module to be called back each time a Mime document has been read. The callback contains a pointer to the content of the Mime document and the application is able to decide if it should take care of that specific buffer or if the buffer should be treated by "metamail".

3.2 Parsing

DIVE is able to read 3D description documents in both its own format [2] and the standard VRML 1.0 format [3].

VRML files have the Mime type "x-world/x-vrml" and the extension ".wrl". Files in the Dive file format have the Mime type "x-world/x-dive" and the extension ".vr". As explained above, Dive is also able to recognize the encodings "x-compress" and "x-gzip" (and their related extensions ".Z" and ".gz"). The GNU program "gzip" is forked if necessary to obtain the uncompressed form.

Note: Dive file format files permit preprocessing with the C preprocessor (cpp). This means that "#include" may be used for macro definitions etc. However, if a Dive file format document is retreived remotely, the preprocessor will be executed in the context: included files will have to be on the local file system.

An URL can be extended with arguments which can be sent to the preprocessor. '?' is used as a separator between the arguments. For example, in the extended URL
<http://www.sics.se/dive/data/dive_town.vr -DLOD=1>, -DLOD=1 is an argument that is passed to the preprocessor.

4. Functional interface

The file input interface offers functions to read the content of an URL in a user-defined way, to spawn external viewers for Mime documents and to create new Dive objects resulting of the parsing of an URL or a buffer in memory.

Reading the contents of an URL and delivering it in raw (uncompressed) content:

http://www.sics.se/dive/manual/cref.d.html#dive_readURL_memory http://www.sics.se/dive/manual/cref.d.html#dive_readURL_callback http://www.sics.se/dive/manual/cref.d.html#dive_readURL_stream

Reading an URL and, by using its MIME type, handle it accordingly. This invloves spwaning an external viewer and calling a registered callback.

http://www.sics.se/dive/manual/cref.d.html#dive_readURL_withMIME http://www.sics.se/dive/manual/cref.m.html#mime_register

Reading an URL or memory buffer and creating and inserting Dive objects by parsing. The two formats that are recognized are DIve (x-world/x-dive) and VRML 1.0 (x-world/x-vrml).

http://www.sics.se/dive/manual/cref.d.html#dive_readURL_add http://www.sics.se/dive/manual/cref.d.html#distr_readURL http://www.sics.se/dive/manual/cref.d.html#dive_extendedURL_to_env http://www.sics.se/dive/manual/cref.d.html#dive_parse_string_add

Entities (worlds, objects, actors) saved to file:

http://www.sics.se/dive/manual/cref.d.html#dive_save_entity

Accessing remote URL with an ANSI C-like stream interface.

http://www.sics.se/dive/manual/cref.u.html#uopen http://www.sics.se/dive/manual/cref.u.html#udopen http://www.sics.se/dive/manual/cref.u.html#uclose http://www.sics.se/dive/manual/cref.u.html#ugetc http://www.sics.se/dive/manual/cref.u.html#ugets http://www.sics.se/dive/manual/cref.u.html#uread http://www.sics.se/dive/manual/cref.u.html#uungetc

5. Examples

The mime-type for the file sweden.gif is image/gif. An external viewer is spawn for this document. In this example the xv tool is started.

5.2 Parsing a file in the DIVE format

In this example, an entity definition (a robot) is read from a file "robot.vr". The robot definition is added as sub to the world (world->id). The identifier to the robot entity is returned in "robot->id".

5.3 Load an extended URL

An example of an extended url is:

The preprocessor is invoked, when parsing "jeep.vr", with the arguments "-DX=-1", "-DY=1" and "-DZ=-3". (X,Y,Z) is the position where the jeep will be placed in the world. The default position is (0.0, 0.0, 0.0) if no arguments are given.

     if (!dive_extendedURL_to_env("http://www.sics.se/dive/data/jeep.vr -DX=-10
                                    -DY=1 -DZ=-3", " \t", &url, &argv, &argc)) 
       dive_readURL_add(&world->id, url, (dive_readURL_add_fn)distr_new_top,
                        NULL, DIVE_OBJ_MASK, argv, NULL);

In this example, the function dive_extendedURL_to_env is invoked with a URL with arguments (separated by a blank), i.e. an extended URL. The arguments are returned in "argv". The URL without the arguments, i.e. "http://www.sics.se/dive/data/jeep.vr", is returned in "url".

Then the function dive_readURL_add is called. The file "http://www.sics.se/dive/data/jeep.vr" is read and parsed. The preprocessor is invoked with the arguments in "argv". The mask "DIVE_OBJ_MASK" is used to filter callbacks. The callback function "distr_new_top" will be invoked for those entities matching the mask, i.e. distr_new_top is invoekd for all dive_objects. The function "distr_new_top" distributes an existing entity hierarchy as a top object and it is defined in the Dive distribution model.

References

[1] Basic HTTP, <http://www.w3.org/hypertext/WWW/Protocols/HTTP/HTTP2.html>

[2] Dive 3.2 file format interface.

[3] The Virtual Reality Modeling Language, Version 1.0c specification, <http://www.vrml.org/VRML1.0/vrml10c.html>