Quintus Prolog Manual Quintus Prolog Manual
G-5: Saving and Loading the Prolog Database
Quintus Prolog Release 3 provides a much more powerful alternative to the traditional save/restore facilities of Prolog. It is now possible to save, and subsequently load, individual predicates, or sets of predicates, or entire modules of predicates, or indeed the complete Prolog database. Such sets of predicates are saved into Quintus' standard Quintus Object Format ("QOF" files). This significantly extends the utility of QOF files which were previously only used to produce runtime systems and stand-alone programs. QOF files are now a fully general way of storing arbitrary Prolog facts and rules in a form that can be quickly and easily used. QOF files contain a machine independent representation of both compiled and dynamic Prolog predicates. This means they are completely portable between different platforms running Quintus Prolog.
QOF files can be generated by:
QOF files can be used by:
QOF saving and loading is available in both Quintus Prolog development systems and runtime systems built for distribution. In a development system, code can be incrementally compiled using the built-in compiler and then saved into a QOF file. In a runtime system, which does not include the built-in compiler, dynamic code can be asserted and then saved. Runtime systems can load QOF files containing previously compiled code.
This combination of the capabilities of the Quintus Prolog Runtime Generator, with the saving and loading facilities of Quintus Prolog Release 3, and the cross-platform portability of QOF files, provides tremendous flexibility that can be used for many purposes. For example:
The facilities for saving and loading QOF files are more than just a convenience when developing programs; they are also a powerful tool that can be used as part of the application itself.
G-5-2: Compatibility with save/restore in previous releases
Unfortunately, it has not been possible to retain the semantics of save/[1,2] available in previous releases of Quintus Prolog. This is regrettable because it means that programs that incorporate code for building saved-states will need to be changed. This section explains why it was necessary to remove these predicates. Note, however, that save_program/1 is available and has the same semantics as previous releases (except for foreign code), although it is based on a new implementation using QOF files. A new predicate save_program/2, described in section G-5-4, has been provided which supports the most common usage of save/[1,2] which was to specify an initial goal for a saved-state to call when run.
The difference between save_program/1 and save/[1,2] in previous releases of Quintus Prolog was that save_program/1 saved only the Prolog database, whereas save/[1,2] saved both the Prolog database and the Prolog execution stacks. It has not been possible to retain the saving of the Prolog execution stacks in a way which is consistent with the Release 3 support of embeddability and the general portability of QOF files. This is why save/[1,2] have been removed. The reasoning goes as follows:
The model that an arbitrary Prolog execution state can be saved thus only works well within a Prolog-only situation. In the complex embedded environments supported by Quintus Prolog Release 3 this model cannot work properly. Hence the removal of the facility.
As mentioned in points 4-7 above, an additional important aspect here is that Prolog no longer makes any attempt to save the state of C (or other foreign language) code. This was a feature of saved-states in previous releases where both the C code and its data structures were saved (as a memory image) into saved-states. This was a feature that caused many problems. A primary problem was that the saved C state was initialized (variables retained their values when restored) and yet the initialized C state could contain many items that were no longer valid in the new process, such as addresses and file descriptors. Such code would often fail when restored. In addition, Prolog was unable to guarantee that it had saved all the necessary foreign code state. With the advent of shared libraries and other complex memory management facilities in the operating system, it became impossible for Prolog to control and manage the states of other tools in the address space.
When one takes a step back and looks at Prolog in the light of the goals of Release 3 (Chapter A-2) - where Prolog code is a component that can be embedded in complex applications written in many languages - it is clearly unreasonable for Prolog to try and control, let alone save, arbitrary non-Prolog state. The Prolog operations for saving and loading QOF files now operate solely on the Prolog database and these operations do not involve making any assumptions about non-Prolog state. This is a much cleaner and more robust approach, and is the most appropriate when Prolog applications become embedded software components.
Prolog QOF files do not contain any foreign code or foreign data structures. As discussed in the previous section, this is different from saved-states in previous releases of Quintus Prolog.
However, QOF files can have dependencies on object files that will be automatically loaded when the QOF file is loaded. These dependencies can arise because:
The qnm utility can be used (from the Unix command level) to see the list of dependencies in a QOF file:
% qnm -D file.qof
It is slower to start up a QOF file with object file dependencies because the object files must be re-linked and re-loaded. This re-linking and re-loading will occur every time the QOF file is loaded and the necessary object files are not yet loaded into the system. If this start-up time becomes a problem then this can be tackled by statically linking your foreign code into the Prolog system. This is described in Chapter H-1. This will make the foreign code become part of the Prolog executable so there is no run-time linking required at all. In addition, on most Unix systems this Prolog code will now be demand-paged from the executable which will improve start-up time and reduce paging (as compared with dynamic linking). We recommend switching over to using static linking for programs with a lot of foreign code. The dynamic foreign language interface can be used for loading code while testing, but once your foreign code is stable it is better to have it statically linked. Furthermore, dynamically loaded foreign code cannot be debugged with a Unix debugger such as dbx or sdb, so you will also need to statically link the foreign code so that the debugger can be used on the resulting executable. This contradicts the fact that foreign code should be dynamically loaded while testing.
All foreign code is either linked into the Prolog executable, or is re-loaded when a QOF file is loaded. This means that when a program is started the foreign code will always be in a uninitialized state. This is exactly the same as any other standard Unix program. However, this is a change from saved-states in previous releases of Quintus Prolog which saved the initialized foreign state. The new semantics is much cleaner, is consistent with standard Unix practice, and avoids previous problems with invalid initializations that were not valid in the new process. This latter problem was particularly problematic for libraries (such as Curses, X Windows, and database interfaces) since the user did not usually have source code for the libraries and how they initialized and what they depended upon could not be easily understood. In Quintus Prolog 3.0 and later, foreign code linked with Prolog, or loaded into Prolog, will work just the same as if it were a separate program independent of Prolog.
It is possible that some of your previous programs relied on the saving of foreign state into saved-states. If you have such programs then they will need to be changed. Usually the change will involve making sure that the foreign code is explicitly initialized each time the application is run. The initialization facilities described below, section G-5-6, may be useful for this.
Saved-states are just a special case of QOF files. The save_program/[1,2] predicate will save the entire Prolog database into a QOF file, and then in addition will make that QOF file executable. The QOF file is made executable by making the first line of the file be a Unix sh(1) script. This script runs the executable that the QOF file was saved from, telling it to load the QOF file.
So, if a saved-state is created as follows:
?- save_program(savedstate).
then if we look at the first line of the file we will see something like the following. Note that '+L' is a Prolog command line option to load a QOF file ('$0' will be the name of the QOF file and '$*' will be any other arguments given when it is run).
% head -1 savedstate
exec /usr/local/quintus/bin3.4/sun4-5/prolog +L $0 $*
(If your version of Unix does not have the "head" command then you could use "sed 1q" instead.) This QOF file can then be run from the command line as follows:
% savedstate
In addition to the user's code from the Prolog database, a saved-state saved by save_program/[1,2] also contains Prolog system information such as flag settings, debugging information and so forth. When the saved-state is loaded this system state is also restored, which is convenient for continued development.
Apart from being made executable, and containing additional Prolog system information, a saved-state saved through save_program/[1,2] is just a standard QOF file. This means that it can be used anywhere you would otherwise use a QOF file, for such things as loading into Prolog, linking together with other QOF files, and linking into executables (see Chapter H-1 for information on these linking capabilities).
A saved-state, or any QOF file, can be restored using the restore/1 predicate from within Prolog:
?- restore(savedstate).
The restore/1 predicate will re-execute the running executable (using the Unix exec(2) system call) in order to obtain a completely new environment, and will then load the QOF file. If the QOF file was saved with save_program/[1,2] then this will restore exactly the same Prolog database state as when the saved-state was saved. In runtime systems, however, it is the application program's responsibility to load the file into the restarted executable, see section L-3-153 and section H-1-8.
Note that the executable that will be re-executed by restore/1 is the one currently running. This may be different from the one named in the first line in the QOF file, if that QOF file was saved from some different executable. To use the executable that originally saved the QOF file you should return to the Unix command line and run the QOF file directly. To use the executable you are currently running, you should use restore/1.
If the loaded QOF file has object file dependencies then those object files will be re-linked and re-loaded as part of loading the QOF file. If the object file cannot be found or linked, then an exception will be raised. Similarly, QOF dependencies are also reloaded at this point.
Note: The QOF file saved by save_program/2 does not contain any of the Prolog code that is statically linked into the executable. Only the Prolog database (both compiled and dynamic) which has been built since the executable started running is saved. This is done to avoid code duplication in the saved-state. However, this does mean that if the QOF-file is loaded into a different executable, then the program may be missing some code that it assumes should be there, because it was present in the original executable. An example would be a saved-state that was saved from an executable containing Quintus' ProWINDOWS add-on product. If that saved-state is loaded into a normal Prolog executable without ProWINDOWS then any calls to ProWINDOWS will not work (they will generate undefined predicate exceptions). The correct thing to do is clearly to make sure that you use either the original executable, or an executable that contains the necessary programs, or you load the necessary programs in addition to loading the saved-state QOF file.
The save_program/2 predicate can be used to specify an initial goal that will be run when the saved-state is re-loaded. This usage of save_program/2 replaces the most common uses of the old save/[1,2] predicates which are no longer available. For example:
?- save_program(saved_state,initial_goal([a,b,c])).
When 'saved_state' is loaded initial_goal/1 will be called. This allows saved-states to be generated which will immediately start running the user's program when they are executed. In addition to this save_program/2 facility there is also a comprehensive facility for programs to set up initializations to be run when they are loaded or otherwise started. This is described below in section G-5-6.
G-5-5: Selective saving and loading of QOF files
The save_program/[1,2] and restore/1 predicates discussed in the previous section are used for saving and restoring the entire Prolog database. To save selected parts of a Prolog database, the predicates save_modules/2 and save_predicates/2 are used.
For example, to save the modules 'user' and 'special' you would use:
?- save_modules([user,special],'file1.qof').
All predicates in those modules will be saved, and in addition any foreign code files previously loaded into these modules will generate an object file dependency in the QOF file. All information in these modules about predicates attached to foreign functions, and also predicates which have been made externally callable from foreign code, is saved as a normal part of the module.
For each module imported by one of the specified modules, a QOF file dependency is included in the QOF file. This means that when you load 'file1.qof' into another Prolog session, it will automatically load any additional QOF files that it needs.
To just save certain predicates you would use:
?- save_predicates([person/2,dept/4],'file2.qof').
This will only save the predicates specified. In this case no additional dependency information is saved into the QOF file. Note that the module information for these predicates is included. When the QOF file is loaded the predicates will be loaded into the same module they were in originally.
Any QOF file, however generated, can be loaded into Prolog with load_files/[1,2]:
?- load_files('file1.qof')
or, equivalently:
?- ['file1.qof'].
The information from each QOF file loaded is incrementally added to the database. This means that definitions from later loads may replace definitions from previous loads. A saved-state QOF file saved with save_program/[1,2] can also be loaded with load_files/[1,2] in which case the contents of the saved-state are just incrementally added to the database as for any other QOF file. The use of load_files/[1,2] for this is different from the use of restore/1 in that restore/1 will re-execute the executable thus reinitializing the database. Using load_files/[1,2] allows the database to be incrementally changed within the same process.
If the loaded QOF file has object file dependencies then those object files will be linked and loaded as part of loading the QOF file unless they have already been loaded. If the object file cannot be found or linked, then an exception will be raised.
The predicates load_files/[1,2] are used for compiling and loading source files as well as QOF files. If 'file1.qof' and 'file1.pl' both exist (and 'file1' does not), then load_files (file 1) will load the source (.pl) or the QOF, whichever is the most recent. Refer to Chapter G-4 for more information on loading programs, and also to the reference page for load_files/[1,2].
Advanced note: Both save_modules/2 and save_predicates/2 will save Prolog code that is statically linked if such modules or predicates are specified. This is different from save_program/[1,2] which will not save statically linked Prolog code. Note that if such a QOF file is loaded back into the same executable which saved it, then the new definitions from the QOF file will replace the statically linked code. There is no problem with this, except that some space will be wasted. The original statically linked code will not be used, but since it is linked into the executable its space cannot be reclaimed. Since static linking is normally used to optimize start-up time and the space usage for code, it is somewhat of a waste to circumvent this by saving and loading a lot of Prolog code that is already in the executable. If the QOF file is to be used for other purposes, such as re-linking the executable, or as a part to be loaded into another program, then, of course, the saving of statically linked code is probably exactly what is required.
G-5-6: Initializing Goals in Saved States
Under the earlier model, a Prolog file could either be compiled into the development system, or compiled to Quintus Object Format by qpc, as shown in Figure G-5-1.
Figure G-5-1: Compilation options: Quintus Prolog 2.5
The ability to save and load QOF files in a development system makes the picture more complicated. Figure G-5-2 shows the ways a Prolog source file can be compiled or saved.
It would be natural to expect a.qof to be the same, however generated. But both the 'save' predicates and qpc offer a rich variety of options, and the reality is less simple (see Figure G-5-2).
Figure G-5-2: Saving and loading options: Quintus Prolog 3.4
G-5-6-1: The Initialization Declaration
The initialization/1 predicate is an important complement to the embedded directive construct ":- Goal" appearing in a file being consulted or compiled, and can in many cases not only replace the directive, but also make the code work better when used in stand-alone programs and runtime systems.
The main reason for this is that :- 'Goal' directive is executed at compile-time, not when the file in which the construct occurs is actually loaded into a running system. This causes no problems within development systems, but if we want to save states and compile programs into qof-files, link them together, and later start them up again, problems arise because:
The initialization/1 predicate, on the other hand, provides a way of letting initialization routines be called when a file is actually loaded or a system containing the file is started up. This allows for correct initialization in stand-alone programs and runtime systems; therefore a recommended programming style is to use initialization/1 instead of a bare ":-Goal" construct whenever appropriate.
In the figure, (Figure G-5-3) Goal_1 might typically be an operator declaration and Goal_2, an initialization predicate which modifies the database.
Figure G-5-3: Embedded directives (goal_1) vs. initialized goals (goal_2)
The initialization goal, Goal_2, is defined to be run when:
A predicate should be declared as volatile if it refers to data that cannot or should not be saved in a QOF file. In most cases a volatile predicate will be dynamic, and it will be used to keep facts about streams or references to C-structures. When a state or a module is saved at run-time, the clauses of all volatile predicates defined in the context will be left unsaved. The predicate definitions will be saved though, which means that the predicates will keep all properties, that is volatile and maybe dynamic or multifile, when the saved state is restored.
For example, if a Prolog application connects to an external database at start up, establishing a connection by an assertion like (A), a volatile declaration would prevent each particular connection from getting saved in the QOF file, as illustrated in Figure G-5-4. A code example is found in the reference page for initialization/1.
assert(db_connection(Connection))
Figure G-5-4: Using the Volatile Property
When used as a compile-time directive, the volatile declaration of a predicate must appear before all clauses of that predicate. The predicate is reinitialized. When used as a callable goal, the only effect on the predicate is that it is set to be volatile.
To tune the initialization of a file or system to be run only when it should be run, volatile/1, in combination with other declarations, give initialization/1 the information necessary to distinguish different loading situations. In the reference pages, we show how some common situations can be programmed using these predicates.
If a source file contains data that is supposed to be transformed according to some complicated rules (which cannot be done with term_expansion/2), and the data after the transformation can be saved into a saved state, we might want the transformation to be done when the file is loaded, but not when a saved state is restored. The following program defines the initialization to be run only when the file is loaded:
:- dynamic do_not_transform/0. % reset fact
:- initialization my_init.
my_init :-
( do_not_transform ->
true
; undo_transform, % remove old data
do_transform,
assert(do_not_transform)
).
In the above example, do_transform/0 and undo_transform/0 are user defined.
Detailed information is found in the reference pages for the following:
initialization/1 save_predicates/2 load_files/[1,2] save_program/[1,2] prolog_load_context/2 volatile/1 restore/1 save_modules/2
contact:
product support
sales information