Most traditional security methods restrict the allowed actions by restricting access to resources. A program run on a computer can be arbitrarily well confined, for instance by emulating a computer to run the program in. In this case the programs are executed in an environment where the security sensitive operations are not available. Unfortunately, this does not by itself solve any problems for programs that need to access any of part of the system to do their work. This is almost any non-trivial program since the reason we use the program in the first place is that we want it to process our data.
By the same token, cryptographic methods do not help us here. The program has, at some point, to have access to unencrypted data. The problem we face is how to know that a program is not misusing its assigned privileges. Digital signatures should be used to authenticate the sender of the program and/or guarantee that the program has not been tampered with. The program's maliciousness, whether deliberate or due to bugs, cannot be decided by any level of cryptography.
Many of the above problems can be remedied by assigning privileges on a per-program-invocation basis. Instead of deducing the program's privileges from the user's privileges, as is common in many file systems today (NFS, AFS etc.), the program is explicitly given a capability to access certain information [32]. The problem with deduction of program privileges from user privileges, called ``the confused deputy problem'', is discussed in [16].
Capabilities make the the assigned privileges more precise but using capabilities gets tedious if the user has to make a decision for every move the program makes. In addition, it is not always easy to grasp whether the program's request for a resource is relevant and proper or what consequences granting of the capability will have.
The assignment of capabilities can be automated to some extent by allowing the access-points of the resources to be programmable. Things that can be automatically taken into consideration is the origin of the program, whether the resource is necessary for this particular kind of task and what other privileges have been granted to this and the other programs. To automatically grant privileges based on different resource consumption profiles or to judge how sensible a resource request is, the system has to have a model of what kind of program is requesting the resource.
Models of behavior are used for supervision of user activities in the Intrusion Detection field of computer security. Intrusion Detection concerns detecting external and internal penetrators of the security system as well as authorized users who misuse their privileges. Traditionally the focus has been on detecting malicious users, but in a networked environment much of the analysis applies to programs as well. Often, a user's actions are compared to a profile of his previous behavior. Statistical measures and inferred rules can be used to detect deviating behavior. Occurrences of particular disallowed actions can be found by explicit rules, e.g occurrences of sequences of events in the collected data constituting a known attack.
A program is often seen as a complete description of its behavior. The program could then, in theory, be analyzed before runtime to see whether it contains any malicious behavior. Completely analyzing the program is computationally very expensive [8]. Large parts of the program might not even be used in a particular execution. In this case a lot of analysis would be in vain. Moreover, the view that a program completely describes its behavior misses the fact that the program, during its execution, will interact with other programs and with the user. What matters is not only what the program can do with its data, but also what data it is fed. Theorem proving for deciding what a program can do is very useful for guaranteeing that parts of the program code lacks side effects and consequently does not require certain runtime checks. It does not, however, solve the whole problem of how to safely use untrusted programs.
Since no definite conclusion of whether a program is malicious or not can be reached before runtime we propose that the supervision is done at runtime. As in Intrusion Detection, the use of models or, to some extent, statistical measures of the program's behavior mark the limits of what the program can do.
To address the issues of how to monitor a program at runtime and how the user can get context sensitive help and advice in security matters we envision a Personal Security Assistant integrated with the user's work environment.
The guiding idea is that when a program is invoked on the computer, the user will have a good general idea of why he/she starts the program and what the program is supposed to do. If this information is communicated to the security assistant, relevant monitoring of the program can be set up.
There are several imaginable ways that this information can be conveyed. If the assistant can extract information about what the user is currently working with, context related security restrictions can be applied. For instance, it probably does not matter as much if your word processor leaks the contents of your shopping list as if it leaks the contents of your business mail.
An even more attractive approach is that the assistant asks the user what type of program was started. If the program is classified by what kind it is, warnings could be given both when certain unexpected behavior occurs and also when some behavior is missing. This would further constrain what the program is allowed to do and could hence provide more evidence that a program might be a Trojan horse. For example an editor rarely reads systemfiles or binaries, and a compiler usually reads header files.
The user should not need to be aware of all the gory details of what it means for a program to belong to a certain category and the category a particular program belongs to has to be sufficiently loose, or the user will be burdened by having to select from too many categories. Categories are discussed further below.
From the information about what the user intends to do the assistant decides what events and potential violations to look for, such as whether communication takes place, with whom and what information is potentially communicated. When a violation of the current constraints is found the assistant automatically notifies the user and gives recommendations about what to do, e.g terminating the program. The assistant should be able to motivate why if finds a particular action suspect and to give the user decision support, e.g by visualizing relevant parts of the system state such as what communication paths are currently open between the programs and what information might be leaked.
Since the number of non-security-experts will only increase in the future, it is important that the assistant is able to communicate with the user in terms of the task the user tries to accomplish. This should make it easier to grasp the risks involved in allowing the violation. In the services of a security assistant can also be included help with the daily security chores such as setting prudent access-privileges. The assistant could be integrated with other user-interface assistants and would then not be perceived as a special supervision tool.
Crucial for the assistant to be effective against new kinds of attacks is how it can learn about new dangers and methods of detection. If the assistants can communicate their findings amongst each other and trade information about known malicious programs, good heuristics and detection strategies the assistant community would quickly and automatically increase its resilience to new attacks. This, of course, brings in the question about the other sites' accountability. Might they be participating in the attack, or what do they gain by releasing this information? This is the same problem as how to trust your business partner on the Internet [34,33] and is not addressed here.
We would like to make use of the intuitive notions of what a program category is. Most people would probably agree that we could expect an editor to read and write to text files opened by the user and that transmitting data over the network is less common, although not necessarily illegal.
Completely defining what features a program category should show would be a very difficult and time consuming process and strict definitions might also hinder inventiveness among the application programmers. It is not likely that we can agree on complete definitions in all cases so the goal when enforcing behavior will rather be to find a balance between how many decisions the user has to make and how well protected he is.
Continuing on this line of reasoning, instead of defining what the categories are, instead only a taxonomy of categories could be defined. Then it would be possible for companies to define and sell programs enforcing the behavior of certain categories. Since no particular standards should need to be enforced, the ability to respond quickly to new threats should increase and monitoring could be tailored to enforce special concerns. These enforcement programs can be used together thereby providing some degree of redundancy and also some protection against implementation bugs
Detection of a maliciously behaving untrusted program would typically consist of the following events:
The program is invoked, either explicitly by the user or found by the system to be appropriate and automatically fetched, installed and run.
The program presents itself to the user and the user selects an appropriate category in terms of what he is currently doing. The selection can be made from a list provided by the security assistant or the program might introduce itself so that the user has only to confirm that this is the appropriate category. Perhaps more information about the context, e.g what task the user is working on, could be used to further constrain the available options. The user selects the category 'simple file visualizer' and what company it comes from, since the program is supposed to, e.g present some product information based on the user's customer profile from some company. The security assistant can now invoke appropriate sensors and set thresholds for examining at the audit data.
When the program reads the user's customer profile for the company the program came from, the security assistant will not complain, since this is an action that the user would want the program to do.
If the program tries to access a file not associated with this company, the sensors will observe the departure from normal 'simple file visualizer' behavior and the security assistant will alert the user of this unexpected behavior. If the user understands why the program does this and is willing to let the program continue with its business, access can be granted to the program. If not, the program is stopped.
If a taxonomy rather than a specification is used, the ability to formally state exactly what you are protected against is traded for an approach where quick market response enables quick adoption of techniques for detecting the current dangers.
There can not be any complete guarantee that nothing bad can happen, but the assistant would be able to gain, rather than loose the user's confidence over time as the monitoring is updated.
The bottom line is that the assistant can give a user more confidence that a program does not misbehave, but the notion that there is an actual risk when a program is granted resources is not disposed of. A security assistant should be seen as a tool for understanding and managing these risks. This service is lacking in end-user environments today and hence blind trust or no trust in the program are the only expressible options.
Essentially, the assistant analyzes a stream of messages describing the system state and system events. Some messages might not be available on certain computers and others might lack some related information. Since a lot of audit data is generated it has to be compressed and clarified. Almost any conceivable measure can be constructed, but to give a good picture of what is going on the measures must not be too low-level. For instance, does 'the-number-of-files-yet-opened' not by itself give much help in assessing the risk. Issues of updatability of what to look for and how to present the information as well as how to handle completely new messages have to be considered in the design. In the context of mobile programs it is also necessary that programs of which there is no prior experience and that are used only for a short while are monitored sensibly. These requirements rule out designs centered around training and matching particular execution profiles. Also, for Neural Nets and Genetic Programs it is difficult to add new parameters and new ways to analyze the data without having to modify and/or retrain the program. It is also hard to make statistical models that take into account the order of the events and that can set additional constraints that help in deducing which program was at err.
We want the assistant to be able to constrain, forbid and assert certain behavior. It also has to be able to refine the data available and should be able to cope with when some information is not available on some platforms.
Albeit more computationally expensive, a model based approach where various rules enforce the expected behavior should be able to fulfill the requirements. The rule language has to be sufficiently general to allow new, innovative rules to be expressed. The most general way to express a rule is as a programmed component. In this case the component can contain state and do arbitrarily complex aggregation and analysis of the available data, even neural-network components are conceivable. Such a component can itself send messages describing the system state from its point of view and make contacts outside of the assistant for additional information. These rules/components are below called sensors to emphasize their reactive behavior.
A blackboard-like architecture, where the sensors are free to act upon any event they deem interesting, would give the requested freedom in how the sensors can exchange information. Some sensors might react to ordinary audit-events, e.g reading a certain file, whereas others sense more complex behaviors like ``ongoing inter-program communication'' by making use of a variety of messages.
However, rather than focusing solely on message-delivering there has to be a versatile way of adding and removing sensors. A sensor has to be able to express what information it needs and what information it delivers so that others can know if they can understand and make use of the sensor. If the rule language allows the sensors to describe their behavior, sensors could find out when they are partly overlapping and share information to reduce redundant work.
If the sensors negotiate with each other about what to tell each other they can ensure that the events required for their proper operation can and will be delivered. The appropriate sensors to use would then be automatically decided each time the monitoring of an untrusted program is set up. In this case new sensors are automatically incorporated in the monitoring as they become available and redundant sensors switched off.
Speech-act oriented protocols are being studied by many software agent researchers. This family of protocols focuses on exchange of knowledge amongst agents. Issues of meta knowledge, such as knowing if one understands the other speaker, as well as concepts for advertising services and various ways of asking for information (ask-one or subscribe etc.) are prominent in the design of these protocols. Speech-act oriented protocols would probably give the sensors much of the above desired communicatability.
: Schematic picture of the Personal Security Assistant architecture.
Figure 1 depicts schematically the suggested architecture of the security assistant. The untrusted program is executed in an environment from which audit data can be extracted.
The data is made available to a set of sensor programs whose task is to enforce the behavior of a certain category. Several sensor programs can be enforcing the same category by looking at different and maybe somewhat redundant information. The sensor programs are autonomous, but in order to increase the efficiency they are able to share work and information. Each sensor program implements three roles to more or less degree. The system specific part gets audit data directly from the audit tool and converts it to some common format. If no-one is interested in a particular event it can be pruned at this level. The functionality providing part can be any general tool for e.g maintaining a set of rules, doing statistical analysis (e.g by a neural net), monitoring a threshold etc. If this role is sufficiently general, the functionality can be made available to other interested sensors. The category enforcing role implements how the particular sensor believes that a well-behaving program of the particular category should behave. This description is used to decide what parts of the other roles are needed for the monitoring. Note that no sensor has any special or privileged status and that there is no predefined hierarchy to fit the sensors into.
