OBJECTIVE

The Atlantis project investigates componentization of workflow modeling and execution systems, particularly synergy of process-centered environment and computer-supported cooperative work components. The components are intended to be interoperable with legacy and off-the-shelf tools and frameworks and indicate requirements on future systems, for a concrete transition path.

Approach

Atlantis is a consortium consisting of the Programming Systems Lab at Columbia University, the Advanced Collaborative Systems Lab at University of Illinois at Urbana-Champaign (now partially located at the Distributed Systems Technology Centre at University of Queensland, Australia) and, formerly, the US Applied Research Lab of Bull HN Information Systems (ARL has shut down; their DARPA contract was cancelled). The two academic groups have been working on their own diverse workflow-related projects for over a decade. They initiated cross-licensing with each other and with Bull to evaluate the prospects for integrating their technologies, culminating in a plan to investigate several important, practical problems not previously being pursued:

Processes (or workflows) can vary substantially across organizations and projects and are “situated”, i.e., dynamically changing in response to a variety of technological, sociological and political stimuli. A process-centered environment (PCE) provides computer-aided support for a range of project-specific processes. The general goals of research in this area are to devise useful paradigms for representing processes, to determine means by which environments may assist teams of users in carrying out processes, and to discover mechanisms that permit in-progress processes to evolve compatibly.

• Groupware. The main theme is to integrate human/human collaboration, studied in the computer-supported cooperative work (CSCW) community, with tool/tool integration, the forte of the software engineering community. The Illinois researchers had addressed process primarily from a CSCW perspective, e.g., Bull used their framework to develop a system for directing human behavior during document inspections. The Columbia lab has been working on multi-user PCEs, particularly enforcement of process constraints and automation of tool invocations to satisfy prerequisites and fulfill consequences of process steps. The open architecture for workflow systems draws from both lines of research, with the results originally intended to be transitioned by Bull into potential products.
• Process transition from current CASE and tool integration technology to PCEs by developing an external process server component that interprets project-specific process definitions. The open architecture supports mediation between such a server and applications, to minimize or avoid changes to pre-existing systems. The process server component provides a rule-based “process assembly language” into which the user organization’s choice of process modeling formalism (e.g., Petri nets, grammars, task graphs) can be translated, to be executed by the corresponding “process virtual machine”, thus lowering the barrier to adoption.
• Collaboration transition from current database transactions and “check-out” technology to collaborative workflow environments by developing an external cooperative transaction manager. It is widely agreed that the classical transaction model is inappropriate for long-duration, interactive and/or cooperative activities, but there is no consensus on the numerous extended transaction models that have been proposed, and it appears that different models may be needed for different applications. Thus the transaction-management component supplies primitives for defining project-specific concurrency control policies, analogous to process modeling. The open architecture enables mediation between the transaction manager and pre-existing systems, mapping task units to transaction-like constructs.
• Geographical distribution. Industrial-scale software development increasingly takes place outside the boundaries of a local area net, often spread across regions and/or independent organizations. Collaborating subcontractors may guard their own proprietary processes and tools, while sharing data subject to security constraints, so a model for “cooperating software processes” is needed. The open architecture extends workflow management and execution technology to interoperability among autonomously defined processes, with an Internet-capable PCE infrastructure.

FY96 Accomplishments

Database management systems (DBMSs) are increasingly used for advanced application domains, such as software development environments, network management, workflow management systems, CAD/CAM, and managed healthcare, where the standard correctness model of serializability is often too restrictive. The Atlantis project at Columbia University introduced the notion of a Concurrency Control Language (CCL) that allows a database application designer to specify concurrency control policies to tailor the behavior of a transaction manager; a well-crafted set of policies defines an extended transaction model. The necessary semantic information required by the CCL run-time interpreter is extracted from a task manager, a (logical) module by definition included in all advanced applications, which stores task models that encode the semantic information about the transactions submitted to the DBMS. Columbia designed a rule-based CCL, called CORD, and implemented a run-time interpreter that can be hooked to a conventional transaction manager to implement the sophisticated concurrency control required by advanced database applications. They developed an architecture for systems based on CORD, integrated the CORD interpreter with their own Pern Cooperative Transaction Manager and with the Exodus Storage Manager from University of Wisconsin, and implemented the well-known Altruistic Locking and Epsilon Serializability extended transaction models. CORD/Pern is being used by DARPA-funded projects at Brown University and University of Colorado.

The Atlantis project at Columbia University developed a process server component, called Amber. New systems could be constructed around the component, or existing non-process environment architectures enhanced with value-added process services (as in Columbia’s integration with Field from Brown University). Synergistic integration with existing process engines enables a degree of heterogeneity (as in Columbia’s coupling with their mockup of the TeamWare system from University of California at Irvine), and previous-generation process enactment facilities in existing environments could be replaced (as Columbia did with their own Oz system and with the ProcessWEAVER product from Cap Gemini). Amber also supports translation of higher-level process modeling formalisms, e.g., Petri nets, grammars and graphs, into its rule-based “process assembly language” for enactment, and permits addition of formalism-specific support to the process engine via an extension and parameterization mechanism. Thus the same process engine can support many different process modeling paradigms. Columbia developed translators for ProcessWEAVER’s Cooperative Procedures (Petri nets), Bill Riddle’s Activity Structures (concurrent regular expressions), and StateMate’s statecharts (finite state automata). Integration of Amber into some foreign system, and extension/parameterization of its process syntax and semantics, are both achieved through callbacks to a mediator consisting of special-purpose “glue” code. The Amber version of Oz is now used for all software development by the Columbia Atlantis group.

Black Box enveloping technology expects the tool integrator to write a special-purpose script to handle the details of interfacing between each COTS tool and the environment framework. Generally, the complete set of arguments from the environment’s data repository is supplied to the tool at its invocation and any results are gathered only when the tool terminates, so the tool execution is encapsulated within an individual task. This does not work very well for: Incremental tools that request parameters and/or return (partial) results in the middle of their execution, e.g., multi-buffer editors and interactive debuggers; Interpretive tools that maintain a complex in-memory state reflecting progress through a series of operations, e.g., “Knowledge-Based Software Assistants”; Collaborative tools that support direct interaction among multiple users, including asynchronous discussion and synchronous conferencing. The Atlantis project at Columbia University introduced a Multi-Tool Protocol (MTP), which enables submission of multiple tasks, either serially or concurrently, to the same executing tool instance on behalf of the same or different users. Single-user tools can thereby be converted to a floor-passing form of groupware, and a user can even send a running task to another user for assistance; these facilities assume X11. MTP also addresses multiple platforms: transmitting tool invocations to machines other than were the user is logged in, e.g., when the tool runs only on particular machine architecture or is licensed only for a specific host. Columbia implemented MTP as part of their Oz process-centered environment.

Componentization involves restructuring a stovepipe system into components that could potentially be reused in other systems, and/or re-engineering an old system to permit replacement of native code with new components. The Atlantis project at Columbia University developed a series of two processes, OzMarvel (running on top of the Marvel process-centered environment) and EmeraldCity (on top of the Oz process-centered environment, the successor to Marvel). Each was intended to support both aspects of componentization, but EmeraldCity addressed one requirement not fully understood when OzMarvel was developed: process support for co-existing components designed as alternatives to each other in plug-n-play style. Columbia used OzMarvel to replace Oz‘s native transaction management subsystem with a transaction manager component (Pern), and used EmeraldCity both to replace Oz‘s native object management system with an object-oriented database management system component (called Darkover) and to replace Oz‘s native process engine with a process server component (Amber).

FY97 Plan

The Atlantis project at Columbia University plans external release of a fully documented and robust version of the Oz process-centered environment framework ported to Solaris 2.5. They already have three “alpha” sites: University of Massachusetts, North Carolina State University, and University of West Virginia, but for an earlier SunOS 4.1.3 version. The official release will be downloaded from the World Wide Web, as their older Marvel 3.1.1 system is now.

Columbia University will complete their currently “proof-of-concept” World Wide Web browser user interface client to their Oz process-centered environment, so that it is fully functional for team software development on Solaris 2.5 and Windows NT. They are developing a general method for accessing legacy client/server applications from standard WWW browsers. An existing client for the system is modified to perform HTTP proxy server duties. Web browser users simply configure their browsers to use this proxy, and can then access the target system via specially encoded URLs that the proxy intercepts and sends to the existing server. The Web-based browser user interface to Oz is one example.

Columbia University will develop mechanisms to structure information so that the view of the World Wide Web, both within and across Web pages, is dynamically customizable. They will investigate an architecture that integrates environment data repositories with WWW to organize such dynamic structures for use in software development environments. Different users, or the same user at different times, could have different views of the Web. Their architecture will provide high flexibility for a wide variety of applications, ranging from managed healthcare to software development environments. A preliminary version will be realized in a “proof-of-concept” system based on Columbia’s Darkover object-oriented database component.

Technology Transition

Oz 1.x has been licensed in alpha form to three institutions (University of Massachusetts, North Carolina State University, and University of West Virginia). The first general release is planned for Fall 1996. Oz is a multi-site process-centered environment where each “site” autonomously defines its own process, data schema and tool wrappers (to integrate COTS tools). An arbitrary number of users connected to the same site participate in the same process. Sites may optionally form alliances that enable multi-process interoperability: Site administrators negotiate Treaties — agreed-upon shared subprocesses, and then the local environments may coordinate Summits — Treaty-defined process steps that involve data and/or users from multiple sites. Multi-site environment instances may involve multiple teams and organizations dispersed across the Internet, e.g., Oz has been run between Columbia and the (now defunct) Bull lab in Billerica, MA. Sites may alternatively co-reside within the same local area network, e.g., a two-process Oz environment is used in-house every day, one process for the shared release area and another for private workspaces, and a three-process Oz environment was previously used when Columbia re-engineered their system to replace the native object management system with an object-oriented database components.

Oz is most commonly used for software development processes, but environment instances have also been developed for document authoring and medical care plan automation (the latter in collaboration with Columbia-Presbyterian Medical Center). The alpha release, Oz 1.1.1, runs on SunOS 4.1.3 and provides XView and tty user interfaces. Oz 1.3, the version planned for release, will run on Solaris 2.5 and include a platform-independent World Wide Web browser-based user interface. Oz has been used for its own support since April 1995.

Oz‘s single-site, single-process (but multi-user) predecessor, Marvel 3.x, has been licensed to approximately 45 institutions (including 13 companies and the Naval Research Laboratory). It is now available for downloading over the World Wide Web (../download.html), so an exact count is no longer possible. Marvel 3.1.1 was the final released version, and has been available since May 1994. Marvel has previously been released for SunOS 4.1.x, Ultrix 4.2 and AIX 3.2, but only the SunOS binaries are currently available. XView and tty user interfaces are provided. Marvel was used for its own support, starting in January 1992, and later for initial Oz development.

Marvel is useful as either a self-contained multi-user process-centered environment (the user organization supplies its own process and tools or tailors the samples provided), or as a component for constructing a proprietary environment. The most serious commercial user is AT&T. AT&T has employed successive versions of Marvel since 1992 through the present as the workflow process engine component of their Provence system. Provence operates in a completely non-intrusive manner, hiding the workflow details from the user, who goes about his/her daily work in the normal manner. However, Provence monitors workflow steps as they occur and automatically performs various bookkeeping chores on behalf of the users to improve their productivity. AT&T is also employing Marvel to model and analyze the processes of several internal business units. For example, Marvel has been used to model the Initial Modification Request initiation and Tracking Process for the 5ESS Switching Software, and to model the Signaling Fault Detection and Tracking Process for the Technology Control Center. The AT&T contact states that both models resulted in better descriptions and formalization of the processes, and in the ability to run simulations and operational scenarios for the purpose of analyzing and visualizing the processes. In one case a significant. money-losing flaw in a process was uncovered during modeling. Click here for public information from AT&T. Contact Dr. Naser Barghouti, AT&T Research, 3D-552, 600 Mountain Avenue, Murray Hill, NJ 07974, 908-582-4384, naser@research.att.com.

A unmoderated user newsgroup is available for Marvel and Oz users and potential users. To be added to this mailing list, send email to majordomo@cs.columbia.edu with the words “subscribe oz-future” in the body of the message. Messages can be posted by sending email to oz-future@cs.columbia.edu (ozzy-n-webbiet@cs.columbia.edu is an alias). Note the newsgroup is NOT moderated: Everything posted will automatically be sent to every member of the mailing list (to which all current licensees were initially subscribed).

Pern 1.x has been licensed in alpha form to two institutions (Brown University and University of Colorado). Columbia plans the first general release for Fall 1996. Pern is a transaction manager component that supports both conventional and policy-based concurrency control, the latter through CORD below, and conventional recovery logging. It works with any kind of data repository that uniquely identifies its entities. Integration with legacy systems, including environment frameworks and database management systems, is implemented via a mediator-based architecture. Pern has been integrated into the Oz prototype above and experimentally interfaced to Cap Gemini’s ProcessWEAVER product and GIE Emeraude’s PCTE product. Pern includes CORD, a rule-based language (no relation to Oz or Marvel process rules) for defining concurrency control policies and extended transaction models (ETMs) that relax conventional serializability. Application-specific semantics are drawn from a task management layer (e.g., a process or workflow engine) via the mediators above and dynamically linked helper functions. Example ETM specifications have included altruistic locking, epsilon serializability, and sagas. CORD can be used independently of Pern, and has been experimentally integrated with U. Wisconsin’s Exodus database management system. Pern/CORD runs on SunOS 4.1.3 and Solaris 2.4+. Pern has been used daily as part of Oz since April 1995.

Potential licensees for Oz or Pern may request information from MarvelUS@cs.columbia.edu. Bugs should also be reported there. Potential Marvel licensees should consult the website mentioned above; Marvel is no longer supported, and users will be encouraged to upgrade to Oz.