Welcome to galaxy

INRIA SOA galaxy^*

* Une Action de Développement Technologique (ADT) * A Technology Development Action (ADT) by

INRIA - Institut National de Recherche en Informatique et en Automatique

INRIA - The French National Institute for Research in Computer Science and Control

Contact: M. Alain Boulze, Dr Philippe Merle
Contributors: The team

Overview

As an INRIA's ADT, galaxy is a collaborative technological development project whose contributors are mostly research project-teams, including ADAM, ASCOLA, OASIS, SARDES, SCORE, TRISKELL, and the TUVALU and SED development teams.

A collaborative project

As an open agile SOA framework, galaxy supports the full lifecycle of SOA and achieves the following goals:

1. galaxy provides SOA architects and developers with open-source software bricks which enhance a complete toolset for designing and monitoring business components and services, and deploying them to service dynamic platforms. These Java software bricks are compliant with the SOA technical standards (Eclipse, BPMN, BPEL, SCA, Grid…) and with technologies hosted by major open-source communities, OW2 for middleware and Eclipse for development environment.
2. galaxy enables an agile SOA approach, and facilitates the collaboration between the different roles and actors involved in the process: the "business" architect, who works close to the business user, the IT architect, in charge of the design of the application and technical architecture and of the deployment to the dynamic service runtime platform, the IT administration people in charge of monitoring the IT resources
3. galaxy offers a monitoring loop which collects information at service and process execution time, and treats and analyses them in order to obtain the best quality of service. Such information can be injected to the toolset used at design time, providing the system architect with events and defaults handled in real-time and useful for the next iterations of the service and process architecture lifecycle. Mechanisms for handling such events and defaults are designed also for interacting with dynamic service and process runtime platforms to enable a dynamic reconfiguration of service binding at runtime.
4. galaxy provides two kinds of open-source software components: Eclipse-based plug-ins and components for tooling the design, architecture and deployment activities; monitoring tools and components for creating monitoring probes, complex event processing, real-time visualizing and historicising events and defaults related to services and processes which are deployed.

Design, Deploy, Execute and Monitor

User documentation

This section explains how building, monitoring and deploying SOA applications with the galaxy platform.

NB: The following documentation contains many links to galaxy source code. To follow these links, you will have to ask an access to M. Alain Boulze or Dr Philippe Merle

Table of Contents

• Illustration by an example (tutorial)
• An application example : Home automation and home assistance to dependent people
  • Description
  • Architecture
  • Building and running the scenario
  • Technologies
• Deploying the galaxy platform
  • How to get the software?
  • Deploying galaxy platform components
• Using galaxy components to build SOA applications
  • Application modeling with Eclipse STP
  • Building application components
  • Building orchestration components
  • Building application front-end
• Using galaxy to deploy/run SOA applications
  • Deploying application components with FDF
• Using galaxy components to monitor SOA applications
  • Complex Event Processing with WildCAT/ESPER
  • Event History with MonFW
  • Visualization
  • Analysis
  • Instrumenting runtimes
  • Writing reconfiguration code

An illustration by the example, and a quick and simple tutorial

How to build the "smart travel" system.

Let's start by a very simple example. We want to build a web application, to be used by American people who want to travel in France, and want to get some useful and practical advice for their travel. More specifically, they want to learn some simple sentences in French to be able to ask people for the best place for dinner,depending on some local conditions (which location in France, what's the current weather, does the currency conversion between US and France favor a simple, i.e. quite cheap, or a most elaborate dinner, i.e. less cheap, …). The IT provider which will expose this service is convinced with SOA, thanks to the capabilities of reusability of business components and services, and agility of the full architecture lifecycle. It means that this application will be designed using service, composition and orchestration paradigms.

• Step 1 => design the business architecture of the application

Smart Travel BPMN process
(click on picture to enlarge it)

• Step 2 => design the functional architecture of the system

BPMN2SCA with STP-IM
(click on picture to enlarge it)

• Step 3 => refining and defining the technical architecture of the system, and deploying

Application deployment with FDF
(click on picture to enlarge it)

Monitoring intent on SCA components
(click on picture to enlarge it)

Monitoring data in editors and web console
(click on picture to enlarge it)

• Step 4 => enhancing the technical architecture (binding to a new service) and deploying again

Alerts & Reconfiguration
(click on picture to enlarge it)

• Step 5 => enhancing the technical architecture (deployment and runtime), and deploying again

Deploying from Eclipse
(click on picture to enlarge it)

IC2D
(click on picture to enlarge it)

• Demo => a short video
  

An application example

"Home automation and home assistance to dependent people"

This scenario has been first designed and developed by the INRIA TRISKELL team to show the benefit of innovative software technologies (used in critical and complex industries such as aerospace for example) and of software code generation in the context of costumer applications. First demonstrators have been shown during the Research Exhibition, held during June 2009, illustrating how to build variable and evolutive software.

Description

This scenario stages assistance actions for dependent people who are supposed to live in an innovative "intelligent and automated house".

Home automation context

Such an house is equipped with many sensors and actuators for lighting, actioning shutters, water leaks detection, detecting a person falling, or the presence of a new person, etc… All these actions will be able to be identified by the information system as many events and contexts, which will trigger as many actions in the information system (or adaptations of the information system).

A touch screen with a graphical user interface allows users to manage easily at home these home equipments. Figure 1 shows the main user interface window. Figure 2 shows a sample of how to switch on/off lights of several rooms in the house.

Figure 1 : Main window
(click on picture to enlarge it)

Figure 2 : Lights
(click on picture to enlarge it)

Some other services are also proposed, such as weather forecast and graphical map. These services are described respectively in figures 3 and 4.

Figure 3 : Weather service
(click on picture to enlarge it)

Figure 4 : Map service
(click on picture to enlarge it)

Home assistance to dependent people

The scenario shows how the galaxy platform can be used for supporting home assistance to dependent people, and helping to facilitate the life of such people. These persons may need medical assistance regularly, so they may be visited by nurses and/or doctors. Generally speaking, such an assistance involves numerous roles and actors, operating from different locations: patients with different pathologies, nurses, doctors, assistance staff and supervisor, maintenance and expert teams, … and needing different kinds of collaboration and information sharing, according to the actual context. The main idea of the scenario is to provide a way to deploy new services for this medical staff and to adapt the user interface (available on the touch screen located at patient's home) according to the context (and the relevant events). For instance, if the nurse comes to patient home for an examination, she may need a service to send an electronic report of the patient health status. This service has to be deployed and made accessible from the graphical user interface automatically. The idea is to detect the external staff profile, automatically deploy the new needed services and finally update the graphical user interface to make this new service accessible.

Figures 5 and 6 show the user interface updated with the nurse service.

Figure 5 : Nurse service
(click on picture to enlarge it)

Figure 6 : Patient report service
(click on picture to enlarge it)

Architecture

Business layer

Figure 7 shows the business process that describes the scenario.

Figure 7 : Business process
(click on picture to enlarge it)

The main requirements are :

• When a patient fall is detected, the system must alert/inform the nurse by SMS to initiate a visit for this patient.
• After patient examination, the nurse should be able to send an electronic report for the patient health status.
• If the patient needs a doctor examination, the nurse requests the system to schedule a doctor visit. The system should then update the doctor's agenda and send him (or her) an information SMS about the added meeting.

IT layer

From the business process described above, several components are needed to implement these requirements. Services such as SMS sending, weather forecast, map, etc are implemented with dedicated components(*). The service coordination is then performed by an orchestration component that references other components. Figure 8 describes the SCA architecture used for this scenario.

Figure 8 : SCA architecture
(click on picture to enlarge it)

(*) To keep the figure simple, these components are grouped into 'Services' component.

As described in section 'Description', the user interface should be updated to make new services accessible to the external staff (nurse, doctor, etc). Behind the scene, the system is dynamically reconfigured to add this new service to the graphical user interface as shown in figure 9.

Figure 9 : UI reconfiguration
(click on picture to enlarge it)

Infrastructure layer

The architecture described in previous section is implemented mainly in java. Weather and map services are web services. The application uses several devices (such as RFID tag to detect external staff profiles, SunSpots to detect patient falls, KNX devices to use/manage home equipments) that have to be configured to be able to run the demo. To keep the scenario simple to run, we provide mock services for these devices to simulate their behavior.

All services used in this application are monitored to keep track of their QoS criteria. This monitoring log may be useful for the system expert to fix technical issues and to know exactly which service(s) causes the problem by reading this log in the monitoring console as described here. Figure 10 show QoS log service which is added to the user interface if the system expert comes to the patient home to fix a problem.

Figure 10 : Expert service
(click on picture to enlarge it)

For more information about different technologies used in Infrastructure layer, please refer to section Technologies .

Building and running the demo

• Build the demo : First you have to checkout the demo source code here. Then, simply run mvn install from the root directory to build the demo.
• Run the demo : run the following commands
  cd demo
mvn -Prun.
  If everything is OK, the main user interface window should appear.

Technologies

Figure 11 is a SCA diagram that describes galaxy technologies used to run this scenario.

Figure 11 : Technologies
(click on picture to enlarge it)

Deploying the galaxy platform

How to get the software?

According to your needs, you may get only some of the following components composing the galaxy platform:

• Design components:
  • Eclipse STP: available with Eclipse updates in the SOA Development category
• Deployment components:
  • FDF: on the INRIA forge
• Runtime components
  • ProActive: on the Activeeon web site (registration needed)
  • FraSCAti: on the OW2 forge, of libs from OW2 maven repository
  • Orchestra: on the OW2 forge
• Monitoring components
  • IC2D: on the Activeeon web site (registration needed)
  • WildCat: on the OW2 forge, of libs from OW2 maven repository (licence notice: WildCAT requires ESPER, which is under dual commercial/GPL licence)
  • MonFW libs: on Tuvalu Maven repository and on the galaxy Hudson server
  • Monitoring console: on the galaxy Hudson server, or from source code, see below.

Deploying galaxy platform components

Requirements

Nearly all galaxy components require Java 6 (except FDF which requires Java 5).
Monitoring console requires a web application container such as Tomcat.
MonFW libs requires a database such as HSQL or MySQL.

Installation/Deployment of design components

Eclipse platform is an archive; to install it, just unzip the ZIP file downloaded from Eclipse Web Site (Eclipse Modeling Tools recommanded). Then to install STP, use the Eclipse Update Site mechanism.

Installation/Deployment of runtimes

Deploying FraSCAti
OW2 FraSCAti can be easily installed by following instructions at http://frascati.ow2.org/doc/1.1.1/ch01.html. This process can be automated with the help of DeployWare/FDF. A FraSCAti FDF personality is available (requires the unreleased 2.2-SNAPSHOT version of FDF, available by downloading the source code: svn checkout svn://scm.gforge.inria.fr/svn/fdf).

Here's an example of how using such a personality in your application deployment description file. This example has a dependency tree which looks like the following picture:

OW2 FraSCAti FDF personality
(click on picture to enlarge it)

To use this example, you will need on the target host(s) a way to connect the target (ssh deamon and a RSA key for example, see FDF documentation for more information).

Deploying ProActive Programming
ProActive Programming requires to be installed on each host on which the GCM/ProActive application must be deployed. There are two ways to install ProActive: manually or using DeployWare/FDF. Both methods require of course at first to download the latest ProActive Programming release on the Activeeon web site.

To install ProActive manually, one should follow the installation instructions provided on the ProActive web site. In two words, installation steps mainly consist to:

• uncompress the archive
• set the JAVA_HOME system property to the current Java home directory (at least JDK 1.5 or later).
• set the PROACTIVE_HOME system property to the new ProActive home directory.

As installing ProActive on many hosts may be tiresome, one could also use DeployWare/FDF to automate installation. To do so, a ProActive FDF personality is available on the unreleased 2.2-SNAPSHOT version of FDF (available by downloading the source code: svn checkout svn://scm.gforge.inria.fr/svn/fdf).

Here's an example of how using such a personality in your ProActive deployment description file. This example deploys both ProActive and a ProActive application. But it can be easily adapted to deploy only ProActive in order to obtain a dependency tree which looks like the following picture:

ProActive Programming FDF personality
(click on picture to enlarge it)

To use such an example, you will need on the target host(s) a way to connect the target (ssh deamon and a RSA key for example, see FDF documentation for more information), and on the source host:

• an archive containing a Java distribution (at least Java5)
• the archive of ProActive Programming

Deploying Orchestra
Orchestra distribution can be downloaded from here with its user guide.

Once Orchestra is started, it's possible to deploy a new process with ant task:

The Orchestra package contains many examples which can be executed on orchestra engine. Each sample contains a build.xml file providing targets to deploy, launch and undeploy the sample.

Example of a build.xml file

Installation/Deployment of monitoring components

galaxy comes with DeployWare/FDF personalities to help monitoring components deployment.
These personalities requires the unreleased 2.2-SNAPSHOT version of FDF, available by downloading the source code: svn checkout svn://scm.gforge.inria.fr/svn/fdf.

Deploying the MonFW and console
The FDF personality for MonFW and console is available on the galaxy forge. It contains the FDF source code to be compiled with the FDF sources to create the FDF-MONFW-2.2-SNAPSHOT.jar file to add in the FDF lib folder.

With this lib, you may use FDF to deploy both MonFW database (requires MySQL), and/or the monitoring console (requires Tomcat).

Here's an example of how using such a personality in your application deployment description file. This example has a dependency tree which looks like the following picture:

MonFW FDF personality
(click on picture to enlarge it)

To use this example, you will need on the target host(s) a way to connect the target (ssh deamon and a RSA key for example, see FDF documentation for more information), and on the source host:

• prerequisite software for Monitoring FW and console: mysql-4.1.22.tar.gz, jre1.6.0_17.tar.gz, and apache-tomcat-6.0.20.tar.gz (all available here)
• for MonFW, DB initialization files, available here
• for Monitoring console, the console webapp (available on the galaxy continuous integration server or you may build by yourself), and a small application that dynamically updates DB IP adress when deploying the webapp WAR (also available in a Maven repository for binaries or from source code).

Deploying WildCAT
The FDF personality for WildCAT is available on the galaxy forge. It contains the FDF source code to be compiled with the FDF sources to create the FDF-WILDCAT-2.2-SNAPSHOT.jar file to add in the FDF lib folder.

As for MonFW and console, here are some examples showing how to use the WildCAT FDF personality. Such a personality allows you to deploy the WildCAT lib in target host(s).

Once wildcat library deployed, the developer can add it to the application classpath to be able to use monitoring aspects implemented with wildcat sensors.

Installation of IC2D
Contrary to ProActive, IC2D does not need to be installed on several hosts, but only on the host where it will be launched (ie. on the monitoring host). Also, the installation is quite simple: download on the Activeeon web site the release version corresponding to the architecture of the monitoring host and uncompress the archive. The documentation is available on the ProActive web site.

Using galaxy components to build SOA applications

Application modeling with Eclipse STP

The tooling part of the galaxy platform leverages the Eclipse environment and uses as well as enhances existing Eclipse SOA editors. These SOA editors, mainly the BPMN and SCA editors, are part of the Eclipse SOA top-level project (TLP), which is a direct evolution of the Eclipse STP TLP. The SOA TLP is a recent addition to the Eclipse Foundation projects and extends the STP TLP by adding runtime-related projects to the already existing set of editors and editor-related functionality.

The Mangrove SOA Modeling Framework project, part of the SOA TLP, is used to "glue" together different editors and runtimes. In galaxy, it is used to transport design information between BPMN and SCA editors, as well as to transport monitoring information from the galaxy monitoring framework into these editors. It is the descendant of the STP Intermediate Model component to which it adds more complete functionality for bridging editors and runtimes.

The video here illustrates transformation capabilities between BPMN and SCA using Mangrove.

Building application components

Application (business) components are SCA components. So, you have to know a bit about SCA. A gentle introduction is available here.

Once you have an idea of core SCA concepts, you can jump into OW2 FraSCAti documentation to know how to write your own SCA components. First steps are described here. Then, to allow remote access to your components, take a look at distributed SCA applications.

Building orchestration components

Orchestration components are GCM/BPEL components. So the first thing to do is to be a bit familiar with ProActive and more specifically GCM/ProActive. The whole documentation of ProActive, which contains concepts, user manual and tutorials, is available here. The GCM/ProActive documentation is available here.

The idea we push forward, is to embed in a GCM composite component one functional component that would be in charge of orchestrating the service composition, acting as a workflow interpreter. This interpreter is in charge of enacting the service graph corresponding to the composite service. In this section we introduce the general architecture we adopted to map and execute a workflow in a flexible way.

General Approach

We rely on the GCM, making our model distributable, evolvable and hierarchical. First, we represent each involved orchestrated service by a component. The set of services is then orchestrated thanks to a dedicated component, the Workflow Manager, in charge of interpreting and enacting the orchestration. This component is to be considered as the central point in the composition, in broad terms as the nervous system of the orchestration from which all necessary service orchestration is dictated. The workflow embedded in a GCM component, characterized by an ADL and a service graph thus becomes an entity that is able to execute and reconfigure itself, benefiting from non-functional features like distribution, reconfiguration, or security.

We focus now on the workflow engine interpreting the service graph. An instance of such an engine is illustrated by the following figure :

The GCM Component corresponding to the ABC orchestration.
(click on picture to enlarge it)

The component interprets the following service graph:

The ABC Orchestration sample graph.
(click on picture to enlarge it)

The composition orchestrates 3 services, A, B and C and implements a function f. The idea is to put in each composite component, one specific component Workflow Manager in charge of interpreting the service graph. In this simple form, the composite would contain exactly 3 components A, B, C besides the Workflow Manager.

As exemplified the graph is mapped on a GCM component containing two kinds of component:

1. The workflow engine interpreting the graph
2. The service invokers and adaptors, acting as proxies towards the involved services (i.e. A, B, and C). These two kinds of components are in the next sections. On the figure graph, A, B and C are linked together according to a composition in time: the interpretation engine has to invoke A then B and C. This orchestration is translated in the component view by the introduction of a workflow interpreter component (the so-called Workflow Manager component); in the GCM view, components A, B and C are not directly bound together according to the composition in space vision, but indirectly through the Workflow Manager (following the composition in time vision).

The ABC Orchestration component offers a non-functional controller allowing the management of the orchestration lifecycle thanks to actions such as deploy, stop, or suspend, a service graph execution:

 public interface OrchestrationController {
    /** Stops the process **/
    public void stop ();
    /** Starts the process **/
    public void start();
    /** Suspends the process **/
    public void suspend ();
    /** Deploys the orchestration on the Workflow Engine **/
    public void deploy (Object workflow);
 }

When the deploy() method of the OrchestrationController interface is invoked, the workflow definition is effectively deployed on the workflow engine and a Service_Invoker_Adaptor component is created for each involved service. The workflow is enacted using the function f(), which is the functional and business method of the interface of the component, it is stopped using the stop() method.

The Workflow Manager Component

The following figure shows the elements composing the Workflow Manager component.

The Workflow Manager Composite Component.
(click on picture to enlarge it)

It is composed of two elements, the Process Manager and the Engine. The Process Manager component is in charge of processing the workflow to create the corresponding service adaptors, and of effectively controlling the workflow deployment and execution. The Engine Component is in charge of effectively enacting the workflow, and enables actions such as start, stop, or suspend to be performed. It is the component and not the workflow engine that interacts effectively with the involved services.

The implementation of the interpretation engine can be either a dedicated and embedded workflow engine, or, a component that delegates the workflow interpretation to an external engine (as we successfully did with the Active BPEL engine for instance by simply invoking web services exposed by the Active BPEL engine).

Service Invoker and Adaptor

One of the main advantages of using a hierarchical and reconfigurable component model is that intermediate components can be introduced easily, and added or reconfigured at runtime. These components can be simple proxies, bound to external services,

A Simple Invoker and Adaptor.
(click on picture to enlarge it)

as well as more sophisticated adapters collaborating to successfully achieve the service invocation:

Example of a complex service Invoker and Adaptor.
(click on picture to enlarge it)

The resulting component corresponding to the ABC orchestration is given by the following figure:

Architecture of the GCM component embedding a workflow.
(click on picture to enlarge it)

A complete example of such an orchestration component is available on the galaxy forge.

Building application front-end

The easiest way to build a front-end (such as a java servlet in this example) for a galaxy SCA application (or to integrate any galaxy SCA application in a non-SCA world) is to expose SCA applications as Web Services.

The way to do this depends on the runtime the SCA application is running on. For example, you can find more detailled explanations in FraSCAti online documentation or in the GCM/ProActive manual.

Once the SCA application is exposed as WS, you have to write the servlet. Here's a servlet code template, with its associated web.xml, index.html and result.jsp files, that we used in the JavaOne galaxy demonstrator. If you use Maven to compile your code, this POM file example may be useful too.

Using galaxy to deploy/run SOA applications

Deploying application components with FDF

For FraSCAti SCA application components
In order to be able to deploy your SCA application with FDF, you have to package it in a Zip archive. This archive must contains:

• a top-level directory (ex: myapp) with all files related to the application inside,
• all SCA composite files to be loaded,
• all your business code packaged in jar files AND its dependencies if any.

Then, you can use the FRASCATI.EXPLORER personality to deploy your SCA application and run the OW2 FraSCAti Explorer (GUI allowing you to dynamically introspect running SCA applications and reconfigure it). You just have to add a few lines to your frascati deployment file:

galaxy-demo = FRASCATI.EXPLORER {
  frascati = /frascati-1;
  archive = FRASCATI.COMPOSITE.ARCHIVE(/path/to/your/app.zip);
}

and to start the deployment with FDF.

For GCM/ProActive application components
As the ProActive platform, a GCM/ProActive application requires to be installed on each host on which it must be deployed. By using DeployWare/FDF, this process is made easier.

To deploy a GCM/ProActive application with FDF, the first step is to package the application in an archive (for example a Zip archive). This archive must contain a top-level directory (ex: myapp) with all files related to the application inside:

• all Fractal files to be loaded,
• all the business code already compiled (classes or jar) AND its dependencies if any,
• the GCM deployment files,
• a SHELL script named deploy.sh which is in charge to launch the GCM deployment and start the application.

Then the ProActive FDF personality available on the unreleased 2.2-SNAPSHOT version of FDF (available by downloading the source code: svn checkout svn://scm.gforge.inria.fr/svn/fdf) has to be used to deploy and launch the GCM/ProActive application through FDF.

Here's an example containing all needed files of how using FDF to deploy and start a GCM/ProActive application.

Using galaxy components to monitor SOA applications

The following picture represents main monitoring components :

Monitoring Global Picture
(click on picture to enlarge it)

Core of monitoring is composed of WildCAT/ESPER (for sensor hierarchy and Complex Event Processing), and MonFW (for event storing and basic event processing).

Monitoring data comes from sensors attached to runtime (see Instrumenting runtime above). It is then processed, transformed and stored by WildCAT/ESPER and MonFW. Result of these computations is used in visualization, analysis and reconfiguration modules.
In this chapter, we will describe new FDF personalities too, helping to the deployment of some monitoring components.

Complex Event Processing with WildCAT/ESPER

WildCAT/ESPER is a central component of the galaxy monitoring. WildCAT/ESPER is a generic framework for context-aware applications:

• the WildCAT front-end provides user with easy access to sensors through a hierarchical organization,
• and the ESPER back-end provides user a power SQL-like language to express conditions upon which to trigger adaptation code.

In galaxy, WildCAT is used to collect and aggregate data coming from sensors (such as service response time or process/activities start/stop), and thanks to ESPER CEP, compute more complex monitoring events (such as timeout leading to raise alert and reconfigure the application in real-time).
WildCAT also sends all low (or more complex) events to the MonFW (through WildCATAdapter component) for more long-term logging and event processing (see picture bellow).

WildCAT/ESPER in galaxy
(click on picture to enlarge it)

One may find find more information about using WildCAT/ESPER on the WildCAT homepage : creating hierarchical data source organization, using PUSH/PULL modes of operation, writting EQL queries, adding/managing sensors, etc.
WildCAT is downloadable on OW2 forge.

Event History with MonFW

The MonFW is a central component of the galaxy monitoring. It aims to log monitoring events, and provide a basic event processing on these events for log-term monitoring.
The following diagram gives a global picture of the MonFW component:

MonFW in galaxy
(click on picture to enlarge it)

On this picture, we distinguish 3 main parts in the MonFW:

• The WildCAT Adapter, transforming monitoring events sent by WildCAT (or any other component handling monitoring events from sensors) into an internal event format. There is no processing nor event-filtering (which should be done by the event sender) here.
• A Data Access layer, offering a Java API to retrieve monitoring events. Listeners use pulling to get last monitoring events, but a publish/subscribe mode is available too for some kind of monitoring events (such as alerts).
• Last but not least, a DB storing monitoring events, and allowing:
  • normalized, scalable and concurrent store for monitoring events
  • long-term monitoring requests (that would not be possible in many CEP because of scalability matters)
  • complex monitoring event building, by mixing long-term events stored in the DB with other monitoring events coming from WildCAT/ESPER.

So WildCAT Adapter and Data Access layers are Java interfaces to the DB, kind of O/R mappers, offering:

• a structured monitoring event hierarchy for event producers,
• a basic and scalable event processing for event consumers.

The DB storing monitoring events is drawn from some BI concepts and rules, and offer 3 kinds of tables: services for response times or SLO/SLA, process for BAM, and alerts (see picture bellow). Both services and process tables have a strong semantic associated to events (separating facts and dimensions in different tables), but the semantic is lighter for alerts (the application is responsible to know what data these events are carrying).

MonFW internal DB schema
(click on picture to enlarge it)

The format of events sent to the MonFW is defined by this one, but try to stay simple (for example needing just response times for services), or relying on standards (such as EVO for BAM events, see the following picture).

EVO (Event Ontology)
(click on picture to enlarge it)

MonFW core libs are available on a Maven reprository (and sources in INRIA forge). Then it depends you are writing an event producer or consumer (some samples may be found in the INRIA galaxy forge).

Installing a database

The MonFW currently supports HSQL and MySQL databases.
Once you have choosen your DB technology, you need to create a DB inside. Then the MonFW offers you three ways to initialize your DB:

• with Java

• with Maven

• with SQL : you have to extract and execute the SQL script that is inside the last monfw-db-1.1*.jar lib of the MonFW.

Writing an event consumer

First, you should add the following dependency in your Maven project:

Then you may create (or use any existing) accessor to the data access layer API. This sample java file and its associated DB connection file may help you.

The current Data Access layer implementation offers the following API:

Writing an event producer

First, you should add the following dependency in your Maven project:

Then you may create (or use any existing) accessor to server-adapter API. This sample java file and its associated DB connection file may help you.
In this class you may overload the void eventArrived(MonitoringEvent event) method to handle your own monitoring events (but notice that if you want to manage attributes that does not currently exist in the DB schema, you will have to modify the MonFW itself).
Currently existing events are summed up in the following picture:

Monitoring Event Hierarchy
(click on picture to enlarge it)

Updating MonFW instance.

When writing an event consumer(/producer), you may want to work with another MonFW instance reading(/storing) events on another databse. To do this, just modify your Maven dependency:

Then you may access the following API:

Visualization

The web console is a web graphical user interface allowing everyone to monitor one or more services thanks to the information moved up by the monitoring framework.

Console overview

A web based console allows an access from every computer with an internet access, and doesn’t depend on an operating system.
The web console is based on Ext-GWT (aka GXT) which is a Java library for building rich internet applications with the Google Web Toolkit (GWT). It supports all major web browsers (Internet Explorer, Firefox, Safari and Opera).

The web console gives service (or process/activities) details (static and dynamic information) as shown in the following picture:

Monitoring services with console
(click on picture to enlarge it)

Developing a custom monitoring console

Console components (widgets) are available in the INRIA forge (source code) and in a Maven repository (binaries), and contains:

• monfw-console-widgets-gxt1 : shared monitoring console code, based on GXT 1.2.4.
• monfw-console-widgets-gxt2 : deprecated code, no more used (based on GXT 2.0.1).

galaxy project offers a way to assemble those widgets into a full service monitoring console (see source code in INRIA forge).

From this source code, use mvn eclipse:eclipse to create an Eclipse project, then mvn clean install to build the console webapp (WAR file will be generated in the target folder).
The console needs to retrieve data from a database, so here you have to start the MonFW database, as for example:

cd x:/<hsqldb_install_folder>/lib
java -cp hsqldb.jar org.hsqldb.Server -database.0 file:galaxydb -dbname.0 galaxydb

Finally, to execute the console in hosted mode (development phase), use mvn gwt:run -DrunTarget=MonitorModule/MonitorModule.html .

Widgets are assembled through "pages", allowing to customized widget positions and the management of a navigation history (back/next button of the web browser).
There are to ways to assemble widget:

• the static one: see examples
• the dynamic one (through portlets) : see Widget2Portlet java class.

Collecting monitoring data from MonFW

There is 2 ways to collect monitoring data: Remote Procedure Call (RPC) and Comet.

Remote Procedure Call (RPC)
GWT provides an RPC mechanism based on Java Servlets to provide access to server side resources. This mechanism includes generation of efficent client and server side code to serialize objects across the network using deferred binding.
With RPC, data comes in one block after an explicit asynchrone call. Here are some examples of servlets implementing RPC functions in the console :

• MonitoringServiceImpl : grouping RPC functions, requesting MonFW data access layer for services (Also check IMonitoringService, MonitoringService, IMonitoringServiceAsync and MonitoringServiceAsync)
• EventServiceImpl :manage data updates, beforing sending to clients with Comet (see below)

Comet
Comet allows retreiving data flow after subscribing a topic, containing different kind of data. Data are then pushed by a server without requesting client polling
We use the gwteventservice library, allowing to use Comet with GWT.
2 important notes about Comet:

• ServiceStateCometHelper class makes easier to handle client connections, assuring not more than one subscription to a given service.
• Comet opens an HTTP connection (and permanently keeps it opened), and the browser generaly doesn’t allow more than two permanent connections to a given domain. It is important that other client libraries do not ask for opening another parallel client connection.

Analysis

Monitoring data may be visualized, or used as input for analysis tools.
In galaxy project, monitoring data is used to bring back runtime service/process data into design editors, and in BPEL2EC tool (not available yet).

Enhanced design editors with MonFW2Editors

The MonFW2Editors is a feature which allows to move up information from the monitoring framework to eclipse editors.

The Eclipse STP project provides several modelers which facilitate the design of SOA applications. In the galaxy project, we manage to leverage these editors by adding QoS information collected by the monitoring, in the modelers themselves. There are two main steps to achieve such a goal:

• Extend the targeted editor to add the graphical container which will display the collected information, by creating an eclipse plug in
• Collect the monitoring information by creating an eclipse plug in as well
  

Code available at source code: svn checkout svn://scm.gforge.inria.fr/svn/galaxy/MonFw2Editors.

The picture below is an overview of the transformation process:

Moving up QoS information to eclipse SCA editor
(click on picture to enlarge it)

BPEL2EC

Not available yet.

Instrumenting runtimes

Generating FraSCAti intents

The intent wizard is an eclipse plugin that allows you to generate skeletons for FraSCAti intents. If you are not familiar with FraSCAti intents, you can find more details here.
Some service QoS policies (such as service repsonse time, service timeout and service availability) were already developed and can be directly used within a SCA application and easily integrated with galaxy monitoring framework.
You can find the source code of this plugin on the galaxy forge and a short screencast explaining how to use it here.

GCM/ProActive monitoring

ProActive provides convenient interfaces to add non-functional policies to components in an aspect oriented fashion thanks to component controllers and interceptors (mainly in org.objectweb.proactive.core.component.interception and org.objectweb.proactive.core.component.control packages). Galaxy uses this mechanism to monitor QoS criteria of components deployed in ProActive runtime.
To add your custom controller/interceptor to GCM/ProActive component:

• Define your new controller type by extending the ProActive abstract controller org.objectweb.proactive.core.component.control.AbstractPAController
• Implement the interceptor interface that corresponds to your needs in an AOP style
• Declare your new custom controller in ProActive xml configuration file ('controllers' element).

You can find an example of QoS policy (response time) added to ProActive in galaxy here.
In this example, the new response time aspect acts as a Wildcat sensor that notifies response time of services provided by GCM/ProActive components for each invocation. Such monitoring events are then recorded to the monitoring framework database and visualized in the monitoring console.

Weaving monitoring aspects to Orchestra core engine

Galaxy provides monitoring aspects written in AspectJ to monitor the execution of bpel processes deployed in OW Orchestra runtime. These aspects can notify life cycle events such as starting/stopping processes and activities instances. Thanks to adapters and interfaces provided by the galaxy monitoring framework, these monitoring aspects can be easily integrated with it to store such events in the database and visualize them in the monitoring console.

To add these monitoring aspects to the orchestra core component :

• Checkout the project source code from galaxy forge
• run the following command to weave monitoring aspects to orchestra core and package dependencies :

mvn package

• rename the generated jar file to "orchestra-core-xxx" where xxx is the version of orchestra core dependency
• copy this renamed file and all the jars included in the dependencies-bin.zip to your orchestra installation lib directory

(For example, if you are using orchestra-tomcat installation, you have to copy these jar files to "/tomcat/webapps/orchestra/WEB-INF/lib")

Monitoring SUPER bus or other 'EVO compliant' events

As seen before, the MonFW handles BAM events, with a Java class hierarchy that partially maps EVO standard (see picture).

EVO (Event Ontology)
(click on picture to enlarge it)

By converting your EVO events (such as ones sent by the SUPER bus), you may store them in the MonFW DB. Here's a sample code and its associated Maven POM file of the SOA4All project: the processWSMLTopEntity method converts a EVO WSMO top entity into a MonFW compliant event. The main method also shows an example of listening SUPER events sent through a JMS connection.

Monitoring PEtALS bus or other 'WSDM compliant' events

WSDM uses Web services as a platform to provide essential distributed computing functionality, interoperability, loose coupling, and implementation independence. The OASIS WSDM working group has defined two specifications which we will use to provide efficient monitoring and management APIs, namely MUWS (Management Using Web services) and MOWS (Management of Web services).

In the SOA4All project, MonFW is used to monitor PEtALS bus, which computes WSDM events. The source code is not available yet (soa4all-dsb-monitoring module), but the idea is to create a WSDMServiceEvent class instance, and send it to the MonFW.

Writing reconfiguration code

OW2 FraSCAti provides reconfiguration ability at runtime. All information dedicated to this topic can be found in the OW2 FraSCAti documentation.