What is the Notification Service?
OpenFusion Notification is the Micro Focus implementation of the OMG's Notification Service Specification, version 1.0.1.
The OMG Notification Service is itself an extension of the earlier OMG Event Service which defined a mechanism that allowed objects to communicate in a decoupled manner. The Event Service defines the notion of suppliers and consumers of events who are connected via a channel.
The Notification Service extends the Event Service in a number of important ways:
- It enables events to be transmitted in a well defined structure in addition to the generic and typed events supported by the Event Service.
- It enables clients to specify exactly the events they are interested in by attaching filters to channels.
- It defines a number of qualities of service that can be configured by clients to tailor the operation of the service.
- It defines an optional event type repository which facilitates the formation of filter constraints by end-users.
When should I use OpenFusion Notification?
Use OpenFusion Notification when you have to solve any of the following problems:
- You want to exploit the flexibility of the message oriented paradigm in a standards-compliant way
- Your applications must handle thousands of events per second between multiple suppliers and consumers
- Your applications need guaranteed once and only once delivery of events
- You need to route messages based on their content
- You need sophisticated control of your messaging software
- Your applications are deployed in diverse hardware and software environments.
OpenFusion Notification can be used in a huge range of applications across all industry sectors. For example, in Telecoms, a common application of OpenFusion Notification is in network management. Alarms and other information produced by switches are sent as messages to alarm management applications and other interested users. Neither the switch manager nor the receiving applications have any direct connection to each other, so they can concentrate on their core tasks without worrying about alarm propagation, receipt or confirmation. OpenFusion Notification takes care of message delivery and propagation - a fire-and-forget strategy.
In Finance, when a stock feed application needs to connect individually to its client (who may not all even be on-line at any one time), real-time performance suffers and the activities of one client could have direct bearing on all the others. When OpenFusion Notification is used, this restriction is removed and performance is maximized.
What is the difference between the Notification Service and the Event Service?
The OMG Notification Service is a powerful mechanism for sending event notifications between applications in a distributed and heterogeneous environment. The Notification Service is an extension and replacement of the original Event Service. It provides backward compatibility with the Event Service and maintains an architecture that allows easy partitioning of an event system into logical groups that communicate using federated event channels.
The Notification Service overcomes a number of limitations with the Event Service. An important extension is the support for event filtering where both structured events and any type events can be filtered using a powerful generic constraint language. Although filters can apply to any part of an event, consumers will typically filter on the filterable body of structured events, which contains a sequence of name/value pairs.
In order to constrain the implementations of the Notification Service specification, a large number of configurable QoS properties are supported. These can be divided into reliability, queue management, event management and event batch management. The Notification Service supports coarse or fine-grained QoS settings as different QoS properties can be applied all the way from the channel level down to individual events.
The Notification Service also informs event suppliers and consumers about changes to the event types required or offered by a channel. The channel obtains information about the event types implicitly via filter objects or explicitly from the connected consumers and suppliers. Intelligent supplier and consumer can use this offer and subscription information to limit the amount of events. An optional part of the Notification Service specification is the event type repository. The repository contains event types and all the filterable properties required for each event type. The repository supports inheritance and import relationships between event types and all information in the repository can be queried and modified at run-time. The event type repository serves two distinct and important purposes. First of all, it is a powerful way of maintaining an evolving set of event types while a complex distributed system is being constructed. More importantly, for systems that evolve during their lifetime, the event repository is a mechanism for new event suppliers and consumers to add additional event types or obtain information about the properties of existing types.
In conclusion, the Notification Service is a powerful event mechanism for distributed systems that directly supports important properties such as flexibility, scalability, and extensibility. With support for different communication models, event filtering, a wide range of QoS properties and the event type repository, the Notification Service covers a very wide range of distributed application requirements - and makes it an obvious candidate for the event notification component of distributed systems.
What are the Quality of Service properties defined by the Notification Service?
Configurable QoS properties are an important feature of the Notification Service. An implementation of the Notification Service is not required to support all the different QoS properties described in the specification, but an implementation that does will cover a wide range of application requirements. There are six categories of QoS properties:
The Notification Service supports two aspects of reliability:
Event Reliability - Events can be delivered according to a best effort policy where no guarantees about delivery are made. Alternatively, guaranteed delivery is supported where events are be kept in a persistent store until they have been successfully delivered to all connected consumers.
Connection Reliability - Connections to event suppliers and consumers can have a reliability of best effort when the Notification Service regards connections as transient and failure to deliver or receive an event will (perhaps after a few attempts to re-connect) result in disconnection. Alternatively, connections can be made persistent, when a supplier or consumer will not be disconnected until the target object no longer exists.
B. QUEUE MANAGEMENT
The Notification Service supports a number of QoS properties related to queue management. Queue management can be based either on the current number of events in a queue or the current memory usage of the machine on which the Notification Service is running.
Maximum queue size - Rather than just leaving the problem of internal buffer overflow as an implementation detail, the Notification Service supports configuration of the maximum number of events that will be stored on behalf of consumers.
Delivery Policy - The delivery policy defines the order in which events are delivered to event consumers. The Notification Service supports a number of policies including FIFO, priority, and time out.
Discard Policy - If an event queue is exhausted, discard policies such as FIFO, LIFO, priority and time out are supported. Also, the Notification Service can be configured to discard new events.
MaxUnacknowledged - This affects DeliveryQueues. MaxUnacknowledged is the number of unacknowledged events after which event delivery to an attached listener should halt pending the next acknowledgment.
MaxMemoryUsage - This setting is used on Event Channels. MaxMemoryUsage is similar in purpose to the property MaxQueueLength, except the size of memory is controlled rather than the number of events. MaxMemoryUsage takes a value of type ulonglong which represents the number of bytes after which attempts should be made to limit memory usage using the current usage policy, which in turn is controlled using the property MaxMemoryUsagePolicy.
MaxMemoryUsagePolicy - This setting is used on Event Channels. MaxMemoryUsagePolicy is the policy by which memory usage is controlled when MaxMemoryUsage is exceeded. It can take one of three values:
- PurgeEvents - if this value is set, then whenever an event is received that takes memory usage above MaxMemoryUsage then that event will be added to the internal queue of the appropriate event channel as normal. Then, in a manner that mirrors discard behavior, the event at the back of the queue will have its data purged from memory. If the event has best-effort delivery semantics then it is effectively discarded and all its used memory should be available for recovery by the garbage collector. However, in the case of a persistent event a placeholder will remain in memory so that the data can be reloaded from persistent store when required. Hence, in the case of a persistent event not all of the used memory is freed so total memory usage will continue to increase. The rate of increase will be greatly reduced however, making this an appropriate policy for dealing with bursts of event delivery.
- DiscardEvents - if this value is set, whenever an event is received that takes memory usage above MaxMemoryUsage threshold, an event is discarded according to the current discard policy.
- RejectEvents - if this value is set then whenever an event is received that takes memory usage above MaxMemoryUsage threshold, an org.omg.CORBA.IMP_LIMIT exception is thrown.
C. EVENT MANAGEMENT
A number of QoS properties related to event management can be set. As mentioned in the previous section, these QoS properties should be set in the variable header of a structured event:
Start time - An absolute start time can be set for each event. An event will not be delivered to any consumer connected to an event channel until the start timer expires.
Time out - A relative time out value for an event can be specified. Alternatively, an absolute stop time can be used. Events will be removed from any delivery queue when the stop timer expires. When events are delivered according to the time out policy, events with the shortest relative time out value will be delivered first.
Priority - The priority of events can be specified. This QoS is used for queues with a priority delivery or discard policy.
D. BATCH HANDLING
In relation to sequence type consumers where events are delivered in batches, a few additional QoS properties are provided:
Maximum batch size - The maximum number of events that a sequence type consumer wishes to receive at a time.
Pacing interval - The maximum time a sequence type consumer will wait for events to be collected. If fewer events than the batch size are available when the pacing timer expires, the Notification Service just delivers whatever events are available.
E. THREAD MANAGEMENT
ThreadPolicy - This QoS can be set at the channel level in order to specify the threading policy that should be used when the Notification Service is delivering events to push consumers. The OpenFusion® Notification Service supports two settings, namely ChannelThreadPool and ThreadPerProxy. The ChannelThreadPool policy uses a thread pool per event channel. This should be used for large numbers of consumers in order to limit the number of concurrent threads that are executing within the Notification Service. The ThreadPerProxy policy creates a single thread per push consumer. This will give good performance for small numbers of consumers. The default policy is ChannelThreadPool.
ThreadPoolSize - This QoS determines the maximum thread pool size. Note that this QoS can only be used when the ThreadPolicy is set to ChannelThreadPool. This QoS property should be set when a large number of consumers are connected in order to limit the number of concurrent thread executing within the Notification Service. The default pool size is zero which means that one thread per proxy will be created.
ThreadIdleTime - This QoS sets the maximum time that a thread will be idle before terminating. This setting should be used in systems where events are sent in bursts. Using this QoS will cause the Notification Service to be responsive at peak times (by dynamically allocating threads up to the ThreadPoolSize) but at the same time release memory resources at times when the event rate is very low. The unit for this QoS is 100 nanoseconds and the default value is zero, i.e. threads will never time out.
As with filters, the QoS properties can be applied in the context of logical groups. As an example, the initial QoS properties of an event channel object will apply to all the administration objects that are subsequently created by the channel. However, fine-grained QoS is also supported, e.g. different delivery or discard policies can be specified for individual proxies.
Notification Service vendors can have additional QoS properties as value added features of an implementation. An important property of the Notification Service is that this can be done without introducing incompatibilities. The QoS interfaces have operations for negotiating and obtaining the QoS settings supported by a particular implementation.
F. BACKWARDS COMPATIBILITY
Two QoS properties have been added to enable clients to behave as specified in earlier versions of the OMG Notification Service Specification:
DisconnectCallback - This affects all proxies. If it is set to true (the default) then when a proxy's disconnect method is called, then the disconnect method on its connected client will also be called. If it is set to false, then a proxy's connected client will not have its disconnect operation invoked when that of the proxy is invoked.
AlwaysPull - This affects EventChannels. If it is set to true (the default), then events will be pulled from connected pull suppliers regardless of whether any consumers are currently connected. If it is set to false then no events are pulled unless there is at least one push consumer attached, or at least one pull consumer that is currently attempting to pull an event.
How do the Reliability QoS properties affect the event delivery?
The Notification Service treats the reliability of specific events, and the reliability of the connections which provide a transport for events between clients of the notification channel and the channel itself, as separate issues, and therefore defines two separate QoS properties to represent them: EventReliability andConnectionReliability.
Each of these properties can take on one of two possible numeric constant values: BestEffort or Persistent. The meanings associated with the settings of these properties are inter-related, and are defined below.
EventReliability=BestEffort & ConnectionReliability=BestEffort:
No specific delivery guarantees are made. In the presence of failures, the event may or may not be received by each of the consumers, and a given consumer may receive the same event multiple times.
EventReliability=BestEffort & ConnectionReliability=Persistent:
The notification channel will maintain all information about its connected clients persistently, implying that connections will not be lost (logically) upon failure of the process within which the notification channel is executing. Any clients who connect to the channel using persistent object references may fail, but unless these object references raise an OBJECT_NOT_EXIST exception, the channel will continue to retry using them. Clients which then re-instantiate objects with these references will (logically) reconnect to their associated proxies. The channel will not, however, store any buffered events persistently. The implication of this combination is that upon restart from a failure of the notification channel server process, the channel will automatically re-establish connections to each of its clients, but will not attempt to retransmit any events that had been buffered at the time the failure occurred.
EventReliability=Persistent & ConnectionReliability=BestEffort:
This combination has no meaning and need not be supported by a conformant Notification Service implementation.
EventReliability=Persisent & ConnectionReliability=Persistent:
Each event is guaranteed to be delivered to all consumers registered to receive it at the time the event was delivered to the channel, within expiry limits. If the connection between the channel and a consumer is lost for any reason, the channel will persistently store any events destined for that consumer until either each event times out due to expiry limits, or the consumer once again becomes available and the channel is subsequently able to deliver the events to all registered consumers. In addition, upon restart from a failure the notification channel will automatically reestablish connections to all clients that were connected to it at the time the failure occurred.
What is the Event Type Repository?
The event type repository contains meta data about event types. For each event type, the repository will contain information about the properties of an event. Because the repository was specifically designed to fulfill the requirement of verifying filter constraints, the repository only contains information about the properties in the filterable body of a structured event. The diagram below shows the UML model for the event type repository.
The repository is a singleton that supports a number of event domains and contains a number of event types. An event type in turn has a domain, a name and a number of properties. Event types can inherit or import the properties of other event types. The model in the diagram is mapped to IDL using the Meta Object Facility, i.e. the interfaces automatically have reflective capabilities.
Event type inheritance means that all the properties in the super type will also be present in the sub type. It is not possible to create an inheritance relationship between two event types if a property name is defined in both event types. Event types can also be imported. This does not create an inheritance relationship but all the properties of the imported type will be present in the importer type. Property names may overlap but only if the type associated with the property is the same in both the imported and importer event types.
An important property of the event type repository is the ability to modify the event types and the relationship between event types at run-time. This allows applications to evolve over time, e.g. an application can create a new event type with additional properties that inherits from an existing event type. New applications can take advantage of the additional information, while existing applications process the event according to the old set of properties.
As the event type repository is populated, it will contain the properties that are expected for the event types used in a distributed application. It is possible for clients to look up event types and investigate what properties are available for filtering. Thus, event suppliers can find out what properties are expected in the events they produce and consumers can use the repository to create meaningful constraint expressions for event filtering.
What are filters used for?
Filters are a means to reduce the number of events received by a consumer to only those that are of interest. For example, if a consumer has connected to an event channel that is forwarding alarm events, the consumer will receive every event that is published to the event channel unless it uses a filter to remove the unwanted events. The filter contains a constraint, which must be evaluated to TRUE for the event to be forwarded to the consumer. In the example of alarm events, the filter could be 'severity > 4'. This would filter out all events that have a severity property of 4 or less, leaving only the most severe alarm event messages to be forwarded to the consumer.
Filters rely upon properties in structured events for filtering. The format of a structured event is:
You can also filter events by advertising subscription types. This is like a primitive filter that only applies to the domain name and type name in the fixed header of structured events.
How do I specify a filter?
Filters can be attached to either proxy objects or administration objects. There are proxy objects and administration objects for both event suppliers and event consumers.
Filters can be attached to proxy consumers, proxy suppliers, supplier administration objects, and consumer administration objects. This provides a very flexible system for specifying filters for groups of consumers, or for individual consumers.
A filter contains a constraint, specified in the Extended Trader Constraint Language. The constraint is evaluated every time an event is received by the object that the filter is attached to e.g. a consumer proxy object. If the constraint evaluates as TRUE, the event is forwarded to the next object in the delivery chain. If the constraint evaluates as FALSE, the event is not forwarded.
More details of filtering and the constraint language can be found in the OpenFusion Developer's Guide.
Can I specify my own filter processors?
The OpenFusion implementation of the Notification Service allows you to implement your own specialized filter and mapping filter object that may, for instance, support a different constraint language. To provide a filter, you must provide an object that implements the Filter interface (or in case of the mapping filter, the MappingFilter interface). Because the functionality for managing constraints is generic, you may wish to extend the OpenFusion filter implementation and only re-implement the match operations. The OpenFusion filter implementation is located in a Java™ class called com.prismt.cos.CosNotification.FilterImpl.
Visit OpenFusion Notification Service web page