RenderX XEP EnMasse User Guide

Competing programs on the same computer fight for resources, and take more time to return a result.

More memory and newer processors help applications run faster, but there are often memory limits. Processors become faster, but their speed is limited; multiple-processor configurations are expensive, and performance does not rise in line with the price and the number of processors.

A solution is to distribute processing among multiple computers. If you have as many computers as there are programs, each program runs as fast as the computer allows it and uses as much memory as you can install. Using multiple computers, programs perform as well as the current level of technology permits, and can achieve a data throughput as high as you can afford; and the price for stock single-CPU PCs is low enough to have sufficient resources to meet your business needs.

If these are connected so that they work all the time, the load is evenly distributed, and even when nodes fail, or need upgrading and are removed, the system still processes all your requests, preserves your data, and the only change noticed is a slight decrease in performance. This is a well-known but not an easy task; and to really use the joint power of grid computing as it is known, you must resolve this issue.

XEP creates elegantly formatted print-ready views of documents. In many deployments, it produces hundreds of thousands of documents on demand. For example:

These are just a few examples. However fast and optimized the program's code, business needs a way to scale up performance so that it handles a growing load.

EnMasse solves this problem for you. For a user or an application programmer, it is a single access point. Through a shared folder, a web form or a network connection (locally on your network, or across the Internet), EnMasse accepts document processing requests and sends them to the XEP formatting engines running on several computers in the local network, then delivers formatted documents back to the user. It monitors the formatting engine performance, notices when they go down and are restored and readjusts the distribution of requests according to the workload. When a server fails in the midst of processing a request, EnMasse re-submits the request to a different server; thus the only impact is a slightly increased response time for that particular document. This provides the security of service needed in today's world.

EnMasse is both opaque and transparent. On one hand, it provides you with single point abstraction such that there is no need to worry about the number of engines running, or about their load; EnMasse dispatches requests to the most appropriate node. On the other hand, both the access point and the processing engines have standard, embedded servers. The system administrator can instantly check the status of the grid, identify problems and take appropriate action. An EnMasse access point takes little memory overhead and processing time, it can be deployed on a loaded intranet server, or even on a workstation and as long as the processing engines run on separate machines, it does not affect the speed or throughput of the grid.

For accounting purposes and performance tuning, EnMasse provides a logging facility. One can adjust the extent of the logging, or completely switch it off. The log files are easy to understand by humans and to process by programs.

EnMasse runs on a wide range of hardware and operating systems, easy to install and requires little maintenance. It has run for weeks on a mix of Linux, FreeBSD and Windows nodes wholly unattended, with multiple access points over the same grid when needed. It has proved to be the solution to many performance problems, providing a reliable service without high levels of support.

Internally, EnMasse distributes formatting jobs, logs activity and monitors grid performance. Whatever the system around it is doing, the role of its core remains constant. For the user, EnMasse provides a choice of ways to submit tasks and receive responses. The three current interfaces are the active folder (Actinia), network server (Toaster) and the SOAP server (Fairy).

The EnMasse distribution contains:

The text files use Unix-style line separator. All XML files are encoded using UTF-8. Documentation is generated from DocBook XML source using DocBook XSL stylesheets. All documentation is prepared using RenderX XEP.

Some filenames in the distribution are in mixed case, sometimes called CamelCase, e.g. ThisIsMixedCase.xml. Please take care to use suitable unpacking or unzipping tools, such as InfoZIP or Winzip which correctly handle mixed case filenames. Depending on the flavor of your operating system, your tools, and security considerations, you may want to change access permissions and ownership of the files in the distribution, but it is important to retain the case sensitive filenames

To run EnMasse, launch an access point on one of the servers and several XEP engines, usually on separate computers. ${instDir}/bin/enmasse is a shell script that launches the access point; it issues the following command:

python ${instDir}/lib/Python/enmasse.py etc/enmasse.conf
      

where enmasse.conf is the configuration file; The configuration syntax is explained in the following section. On a platform which supports Bourne shell scripts, use the convenience features the script provides (execute bin/enmasse -help for usage instructions), otherwise just run the command above.

${instDir}/bin/engine launches an XEP engine; it is a call (to a Java program):

java com.renderx.xepx.cliser.Engine -DCONFIG=/path/to/xep.xml
        

(replace the path to xep.xml with the actual location of XEP configuration). Additional command-line switches are:

Both the access point and the engines run embedded HTTP servers; the servers display the current state, to monitor activity and help performance tuning. By default, the HTTP ports are 6590 for Actinia, 6595 for Toaster, 6597 for Fairy, and 6580 for XEP.

config = actinia | toaster | fairy
actinia = element actinia {
  actinia-folders & settings
}

toaster = element toaster {
  toaster-folders  & settings
}

fairy = element fairy {
  fairy-folders & settings
}
settings = options & servers & cliser

actinia-folders = element folders {
  attribute input {string},
  attribute output {string},
  attribute quarantine {string},
  attribute temporary {string}
}

toaster-folders = element folders {
  attribute temporary {string}
}

fairy-folders = element folders {
  attribute temporary {string}
}

options = element option {
  attribute name {token},
  attribute value {string}
}*

servers = servers {
  element server {
    attribute host {token}?,
    attribute port {token}?
  }+
}

cliser = element cliser {
  attribute format {token}?,
  options
}

<cliser format="pdf">
  <option name="FRM:VALIDATE" value="'true'"/>
  <option name="GEN:pdf:COMPRESS" value="'false'"/>
</cliser>

Toaster is one of the parts of EnMasse which requires that you write a program to use it. Since Toaster accepts requests over a network TCP socket, and implements a simple protocol, you must implement the protocol in your language of choice and embed it in your client-side application, such as a web form, or an authoring tool. An example of protocol CGI script calling toaster to format a document submitted via a WWW page is provided in ${instDir}/Python/wet.py, and a sample WWW page is in ${instDir}/etc/wet.html. Additionally, examples of Java Server Pages and ASP.NET are included into the distribution.

The protocol involves one request and one response. The client sends the request, in the form

RECEIVE data-sizesystemId

followed by a zero byte ('\0' in C), and then by the data of data-size bytes in length itself. Toaster transforms the document into PDF or PostScript and returns it: it sends

RECEIVE data-sizeformat

followed by a zero byte and by the formatted document.

If EnMasse cannot format the document, it sends

ERROR message-size None
      

followed by a zero byte and then by the error message. The message contains XEP's diagnostics and helps identify the problem.

To shutdown Toaster, send message STOP to the data port.

Fairy also requires you to write a program to use it. Since Fairy accepts SOAP requests you can use any SOAP toolkit to access it.

Fairy, as SOAP service, provides two methods: to format and to stop. If not otherwise specified in configuration methods names are respectively format and stop.

Fairy provides WSDL document, describing the service. For example, if Fairy is running at host yourhost with default configuration, you can get WSDL document, with GET /fairy?WSDL HTTP request to yourhost:6577 (say, just by typing http://yourhost:6577/fairy?WSDL in browser's address field).

...
<format>
  <systemId>SYSTEMID</systemId>
  <xml>XML_DATA_METASYMBOLS_REPLACED</xml>
</format>
...
        

...
<format>
  <systemId xsi:type="xsd:base64">SYSTEMID_BASE64_ENCODED</systemId>
  <xml xsi:type="xsd:base64">XML_DATA_BASE64_ENCODED</xml>
</format>
...