Engroup document management system overview

• Technology: engroup document system is developed base on excellent java frameworks and libaries as Spring, Spring Modules, Jackrabbit, Lucene at back-end and Flex at front-end.
• Development Time: first line of codes are written when we write the blog post http://blog.esofthead.com/jackbrabbit-ocm-and-spring. By time, the source code have many changes when we have more knowledge of using Jackrabbit, Luncene and product requirements, java libraries are more stable.
• Status: engroup document management (EDM) are maintained and bug-fixes phase. engroup 1.5.1 released on 10/7/2009 has 3 bug-fixes for EDM and we continue to fix any discovered defects of EDM reports from our customer and our QC team. With the workflow process applied to all engroup modules since 1.7.0 version (our roadmap), EDM will support workflow as well (what are the rules people must follow while access document?) also we improve searching criteria and back-up records system and user collaboration.

Benefits of using Engroup Document System

Increased Productivity / Efficiencies

• All documents are stored in central system. People does not need to search document in many sources: mail, network drive, local files etc
• Access heavy use documents instantly by bookmark menu.
• Work with document everywhere and anytime. You only need a web browser to do that.
• Get all documents in the same category fastly.
• Keep all previous documents in versionable form. You can get previous work easily as you work with current document.

Improved, faster and more flexible search

Engroup document system can retrieve files by any word or phrase in the document - known as full text search or browse documents using taxonomies - things are not presented in network sharing.

Secure your data

Most of user are familar with operating file system permission, engroup developers make our system familar with what you are using. Engroup provides permission rules are similar with file permissions. By default, each engroup user has his personal folder with full access right, and he can not access other folders (except administrator can access all folders). Engroup administrator will provide the rights for users, group of users (roles) access some folders/files with appropriate permissions. Engroup search only display results of documents can be accessible by user.

Data Integration with other engroup modules

Engroup modules saved documents in the same central place, it means you can access employee CVs, Sales documents or project attachments in one storage only. When you perform search content document, the system will list all matched document in all places: project management, CRM, HR etc which you have authorization to access.

Here are the new user features of engroup 1.5.1, the detail changes can be read at http://esofthead.com/node/51.

1. Project Management module:

• Helps enterprise manage their works easily with hierarchy tasks, deadlines.
• Collaborates project members, store project information with messages and their comments, attachments.
• Manages project resources and statuses such as project members, risks and problems.
• Provides automotive scheduling base on various variables start date, end date, predecessor tasks.
• Integrations with Microsoft Project: import/export data to/from MS Project format.

2. Connection module: is extracted from previous Dashboard module

• Private messages.
• Members List.

3 User Profile module: is extracted from previous Dashboard module

• Uploads user picture to profile section.
• Crop picture to profile picture.
• Displays profile picture.

We will write the usage of engroup modules such as document management, project management in future posts. Stay tuned.

'Engroup is a information, knowledge center and communication tool for groups and companies. Engroup promotes both informal and formal communication by combining various communication methods like private message (one-to-one, simple protocol), chat or email. In addition, Engroup provides many enterprise features like Human Resource Management, Customer Relationship, Document Management and Project Management modules. By utilizing the strength of Java language at server side and Flex on Flash runtime at client side, Engroup provides the fast and stability platform with user friendly interface'.

Engroup is the right tool for you if you want a software for one or many following needs:

• Manage your tasks.
• Read new channels.
• Communicate with others: private message, chat, mail, forum.
• Manage your company knowledge and information: wiki, document management.
• Manage company employee information.
• Manage customer information.
• Manage project: problems, risks, resources and documents.

Engroup milestone 3 is the scrum iteration release (1 month after the second milestone), it includes bug fixes, major improvements in Customer Relationship Management module and a new Project Management module is added. For the detail, you should check out our RELEASE NOTES at http://esofthead.com/node/49

- If any engroup modules need to tag some resource, which resource can be a document or a CRM Lead etc, it does not call tag service (if it do so, tag service must be available and it depends on tag service) but send a message with necessary information to a specific channel name 'add.tag'. If a tag service is available, it will receive message from that channel and save tag as requested.

- When a new user register to engroup system, engroup must do: 1. Create new user document workspace 2. Synchronize user information with its forum and wiki and several tasks later we will add, for example create a account with the same name in chat server, email server. Instead of adding code in register user part and modifying code when there is new added action when register user, we simply send a message to 'add.user' channel, any interested parties would listen this channel and do their tasks.

Engroup runs in Spring Osgi platform that we can control the dependencies among engroup modules and engroup external libraries. With the goal to make engroup modules are loose coupling, Engroup uses publish-subscribe pattern to make two modules have no dependencies in code can talk together with the help from ActiveMQ (messaging server) and Spring Integration framework.

Here are an example that we use all of these tools in engroup: we have two modules engroup document management module and engroup tag resource. Each document could have many tags, when a document is saved into engroup data-store, it will post message to appropriate channel to notify engroup tag module to save its tag. The similar process is applied for any module support tag source in engroup.

• Set up embedded ActiveMQ server:

For deployment simplicity, engroup uses embedded ActiveMQ instance in its community edition.

        <!--  lets create an embedded ActiveMQ Broker -->
        <amq:broker useJmx="false" persistent="false">
                <amq:transportConnectors>
                        <amq:transportConnector uri="tcp://localhost:0" />
                </amq:transportConnectors>
        </amq:broker>

        <!--  ActiveMQ destinations to use  -->

        <!-- JMS ConnectionFactory to use, configuring the embedded broker using XML -->
        <amq:connectionFactory id="jmsFactory" brokerURL="vm://localhost" />

        <bean id="connectionFactory"
                class="org.springframework.jms.connection.SingleConnectionFactory">
                <property name="targetConnectionFactory">
                        <ref local="jmsFactory" />
                </property>
        </bean>

• Using Spring JMS to send message:

          <!--Convert java object to Json message format-->
          <bean id="messageConverter"
                  class="com.engroup.integration.jms.converter.JsonMessageConverter" />
  
          <bean id="jmsTemplate"
                  class="org.springframework.jms.core.JmsTemplate">
                  <property name="connectionFactory" ref="connectionFactory" />
                  <property name="messageConverter" ref="messageConverter"></property>
          </bean>
  
          <bean id="messageDispatcher"
                  class="com.engroup.integration.dispatcher.jms.JmsMessageDispatcher">
                  <property name="jmsTemplate" ref="jmsTemplate" />
          </bean>

• Define message destinations:

In engroup tag module, we define two message destinations: tagRemove and tagAdd

        <!--Import spring service connection factory from engroup esb module-->
        <osgi:reference id="connectionFactory"
                interface="javax.jms.ConnectionFactory" />

        <bean id="tagRemoveDest"
                class="org.apache.activemq.command.ActiveMQQueue">
                <constructor-arg value="tag.remove" />
        </bean>

        <bean id="tagAddDest"
                class="org.apache.activemq.command.ActiveMQQueue">
                <constructor-arg value="tag.add" />
        </bean>

• Define service activator for channels:

Now, we define service activator to handle tag add request

        <bean id="tagServiceActivator"
                class="com.engroup.module.tag.integration.TagServiceActivator">
                <property name="tagService" ref="tagService" />
        </bean>

        <!-- Establish add tag channel and related things -->

        <jms:inbound-gateway id="jmsTagAdd" request-destination="tagAddDest"
                request-channel="tag.addChannel" reply-timeout="5000"
                request-timeout="5000" />

        <integration:channel id="tag.addChannel" />

        <integration:channel id="tag.addChannelAfterTransformer" />

        <integration:channel id="tag.result" />

        <engroup-esb:json-to-map-transformer input-channel="tag.addChannel"
                output-channel="tag.addChannelAfterTransformer" />

        <!--Method addTag is used to receive message from channel &#39;tag.addChannelAfterTransformer&#39;,
process and return message to channel &#39;tag.result&#39;-->

        <integration:service-activator ref="tagServiceActivator"
                method="addTag" input-channel="tag.addChannelAfterTransformer"
                output-channel="tag.result">
        </integration:service-activator>

Here is a part of code of TagServiceActivator:

public void addTag(Map tagInfo) {
        String contentId = (String) tagInfo.get("contentId");
        String[] tags = (String[]) tagInfo.get("tags");

        tagService.deleteTagRelationshipOfContent(contentId);

        for (String tag : tags) {
            tagService.saveTagRelationship(contentId, tag.trim());
        }
    }

Remember to add dependency Spring integration packages into MANIFEST.MF file, otherwise you will get ClassNotFoundException while
running Osgi.

• Send message to tag module channels

We go to the code of any modules are interesting in tag resource, in our example it is document module. First, we import messageDispatcher to document service bean:

         <osgi:reference id="connectionFactory"
                interface="javax.jms.ConnectionFactory" />

        <osgi:reference id="messageDispatcher"
                interface="com.engroup.integration.dispatcher.MessageDispatcher" />

        <bean id="internalAccessValidatorFileService"
                class="com.engroup.module.ecm.service.impl.AccessValidatorFileSystemServiceImpl">
                <property name="jcrTemplate" ref="jcrMappingTemplate" />
                <property name="transactionService"
                        ref="contentTransactionService">
                </property>
                <property name="messageDispatcher" ref="messageDispatcher"></property>
        </bean>

When we want to send a message to add tag channel of engroup tag module, we just simply construct a message and messageDispatcher will do the rest:

    private void sendSaveTagRequest(String contentId, String[] tags) {
        Dictionary
 props = new Hashtable();
        props.put("contentId", contentId);
        props.put("tags", tags);
        messageDispatcher.dispatchObject(AvailableDestinationNames.TAG_ADD,
                props);
    }

Osgi can help you control strictly dependencies of a module with external packages if we control well, we can limit dependencies of one engroup module with other module. Spring Integration framework provides solution to integrate data among modules without modifying existing code base. At the beginning, we feel it is complex solution while combining Spring Integration with Osgi platform. However, after number of modules is grown continuously also data exchange among modules are increased, such complex solution become a simple one on both development and deployment. Could you have other solutions?

The following screen-shots are result of displaying tag in document module.

Engroup 1.5.07082009 has the following new features and improvements according with the previous one:

• eSoftHead replaces Engroup AIR client application by Flex client browser. In the past, several clients complained the security warning while they install Engroup client and additional steps they must do to install client application. With engroup web client, such obstacles are disappeared. eSoftHead will only distribute Engroup AIR client application in its commercial delivery since Engroup 1.6. It will interact with native application, processes and file system that a web application cannot do.
• Document Management Module:
• • Bookmark content: user can bookmark their favorite content and manage bookmarks and its folders easily.
  • Access control: it is the key feature of this release. Now, Engroup Document Management manage contents like Operating System manages its file systems.
  • Improve performance of listing/reading contents.
• Improving caching model and create web session-like variable to manage user sessions.
• Various bug fixes of other modules

All your suggestions/comments will be highly appreciated. Please do not hesitate to drop some lines to us at info@esofthead.com.

In our last project, we faced 2 challenges of how to build the SaaS platform including various modules; some are developed by ourselves and some are third parties (Forum, Wiki, Mail, etc.). With third-parties, we limit modifying their code base and database schema due to client's requirements. Here are our challenges:

Challenge 1: When a new user is registered, some our modules need to acknowledge and have re-actions. For example:

• Document Management module creates a personal document workspace for that user.
• Calendar module creates a personal calendar for that user.
• Forum and Wiki register that user to their user database because Engroup supports Single Sign On (SSO) mechanism. That means any user registered to Engroup user database can access Forum and Wiki resources without registering their user database.

Since our application follows SaaS model, the modules acknowledging and having re-actions to that registered user are dynamically discovered at runtime. This prohibits us from invoking module API in static manner.

Challenge 2: We want to implement audit trail process. Each module installed into our platform must follow the rules of tracking our user activities. Our goal is to build a common API sending user activities to reporting server but all our modules must loose coupling with the reporting API. This design helps us to configure reporting server in different machines easily.

We found out the solution for 2 application integration problems above - service bus. After evaluating 3 open source enterprise service bus (ESB) frameworks: Spring Integration, Mule, and Service Mix, Spring Integration finally is our choice, based on the criterion we set as follows:

According to the Engroup 1.5 roadmap posted at http://blog.esofthead.com/engroup-15-roadmap/, the new features of Engroup 1.5 M1 are as follows:

• Upload single/multiple contents.
• Download single/multiple contents.
• Categorize contents via document type, status, and tag.
• Display simple transaction view.

If you want to view some screenshots before trying Engroup 1.5 M1, you could check them at http://www.flickr.com/photos/34441795@N06/sets/72157616946782473/. Besides adding Document Management module, Engroup 1.5 added some technical features:

• Improve caching by utilizing ehcache API that manages session-like variables.
• Utilize system resources and improve system responsive time by using Threadpool and asynchronous invocations.
• Engroup 1.0 promotes modular architecture by using Osgi event library to exchange messages among its modules. This approach does not persist messages if the server is down. Engroup 1.5 embeds messaging server (ActiveMQ - http://activemq.apache.org/) to provide persistent message and enterprise integration provider features
• Engroup 1.5 embeds a lightweight enterprise service bus - Spring Integration library. It applies Enterprise Integration patterns to establish message channels (direct, publish-suscribable) to exchange/transform messages among channels.
• RESTeasy is used to enable REST support to integrate data between third-party and Engroup.

All your suggestions/comments will be highly appreciated. Please do not hesitate to drop some lines to us at info@esofthead.com.

Our sample project is an existing application that uses Spring as IOC container, we would like to expose service method to third-party applications. We have the interface and its implementation already.

public interface FileSystemService {
    SimpleFile getFileByPath(String filePath);
    ...
}

Enable Restful service

The request /filesystem/getFile/<path of file> will return the file information if it is existed. To make it happen, we add java REST annotation to our service:

@Path("/filesystem")
public interface FileSystemService {
    @GET
    @Path("/getFile/{filePath:.*}")
    @Produces("application/xml")
    SimpleFile getFileByPath(@PathParam("filePath") String filePath);
    ...
}

Because the path of file may contains the '/' character, '/' is also the special characters of Http request, thus in line 4 we must use regular expression * to tell RESTeasy that all requests starts with /getFille is mapped to method getFileByPath (for example: /getFile/content/A or /getFile/content/A/B/C is valid path). @Produces("application/xml") annotation tells RESTeasy that the output must be in XML form.

Content marshalling/unmarshalling

Class SimpleFile is the complex type, to force RESTeasy produces XML data to client you must instruct RESTeasy how to marshal SimpleFile class.

@XmlRootElement(namespace="http://www.esofthead.com/engroup")
public class SimpleFile extends Content {
    private long size;
        ...
}

Installation

It is easy to install RESTeasy in java application, read the Installation section of RESTeasy documentation. Read more Spring integration section if you want to integrate RESTeasy with Spring. And we finish enable RESTful support in our Java application! Like its name, it is easy to deploy Restful solution by using RESTeasy.

Unit test

After all above tasks, it is time to look unit test solution. RESTeasy provides two approaches to write unit test:

• Embedded Container:RESTeasy packages TJWS embeddable servlet container with JAX-RS. You can start the embedded server in your unit test and use Apache HttpClient to test your code. See the sample in attachment files for detail.
• Server side - Mock framework:If you are not comfortable with starting the embedded server and would like a simpler solution, RESTeasy Mock framework would be a good option.

  Create Dispatcher mock object and integrate RESTeasy with Spring

  dispatcher = MockDispatcherFactory.createDispatcher();
  SpringBeanProcessor processor = new SpringBeanProcessor(dispatcher,
                  null, null);
  factory = new ClassPathXmlApplicationContext("ecm-context-test.xml");
  factory.addBeanFactoryPostProcessor(processor);
   
  SpringResourceFactory noDefaults = new SpringResourceFactory(
                  "fileSystemService", factory, FileSystemService.class);
  dispatcher.getRegistry().addResourceFactory(noDefaults);

  Send the request and test expected output

  MockHttpRequest request = MockHttpRequest
                  .get("/filesystem/getFile/content/A");
  MockHttpResponse response = new MockHttpResponse();
  dispatcher.invoke(request, response);
   
  Assert.assertEquals(HttpServletResponse.SC_OK, response.getStatus());
  String responseBodyAsString = response.getContentAsString();
   
  JAXBContext jaxbContext = JAXBContext.newInstance(SimpleFile.class);
  Unmarshaller unmarshaller = jaxbContext.createUnmarshaller();
  SimpleFile resultFile = (SimpleFile) unmarshaller
                  .unmarshal(new StringReader(responseBodyAsString));
   
  Assert.assertEquals(ContentConstants.FILE_ROOT_PATH + "/A", resultFile
                  .getPath());

  The result of request method is a XML string, the result of marshaling SimpleFile object to XML. To avoid testing base on XML data, we use JAXB to unmarshal XML data to SimpleFile object and validate data on this POJO.

I include two samples unit test for two approaches FileSystemRestTest-EmbeddedServer.java and FileSystemRestTest-MockFramework.java.This unit test sample is gotten from our engroup ECM module (you can get the Engroup product at http://esofthead.com/node/25), the unit test can not run because it lacks some classes of Engroup ECM module (this module is released in upcoming Engroup 1.5 version) but it provides the detail how to write unit test with RESTeasy.