- If any engroup modules need to tag some resource, which resource can be a document or a CRM Lead etc, it does not call tag service (if it do so, tag service must be available and it depends on tag service) but send a message with necessary information to a specific channel name 'add.tag'. If a tag service is available, it will receive message from that channel and save tag as requested.

- When a new user register to engroup system, engroup must do: 1. Create new user document workspace 2. Synchronize user information with its forum and wiki and several tasks later we will add, for example create a account with the same name in chat server, email server. Instead of adding code in register user part and modifying code when there is new added action when register user, we simply send a message to 'add.user' channel, any interested parties would listen this channel and do their tasks.

Engroup runs in Spring Osgi platform that we can control the dependencies among engroup modules and engroup external libraries. With the goal to make engroup modules are loose coupling, Engroup uses publish-subscribe pattern to make two modules have no dependencies in code can talk together with the help from ActiveMQ (messaging server) and Spring Integration framework.

Here are an example that we use all of these tools in engroup: we have two modules engroup document management module and engroup tag resource. Each document could have many tags, when a document is saved into engroup data-store, it will post message to appropriate channel to notify engroup tag module to save its tag. The similar process is applied for any module support tag source in engroup.

• Set up embedded ActiveMQ server:

For deployment simplicity, engroup uses embedded ActiveMQ instance in its community edition.

        <!--  lets create an embedded ActiveMQ Broker -->
        <amq:broker useJmx="false" persistent="false">
                <amq:transportConnectors>
                        <amq:transportConnector uri="tcp://localhost:0" />
                </amq:transportConnectors>
        </amq:broker>

        <!--  ActiveMQ destinations to use  -->

        <!-- JMS ConnectionFactory to use, configuring the embedded broker using XML -->
        <amq:connectionFactory id="jmsFactory" brokerURL="vm://localhost" />

        <bean id="connectionFactory"
                class="org.springframework.jms.connection.SingleConnectionFactory">
                <property name="targetConnectionFactory">
                        <ref local="jmsFactory" />
                </property>
        </bean>

• Using Spring JMS to send message:

          <!--Convert java object to Json message format-->
          <bean id="messageConverter"
                  class="com.engroup.integration.jms.converter.JsonMessageConverter" />
  
          <bean id="jmsTemplate"
                  class="org.springframework.jms.core.JmsTemplate">
                  <property name="connectionFactory" ref="connectionFactory" />
                  <property name="messageConverter" ref="messageConverter"></property>
          </bean>
  
          <bean id="messageDispatcher"
                  class="com.engroup.integration.dispatcher.jms.JmsMessageDispatcher">
                  <property name="jmsTemplate" ref="jmsTemplate" />
          </bean>

• Define message destinations:

In engroup tag module, we define two message destinations: tagRemove and tagAdd

        <!--Import spring service connection factory from engroup esb module-->
        <osgi:reference id="connectionFactory"
                interface="javax.jms.ConnectionFactory" />

        <bean id="tagRemoveDest"
                class="org.apache.activemq.command.ActiveMQQueue">
                <constructor-arg value="tag.remove" />
        </bean>

        <bean id="tagAddDest"
                class="org.apache.activemq.command.ActiveMQQueue">
                <constructor-arg value="tag.add" />
        </bean>

• Define service activator for channels:

Now, we define service activator to handle tag add request

        <bean id="tagServiceActivator"
                class="com.engroup.module.tag.integration.TagServiceActivator">
                <property name="tagService" ref="tagService" />
        </bean>

        <!-- Establish add tag channel and related things -->

        <jms:inbound-gateway id="jmsTagAdd" request-destination="tagAddDest"
                request-channel="tag.addChannel" reply-timeout="5000"
                request-timeout="5000" />

        <integration:channel id="tag.addChannel" />

        <integration:channel id="tag.addChannelAfterTransformer" />

        <integration:channel id="tag.result" />

        <engroup-esb:json-to-map-transformer input-channel="tag.addChannel"
                output-channel="tag.addChannelAfterTransformer" />

        <!--Method addTag is used to receive message from channel &#39;tag.addChannelAfterTransformer&#39;,
process and return message to channel &#39;tag.result&#39;-->

        <integration:service-activator ref="tagServiceActivator"
                method="addTag" input-channel="tag.addChannelAfterTransformer"
                output-channel="tag.result">
        </integration:service-activator>

Here is a part of code of TagServiceActivator:

public void addTag(Map tagInfo) {
        String contentId = (String) tagInfo.get("contentId");
        String[] tags = (String[]) tagInfo.get("tags");

        tagService.deleteTagRelationshipOfContent(contentId);

        for (String tag : tags) {
            tagService.saveTagRelationship(contentId, tag.trim());
        }
    }

Remember to add dependency Spring integration packages into MANIFEST.MF file, otherwise you will get ClassNotFoundException while
running Osgi.

• Send message to tag module channels

We go to the code of any modules are interesting in tag resource, in our example it is document module. First, we import messageDispatcher to document service bean:

         <osgi:reference id="connectionFactory"
                interface="javax.jms.ConnectionFactory" />

        <osgi:reference id="messageDispatcher"
                interface="com.engroup.integration.dispatcher.MessageDispatcher" />

        <bean id="internalAccessValidatorFileService"
                class="com.engroup.module.ecm.service.impl.AccessValidatorFileSystemServiceImpl">
                <property name="jcrTemplate" ref="jcrMappingTemplate" />
                <property name="transactionService"
                        ref="contentTransactionService">
                </property>
                <property name="messageDispatcher" ref="messageDispatcher"></property>
        </bean>

When we want to send a message to add tag channel of engroup tag module, we just simply construct a message and messageDispatcher will do the rest:

    private void sendSaveTagRequest(String contentId, String[] tags) {
        Dictionary
 props = new Hashtable();
        props.put("contentId", contentId);
        props.put("tags", tags);
        messageDispatcher.dispatchObject(AvailableDestinationNames.TAG_ADD,
                props);
    }

Osgi can help you control strictly dependencies of a module with external packages if we control well, we can limit dependencies of one engroup module with other module. Spring Integration framework provides solution to integrate data among modules without modifying existing code base. At the beginning, we feel it is complex solution while combining Spring Integration with Osgi platform. However, after number of modules is grown continuously also data exchange among modules are increased, such complex solution become a simple one on both development and deployment. Could you have other solutions?

The following screen-shots are result of displaying tag in document module.

Engroup 1.5.07082009 has the following new features and improvements according with the previous one:

• eSoftHead replaces Engroup AIR client application by Flex client browser. In the past, several clients complained the security warning while they install Engroup client and additional steps they must do to install client application. With engroup web client, such obstacles are disappeared. eSoftHead will only distribute Engroup AIR client application in its commercial delivery since Engroup 1.6. It will interact with native application, processes and file system that a web application cannot do.
• Document Management Module:
• • Bookmark content: user can bookmark their favorite content and manage bookmarks and its folders easily.
  • Access control: it is the key feature of this release. Now, Engroup Document Management manage contents like Operating System manages its file systems.
  • Improve performance of listing/reading contents.
• Improving caching model and create web session-like variable to manage user sessions.
• Various bug fixes of other modules

All your suggestions/comments will be highly appreciated. Please do not hesitate to drop some lines to us at info@esofthead.com.

According to the Engroup 1.5 roadmap posted at http://blog.esofthead.com/engroup-15-roadmap/, the new features of Engroup 1.5 M1 are as follows:

• Upload single/multiple contents.
• Download single/multiple contents.
• Categorize contents via document type, status, and tag.
• Display simple transaction view.

If you want to view some screenshots before trying Engroup 1.5 M1, you could check them at http://www.flickr.com/photos/34441795@N06/sets/72157616946782473/. Besides adding Document Management module, Engroup 1.5 added some technical features:

• Improve caching by utilizing ehcache API that manages session-like variables.
• Utilize system resources and improve system responsive time by using Threadpool and asynchronous invocations.
• Engroup 1.0 promotes modular architecture by using Osgi event library to exchange messages among its modules. This approach does not persist messages if the server is down. Engroup 1.5 embeds messaging server (ActiveMQ - http://activemq.apache.org/) to provide persistent message and enterprise integration provider features
• Engroup 1.5 embeds a lightweight enterprise service bus - Spring Integration library. It applies Enterprise Integration patterns to establish message channels (direct, publish-suscribable) to exchange/transform messages among channels.
• RESTeasy is used to enable REST support to integrate data between third-party and Engroup.

All your suggestions/comments will be highly appreciated. Please do not hesitate to drop some lines to us at info@esofthead.com.

ApplicationContext ctx = new ClasspathApplicationContext("example.xml");
BeanEx beanEx = (BeanEx)ctx.getBean("beanEx");

However, such calling will throw an exception in Spring DM platform because Spring requires the bundle context is existed. To overcome this issue, you just simply retrieving Spring-Osgi bean in the Activator class. Here are the example:

public class Activator implements BundleActivator {
 
    static ForumAuthenticationService forumService;
 
    static BundleContext bundleContext;
 
    public void start(BundleContext bundleContext) throws Exception {
        Activator.bundleContext = bundleContext;
    }
 
    public void stop(BundleContext arg0) throws Exception {
        forumService = null;
 
    }
 
    public static ForumAuthenticationService getAuthenticationService() {
        if (forumService == null) {
            OsgiBundleXmlApplicationContext context = new OsgiBundleXmlApplicationContext(
                    new String[] { "META-INF/spring/forum-context.xml" });
            context.setBundleContext(bundleContext);
            context.refresh();
            forumService = (ForumAuthenticationService) context
                    .getBean("forumAuthenticationService");
        }
 
        return forumService;
    }
 
}

In the above example, we need to get the spring service to make authenticate user in the legacy class and it is the simple solution how to retrieve the Spring-osgi service from some class in Osgi platform.

I am very pleased to announce that Engroup 1.0 after more than one month of previous release date. It is the bug fix release for the previous one, Engroup 1.0 beta. You can download the Engroup at eSoftHead site and see the last changes at here.

It is the first time eSoftHead releases the Engroup source codes under GPL license, and it is the major difference between the current release and the previous one. Releasing the source code to community is the key factor to help us continuously improve our source code quality, share knowledge and help other non-commercial organizations build RIA application using Adobe technology faster base on Engroup codebase. These benefits are over the threat that some guys may steal our source code and release the product with their copyright. We faced many issues to manage such big project, overcome the limitation of external libraries and tools and continuously optimize our source code, so we are welcome any your valuable comments to help us work better. You could download Engroup source code at eSoftHead site and read carefully the instruction at here.

Our development team are developing the next Engroup version, it focus on Document Management area and integrate Enterprise documents with varies Engroup modules such as HR, CRM. I will write the road-map for the next Engroup version in next several days and the first release of Engroup 1.0 will be planned in the next 2-3 weeks. Remember to keep update Engroup information on our website and this blog.

• Engroup is the mix project of Java and Flash/Flex with server side is written by Java and front-end is developed by Flex.
• Now, there are many projects:
• • Engroup server has 29 sub-projects (include test projects for Engroup modules and documentation project). The number of projects would be increased in future. Maven is the main tool to build Engroup server projects. In some cases, we combine with Ant via maven-ant tasks to utilize Ant power to support doing tasks that Maven can't.
  • Engroup client has 18 sub-projects. Ant is the main tool to build Engroup client projects by mainly using Flex-Ant task also our custom tasks to generate the code from Engroup server.
• Developers commit code daily.
• Long list of project dependencies: Engroup module projects depend on Engroup core and common modules.
• Back-end services must have unit test for its implementation.
• Database backup must be generated in every build and Engroup installer must be available in every successfully build.

Related build tools:

• Hudson (https://hudson.dev.java.net/): one of the best CI Java build server. Simple to run, simple to use. Have many plug-ins and a clean dashboard to follow the build status.
• Nexus (http://nexus.sonatype.org/): a powerful and robust Maven manager. Nexus is very easy to use and we deploy our Engroup server modules to our nexus server.
• Maven (http://maven.apache.org): an excellent build tool. A must tool for any Java projects.
• Ant (http://ant.apache.org): another excellent Java build tool. A must tool for any Java projects. A combination of Ant and Maven strengthen the power of each tool.

Our standard build process:

1. CI server (Hudson) will get the latest sources from SVN server.
2. Build Engroup projects order depends on project dependencies.
3. Run unit test.
4. Make the reports (thanks for the help of Maven): checkstyle, pmd, junit, cobertura.
5. Deploy Engroup modules to maven repository manager if the build is success.
6. Back up database if all module builds are success.
7. Make the Engroup installer.

Manage projects and build process by CI server (Hudson)

It is fairly easy to create the new build job on Hudson, create 'Create Job' link at Hudson CI server and simply fill necessary information

You can integrate Unit test, checkstyle, cobertura, pmd to project dashboard by selecting them in build settings section.

And you could manage builds and its status on dashboard page, for multiple projects it is better if you customize the list view of modules. For example, in our project we make the customization view for client and server view also view of common artifacts (documentations, installer).

Monitor project health
We can manage the health of project via the following factors:

• Build status: how many failure or success of builds.
• Build warnings: compiler warnings, code warnings (checkstyle, violations) or defined threshold of checking tools (unit test coverage, findbugs)
• Other information: code metrics, duplicate code, source code.
• Project information: development information, license, source repository etc

Go to the Maven site to see project information:

and we can visit some sections to see the detail:

The values of applying CI in our project:

Applying continuous integration in our project give us many advantages:

• The whole process is automated (even backup database task) - no manual task in our build process. No any human mistake causes the build broken, no build check-list is needed any longer.
• Feedback to development status as soon as possible via unit test result and code warnings and some useful metrics.
• Give the project status as you need.
• Managing multiple projects status in one place.
• Developer could get the latest other works as soon as possible (thanks for nexus - open source maven repository).

Applying CI in your project should be the good option for your project as we get in our projects. Just takes some hours to make all things are run properly but the benefits you have is far greater than the effort you spent. Let's try, see and enjoy!

• Start the OSGi framework (Equinox, Knopflerfish, Felix)

• install and start any specified bundles required for the test

• package the test case itself into a on the fly bundle, generate the manifest (if none is provided) and install it in the OSGi framework

• execute the test case inside the OSGi framework

• shut down the framework

• passes the test results back to the originating test case instance that is running outside of OSGi

Setting the integration test environment

Make sure that the following necessary files belong your class-path:

• spring-osgi-core-1.1.0
• spring-osgi-extender-1.1.0
• spring-osgi-io-1.1.0
• spring-osgi-test-1.1.0

Creating the first Osgi integration test

All integration test class must inherit the org.springframework.osgi.test.AbstractConfigurableBundleCreatorTest. The first step is creating our base class for all osgi integration testing:

/**
 * The base class of integration testing of Engroup. All integration testing
 * classes must be derived from this class
 *
 */
public class AbstractEngroupOsgiTest extends
  AbstractConfigurableBundleCreatorTests {
   ...
}

Configuring boot bundles of Spring DM

By default, Spring DM testing frameworks will load some default bundles before loading specific bundles for your test. You can see the default bundles loaded by Spring DM testing framework at org.springframework.osgi.test.internal.boot-bundles of spring-test project. In some cases, you need to replace the boot bundles of Spring DM testing framework by your bundles. The typical scenario that you need to do that is case you already have some bundles, in these bundles you add some extra osgi meta data (such as DynamicImport-Package field to allow the bundle can dynamic load class at runtime). Having the same package with the same version is prohibited. In addition, we are doing integration test, so make the test environment is similar with real environment is a must. Spring DM allows you can inject the new configuration of boot bundles by simply override the method getTestingFrameworkBundlesConfiguration of class AbstractConfigurableBundleCreatorTest.

Example 1: Inject the new configuration of boot bundles for Spring DM testing

private static final String TEST_FRRAMEWORK_BUNDLES_CONF_FILE = "/META-INF/boot-bundles.properties";
...
@Override
protected Resource getTestingFrameworkBundlesConfiguration() {
  return new InputStreamResource(AbstractEngroupOsgiTest.class
    .getResourceAsStream(TEST_FRRAMEWORK_BUNDLES_CONF_FILE));
}

Customizing the bundle content

Due to your test class is bundled in on-the-fly bundle. You can customize the manifest file of this bundle. In some cases, it is the must for complex case while the test class use the class of other bundles run during test executing. To do this, you simply override the method getManifestLocation (note that you can change the content of manifest file programatically by override the method getManifest, however we prefer to use the manifest file)

Example 2: Customizing the test manifest file

@Override
protected String getManifestLocation() {
    return "classpath:META-INF/MANIFEST.MF";
}

Example 3: Customized manifest file

Manifest-Version: 1.0
Embed-Directory: lib
Implementation-Title: Engroup Osgi Integration Test
Spring-Version: 2.5.5
Bundle-Activator: org.springframework.osgi.test.JUnitTestActivator
Implementation-Version: 2.5.5
Tool: Bnd-0.0.160
Bundle-Name: engroup-integration-test
Created-By: 1.6.0 (Sun Microsystems Inc.)
Bundle-Version: 0.0.1
Bnd-LastModified: 1207763595575
Bundle-ManifestVersion: 2
Import-Package:
 com.engroup.module.common.domain,
 com.engroup.module.common.service,
 ...
 com.mysql.jdbc,
 javax.jcr,
 javax.naming,
 javax.naming.directory,
 javax.naming.ldap,
 javax.sql,
 junit.framework,
 org.dbunit,
 org.dbunit.database,
 org.dbunit.dataset,
 org.dbunit.dataset.xml,
 org.dbunit.operation,
 org.osgi.framework,
 org.springframework.context,
 ...
Bundle-ClassPath: .,
 {src/test/resources}
Bundle-SymbolicName: engroup.server.engroup-integration-test
Include-Resource: {srctestresources}

Some tips of writing Osgi Integration Test

1. Set the default wait timer of creating spring context:
   Waiting time is set to wait until all dependencies are resolved before running test execution. Base on practice the waiting time is varied but limit under 20s (in our project). However, to support debugging while running integration test, the longer value should be set to prevent Spring DM interupt the test while developers are debugging the test program.Example 4: Set the wait time override the default value of Spring DM

       @Override
       protected long getDefaultWaitTime() {
           return 3000;
       }

2. Inject spring beans in test class:
   We use Spring DM with the purpose that Spring beans could be exposed as the osgi services and used in other bundles. Of course, we would like to test spring beans in integration testing. To use spring, make sure the integration test class load the spring context files firstExample 5: Load the spring context files

        @Override
        protected String[] getConfigLocations() {
            return new String[] {
                "classpath:META-INF/spring/hr-context-osgi-test.xml",
                "classpath:META-INF/spring/common-context-osgi-test.xml",
                "META-INF/spring/db-context-osgi-test.xml" };
        }

   Note: All spring context files should only declare only spring beans are exposed by other bundles and remember to import the necessary packages in your manifest file

   Example 6: Spring context file declares the spring bean service exported by other bundles and they serve for testing purpose

   <beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:osgi="http://www.springframework.org/schema/osgi"
    xsi:schemaLocation=
     "http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
     http://www.springframework.org/schema/osgi http://www.springframework.org/schema/osgi/spring-osgi.xsd">
    <osgi:reference id="companyService"
     interface="com.engroup.module.hr.service.CompanyService" />
   
    <osgi:reference id="divisionService"
     interface="com.engroup.module.hr.service.DivisionService" />
   ...

   Now, it is ready to inject our spring beans to test class by injecting the spring bean to protected objects of test class

   Example 7: Inject the spring bean to test class protected fields

      protected CompanyService companyService;
   
      public AbstractEngroupOsgiTest() {
          setPopulateProtectedVariables(true);
      }

   Now, you can use companyService spring bean is exposed from other bundles in your test method.

• Better separation of application logic into modules, with runtime enforcement of module boundaries (Osgi provides)
• The ability to deploy multiple versions of a module (or library) concurrently (Osgi provides)
• The ability to dynamically discover and use services provided by other modules in the system (Osgi provides)
• The ability to dynamically install, update and uninstall modules in a running system (Osgi provides)
• Use of the Spring Framework to instantiate, configure, assemble, and decorate components within and across modules. (Spring + Osgi)
• A simple and familiar programming model for enterprise developers to exploit the features of the OSGi platform. (Spring + Osgi)

The two most important benefits that Spring Dynamic Modules bring to us is the last two items of above list. Let investigate them with our real example:

We are developing the enterprise application base on SaaS model with many modules such as human resources, project management, timesheet, chat, video conference etc. All public services are exposed via Spring beans. To make the loose coupling among services, we define the rules at services of different component could be use the other services via interface (of course, it is the rules that all
developers are aware but junior developers can eventually break the rules by calling the implementation class). We faced the following problems that encourage us to choose Osgi:

• Our application must integrate with other applications such as forum, wiki, calendar that have many third-party jar files and some of them are conflicted in version (and usage). For example, our main module use lucence (2.3.2 version) for search function. However, jackrabbit module can not work with this version of lucene but the lower version one 2.2.0.
• Each module of engroup application could be installed/updated/removed without requiring application reboot.

Without osgi, you could develop and maintain your own created class loader and meta data to resolve the first one and create the framework that manage your services for the second one. With osgi, you do not need to do anything except modify the manifest file follow osgi rules with the great help of maven osgi plugin (thanks for bnd tool and felix team to provide the community this tool). Ok, we finish the first step that bundle our services into bundles and expose the necessary interfaces for other use.

Another problem is we already developed many services under Spring bean form, and how other bundle can recorgnize the spring bean service locates from other bundle. Spring Dynamic Modules for Osgi solves this issue very well. Just by declaration in spring context file, my spring service can use other spring service of other bundle! Great! I can define the datasource spring bean in shared module and other module could use this datasource in its bundle (not need to use import tag in spring context file) but treat it as exposed service of shared bundle.

And the great consequence that Spring Dynamic Modules can provide to us that we only exposed the service under the Spring bean form, not need to register our service into service registry in bundle activator, if we need to expose our service to other bundle, we define the service in spring osgi context file (we only write the bundle activator for special case only), also we do not manage the life cycle of bundles but we manage the life cycle of spring beans. For example, before starting the human resource module, we need to create the neccessary tables for this module, instead of writing the initializing database in bundle activator, we do it in human resource spring bean (implement the LifeCycle interface). By doing this, we make our business code loose coupling totally with Osgi framework (though we depends our code in spring - but we do not have any plan to use another IoC except Spring!)

With the great features of Spring Dynamic Modules (now we are upgrading our integration test with Spring Module version 1.1.0), it is the natural fit with Spring and Osgi in one platform. Developing application on it is the great interest with the development team. And thanks god, it saves us much effort of integration and use Spring in Osgi environment.