Tuesday, January 15, 2013

Functional testing in SOA environment

My first project in a service oriented environment has reached functional testing stage. The product involves processing large amounts of business objects whose reference data is obtained from other subsystems through service calls, in production.

While devising the strategy for functional testing, the issue was how to get this reference data in a test environment.

Unit Testing

During unit testing, the service calls were mocked using a mocking library (Mockito in our case), and the expected responses were returned using when - thenReturn construct, as follows:


import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;

import org.mockito.MockitoAnnotations;
import org.mockito.Mock;
import org.junit.Before;
import org.junit.Test;

public class SweepTest 
{
    /*
     * Specify that the service wrapper should be mocked.
     */
    @Mock
    private XServiceWrapper xServiceWrapper;

    /*
     * SomeBusinessObjectManager has reference to XServiceWrapper and we want the 
     * mocked instance to be injected instead of the normal implementation.
     */
    @InjectMocks
    @Autowired
    private SomeBusinessObjectManager boManager;

    @Before
    public void setup()
    {
         MockitoAnnotations.initMocks(this);

         /*
          * Mock the response object from the service.
          */
         XServiceResponse serviceResponse = mock(XServiceResponse.class);
         when(serviceResponse.getXValue()).thenReturn(myValue);

         /*
          * Mock the service call and return mocked response object.
          */
         when(xServiceWrapper.serviceCall()).thenReturn(serviceResponse);
    }
}


Functional Testing

During functional testing, we identified golden set of use cases that necessarily and sufficiently cover the behavior of the system in various scenarios. The data for various internal objects for these golden set of use cases is prepared. But, it is not feasible to get relevant data from the external services that would suit the input data for a given use case. So, the following changes were done to the project w.r.t interacting with external services:
  1. Define an ExternalServiceFacade interface that would define all the calls to be made to different external services.
  2. /**
     * ExternalServiceFacade.java
     * 
     * $Source$
     */
    /*
     * Copyright (c) 2013 Krovi.com, Inc. All rights reserved.
     */
    
    package com.krovi.app.manager;
    
    import com.krovi.services.BusinessObjectA;
    import com.krovi.services.BusinessObjectB;
    import com.krovi.services.BusinessObjectC;
    
    /**
     * This class follows the Facade pattern and provides APIs to
     * interact with various external services that provide reference data for the given object IDs.
     * 
     * @author krovi
     * @version $Id$
     */
    public interface ExternalServiceFacade
    {
        BusinessObjectA lookupBusinessObjectA(String referenceId);
    
        BusinessObjectB lookupBusinessObjectB(String referenceId1, String referenceId2);
    
        BusinessObjectC lookupBusinessObjectC(String referenceId1, String referenceId2, Long referenceValue3);
    
    }
    
    
  3. Provide an implementation that delegates the calls to the actual services and obtains the responses. This implementation will be used in production scenarios.
  4. /**
     * ExternalServiceFacadeImpl.java
     * 
     * $Source$
     */
    /*
     * Copyright (c) 2013 Krovi.com, Inc. All rights reserved.
     */
    
    package com.krovi.app.manager;
    
    import com.krovi.services.BusinessObjectA;
    import com.krovi.services.BusinessObjectAService;
    import com.krovi.services.BusinessObjectAServiceClient;
    import com.krovi.services.BusinessObjectB;
    import com.krovi.services.BusinessObjectBService;
    import com.krovi.services.BusinessObjectBServiceClient;
    import com.krovi.services.BusinessObjectC;
    import com.krovi.services.BusinessObjectCService;
    import com.krovi.services.BusinessObjectCServiceClient;
    
    /**
     * @author krovi
     * @version $Id$
     */
    public class ExternalServiceFacadeImpl implements ExternalServiceFacade
    {
        @Autowired
        private BusinessObjectAService boAService;
    
        @Autowired
        private BusinessObjectBService boBService;
    
        @Autowired
        private BusinessObjectCService boCService;
    
        @Override
        public BusinessObjectA lookupBusinessObjectA(String referenceId)
        {
            BusinessObjectAServiceClient client = boAService.getClient();
            return client.lookupBusinessObjectA(referenceId);
        }
    
        @Override
        public BusinessObjectB lookupBusinessObjectB(String referenceId1, String referenceId2)
        {
            BusinessObjectBServiceClient client = boBService.getClient();
            return client.lookupBusinessObjectB(referenceId1, referenceId2);
        }
    
        @Override
        public BusinessObjectC lookupBusinessObjectC(String referenceId1, String referenceId2, Long referenceValue3)
        {
            BusinessObjectCServiceClient client = boCService.getClient();
            return client.lookupBusinessObjectC(referenceId1, referenceId2, referenceValue3);
        }
    
    }
    
    
  5. Provide a mock implementation that will connect to a local data store that is populated with the expected data and returns it. This implementation will be used in functional testing scenario.
  6. /**
     * MockExternalServiceFacadeImpl.java
     * 
     * $Source$
     */
    /*
     * Copyright (c) 2013 Krovi.com, Inc. All rights reserved.
     */
    
    package com.krovi.app.manager;
    
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.jdbc.core.JdbcTemplate;
    
    import com.krovi.services.BusinessObjectA;
    import com.krovi.services.BusinessObjectB;
    import com.krovi.services.BusinessObjectC;
    
    /**
     * @author krovi
     * @version $Id$
     */
    public class MockExternalServiceFacadeImpl implements ExternalServiceFacade
    {
        @Autowired
        private JdbcTemplate jdbcTemplate;
    
        public MockExternalServiceFacadeImpl()
        {
            setupDBObjects();
        }
    
        private void setupDBObjects()
        {
            /**
             * 1. This method creates DB tables corresponding to the structures of BusinessObject* classes. 2. The test
             * harness would insert test data into these tables as per the use cases.
             */
        }
    
        @Override
        public BusinessObjectA lookupBusinessObjectA(String referenceId)
        {
            // Look up the database instead of making a service call. The database would contain the appropriate object
            // required for the test case that is being currently run.
            return null;
        }
    
        @Override
        public BusinessObjectB lookupBusinessObjectB(String referenceId1, String referenceId2)
        {
            // Look up the database instead of making a service call. The database would contain the appropriate object
            // required for the test case that is being currently run.
            return null;
        }
    
        @Override
        public BusinessObjectC lookupBusinessObjectC(String referenceId1, String referenceId2, Long referenceValue3)
        {
            // Look up the database instead of making a service call. The database would contain the appropriate object
            // required for the test case that is being currently run.
            return null;
        }
    
    }
    
    
  7. Modify all calls to services in the entire code base to go through ExternalServiceFacade.
  8. The spring configuration files for production and testing stages are modified to inject the appropriate implementation of the ExternalServiceFacade.

Wednesday, October 17, 2012

@Memoize in Java using Spring and Aspects

Recently, while writing an application in service oriented architecture, I had a couple of places in the code where I needed to make a service call to get the details by a particular application key. I was wondering about duplicate calls, i.e. multiple service calls made to get the details for the same application key. As we all know, service calls are costly with all the marshalling, unmarshalling, cache lookups, database lookups involved, I wanted to cache the results of a service call by application key or keys and this is a clear cross-cutting concern. Having programmed in Perl and Python that have the memoize option, I thought a Memoize aspect would be most appropriate here. As always, I was looking for Spring's Aspect Oriented Programming APIs and though it was tricky to get the set up right, I ended up with a very small amount of code that does the basic memoization. Since the set up was tricky, I thought I would document here for reference:


  1. Set up Spring, AspectJ, CGLib and ASM for the application.
    1. The following Spring jars are needed:
      1. spring-aop (v3.3.1)
      2. spring-aspects (v3.3.1)
      3. spring-beans (v3.3.1)
      4. spring-context (v3.3.1)
      5. spring-context-support (v3.3.1)
      6. spring-core (v3.3.1)
      7. spring-test (v3.3.1)
      8. spring-expression (v3.3.1)
      9. spring-asm (v3.3.1)
    2. The following AspectJ jars are needed for load-time-weaving and aspectj runtime.
      1. aspectj-rt (v1.6.9)
      2. aspectj-weaver (v1.5.4)
      3. aop-alliance (v1.0)
    3. The following CGLib jars are needed for code generation
      1. cglib (v2.2.3)
    4. The following ASM jars are needed for byte-code manipulation
      1. asm (v3.3.1)
    5. Commons logging jar needed by Spring
      1. commons-logging (v1.1.1)
  2. The CGLib and ASM libraries should be compatible with each other. So, if you choose a different version of any of these jars, make sure to select the compatible other one. Otherwise, you will end up with a runtime error (NoSuchMethodError).
  3. Enable aspects in your spring application using the following configuration:
    <?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
        xmlns:aop="http://www.springframework.org/schema/aop"
        xsi:schemaLocation="
                http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd 
                http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
                http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.0.xsd
                ">
    
        <context:component-scan base-package="com.vikdor.aspects" />
        <aop:aspectj-autoproxy proxy-target-class="true" />
    </beans>
    
  4. Define the annotation that is going to be used in the advice:
    import java.lang.annotation.Retention;
    
    @Retention(RetentionPolicy.RUNTIME)
    public @interface Memoize
    {
    
    }
    
  5. Write a simple Aspect class, annotate it with @Aspect and define an @Around advice for all the methods that use the @Memoize annotation, as follows:
    @Aspect
    @Component
    public class MemoizeAspect
    {
        private WeakHashMap<Object[], Object> cache = new WeakHashMap<Object[], Object>();
    
        @Around("@annotation(com.vikdor.aspects.Memoize)")
        public Object handledMemoize(ProceedingJoinPoint pjp) throws Throwable
        {
            System.out.println("Handling memoize");
            Object[] args = pjp.getArgs();
    
            if (args != null)
            {
                Object response = cache.get(Arrays.asList(args));
                if (response != null)
                {
                    return response;
                }
            }
            Object obj = pjp.proceed();
            if (args != null)
            {
                cache.put(Arrays.asList(args), obj);
            }
            return obj;
        }
    }
    
Now, add @Memoize to any method whose request and response should be cached and it just works. BTW, this is a very naive Memoize implementation and is not production ready at all ;-)

Sunday, October 7, 2012

Connecting to SQL Server Express through JDBC

I came across this question in a decent number of forums where the subtleties of connecting to an instance of SQL Server Express using JDBC are missed out. So, thought it would be good to just document how I approached it, here for my reference and for others, if they find useful.
Once you install the SQL Server Express edition, you need to configure the TCP/IP settings for the JDBC drivers to connect to the SQLEXPRESS database instance.

Configuring the TCP/IP:

  1. Configure TCP/IP communication with SQL Express
    1. Open SQL Server Configuration Manager.
    2. Go to SQL Server Network Configuration -> Protocols for SQLEXPRESS
    3. Set the status of TCP/IP protocol to "Enabled" (if it is already not).
    4. Open Properties window for TCP/IP, go to IP Addresses section.
    5. Go to the bottom of this property page and set the TCP Port under `IPAll` to 1433.
  2. Connect to the SQLExpress instance using `Microsoft's JDBC driver for SQL Server`
    1. JDBC URL: `jdbc:sqlserver://localhost;instance=SQLEXPRESS;databaseName=<your DB>;user=<your User>;password=<your Passwd>`

Using Windows Integrated Authentication:
  1. Create a windows account for the application that would be used to run your programs. This account's credentials will be used to connect to the server.
  2. Get Microsoft JDBC Driver for SQL Server from here.
  3. Configure the JDBC URL as follows:

    jdbc:sqlserver://<hostname>;databaseName=<DBName>;integratedSecurity=true
  4. Configure the launcher that run the Java programs from command line to include the following JVM parameter:

    -Djava.library.path="<jdbc driver dll location>"

    where the location is the directory where the JDBC driver downloaded earlier is installed or extracted. It was C:\Program Files\sqljdbc_4.0.2206.100_enu\sqljdbc_4.0\enu\auth\x64 in my case. The dll should be picked based on the JVM used for running these programs.

With the above configuration, the connection established to SQL Server would use the Windows Authentication Credentials of the domain user running the java program/process.

Sample Code:

public class SQLServerEx
{
    static enum Drivers
    {
        MICROSOFT(
                  SQLServerDriver.class.getName(),
                  "jdbc:sqlserver://"
                      + "localhost;"
                      + "instance=SQLEXPRESS;databaseName=vikdor_test;user=vikdor_user;password=vikdor_user"),
        MICROSOFT_INTEG(
                        SQLServerDriver.class.getName(),
                        "jdbc:sqlserver://"
                            + "localhost;"
                            + "instance=SQLEXPRESS;databaseName=vikdor_test;integratedSecurity=true");

        private String driverClass;
        private String driverURL;

        private Drivers(String driverClass, String driverURL)
        {
            this.driverClass = driverClass;
            this.driverURL = driverURL;
        }

        public String getDriverClass()
        {
            return driverClass;
        }

        public String getDriverURL()
        {
            return driverURL;
        }
    }

    public static void main(String[] args) throws Exception
    {
        Class.forName(Drivers.MICROSOFT_INTEG.getDriverClass());
        Connection con =
            DriverManager.getConnection(Drivers.MICROSOFT_INTEG.driverURL);
        PreparedStatement ps =
            con.prepareStatement(
                "insert into auto_table(name, role) values(?, ?)",
                Statement.RETURN_GENERATED_KEYS);
        ps.setString(1, "vikdor");
        ps.setString(2, "sde");
        ps.executeUpdate();
        ResultSet rs = ps.getGeneratedKeys();
        while (rs.next())
        {
            System.out.println(rs.getInt(1));
        }
        rs.close();
        ps.close();
        con.close();
    }
}

Saturday, October 6, 2012

Using Collection Merging in Spring to re-use a list

Recently, I came across a requirement where I had a set of rules to be run on various sub-types of a business object--Invoice. All sub-types shared a common set of rules and then each sub-type has some exclusive set of rules to be run along with those in the common set. I was using Spring for dependency injection and did it as follows, initially:

<bean id="rule1" class="com.vikdor.rules.Rule1" />
<bean id="rule2" class="com.vikdor.rules.Rule2" />
<bean id="rule3" class="com.vikdor.rules.Rule3" />
<bean id="rule4" class="com.vikdor.rules.Rule4" />
<bean id="rule5" class="com.vikdor.rules.Rule5" />

<util:list id="normalInvRules">
    <ref bean="rule1" />
    <ref bean="rule3" />
    <ref bean="rule5" />
    <ref bean="rule4" />
</util:list>

<util:list id="prepaidInvRules">
    <ref bean="rule1" />
    <ref bean="rule3" />
    <ref bean="rule5" />
    <ref bean="rule2" />
</util:list>
This obviously was not going to scale in my use case where I have good number of rules to be run and the number of sub-types are also more. I was wondering if there is a way to define the common list and then re-use it while defining the actual list of rules for a given type of invoice and then I came across Collection Merging concept in Spring where we can define a parent element and then have child elements inherit the properties from the parent element and override them (i.e. when there is a conflict, the child property definition takes precedence) and modified my configuration as follows and it works great!
<bean id="rule1" class="com.krovi.rules.Rule1" />
    <bean id="rule2" class="com.krovi.rules.Rule2" />
    <bean id="rule3" class="com.krovi.rules.Rule3" />
    <bean id="rule4" class="com.krovi.rules.Rule4" />
    <bean id="rule5" class="com.krovi.rules.Rule5" />

    <bean id="commonList"
        class="org.springframework.beans.factory.config.ListFactoryBean">
        <property name="sourceList">
            <list>
                <ref bean="rule1" />
                <ref bean="rule2" />
                <ref bean="rule3" />
            </list>
        </property>
    </bean>
    <bean id="firstExecutorRules" parent="commonList"
        class="org.springframework.beans.factory.config.ListFactoryBean">
        <property name="sourceList">
            <list merge="true">
                <ref bean="rule4" />
            </list>
        </property>
    </bean>
    <bean id="secondExecutorRules" parent="commonList"
        class="org.springframework.beans.factory.config.ListFactoryBean">
        <property name="sourceList">
            <list merge="true">
                <ref bean="rule5" />
            </list>
        </property>
    </bean>

Tuesday, August 14, 2012

Using @Configurable to inject resources into objects created using "new" operator (domain objects)

I am currently developing a query language based on a context-free grammar. The library contains a parser that parses the queries accepted by the language and returns a tree of parse entries. So, these parse entries essentially form the domain objects of the application. The library also provides the evaluator framework to evaluate the state of an object against a parse tree. A parse entry contains a reference to the evaluator that should be used to evaluate the state of an object against itself.
Since the parse entries are created using the new operator during the parse process and since it is not a good idea to hard-wire the evaluators into the parse entry classes, I was looking for a way to inject them. Then I came across the @Configurable annotation provided by Spring framework to inject beans into objects of a class whose objects are instantiated using the new operator instead of getting them from the Spring framework. I followed the Load Time Weaving approach to get this working, in a sample project, as follows:
  1. Configure the maven repositories required:
        
        
            com.springsource.repository.bundles.external
            SpringSource Enterprise Bundle Repository - External Bundle Releases
            http://repository.springsource.com/maven/bundles/external
        
    
  2. Configure the maven dependencies required:
    
        
            org.springframework
            spring-aspects
            3.1.1.RELEASE
        
        
            org.springframework
            spring-context
            3.1.1.RELEASE
        
        
            org.springframework
            spring-test
            3.1.1.RELEASE
        
        
            org.aspectj
            com.springsource.org.aspectj.weaver
            1.6.12.RELEASE
        
        
            junit
            junit
            4.10
        
    
    
    
  3. Created a sample domain object--Account.java, mark it as Configurable, and add a reference to the yet to be created AccountDAO.
    @Configurable
    public class Account
    {
        protected AccountDAO accountDAO;
    
        private int accountId;
        private String accountHolderName;
        private double balance;
    
        public Account(int accountId, String holderName, double balance)
        {
            this.accountId = accountId;
            this.accountHolderName = holderName;
            this.balance = balance;
        }
    
        public AccountDAO getAccountDAO()
        {
            return accountDAO;
        }
    
        /**
         * @return the accountId
         */
        public int getAccountId()
        {
            return accountId;
        }
    
        /**
         * @return the accountHolderName
         */
        public String getAccountHolderName()
        {
            return accountHolderName;
        }
    
        /**
         * @return the balance
         */
        public double getBalance()
        {
            return balance;
        }
    
        public void setAccountDAO(AccountDAO accountDAO)
        {
            this.accountDAO = accountDAO;
        }
    
        /**
         * @param accountId the accountId to set
         */
        protected void setAccountId(int accountId)
        {
            this.accountId = accountId;
        }
    
        /**
         * @param accountHolderName the accountHolderName to set
         */
        protected void setAccountHolderName(String accountHolderName)
        {
            this.accountHolderName = accountHolderName;
        }
    
        /**
         * @param balance the balance to set
         */
        protected void setBalance(double balance)
        {
            this.balance = balance;
        }
    
        public void persist()
        {
    
        }
    }
    
    
    The setter method for accountDAO must be public for Spring to find it and inject it(I found this strange as I thought the reflection based frameworks can access protected methods as well, which is the case with Hibernate.)
  4. Create the DAO and sample DAOImpl for the Account class.
    public interface AccountDAO
    {
        public Account createAccount(Account account);
        
        public Account updateAccount(Account account);
        
        public void delete(Account account);
    }
    
    public class AccountDAOImpl implements AccountDAO
    {
        public Account createAccount(Account account)
        {
    
            System.out.println("Created....");
            return account;
        }
    
        public Account updateAccount(Account account)
        {
            System.out.println("Update....");
            return account;
        }
    
        public void delete(Account account)
        {
            System.out.println("Deleted....");
        }
    }
    
  5. The application-config.xml file:
    
    
        
        
        
    
    
  6. Write a sample test class TestAccount
    @RunWith(SpringJUnit4ClassRunner.class)
    @ContextConfiguration(locations =
    { "file:META-INF/application-config.xml" })
    public class TestAccount
    {
    
        @Test
        public void test()
        {
            Account account = new Account(-1, "Krovi Sarma", 1630000.00);
            assertNotNull(account.getAccountDAO());
        }
    
    }
    
  7. Run the test class with the JVM argument -javaagent:/path/to/org.springframework.instrument.jar