Tuesday, October 27, 2015

Using SpringFox for API document generation.

While working on Spring MVC based RESTful services that expose APIs for other developers to use, I needed a library that could read the API specifications in the Controller classes, generate API documentation (so that it is always up-to-date with the changes to the API spec) and better if it can provide a way to invoke these APIs. Springfox's implementation of Swagger appeared to fit these requirements perfectly.

This post is a log of changes I made to my Spring MVC RESTful service to integrate with Springfox's Swagger implementation.
  1. Included the maven dependencies for swagger-springfox to expose the API details and swagger-springfox-ui to render the documentation and provide a "Try it out" interface to invoke these APIs.
  2.     
            io.springfox
            springfox-swagger2
            2.2.2
        
        
            io.springfox
            springfox-swagger-ui
            2.2.2
        
    
  3. Created a Configuration class to initialize SpringFox
  4. import static com.google.common.collect.Lists.newArrayList;
    import static springfox.documentation.schema.AlternateTypeRules.newRule;
    
    import org.joda.time.LocalDate;
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.context.annotation.Bean;
    import org.springframework.context.annotation.Configuration;
    import org.springframework.http.ResponseEntity;
    import org.springframework.web.bind.annotation.RequestMethod;
    import org.springframework.web.context.request.async.DeferredResult;
    
    import com.fasterxml.classmate.TypeResolver;
    
    import springfox.documentation.builders.PathSelectors;
    import springfox.documentation.builders.RequestHandlerSelectors;
    import springfox.documentation.builders.ResponseMessageBuilder;
    import springfox.documentation.schema.ModelRef;
    import springfox.documentation.schema.WildcardType;
    import springfox.documentation.spi.DocumentationType;
    import springfox.documentation.spring.web.plugins.Docket;
    import springfox.documentation.swagger2.annotations.EnableSwagger2;
    
    @Configuration
    @EnableSwagger2
    public class SwaggerConfig
    {
        @Autowired
        private TypeResolver typeResolver;
    
        @Bean
        public Docket petApi()
        {
            return new Docket(DocumentationType.SWAGGER_2)
                .select()
                .apis(RequestHandlerSelectors.any())
                .paths(PathSelectors.any())
                .build()
                .pathMapping("/")
                .directModelSubstitute(LocalDate.class, String.class)
                .genericModelSubstitutes(ResponseEntity.class)
                .alternateTypeRules(
                    newRule(
                        typeResolver.resolve(
                            DeferredResult.class,
                            typeResolver.resolve(ResponseEntity.class, WildcardType.class)),
                        typeResolver.resolve(WildcardType.class)))
                .useDefaultResponseMessages(false)
                .globalResponseMessage(
                    RequestMethod.GET,
                    newArrayList(
                        new ResponseMessageBuilder()
                            .code(500)
                            .message("500 message")
                            .responseModel(new ModelRef("Error"))
                            .build()))
                .enableUrlTemplating(false);
        }
    
    }
    
  5. With the addition of Springfox Configuration above, the test cases started failing due to missing ServletContext bean. Added the @WebAppConfiguration annotation to the parent test class that also has the test spring configuration initialized.
  6. import org.junit.Ignore;
    import org.junit.runner.RunWith;
    import org.springframework.test.annotation.DirtiesContext;
    import org.springframework.test.annotation.DirtiesContext.ClassMode;
    import org.springframework.test.context.ContextConfiguration;
    import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
    import org.springframework.test.context.web.WebAppConfiguration;
    
    @RunWith(SpringJUnit4ClassRunner.class)
    @ContextConfiguration(locations = { "classpath:test-application-spring-config.xml" })
    @DirtiesContext(classMode = ClassMode.AFTER_CLASS)
    @WebAppConfiguration
    @Ignore("Ingoring an abstract test class")
    public abstract class AbstractIntegrationTest
    {
    
    }
    
    
  7. Added the following resource mappings to serve the swagger related resources from springfox-swagger-ui.jar.
  8.     
    
Notes:
  1. Initially, I set the "enableUrlTemplating" attribute to "true" in SwaggerConfiguration. Due to this the URLs in "Try it out" section included placeholders for the request parameters, making this feature use less.

    E.g.
    URL when the attribute is set to true:
            http://localhost:8080/my-api{?param1,param2}?param1=&param2=&

    URL after the attribute value is changed to false:
            http://localhost:8080/my-api?param1=&param2=&
  2. At first, I included the mvc resource mappings in the SwaggerConfiguration class itself, by extending the WebMvcConfigurerAdapter class, but access to swagger-ui.html returned 404 error. So, I removed this from here and added them to the XML configuration and then the swagger-ui.html came up properly.
  3.     @Override
        public void addResourceHandlers(final ResourceHandlerRegistry registry)
        {
            registry.addResourceHandler("swagger-ui.html").addResourceLocations("classpath:/META-INF/resources/");
            registry.addResourceHandler("/webjars/**").addResourceLocations("classpath:/META-INF/resources/webjars/");
        }
    
  4. I tried to render swagger-ui.html under a different URL (e.g. /my-service-api) but, due to hard coded lookup for swagger-ui.html in the springfox.js, even though the page loaded, it couldn't proceed further to make call to the /v2/api-docs to get the JSON and render it.

Sunday, December 21, 2014

Configuring Log4J with Slf4J in Spring-based application

While working with a Spring Boot based application, the spring-boot jar brings in the sl4j related jars which seems to be configured to jakarta-commons-logging by default. So, the log4j.properties in the classpath was never read by the log4j system.

To make log4j work with sl4j in this scenario, the sl4j dependencies coming from spring-boot have been excluded as follows:


 org.springframework.boot
 spring-boot-starter-data-rest
 ${spring-boot-version}
 
  
   org.slf4j
   slf4j-api
  
  
   org.slf4j
   jul-to-slf4j
  
  
   org.slf4j
   log4j-over-slf4j
  
  
   org.slf4j
   jcl-over-slf4j
  
  
   ch.qos.logback
   logback-classic
  
 

Then, the sl4j and log4j dependencies are pulled in separately, as follows:
        
            org.slf4j
            slf4j-api
            ${slf4j.version}
        
        
            org.slf4j
            slf4j-log4j12
            ${slf4j.version}
        
        
            log4j
            log4j
            1.2.16
            runtime
        
The Spring Test library seems to require JCL (jakarta-commons-logging) as it was throwing the following exception:

java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
 at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.clinit(SpringJUnit4ClassRunner.java:86)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
 at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
 at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
 at java.lang.reflect.Constructor.newInstance(Unknown Source)


Included the jcl-over-slf4j dependency to fix the above exception:

    org.slf4j
    jcl-over-slf4j
    ${slf4j.version}

Now, the log4j.properties file in src/main/resources is read by the log4j system, without any additional configuration.

Tuesday, January 15, 2013

Functional testing in SOA environment

My first project in a service oriented environment has reached functional testing stage. The product involves processing large amounts of business objects whose reference data is obtained from other subsystems through service calls, in production.

While devising the strategy for functional testing, the issue was how to get this reference data in a test environment.

Unit Testing

During unit testing, the service calls were mocked using a mocking library (Mockito in our case), and the expected responses were returned using when - thenReturn construct, as follows:


import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;

import org.mockito.MockitoAnnotations;
import org.mockito.Mock;
import org.junit.Before;
import org.junit.Test;

public class SweepTest 
{
    /*
     * Specify that the service wrapper should be mocked.
     */
    @Mock
    private XServiceWrapper xServiceWrapper;

    /*
     * SomeBusinessObjectManager has reference to XServiceWrapper and we want the 
     * mocked instance to be injected instead of the normal implementation.
     */
    @InjectMocks
    @Autowired
    private SomeBusinessObjectManager boManager;

    @Before
    public void setup()
    {
         MockitoAnnotations.initMocks(this);

         /*
          * Mock the response object from the service.
          */
         XServiceResponse serviceResponse = mock(XServiceResponse.class);
         when(serviceResponse.getXValue()).thenReturn(myValue);

         /*
          * Mock the service call and return mocked response object.
          */
         when(xServiceWrapper.serviceCall()).thenReturn(serviceResponse);
    }
}


Functional Testing

During functional testing, we identified golden set of use cases that necessarily and sufficiently cover the behavior of the system in various scenarios. The data for various internal objects for these golden set of use cases is prepared. But, it is not feasible to get relevant data from the external services that would suit the input data for a given use case. So, the following changes were done to the project w.r.t interacting with external services:
  1. Define an ExternalServiceFacade interface that would define all the calls to be made to different external services.
  2. /**
     * ExternalServiceFacade.java
     * 
     * $Source$
     */
    /*
     * Copyright (c) 2013 Krovi.com, Inc. All rights reserved.
     */
    
    package com.krovi.app.manager;
    
    import com.krovi.services.BusinessObjectA;
    import com.krovi.services.BusinessObjectB;
    import com.krovi.services.BusinessObjectC;
    
    /**
     * This class follows the Facade pattern and provides APIs to
     * interact with various external services that provide reference data for the given object IDs.
     * 
     * @author krovi
     * @version $Id$
     */
    public interface ExternalServiceFacade
    {
        BusinessObjectA lookupBusinessObjectA(String referenceId);
    
        BusinessObjectB lookupBusinessObjectB(String referenceId1, String referenceId2);
    
        BusinessObjectC lookupBusinessObjectC(String referenceId1, String referenceId2, Long referenceValue3);
    
    }
    
    
  3. Provide an implementation that delegates the calls to the actual services and obtains the responses. This implementation will be used in production scenarios.
  4. /**
     * ExternalServiceFacadeImpl.java
     * 
     * $Source$
     */
    /*
     * Copyright (c) 2013 Krovi.com, Inc. All rights reserved.
     */
    
    package com.krovi.app.manager;
    
    import com.krovi.services.BusinessObjectA;
    import com.krovi.services.BusinessObjectAService;
    import com.krovi.services.BusinessObjectAServiceClient;
    import com.krovi.services.BusinessObjectB;
    import com.krovi.services.BusinessObjectBService;
    import com.krovi.services.BusinessObjectBServiceClient;
    import com.krovi.services.BusinessObjectC;
    import com.krovi.services.BusinessObjectCService;
    import com.krovi.services.BusinessObjectCServiceClient;
    
    /**
     * @author krovi
     * @version $Id$
     */
    public class ExternalServiceFacadeImpl implements ExternalServiceFacade
    {
        @Autowired
        private BusinessObjectAService boAService;
    
        @Autowired
        private BusinessObjectBService boBService;
    
        @Autowired
        private BusinessObjectCService boCService;
    
        @Override
        public BusinessObjectA lookupBusinessObjectA(String referenceId)
        {
            BusinessObjectAServiceClient client = boAService.getClient();
            return client.lookupBusinessObjectA(referenceId);
        }
    
        @Override
        public BusinessObjectB lookupBusinessObjectB(String referenceId1, String referenceId2)
        {
            BusinessObjectBServiceClient client = boBService.getClient();
            return client.lookupBusinessObjectB(referenceId1, referenceId2);
        }
    
        @Override
        public BusinessObjectC lookupBusinessObjectC(String referenceId1, String referenceId2, Long referenceValue3)
        {
            BusinessObjectCServiceClient client = boCService.getClient();
            return client.lookupBusinessObjectC(referenceId1, referenceId2, referenceValue3);
        }
    
    }
    
    
  5. Provide a mock implementation that will connect to a local data store that is populated with the expected data and returns it. This implementation will be used in functional testing scenario.
  6. /**
     * MockExternalServiceFacadeImpl.java
     * 
     * $Source$
     */
    /*
     * Copyright (c) 2013 Krovi.com, Inc. All rights reserved.
     */
    
    package com.krovi.app.manager;
    
    import org.springframework.beans.factory.annotation.Autowired;
    import org.springframework.jdbc.core.JdbcTemplate;
    
    import com.krovi.services.BusinessObjectA;
    import com.krovi.services.BusinessObjectB;
    import com.krovi.services.BusinessObjectC;
    
    /**
     * @author krovi
     * @version $Id$
     */
    public class MockExternalServiceFacadeImpl implements ExternalServiceFacade
    {
        @Autowired
        private JdbcTemplate jdbcTemplate;
    
        public MockExternalServiceFacadeImpl()
        {
            setupDBObjects();
        }
    
        private void setupDBObjects()
        {
            /**
             * 1. This method creates DB tables corresponding to the structures of BusinessObject* classes. 2. The test
             * harness would insert test data into these tables as per the use cases.
             */
        }
    
        @Override
        public BusinessObjectA lookupBusinessObjectA(String referenceId)
        {
            // Look up the database instead of making a service call. The database would contain the appropriate object
            // required for the test case that is being currently run.
            return null;
        }
    
        @Override
        public BusinessObjectB lookupBusinessObjectB(String referenceId1, String referenceId2)
        {
            // Look up the database instead of making a service call. The database would contain the appropriate object
            // required for the test case that is being currently run.
            return null;
        }
    
        @Override
        public BusinessObjectC lookupBusinessObjectC(String referenceId1, String referenceId2, Long referenceValue3)
        {
            // Look up the database instead of making a service call. The database would contain the appropriate object
            // required for the test case that is being currently run.
            return null;
        }
    
    }
    
    
  7. Modify all calls to services in the entire code base to go through ExternalServiceFacade.
  8. The spring configuration files for production and testing stages are modified to inject the appropriate implementation of the ExternalServiceFacade.

Wednesday, October 17, 2012

@Memoize in Java using Spring and Aspects

Recently, while writing an application in service oriented architecture, I had a couple of places in the code where I needed to make a service call to get the details by a particular application key. I was wondering about duplicate calls, i.e. multiple service calls made to get the details for the same application key. As we all know, service calls are costly with all the marshalling, unmarshalling, cache lookups, database lookups involved, I wanted to cache the results of a service call by application key or keys and this is a clear cross-cutting concern. Having programmed in Perl and Python that have the memoize option, I thought a Memoize aspect would be most appropriate here. As always, I was looking for Spring's Aspect Oriented Programming APIs and though it was tricky to get the set up right, I ended up with a very small amount of code that does the basic memoization. Since the set up was tricky, I thought I would document here for reference:


  1. Set up Spring, AspectJ, CGLib and ASM for the application.
    1. The following Spring jars are needed:
      1. spring-aop (v3.3.1)
      2. spring-aspects (v3.3.1)
      3. spring-beans (v3.3.1)
      4. spring-context (v3.3.1)
      5. spring-context-support (v3.3.1)
      6. spring-core (v3.3.1)
      7. spring-test (v3.3.1)
      8. spring-expression (v3.3.1)
      9. spring-asm (v3.3.1)
    2. The following AspectJ jars are needed for load-time-weaving and aspectj runtime.
      1. aspectj-rt (v1.6.9)
      2. aspectj-weaver (v1.5.4)
      3. aop-alliance (v1.0)
    3. The following CGLib jars are needed for code generation
      1. cglib (v2.2.3)
    4. The following ASM jars are needed for byte-code manipulation
      1. asm (v3.3.1)
    5. Commons logging jar needed by Spring
      1. commons-logging (v1.1.1)
  2. The CGLib and ASM libraries should be compatible with each other. So, if you choose a different version of any of these jars, make sure to select the compatible other one. Otherwise, you will end up with a runtime error (NoSuchMethodError).
  3. Enable aspects in your spring application using the following configuration:
    <?xml version="1.0" encoding="UTF-8"?>
    <beans xmlns="http://www.springframework.org/schema/beans"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
        xmlns:aop="http://www.springframework.org/schema/aop"
        xsi:schemaLocation="
                http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd 
                http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
                http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.0.xsd
                ">
    
        <context:component-scan base-package="com.vikdor.aspects" />
        <aop:aspectj-autoproxy proxy-target-class="true" />
    </beans>
    
  4. Define the annotation that is going to be used in the advice:
    import java.lang.annotation.Retention;
    
    @Retention(RetentionPolicy.RUNTIME)
    public @interface Memoize
    {
    
    }
    
  5. Write a simple Aspect class, annotate it with @Aspect and define an @Around advice for all the methods that use the @Memoize annotation, as follows:
    @Aspect
    @Component
    public class MemoizeAspect
    {
        private WeakHashMap<Object[], Object> cache = new WeakHashMap<Object[], Object>();
    
        @Around("@annotation(com.vikdor.aspects.Memoize)")
        public Object handledMemoize(ProceedingJoinPoint pjp) throws Throwable
        {
            System.out.println("Handling memoize");
            Object[] args = pjp.getArgs();
    
            if (args != null)
            {
                Object response = cache.get(Arrays.asList(args));
                if (response != null)
                {
                    return response;
                }
            }
            Object obj = pjp.proceed();
            if (args != null)
            {
                cache.put(Arrays.asList(args), obj);
            }
            return obj;
        }
    }
    
Now, add @Memoize to any method whose request and response should be cached and it just works. BTW, this is a very naive Memoize implementation and is not production ready at all ;-)

Sunday, October 7, 2012

Connecting to SQL Server Express through JDBC

I came across this question in a decent number of forums where the subtleties of connecting to an instance of SQL Server Express using JDBC are missed out. So, thought it would be good to just document how I approached it, here for my reference and for others, if they find useful.
Once you install the SQL Server Express edition, you need to configure the TCP/IP settings for the JDBC drivers to connect to the SQLEXPRESS database instance.

Configuring the TCP/IP:

  1. Configure TCP/IP communication with SQL Express
    1. Open SQL Server Configuration Manager.
    2. Go to SQL Server Network Configuration -> Protocols for SQLEXPRESS
    3. Set the status of TCP/IP protocol to "Enabled" (if it is already not).
    4. Open Properties window for TCP/IP, go to IP Addresses section.
    5. Go to the bottom of this property page and set the TCP Port under `IPAll` to 1433.
  2. Connect to the SQLExpress instance using `Microsoft's JDBC driver for SQL Server`
    1. JDBC URL: `jdbc:sqlserver://localhost;instance=SQLEXPRESS;databaseName=<your DB>;user=<your User>;password=<your Passwd>`

Using Windows Integrated Authentication:
  1. Create a windows account for the application that would be used to run your programs. This account's credentials will be used to connect to the server.
  2. Get Microsoft JDBC Driver for SQL Server from here.
  3. Configure the JDBC URL as follows:

    jdbc:sqlserver://<hostname>;databaseName=<DBName>;integratedSecurity=true
  4. Configure the launcher that run the Java programs from command line to include the following JVM parameter:

    -Djava.library.path="<jdbc driver dll location>"

    where the location is the directory where the JDBC driver downloaded earlier is installed or extracted. It was C:\Program Files\sqljdbc_4.0.2206.100_enu\sqljdbc_4.0\enu\auth\x64 in my case. The dll should be picked based on the JVM used for running these programs.

With the above configuration, the connection established to SQL Server would use the Windows Authentication Credentials of the domain user running the java program/process.

Sample Code:

public class SQLServerEx
{
    static enum Drivers
    {
        MICROSOFT(
                  SQLServerDriver.class.getName(),
                  "jdbc:sqlserver://"
                      + "localhost;"
                      + "instance=SQLEXPRESS;databaseName=vikdor_test;user=vikdor_user;password=vikdor_user"),
        MICROSOFT_INTEG(
                        SQLServerDriver.class.getName(),
                        "jdbc:sqlserver://"
                            + "localhost;"
                            + "instance=SQLEXPRESS;databaseName=vikdor_test;integratedSecurity=true");

        private String driverClass;
        private String driverURL;

        private Drivers(String driverClass, String driverURL)
        {
            this.driverClass = driverClass;
            this.driverURL = driverURL;
        }

        public String getDriverClass()
        {
            return driverClass;
        }

        public String getDriverURL()
        {
            return driverURL;
        }
    }

    public static void main(String[] args) throws Exception
    {
        Class.forName(Drivers.MICROSOFT_INTEG.getDriverClass());
        Connection con =
            DriverManager.getConnection(Drivers.MICROSOFT_INTEG.driverURL);
        PreparedStatement ps =
            con.prepareStatement(
                "insert into auto_table(name, role) values(?, ?)",
                Statement.RETURN_GENERATED_KEYS);
        ps.setString(1, "vikdor");
        ps.setString(2, "sde");
        ps.executeUpdate();
        ResultSet rs = ps.getGeneratedKeys();
        while (rs.next())
        {
            System.out.println(rs.getInt(1));
        }
        rs.close();
        ps.close();
        con.close();
    }
}