Thursday, June 1, 2017

Handling Concurrently Executions of Expensive tasks

In order to handle concurrently executions of expensive tasks, Brian Goetz provided an example of Memorizer in his book: “Java Concurrency In Practice”. The following is a slightly modified version of the class.


public class ConcurrentResultCache<A, V> implements Computable<A, V> {
      private static final Logger logger = LoggerFactory.getLogger(ConcurrentResultCache.class);
      private final ConcurrentMap<A, Future<V>> cache = new ConcurrentHashMap<A, Future<V>>();
      private final Computable<A, V> c;

      public ConcurrentResultCache(final Computable<A, V> c) {
            this.c = c;
      }

      public V compute(final A arg) throws InterruptedException {
            logger.trace("entered compute...");
            // waiting result to be available. intercept any CacellationException.
            while (true) {
                  logger.trace("before first get");
                  Future<V> f = cache.get(arg);
                  logger.trace("after first get");
                  if (f == null) {
                        logger.trace("f is null");
                        // Creating a new task wrapped in a Callable.
                        Callable<V> eval = new Callable<V>() {
                              public V call() throws InterruptedException {
                                    return c.compute(arg);
                              }
                        };
                        FutureTask<V> ft = new FutureTask<V>(eval);
                        f = cache.putIfAbsent(arg, ft);
                        logger.trace("after putIfAbsent()");
                        if (f == null) {
                              // if the task is put into the cache first time, run the task in the SAME thread.
                              f = ft;
                              logger.trace("before run()");
                              ft.run();
                        }
                  }
                  try {
                        logger.trace("before second get");
                        return f.get();
                  } catch (CancellationException e) {
                        cache.remove(arg, f);
                  } catch (ExecutionException e) {
                        throw new IllegalStateException("Task execution exception caught.", e);
                  }
            }
      }
     
      public void remove(final String key) {
            this.cache.remove(key);
      }
}
     
      public void remove(final String key) {
            this.cache.remove(key);
      }
}


As an example, if a Properties need to be loaded in a high traffic application, it could implement the Computable interface.

public class PropertyLoader implements Computable<String, Properties> {
      private static final Logger logger = LoggerFactory.getLogger(PropertyLoader.class);

      @Override
      public Properties compute(final String filePath) throws InterruptedException {
            logger.trace("entered PropertyLoader compute()");
            return readProperties(filePath);
      }

      private Properties readProperties(final String filePath) {
            final String propFile = System.getProperty(filePath);
            final Properties properties = new Properties();
            try {
                  properties.load(new FileReader(propFile));
            } catch (IOException e) {
                  logger.error("Failed to load properties.", e);
            }
            return properties;
      }
}


The usage of the class:

private static ConcurrentResultCache<String, Properties> propertiesCache = new ConcurrentResultCache<String, Properties>(new PropertyLoader());

public Properties getClientThrottlingRateMap() {
    final Properties properties = propertiesCache.compute(propertiesLocation);


Wednesday, May 31, 2017

HTTP Status Codes in Restful API


Here is a list of most common HTTP codes that are used in Restful API designs:

  • 200 - OK
  • 400 - Bad Request (Client Error) - a json with error and more details of the errors.
  • 401 - Unauthorized
  • 500 - Internal Server Error - a json with an error may return to the client without details to avoid security risk

  • 202 (Accepted) - when no response returns, e.g. asyncronous processing is used
  • 401 (Unauthorized) - when there is a problem with the client’s credentials
  • 403 (Forbidden) - forbid access regardless of authorization state
  • 404 (Not Found) - when a client’s URI cannot be mapped to a resource

Sample of 400 response:


{
  "errors": [
    {
      "error": {
        "code": "100000400",
        "field": "transactionId",
        "message": "transaction is required."
        "link": "https://url link to a page with explanation"
      }
    }
  ]
}


Sample of 500 response:


{
  "error": {
    "code": "100000500",
    "reference": "3344c2e4-d54c-5caf-a4ea-c342cb4cccad",
    "message": "Server error."
  }
}
















































HTTP Status Codes Usage Examples

APIStatus Codes
Twitter200, 304, 400, 401, 403, 404, 406, 410, 420, 422, 429, 500, 502, 503, 504
Stripe200, 400, 401, 402, 404, 429, 500, 502, 503, 504
Github200, 400, 422, 301, 302, 304, 307, 401, 403
Pagerduty200, 201, 204, 400, 401, 403, 404, 408, 500
NewRelic Plugins200, 400, 403, 404, 405, 413, 500, 502, 503, 503
Etsy200, 201, 400, 403, 404, 500, 503
Dropbox200, 400, 401, 403, 404, 405, 429, 503, 507
Spotify200, 201, 204, 304, 400, 401, 403, 404, 429, 500, 502, 503
Google Cloud200, 301, 304, 307, 308, 400, 401, 403, 404, 405, 409, 411, 412, 416, 429, 500, 501, 503
HipChat200, 201, 202, 400, 401, 403, 404, 405, 429, 500, 503
Telegram200, 303, 400, 401, 403, 404, 420, 500
Pocket200, 400, 401, 403, 503
Uber200, 201, 400, 401, 403, 404, 406, 409, 422, 429, 500



Sunday, May 28, 2017

Maven Reporting Customization: PMD, findbug, checkstyle

How to customize Maven PMD, findbug, checkstype configuration with Maven build extension and reporting plugin

checkstyle-suppressions.xml


<?xml version="1.0"?>

<!DOCTYPE suppressions PUBLIC
"-//Puppy Crawl//DTD Suppressions 1.1//EN"
"http://www.puppycrawl.com/dtds/suppressions_1_1.dtd">

<suppressions>
       <!-- Supress scanning generated files -->
             <suppress files="com[\\/]americanexpress[\\/]sps[\\/]service" checks=".*" />
             <suppress files="com[\\/]americanexpress[\\/]sps[\\/]service2" checks=".*" />
      
       <!-- Supress binaries -->
             <suppress files=".+\.(?:jar|zip|war|class|tar|bin)$" checks=".*" /> 
      
       <!-- Supress FileLength checks -->
             <suppress files="com[\\/]americanexpress[\\/]sps[\\/].*.java" checks="FileLength" />
</suppressions>


findbugs-exclude.xml


<FindBugsFilter>
       <Match>
             <!-- Exclude package com.americanexpress.sps.service2 -->
             <Package name="~com\.americanexpress\.sps\.service\.*" />


       </Match>
       <Match>
             <!-- Exclude package com.americanexpress.sps.service2 -->

             <Package name="~com\.americanexpress\.sps\.service2\.*" />


       </Match>

       <Match>
             <!-- Exclude package com.americanexpress.sps.intranet -->

             <Package name="~com\.americanexpress\.sps\.intranet\.*" />

       </Match>
       <Match>
             <Class name="~.*Test$" />
       </Match>
       <Match>
             <Package name="~test\..*" />
       </Match>

</FindBugsFilter>

sun_checks.xml


<?xml version="1.0"?>
<!DOCTYPE module PUBLIC
    "-//Puppy Crawl//DTD Check Configuration 1.2//EN"
    "http://www.puppycrawl.com/dtds/configuration_1_2.dtd">

<!--

  Checkstyle configuration that checks the sun coding conventions from:

    - the Java Language Specification at
      http://java.sun.com/docs/books/jls/second_edition/html/index.html

    - the Sun Code Conventions at http://java.sun.com/docs/codeconv/

    - the Javadoc guidelines at
      http://java.sun.com/j2se/javadoc/writingdoccomments/index.html

    - the JDK Api documentation http://java.sun.com/j2se/docs/api/index.html

    - some best practices

  Checkstyle is very configurable. Be sure to read the documentation at
  http://checkstyle.sf.net (or in your downloaded distribution).

  Most Checks are configurable, be sure to consult the documentation.

  To completely disable a check, just comment it out or delete it from the file.

  Finally, it is worth reading the documentation.

-->


<module name="Checker">
    <!--
        If you set the basedir property below, then all reported file
        names will be relative to the specified directory. See
        http://checkstyle.sourceforge.net/5.x/config.html#Checker

        <property name="basedir" value="${basedir}"/>
    -->

    <!-- Checks that each Java package has a Javadoc file used for commenting. -->
    <!-- See http://checkstyle.sf.net/config_javadoc.html#JavadocPackage       -->
    <module name="JavadocPackage">
      <property name="allowLegacy" value="true"/>
    </module>

    <!-- Checks whether files end with a new line.                        -->
    <!-- See http://checkstyle.sf.net/config_misc.html#NewlineAtEndOfFile -->
    <module name="NewlineAtEndOfFile"/>

    <!-- Checks that property files contain the same keys.         -->
    <!-- See http://checkstyle.sf.net/config_misc.html#Translation -->
    <module name="Translation"/>

    <module name="FileLength"/>

    <!-- Following interprets the header file as regular expressions. -->
    <!-- <module name="RegexpHeader"/>                                -->

    <module name="FileTabCharacter">
        <property name="eachLine" value="false"/>
    </module>

    <module name="RegexpSingleline">
        <!-- \s matches whitespace character, $ matches end of line. -->
        <property name="format" value="\s+$"/>
        <property name="message" value="Line has trailing spaces."/>
    </module>

    <module name="TreeWalker">

        <property name="cacheFile" value="${checkstyle.cache.file}"/>

        <!-- required for SuppressWarningsFilter (and other Suppress* rules not used here) -->
        <!-- see http://checkstyle.sourceforge.net/config_annotation.html#SuppressWarningsHolder -->
        <module name="SuppressWarningsHolder"/>

        <!-- Checks for Javadoc comments.                     -->
        <!-- See http://checkstyle.sf.net/config_javadoc.html -->
        <module name="JavadocMethod"/>
        <module name="JavadocType"/>
        <module name="JavadocVariable"/>
        <module name="JavadocStyle"/>


        <!-- Checks for Naming Conventions.                  -->
        <!-- See http://checkstyle.sf.net/config_naming.html -->
        <module name="ConstantName"/>
        <module name="LocalFinalVariableName"/>
        <module name="LocalVariableName"/>
        <module name="MemberName"/>
        <module name="MethodName"/>
        <module name="PackageName"/>
        <module name="ParameterName"/>
        <module name="StaticVariableName"/>
        <module name="TypeName"/>


        <!-- Checks for Headers                                -->
        <!-- See http://checkstyle.sf.net/config_header.html   -->
        <!-- <module name="Header">                            -->
            <!-- The follow property value demonstrates the ability     -->
            <!-- to have access to ANT properties. In this case it uses -->
            <!-- the ${basedir} property to allow Checkstyle to be run  -->
            <!-- from any directory within a project. See property      -->
            <!-- expansion,                                             -->
            <!-- http://checkstyle.sf.net/config.html#properties        -->
            <!-- <property                                              -->
            <!--     name="headerFile"                                  -->
            <!--     value="${basedir}/java.header"/>                   -->
        <!-- </module> -->


        <!-- Checks for imports                              -->
        <!-- See http://checkstyle.sf.net/config_import.html -->
        <module name="AvoidStarImport"/>
        <module name="IllegalImport"/> <!-- defaults to sun.* packages -->
        <module name="RedundantImport"/>
        <module name="UnusedImports"/>


        <!-- Checks for Size Violations.                    -->
        <!-- See http://checkstyle.sf.net/config_sizes.html -->
        <module name="LineLength">
             <property name="max" value="200" />
        </module>
        <module name="MethodLength"/>
        <module name="ParameterNumber"/>


        <!-- Checks for whitespace                               -->
        <!-- See http://checkstyle.sf.net/config_whitespace.html -->
        <module name="EmptyForIteratorPad"/>
        <module name="MethodParamPad"/>
        <module name="NoWhitespaceAfter"/>
        <module name="NoWhitespaceBefore"/>
        <module name="OperatorWrap"/>
        <module name="ParenPad"/>
        <module name="TypecastParenPad"/>
        <module name="WhitespaceAfter"/>
        <module name="WhitespaceAround"/>


        <!-- Modifier Checks                                    -->
        <!-- See http://checkstyle.sf.net/config_modifiers.html -->
        <module name="ModifierOrder"/>
        <module name="RedundantModifier"/>


        <!-- Checks for blocks. You know, those {}'s         -->
        <!-- See http://checkstyle.sf.net/config_blocks.html -->
        <module name="AvoidNestedBlocks"/>
        <module name="EmptyBlock"/>
        <module name="LeftCurly"/>
        <module name="NeedBraces"/>
        <module name="RightCurly"/>


        <!-- Checks for common coding problems               -->
        <!-- See http://checkstyle.sf.net/config_coding.html -->
        <module name="AvoidInlineConditionals"/>
        <module name="EmptyStatement"/>
        <module name="EqualsHashCode"/>
        <module name="HiddenField"/>
        <module name="IllegalInstantiation"/>
        <module name="InnerAssignment"/>
        <module name="MagicNumber"/>
        <module name="MissingSwitchDefault"/>
        <module name="SimplifyBooleanExpression"/>
        <module name="SimplifyBooleanReturn"/>

        <!-- Checks for class design                         -->
        <!-- See http://checkstyle.sf.net/config_design.html -->
        <module name="DesignForExtension"/>
        <module name="FinalClass"/>
        <module name="HideUtilityClassConstructor"/>
        <module name="InterfaceIsType"/>
        <module name="VisibilityModifier"/>


        <!-- Miscellaneous other checks.                   -->
        <!-- See http://checkstyle.sf.net/config_misc.html -->
        <module name="ArrayTypeStyle"/>
        <module name="FinalParameters"/>
        <module name="TodoComment"/>
        <module name="UpperEll"/>

    </module>

    <!-- Support @SuppressWarnings (added in Checkstyle 5.7) -->
    <!-- see http://checkstyle.sourceforge.net/config.html#SuppressWarningsFilter -->
    <module name="SuppressWarningsFilter"/>

    <!-- Checks properties file for a duplicated properties. -->
    <!-- See http://checkstyle.sourceforge.net/config_misc.html#UniqueProperties -->
    <module name="UniqueProperties"/>

</module>


in project pom.xml


<build>
       <extensions>
             <extension>
                    <groupId>package</groupId>
                    <artifactId>reportConfig</artifactId>
                    <version>${version}</version>
             </extension>
       </extensions>
</build>


in root pom.xml


<reporting>
    <plugins>
        <!-- CheckStyle Report -->
        <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-checkstyle-plugin</artifactId>
        <configuration>
            <!-- <configLocation>./maven-config/checkstyle.xml</configLocation> -->
            <suppressionsLocation>maven-config/checkstyle-suppressions.xml</suppressionsLocation>
            <suppressionsFileExpression>checkstyle.suppressions.file</suppressionsFileExpression>
        </configuration>
        </plugin>

        <!-- FindBugs Report -->
        <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>findbugs-maven-plugin</artifactId>
        <version>3.0.4</version>
        <configuration>
            <findbugsXmlOutput>true</findbugsXmlOutput>
            <findbugsXmlWithMessages>true</findbugsXmlWithMessages>
            <xmlOutput>true</xmlOutput>
            <excludeFilterFile>maven-config/findbugs-exclude.xml</excludeFilterFile>
        </configuration>
        </plugin>

        <!-- PMD Report -->
        <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-pmd-plugin</artifactId>
        <version>3.7</version>
        <configuration>
            <linkXref>true</linkXref>
            <sourceEncoding>utf-8</sourceEncoding>
            <minimumTokens>100</minimumTokens>
            <targetJdk>1.5</targetJdk>
            <excludes>
            <exclude>**/*SomeBean.java</exclude>
            <exclude>**/generated/*.java</exclude>
            </excludes>
            <excludeRoots>
            <excludeRoot>target/generated-sources/stubs</excludeRoot>
            </excludeRoots>
        </configuration>
        </plugin>
</reporting>