Friday, January 23, 2015

Add Java Decompiler to RAD 8.0.4.3

The update site:

 http://jd.benow.ca/jd-eclipse/update

0.15 works for me.
Posted by at No comments:

Add Subclipse plugin for RAD 8.0.4.3

Subclipse update url:

subclipse - http://subclipse.tigris.org/update_1.6.x

Don't choose the Subclipse Integration for Mylyn 3.x


Posted by at No comments:

Wednesday, January 14, 2015

WebSphere Commerce Search NullPointerException and Root Causes

You may see NullPointerException in Search server log when there are configuration problem.

The following is a list of the problems I have had so far:

1. wc-search.xml configuration problem:

If you did a APAR update or similar, the wc-search.xml in the com.ibm.comerce.catalog-ext maybe modified after the setupsearchindex.sh was executed

    <_config:server name="AdvancedConfiguration">
        <_config:common-http URL="http://localhost:10080/solr"
            allowCompression="true" connectionTimeout="15000"
            defaultMaxConnectionsPerHost="600" followRedirects="false"
            maxRetries="1" maxTotalConnections="600"
            reference="com.ibm.commerce.foundation.server.services.search.url"
            retryTimeInterval="1000" soTimeout="15000"/>
    </_config:server>
    <_config:cores>
        <_config:core catalog="11101" indexName="CatalogEntry"
            language="en_US" name="MC_11101_CatalogEntry_en_US"
            path="/MC_11101/en_US/CatalogEntry" serverName="BasicConfiguration"/>

then the serverName above does not match the one set in the _config:server, IBM code will throw a NullPointerException:

 [1/14/15 11:48:27:150 PST] 000000b7 AbstractBaseR E com.ibm.commerce.foundation.rest.resourcehandler.AbstractBaseResourceHandler handleException(String, Exception, String) Caught unexpected exception, error is: {errors=[{errorCode=CWXFR0230E, errorKey=ERR_INTERNAL_SERVER_ERROR, errorLevel=SEVERE, errorMessage=CWXFR0230E: Internal server error. Details will be stored within the server logs.}]}
                                 com.ibm.commerce.foundation.internal.common.exception.FoundationApplicationException: An error has occurred.  If this problem persists, contact product support.
at com.ibm.commerce.foundation.server.services.search.SearchServiceFacade.performSearch(SearchServiceFacade.java:200)
at com.ibm.commerce.rest.search.handler.v2_0.AbstractSearchResourceHandler.executeSearch(AbstractSearchResourceHandler.java:333)
at com.ibm.commerce.rest.search.handler.v2_0.AbstractSearchResourceHandler.performSearch(AbstractSearchResourceHandler.java:165)
at com.ibm.commerce.rest.search.handler.v2_0.CategoryViewHandler.findSubCategories(CategoryViewHandler.java:482)
at com.ibm.commerce.rest.search.handler.v2_0.CategoryViewHandler.findTopCategories(CategoryViewHandler.java:268)
...
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1660)
Caused by: java.lang.NullPointerException
at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:946)
at com.ibm.commerce.foundation.server.services.rest.search.processor.solr.SolrRESTSearchExpressionProcessor.performSearch(SolrRESTSearchExpressionProcessor.java:467)
at com.ibm.commerce.foundation.server.services.search.SearchServiceFacade.performSearch(SearchServiceFacade.java:193)
... 84 more

2. ibm-web-ext.xml flag setting


If the flag of the enable-file-serving is not set to true in the ibm-web-ext.xml file, and you customized the Search-Solr.war and tries to access files, you may have NullPointerException from Apache Solr package too, since the Apache Solr code fails to retrieve request path from ServletRequest: req.getServletPath()

[1/15/15 9:16:55:431 PST] 00000028 SolrDispatchF E org.apache.solr.common.SolrException log null:java.lang.NullPointerException
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:190)

/usr/IBM/WebSphere/AppServer/v70/profiles/"instance name"/config/cells/cell01/applications/Search.ear/deployments/Search/Search-Solr.war/WEB-INF/ibm-web-ext.xml

 <?xml version="1.0" encoding="UTF-8"?>

<web-ext xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://websphere.ibm.com/xml/ns/javaee"
    xsi:schemaLocation="http://websphere.ibm.com/xml/ns/javaee http://websphere.ibm.com/xml/ns/javaee/ibm-web-ext_1_0.xsd" version="1.0">
  <jsp-attribute name="reloadEnabled" value="true"/>
  <jsp-attribute name="reloadInterval" value="10"/>
  <reload-interval value="3"/>
  <enable-directory-browsing value="false"/>
  <enable-file-serving value="true"/>
  <enable-reloading value="false"/>
  <enable-serving-servlets-by-class-name value="false"/>
  <context-root uri="/solr"/>

</web-ext>

Here is a sample of full stack trace:
[2/11/15 17:17:42:268 PST] 0000002b SolrDispatchF E org.apache.solr.common.SolrException log null:java.lang.NullPointerException
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
        at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:190)
        at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:125)
        at com.ibm.ws.webcontainer.filter.WebAppFilterChain._doFilter(WebAppFilterChain.java:80)
        at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:908)
        at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:997)
        at com.ibm.ws.webcontainer.webapp.WebApp.invokeFilters(WebApp.java:4044)
        at com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:3971)
        at com.ibm.ws.webcontainer.webapp.WebGroup.handleRequest(WebGroup.java:276)
        at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:945)
        at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1592)
        at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:191)
        at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:453)
        at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:515)
        at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:306)
        at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:277)
        at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214)
        at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113)
        at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
        at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
        at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
        at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
        at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
        at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
        at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
        at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1656)
Posted by at No comments:

Friday, January 9, 2015

WebSphere Commerce Inventory Indexing Notes

Notes:

The di-proprocess script will do nothing for a full index build, if you check the wc-dataimport-preprocess-fullbuild.xml, the sql="".

In di-buildindex process, the inventory data is retrieved from INVENTORY table directly:

in the wc-data-config.xml file under the ...\IBM\WCDE_ENT70\search\solr\home\MC_10001\generic\CatalogEntry\Inventory\conf\wc-data-config.xml

for ItemBean:

SELECT 
    INVENTORY.CATENTRY_ID, 
    INVENTORY.STORE_ID, 
    INVENTORY.FFMCENTER_ID, 
    INVENTORY.QUANTITY
FROM INVENTORY
WHERE INVENTORY.QUANTITYMEASURE = 'C62' -- This is an ISO standard UOM: 1
AND INVENTORY.CATENTRY_ID IN (
    SELECT CATENTRY.CATENTRY_ID
    FROM CATENTRY, CATGPENREL
    WHERE CATGPENREL.CATALOG_ID = 10001
    AND CATENTRY.MARKFORDELETE = 0
    AND CATENTRY.CATENTRY_ID = CATGPENREL.CATENTRY_ID
    AND CATENTRY.CATENTTYPE_ID = 'ItemBean')
ORDER BY INVENTORY.CATENTRY_ID, INVENTORY.STORE_ID, INVENTORY.FFMCENTER_ID

for ProductBean:

SELECT 
    CATENTREL.CATENTRY_ID_
    PARENT CATENTRY_ID, 
    INVENTORY.STORE_ID, 
    INVENTORY.FFMCENTER_ID, 
    SUM(INVENTORY.QUANTITY) TOTAL_QUANTITY
FROM INVENTORY, CATENTRY, CATENTREL, CATGPENREL
WHERE CATGPENREL.CATALOG_ID = 10001
AND CATENTRY.CATENTRY_ID = CATGPENREL.CATENTRY_ID
AND CATENTRY.CATENTRY_ID = CATENTREL.CATENTRY_ID_CHILD
AND INVENTORY.QUANTITYMEASURE = 'C62'
AND CATENTRY.CATENTTYPE_ID = 'ItemBean'
AND CATENTREL.CATENTRY_ID_CHILD = INVENTORY.CATENTRY_ID
GROUP BY CATENTREL.CATENTRY_ID_PARENT, INVENTORY.STORE_ID, INVENTORY.FFMCENTER_ID
ORDER BY CATENTREL.CATENTRY_ID_PARENT, INVENTORY.STORE_ID, INVENTORY.FFMCENTER_ID
Posted by at No comments:

WebSphere Commerce Search Test URLs

Solr Search Engine

Solr Admin Console

http://localhost:8080/solr/admin.html#/

Default Core

http://localhost:8080/solr/Default/select?q=*%3A*

Category, Catalog Entry and Inventory Cores

http://localhost:8080/solr/MC_10001_CatalogEntry_en_US/select?q=*%3A*
http://localhost:8080/solr/MC_10001_CatalogGroup_en_US/select?q=*%3A*
http://localhost:8080/solr/MC_10001_CatalogEntry_Inventory_generic/select?q=*%3A*

Solr Rest Search APIs

Category View

http://localhost:8080/search/resources/store/10053/categoryview/@top?catalogId=10001&langId=-1&currency=USD&responseFormat=json&depthAndLimit=-1,-1,0

Product View

http://localhost:8080/search/resources/store/10053/productview/bySearchTerm/water?catalogId=10001&langId=-1&currency=USD&responseFormat=json&pageSize=20&pageNumber=1&depthAndLimit=*&profileName=IBM_findProductsBySearchTerm

Posted by at No comments:

Tuesday, January 6, 2015

WebSphere Commerce Solr Admin Console and Debugging and Monitoring Tools

Original IBM Link

Steps to enable Solr Admin Console:
  1. Import the Search_eardir/Search-Solr.war project into your workspace.
  2. Copy the following directories and files from the downloaded Solr.war file into the Search-Solr/WebContent directory:
    • css
    • img
    • js
    • tpl
    • admin.html
    • favicon.ico
  3. Open the Search-Solr/WebContent/WEB-INF/ibm-web-ext.xml file for editing. (Note: in a production environment, the file that should be modified is in the following location: .../IBM/WebSphere/AppServer/v70/profiles/${profile}/config/cells/cell01/applications/Search.ear/deployments/${instance}/Search-Solr.war/WEB-INF/ibm-web-ext.xml, NOT the one in this location: .../IBM/WebSphere/AppServer/v70/profiles/{profile}/installedApps/${cell}/Search.ear/Search-Solr.war/WEB-INF/ibm-web-ext.xml 
  4. Find the enable-file-serving flag and set it to true.
    For example, in bold:
    <web-ext
   xmlns="http://websphere.ibm.com/xml/ns/javaee"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xsi:schemaLocation="http://websphere.ibm.com/xml/ns/javaee http://websphere.ibm.com/xml/ns/javaee/ibm-web-ext_1_0.xsd"
   version="1.0">

   <reload-interval value="3"/>
   <context-root uri="/solr" />
   <enable-directory-browsing value="false"/>
   <enable-file-serving value="true"/>
   <enable-reloading value="false"/>
   <enable-serving-servlets-by-class-name value="false" />

</web-ext>
  5. Save your changes and close the file.
  6. Deploy the Search-Solr project; Right-click the project and select Java EE > Prepare for Deployment.
  7. Update the following files:
    • (Developer)Search-Solr.war/WEB-INF/web.xml
    • Search-Solr.war/WEB-INF/web.xml
    • Solr_cell/applications/solr.ear/deployments/solr/Search-Solr.war/WEB-INF/web.xml
  8. Change the URL pattern for the SolrSearchServlet servlet:
    • (FEP7)Change it to /Search-Solr
    • (FEP8)Change it to /solr
    For example:                                                                                   (FEP7) (Note: I used the /solr as FEP8 and it works)
    <servlet-mapping>
     <servlet-name>SolrSearchServlet</servlet-name>
       <url-pattern>/Search-Solr</url-pattern>
</servlet-mapping>
    (FEP8)
    <servlet-mapping>
     <servlet-name>SolrSearchServlet</servlet-name>
       <url-pattern>/solr</url-pattern>
</servlet-mapping>
  9. Save your changes and close the files.
  10. Publish the search EAR.
  11. Use the following URL to access the Solr administrative interface:
    • (FEP7)http://localhost/Search-Solr/admin.html
    • (FEP8)http://hostname:3737/solr/admin.html
For more information, see SolrAdminGUI.
Lucene Index Toolbox (Luke)
Luke is a development and diagnostic tool for search indexes. It enables you to display and modify search index content. For more information, seeLuke - Lucene Index Toolbox.
WebSphere Application Server JMX clients
JMX clients can read runtime statistics from Solr.
To set up the client:
  1. Add the JMX registry in the Solr core configuration file, solrconfig.xml:
    <jmx serviceURL="service:jmx:iiop://host_name:2809/jndi/JMXConnector"/>
  2. Use jconsole in Rational Application Developer to connect to the runtime JMX.
    When the Solr core is initialized, you can use jconsole to view information from JMX, such as statistics information for caches.
For more information, see SolrJmx.

Note:

1. If you don't set the 
<enable-file-serving value="true"/>
as instructed in step 4 (remember the location is: 
.../IBM/WebSphere/AppServer/v70/profiles/${profile}/config/cells/cell01/applications/Search.ear/deployments/${instance}/Search-Solr.war/WEB-INF/ibm-web-ext.xml) , you may get this exception:

[1/1/15 12:00:00:000 PST] 0000003d SolrDispatchF E org.apache.solr.common.SolrException log null:java.lang.NullPointerException at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:186)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:125)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain._doFilter(WebAppFilterChain.java:80)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:908)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:997)
at com.ibm.ws.webcontainer.webapp.WebApp.invokeFilters(WebApp.java:4044)
at com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:3971)
at com.ibm.ws.webcontainer.webapp.WebGroup.handleRequest(WebGroup.java:276)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:945)
at com.ibm.ws.webcontainer.WSWebContainer.handleRequest(WSWebContainer.java:1592)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:191)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:453)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:515)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:306)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:277)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1660)
2. If you would like to let the web app handle some static pages, you may add the following to the web.xml file (again, find the right location in the config directory):

         <servlet>
                <servlet-name>FileServlet</servlet-name>
                <servlet-class>com.ibm.ws.webcontainer.servlet.SimpleFileServlet</servlet-class>
                <load-on-startup>1</load-on-startup>
        </servlet>
        <servlet-mapping>
                <servlet-name>FileServlet</servlet-name>
                <url-pattern>hello.html</url-pattern>
        </servlet-mapping>

Posted by at No comments:

Monday, January 5, 2015

WebSphere Commerce Search Server Web Content Access Control File in a Clustered Environment




The file that is used is in the following location:

 /usr/IBM/WebSphere/AppServer/v70/profiles/"instance name"/config/cells/cell01/applications/Search.ear/deployments/Search/Search-Solr.war/WEB-INF/ibm-web-ext.xml

 <?xml version="1.0" encoding="UTF-8"?>
<web-ext xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://websphere.ibm.com/xml/ns/javaee"
    xsi:schemaLocation="http://websphere.ibm.com/xml/ns/javaee http://websphere.ibm.com/xml/ns/javaee/ibm-web-ext_1_0.xsd" version="1.0">
  <jsp-attribute name="reloadEnabled" value="true"/>
  <jsp-attribute name="reloadInterval" value="10"/>
  <reload-interval value="3"/>
  <enable-directory-browsing value="false"/>
  <enable-file-serving value="true"/>
  <enable-reloading value="false"/>
  <enable-serving-servlets-by-class-name value="false"/>
  <context-root uri="/solr"/>

</web-ext>

Posted by at No comments:
Labels:
Subscribe to: Posts (Atom)