Monday, May 18, 2015

Solve Spring NoUniqueBeanDefinitionException

I encountered the NoUniqueBeanDefinitionException where-in Spring was unable to which class to inject for an Autowired variable even though the it was properly qualified!

This happened because the Qualifier did not get post-processed and so multiple implementations of the Interface type of the variable being injected were now available in the classpath. So Spring found more that one instance and was unable to decide which instance to inject for the particular Autowired target.

The turn on the default AutowiredAnnotationBeanPostProcessor which processes the @Qualifier , you need to add the line to the beans section of your spring configuration. Since this was missing from the beans section of the configuration file, the Qualifier annotation was ignored in this code:

    @Autowired
    public void setDao(@Qualifier("myDaoImpl") MyDAO dao) {

And here is a snippet of the exception that was logged:

 java.lang.RuntimeException: org.springframework.beans.factory.BeanCreationException: Injection of autowired dependencies failed for class [class my.package.impl.name.MyBeanClassName]; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire method: public void my.package.impl.name.MyBeanClassName.setDao(my.package.intf.name.MyDAO); nested exception is org.springframework.beans.factory.NoUniqueBeanDefinitionException: No qualifying bean of type [my.package.intf.name.MyDAO] is defined: expected single matching bean but found 4: myDaoImpl,secondDaoImpl,thirdDaoImpl,fourthDaoImpl

This error was resolved after adding the below line to the beans section of spring configuration:

<context:annotation-config></context:annotation-config>

Friday, May 15, 2015

Upgrade Hibernate 3..x to Hibernate 4.3.x

NOTE: This is a partial guide to upgrade your Hibernate 3 project to Hibernate 4.3.x

Hibernate has a come a long way from version 3.x to the latest version Hibernate 4.3.x with lots of major changes, bug fixes and enhancements:
  • Started using gradle for builds
  • Redesign the way SessionFactory is built
  • Improved metamodel
  • Initial osgi-fication by package splitting (public, internal, spi)
  • Migration to i18n logging framework (using jboss logging)
  • JDK 1.6 (JDBC4) as baseline

The following hibernate jars are required to be downloaded from the Maven Repository:

  1. hibernate-core-4.3.9.Final.jar
  2. hibernate-annotations-3.5.6-Final.jar
  3. hibernate-commons-annotations-3.2.0.Final.jar (DO NOT USE 3.3.0.ga version)
  4. hibernate-entitymanager-4.3.9.Final.jar
  5. hibernate-validator-5.1.3.Final.jar
  6. jboss-logging-3.1.3.GA.jar
Do not use hibernate-commons-annotations-3.3.0.ga.jar as it is a release mistake and will create more problems.

You can download the corresponding javadoc and source files by going to the Maven website and searching for: 
g:"org.hibernate" AND a:"hibernate-core" AND v:"4.3.9.Final"

Now that all the required jars are downloaded, hibernate3.jar will need to be replaced by hibernate-core jar; here is a simple guide to when to include rest of the jars.

To resolve the below error, add hibernate-commons-annotations-3.2.0.Final.jar to classpath and the JBoss library path:
NOTE: DO NOT USE hibernate-commons-annotations-3.3.0.ga.jar as it is a release mistake
java.lang.NoClassDefFoundError: org/hibernate/annotations/common/reflection/MetadataProvider
at org.jboss.hibernate.jmx.Hibernate.buildConfiguration(Hibernate.java:194)
at org.jboss.hibernate.jmx.Hibernate.buildSessionFactory(Hibernate.java:228)
at org.jboss.hibernate.jmx.Hibernate.startService(Hibernate.java:155)
at org.jboss.system.ServiceMBeanSupport.jbossInternalStart(ServiceMBeanSupport.java:289)
at org.jboss.system.ServiceMBeanSupport.jbossInternalLifecycle(ServiceMBeanSupport.java:245)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

To resolve the below error, add jboss-logging-3.2.1.Final.jar to classpath and the JBoss library path:
Not resheduling failed loading task, loadTask=org.jboss.mx.loading.ClassLoadingTask@1219665{classname: org.hibernate.internal.CoreMessageLogger, requestingThread: Thread[main,5,jboss], requestingClassLoader: org.jboss.mx.loading.UnifiedClassLoader3@1b06041{ url=null ,addedOrder=2}, loadedClass: nullnull, loadOrder: 2147483647, loadException: java.lang.NoClassDefFoundError: org/jboss/logging/BasicLogger, threadTaskCount: 0, state: 1, #CCE: 1}
java.lang.NoClassDefFoundError: org/jboss/logging/BasicLogger
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)

Follow the Hibernate Code Migration Guide to make the appropriate changes as required:

  1. Initial move to ServiceRegistry.  For now, see design wikis or sources for more information.  Not all "services" have been migrated to this model yet.  The main ones (JDBC and transaction stuff) as well as lowever level one (classloading and such) have been migrated.  The rest will be moved during Alpha2 development.
  2. In an initial push toward osgi we started splitting up packages a little bit differently in this release. 
    1. The reason is to identify classes which are intended as
      1. public API, which are fully expected to be used in application code.
      2. internal implementation details, which are only intended for Hibernate use.
      3. SPI contracts, whch define
        1. extension contracts
        2. contracts with Hibernate internals exposed to these extensions
    2. This will potentially lead to some  packaging changes needed in user code:
      1. org.hibernate.dialect.resolver.DialectResolver -> org.hibernate.service.jdbc.dialect.spi.DialectResolver
  3. Deprecated methods that have been removed:
    1. References to org.hibernate.type.AbstractSingleColumnStandardBasicType and org.hibernate.type.SingleColumnType methods should be changed as indicated:
      1. nullSafeGet(ResultSet rs, String name) should be changed to 
        nullSafeGet(ResultSet rs, String name, SessionImplementor session)
      2. get(ResultSet rs, String name) should be changed to 
        get(ResultSet rs, String name, SessionImplementor session)
      3. nullSafeSet(PreparedStatement st, T value, int index) should be changed to nullSafeSet(PreparedStatement st, Object value, int index, SessionImplementor session)
      4. set(PreparedStatement st, T value, int index) should be changed to set(PreparedStatement st, T value, int index, SessionImplementor session)
    2. References to org.hibernate.usertype.UserType methods should be changed as indicated:
      1. nullSafeGet(ResultSet rs, String[] names, Object owner) should be changed to   
        nullSafeGet(ResultSet rs, String[] names, SessionImplementor session, Object owner)
      2. nullSafeSet(PreparedStatement st, Object value, int index) should be changed to nullSafeSet(PreparedStatement st, Object value, int index, SessionImplementor session)
    3. Session.reconnect() - manual disconection and reconnection is now only supported for user-supplied-connection scenarios (JDBC Connection passed in while opening the Session)
    4. Session.connection() - use Session.doWork(), Session.doReturningWork() or Session.sessionWithOptions()...openSession() as replacement depending on need
    5. Most of the overloaded SessionFactory.openSession methods.  Use SessionFactory.withOptions()...openSession() instead
  4. Deprecated classes/interfaces that have been removed:
    1. org.hibernate.classic.Session
    2. org.hibernate.classic.Validatable
    3. org.hibernate.classic.ValidationException
  5. org.hibernate.jdbc.BatcherFactory, Batcher, and their implementations have been replaced by org.hibernate.engine.jdbc.batch.spi.BatchBuilder and Batch, with default implementations in org.hibernate.engine.jdbc.batch.internal. You can override the default BatchBuilder by defining the  "hibernate.jdbc.batch.builder" property as the name of a BatchBuilder implementation, or by providing a BatchBuilder in a custom ServiceRegistry.
  6. hibernate.cfg.xml no longer supported as means of specifying listeners.  New approach invloves using an org.hibernate.integrator.spi.Integrator which works based on "service discovery". 




Thursday, May 14, 2015

Upgrade from Spring 2.5.x to Spring 4.1.x - JBoss

After completing the steps mentioned in my previous post for Upgrading Spring 2.5.x to Spring 4.1.x, the project compiles without problems, however, JBoss deployment does not work as it does not get all the required new Spring Jars.

The Spring component jars also need to be added to the JBoss classpath. However, here are a few specific exceptions that point to requiring specific jars in the JBoss classpath as below.

To resolve the below error, add spring-context.jar to JBoss library path:
java.lang.ArrayStoreException: sun.reflect.annotation.TypeNotPresentExceptionProxy
at sun.reflect.annotation.AnnotationParser.parseClassArray(AnnotationParser.java:653)
at sun.reflect.annotation.AnnotationParser.parseArray(AnnotationParser.java:460)
at sun.reflect.annotation.AnnotationParser.parseMemberValue(AnnotationParser.java:286)
at sun.reflect.annotation.AnnotationParser.parseAnnotation(AnnotationParser.java:222)
at sun.reflect.annotation.AnnotationParser.parseAnnotations2(AnnotationParser.java:69)
at sun.reflect.annotation.AnnotationParser.parseAnnotations(AnnotationParser.java:52)
at java.lang.Class.initAnnotationsIfNecessary(Class.java:3079)


To resolve the below error, add spring-beans.jar to JBoss library path
java.lang.NoClassDefFoundError: org/springframework/beans/factory/BeanFactory
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2436)
at java.lang.Class.getDeclaredMethods(Class.java:1793)
at org.jboss.ejb3.interceptor.InterceptorInfoRepository$AnnotationInitialiser.getInfo(InterceptorInfoRepository.java:704)
at org.jboss.ejb3.interceptor.InterceptorInfoRepository.initialiseFromAnnotations(InterceptorInfoRepository.java:469)
at org.jboss.ejb3.interceptor.InterceptorInfoRepository.getOrInitialiseFromAnnotations(InterceptorInfoRepository.java:451)
at org.jboss.ejb3.interceptor.InterceptorInfoRepository.getInterceptorsFromAnnotation(InterceptorInfoRepository.java:341)
at org.jboss.ejb3.interceptor.InterceptorInfoRepository.getClassInterceptors(InterceptorInfoRepository.java:139)
at org.jboss.ejb3.EJBContainer.initialiseInterceptors(EJBContainer.java:737)
at org.jboss.ejb3.EJBContainer.getClassInterceptors(EJBContainer.java:429)

Hopefully this should resolve all issues for a successful upgrade of Spring 2.5.x to Spring 4.1.x!


Wednesday, May 6, 2015

EasyMock Exceptions when mocking Classes instead of Interfaces

EasyMock as the name suggests, provides an easy way to mock objects for testing where-in you mock interfaces of the classes you want to test. EasyMock then generates mock objects on the fly using Java's proxy mechanism and then simulates it in a simple way and also verifies whether it is used as expected.
When creating some new mock objects for testing using EasyMock 2.5.x, I noticed the below exceptions:
java.lang.IllegalArgumentException: org.hibernate.dialect.Dialect is not an interface
 at java.lang.reflect.Proxy.getProxyClass0(Proxy.java:470)
 at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:690)
 at org.easymock.internal.JavaProxyFactory.createProxy(JavaProxyFactory.java:12)
 at org.easymock.internal.MocksControl.createMock(MocksControl.java:37)
 at org.easymock.EasyMock.createMock(EasyMock.java:43)

To solve this, change your import as:
org.easymock.EasyMock.createMock => org.easymock.classextension.EasyMock.createMock

And another set of exceptions popped up:
java.lang.IllegalArgumentException: not a proxy instance
 at java.lang.reflect.Proxy.getInvocationHandler(Proxy.java:769)
 at org.easymock.EasyMock.getControl(EasyMock.java:1336)
 at org.easymock.EasyMock.replay(EasyMock.java:1280)

To solve this, change your imports as:
org.easymock.EasyMock.replay => org.easymock.classextension.EasyMock.replay
org.easymock.EasyMock.verify => org.easymock.classextension.EasyMock.verify

Viola! and now we are good!
When mocking classes that are not an interface, we need to use the createMock, replay, and verify methods FROM the class org.easymock.classextension.EasyMock and NOT the ones we regularly use from org.easymock.EasyMock
However, since EasyMock 3.0, the separate classextension package has been deprecated and left in place for backward compatibility and the above changes are not required. So it is recommended that if possible, one should upgrade to the latest 3.x version of EasyMock instead of going for the above changes.

Tuesday, May 5, 2015

Oracle Date / Timestamp difference in Hours, Minutes & Seconds

Figuring out the exact time difference that has elapsed between between two Date / Timestamp columns in Oracle using SQL is not as easy as it sounds.
A simple subtraction the those values using this query yields:
SQL> select runnumber, starttime, endtime, (endtime-starttime) TotalMinutes0 from run_log;

 RUN STARTTIME            ENDTIME              TOTALMINUTES0 
---- -------------------- -------------------- ------------- 
   3 24-APR-15 12:07:49   24-APR-15 12:58:37   0.03527777778  
   2 24-APR-15 11:00:47   24-APR-15 11:55:11   0.03777777778  
   1 24-APR-15 08:00:13   24-APR-15 08:52:39   0.03641203704  


So, I convert the difference to minutes by multiplying it by 1440 (24 Hours X 60 Minutes) using this below query:
SQL> select runnumber, starttime, endtime, (endtime-starttime) TotalMinutes0, 
        to_char((endtime-starttime)*1440,'99.99') TotalMinutes1 -- OR (24 Hours X 60 Minutes)
     from run_log order by starttime desc ;

 RUN STARTTIME            ENDTIME              TOTALMINUTES0 TOTALMINUTES1 
---- -------------------- -------------------- ------------- ------------- 
   3 24-APR-15 12:07:49   24-APR-15 12:58:37   0.03527777778  50.80         
   2 24-APR-15 11:00:47   24-APR-15 11:55:11   0.03777777778  54.40         
   1 24-APR-15 08:00:13   24-APR-15 08:52:39   0.03641203704  52.43         


That's great progress, but now the seconds part is a decimal fraction instead of seconds. This necessitates the wielding of the Oracle "numtodsinterval" built-in function using this below query:
SQL> select runnumber, starttime, endtime, (endtime-starttime) TotalMinutes0,
         to_char((endtime-starttime)*1440,'99.99') TotalMinutes1,
         numtodsinterval((endtime-starttime),'day') TotalMinutes2
     from run_log order by starttime desc ;

 RUN STARTTIME            ENDTIME              TOTALMINUTES0 TOTALMINUTES1 TOTALMINUTES2 
---- -------------------- -------------------- ------------- ------------- ------------- 
   3 24-APR-15 12:07:49   24-APR-15 12:58:37   0.03527777778  50.80        0 0:50:48.0   
   2 24-APR-15 11:00:47   24-APR-15 11:55:11   0.03777777778  54.40        0 0:54:24.0   
   1 24-APR-15 08:00:13   24-APR-15 08:52:39   0.03641203704  52.43        0 0:52:26.0   


The output is good and we now only need to trim it a bit as in the below query:
SQL> select runnumber, starttime, endtime, (endtime-starttime) TotalMinutes0,
         to_char((endtime-starttime)*1440,'99.99') TotalMinutes1,
         numtodsinterval((endtime-starttime),'day') TotalMinutes2,
         substr(numtodsinterval((endtime-starttime),'day'), 4, 8) TotalMinutes3 -- Ignore hours
     from run_log order by starttime desc ;

 RUN STARTTIME            ENDTIME              TOTALMINUTES0 TOTALMINUTES1 TOTALMINUTES2 TOTALMINUTES3 
---- -------------------- -------------------- ------------- ------------- ------------- ------------- 
   3 24-APR-15 12:07:49   24-APR-15 12:58:37   0.03527777778  50.80        0 0:50:48.0   0000000       
   2 24-APR-15 11:00:47   24-APR-15 11:55:11   0.03777777778  54.40        0 0:54:24.0   0000000       
   1 24-APR-15 08:00:13   24-APR-15 08:52:39   0.03641203704  52.43        0 0:52:26.0   0000000       


Now that's really very strange - the trimming substring does not get me the expected value - wonder what's going on? To check that, we use "to_char" function to see what is happening during the conversion to string as in the below query:
SQL> select runnumber, starttime, endtime, (endtime-starttime) TotalMinutes0,
         to_char((endtime-starttime)*1440,'99.99') TotalMinutes1,
         numtodsinterval((endtime-starttime),'day') TotalMinutes2,
         substr(numtodsinterval((endtime-starttime),'day'), 4, 8) TotalMinutes3, -- Ignore hours
         to_char(numtodsinterval((endtime-starttime),'day')) TotalMinutes4
     from run_log order by starttime desc ;

 RUN STARTTIME            ENDTIME              TOTALMINUTES0 TOTALMINUTES1 TOTALMINUTES2 TOTALMINUTES3 TOTALMINUTES4                  
---- -------------------- -------------------- ------------- ------------- ------------- ------------- ------------------------------ 
   3 24-APR-15 12:07:49   24-APR-15 12:58:37   0.03527777778  50.80        0 0:50:48.0   0000000       +000000000 00:50:48.000000000  
   2 24-APR-15 11:00:47   24-APR-15 11:55:11   0.03777777778  54.40        0 0:54:24.0   0000000       +000000000 00:54:24.000000000  
   1 24-APR-15 08:00:13   24-APR-15 08:52:39   0.03641203704  52.43        0 0:52:26.0   0000000       +000000000 00:52:26.000000000  


Ahh ha! Substring converts the given value to string before getting us a part of it and the conversion to string gives us a totally different & larger string! So now we use trim the result as in the below query to get the executed final result:
SQL> select runnumber, starttime, endtime, (endtime-starttime) TotalMinutes0,
         to_char((endtime-starttime)*1440,'99.99') TotalMinutes1,
         numtodsinterval((endtime-starttime),'day') TotalMinutes2,
         substr(numtodsinterval((endtime-starttime),'day'), 4, 8) TotalMinutes3, -- Ignore hours
         to_char(numtodsinterval((endtime-starttime),'day')) TotalMinutes4,
         substr(numtodsinterval((endtime-starttime),'day'), 15, 8) TotalMinutes5 -- Ignore hours
     from run_log order by starttime desc ;

 RUN STARTTIME            ENDTIME              TOTALMINUTES0 TOTALMINUTES1 TOTALMINUTES2 TOTALMINUTES3 TOTALMINUTES4                  TOTALMINUTES5
---- -------------------- -------------------- ------------- ------------- ------------- ------------- ------------------------------ -------------
   3 24-APR-15 12:07:49   24-APR-15 12:58:37   0.03527777778  50.80        0 0:50:48.0   0000000       +000000000 00:50:48.000000000  50:48.00      
   2 24-APR-15 11:00:47   24-APR-15 11:55:11   0.03777777778  54.40        0 0:54:24.0   0000000       +000000000 00:54:24.000000000  54:24.00      
   1 24-APR-15 08:00:13   24-APR-15 08:52:39   0.03641203704  52.43        0 0:52:26.0   0000000       +000000000 00:52:26.000000000  52:26.00      


Finally! The time difference is displayed in the format HH24:MI:SS.NNN

So after five iterations, I finalize that this exercise requires using the Oracle built-in function "numtodsinterval" which takes a number of a given interval unit and converts it to an INTERVAL DAY TO SECOND literal. The interval value indicates the unit of the specified number and the case-insensitive valid values are:

  • 'DAY'
  • 'HOUR'
  • 'MINUTE'
  • 'SECOND'

In our case, the number in question is the simple difference of the two time values and its' unit is a day so the interval value is "Day".

Wednesday, April 22, 2015

Upgrade from Spring 2.5.x to Spring 4.1.x

I am upgrading Spring libraries of a critical project from Spring 2.5.x to the latest available release which is Spring 4.1.6 right now. One of the major issues I faced is that the packaging strategy was changed in Spring 3.0 release. Earlier, a single spring.jar with all jars and required libraries was provided, but this was discontinued from the 3.0 release. Now we would need to include each jar individually as per requirements - one jar for all situations doesn't exist anymore.

So, now you need to download individual spring component jars from the Maven Repository. The jars required are:
  1. spring-aop.jar
  2. spring-beans.jar
  3. spring-context.jar
  4. spring-core.jar
  5. spring-expression.jar
  6. aopalliance-1.0.jar - third-party jar
The third-party jar aopalliance-1.0 is required only if you use org.springframework.aop and it can be downloaded from here. This was earlier on bundled in the spring.jar and so now has to be added separately as required.

You can download the corresponding javadoc and source files by going to the Maven website and searching for:
  • g:"org.springframework" AND v:"4.1.6.RELEASE" - for Spring components
  • g:"aopalliance" - for aopalliance


Now that all the required jars are downloaded, spring-core jar will need to be included in most projects; here is a simple guide to when to include rest of the jars.

Add spring-beans.jar to classpath when you see the below or similar error during compilation:
Caused by: java.lang.ClassNotFoundException: org.springframework.beans.factory.NoSuchBeanDefinitionException
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)


Add spring-context.jar to classpath when you see the below or similar error during compilation:
Caused by: java.lang.ClassNotFoundException: org.springframework.context.access.ContextSingletonBeanFactoryLocator
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)


Add spring-expression.jar to classpath when you see the below or similar error during compilation:
Caused by: java.lang.ClassNotFoundException: org.springframework.expression.ParserContext
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)


Add spring-aop.jar to classpath when you see the below or similar error during compilation:
The type org.springframework.aop.TargetSource cannot be resolved. It is indirectly referenced from required .class files
OR

Caused by: java.lang.ClassNotFoundException: org.springframework.aop.framework.ProxyFactoryBean


Add aopalliance-1.0.jar to classpath when you see the below or similar error during compilation:
The type org.aopalliance.aop.Advice cannot be resolved. It is indirectly referenced from required .class files
OR

Caused by: java.lang.ClassNotFoundException: org.aopalliance.intercept.MethodInterceptor


That's it - you should be all set and upgraded to the latest version of Spring after this exercise - no code change required!

Friday, November 7, 2014

MongoDB keyfile permissions are too open error

Creating an authenticated replicated set requires setting up of a secret key file on all the nodes.
In spite of creating the key file and putting the same file on all the nodes, you may encounter error stating that the permissions are too open:

permissions on /var/mongodb/conf/mongodb-keyfile are too open

 To solve this, here are the two things that you need to do to solve it:

  1.  Check that the owner of the key file and the user running mongod are same.
     Use chown command to change ownership to match if required.
      chown
  2.  Check permissions of the key file - should be 600 or 700.
     Use chmod command to make permissions more restrictive.
     chmod 600

Amazon Echo - your own personal digital concierge!

Amazon has launched nifty device named Echo - which is like a stand-alone "Siri" - but more like a personal digital concierge for the family. The cost is pretty reasonable at $199 and Amazon is running a promotion right now where Prime members can get it for only $99.

Check out the Echo on Amazon - looks pretty interesting but is available by invite only.

Friday, September 5, 2014

MongoDB warning for running on NUMA machine

When running MongoDB on a Linux server (having multiple physical processors as most servers have these days), you will see the below warning in your log file on startup:

2014-09-04T13:13:35.245-0400 [initandlisten] ** WARNING: You are running on a NUMA machine.
2014-09-04T13:13:35.245-0400 [initandlisten] **          We suggest launching mongod like this to avoid performance problems:
2014-09-04T13:13:35.245-0400 [initandlisten] **              numactl --interleave=all mongod [other options]

This check for NUMA based system is done on startup since MongoDB2.0 as running MongoDB or for that matter any database applications can cause poor performance due to excessive swapping.

There is no ideal solution to this problem yet, but a good solution would be to switch off zone reclaim and set a memory interleave policy before starting mongod process. Here are the two steps, both of which need to be done to fully disable NUMA:

  1. Switch off zone reclaim:
    echo 0 > /proc/sys/vm/zone_reclaim_mode
  2. Flush Linux buffer caches just before starting mongod:
    sysctl -q -w vm.drop_caches=3
  3. Set interleave policy for mongod process when starting it:
    numactl --interleave=all <$MONGO_HOME>/mongod <options>

This should solve your problem - but if you want to understand what the issue is and what's really going on - then continue to read below!

Older multi-processor systems were designed to have Uniform Memory Access (UMA) - this essentially means is that all memory is common to all processors and accessed via a common bus. Thus the latency and performance of all processors was the same. This is also referred to as Symmetric Multi-Processing (SMP).

New multi-processor systems were designed to have Non-Uniform Memory Access (NUMA) - each processor has a local bank of memory to which it has very fast access & low latency and it also has access to memory banks of other processors though with a poorer performance & higher latency. This is sometimes also referred to as Cache-Coherent NUMA (ccNUMA).

Linux is aware of its running on a NUMA system and does the following:

  • Probes the system to figure out the physical layout
  • Attaches the memory module to its' corresponding processor to create a node
  • Creates a map (table/matrix) of the cost or weight of inter-node communication between all the nodes
  • Allocates a preferred node for each thread to run on and allocates memory to it, preferably from the same node or from another node if available. If memory is not available on the same node, it is allocated from the available node with the lowest cost
  • Uses zone reclaim to reclaim memory when a node runs out of memory

Zone Reclaim allows Linux SysAdmins to set a more or less aggressive approach to reclaim memory when a zone runs out of memory. If it is set to zero, which is the default, then no zone reclaim occurs. Allocations will be satisfied from other zones / nodes in the system.

Now that you have a simplistic view of what NUMA is, let's understand what our solution actually did.

  1. We switched off zone reclaim - this would reduce paging as allocation of memory is now satisfied from other nodes if required
  2. Flushed Linux's buffer caches - This helps to ensure allocation fairness, even if the daemon is restarted while significant amounts of data are in the operating system buffer cache.
  3. We set memory allocation to interleave for mongod - this spreads allocation of memory evenly on a round-robin basis across all nodes thus spreading out memory usage and reducing paging

For a more in-depth analysis and further reading, check out Jeremy Cole's blog posts:

Wednesday, April 16, 2014

Maven Compile error

Maven Compile Plugin by default uses version 2.0.2 and JDK1.3 as the target for the compilation of Java code if you do not specify the compiler plugin version and the Java source & target versions. Due to this, you may get the below error when compiling:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile (default-compile) on project scratch: Compilation failure
[ERROR] /home/projects/java/scratch/src/main/java/com/test/MavenCompileTest.java:[50,26] error: for-each loops are not supported in -source 1.3

To resolve this problem, you need to specify the compiler plugin version and the Java -source and -target versions in the pom.xml of your project as below:


  
    
      org.apache.maven.plugins
      maven-compiler-plugin
      3.1
      
        1.7
        1.7
      
    
  


You can figure out the latest version of the Compiler Plugin by reading the version number on the top right of the Compiler Plugin page.

LinkWithin

Related Posts Plugin for WordPress, Blogger...