NoClassDefFoundError in Blackberry

Background
Since lastweek I spent couple of days trying to fix an issue: The application keep throwing NoClassDefFoundError Exception in a callback method. Through JDE debugger, I can figure out exactly where the exception is threw and which object cause the exception, but when I check the code back and forth, I could not find the problem, since the class file is there. Finally I realized that I also changed the build script: before I use antenna wtkbuild task to generate the library, the new build I use regular javac task, since I used the blackberry native API. Will it cause the problem? It does not make any sense to me. I tried to switch back the build script using twkbuild task and adding the rim api in the class path, this time everything works! No more NoClassDefFoundError any more!

Finding the root cause
I compared the wtktask and the javac task with blackberry build script:

build target(with issue):

<target name="compile_rim" depends="init">
 
  <javac destdir="${outputclasses}" encoding="${src.encoding}" source="1.3" srcdir="${src.dir}:${rim.src.dir}">
    <classpath location="${rimlib.loc}"/>
   </javac>
</target> 

Created by Pretty R at inside-R.org


wtk build ( no issue)



Form the above, I can not tell the difference, and can not figure out why there is an issue. So I have to look at the antenna source code. When I looked at the WtkBuild.java, I noticed that antenna sets the javac target to 1.1 by default! This is interesting.

Check the ant document for javac task, for target attribute, it says:

Generate class files for specific VM version (e.g., 1.1 or 1.2). Note that the default value depends on the JVM that is running Ant. In particular, if you use JDK 1.4+ the generated classes will not be usable for a 1.1 Java VM unless you explicitly set this attribute to the value 1.1 (which is the default value for JDK 1.1 to 1.3). We highly recommend to always specify this attribute.

So based on the above statement, I suspect that the Ant require us to specify the target version, if we don't specify ant will use the default version.and the default target version cause the blackberry device throws the exception.

What is the target version for J2ME and Blackberry
In Antenna, the target is set to 1.1, is this the correct value for J2ME and Blackberry? To prove this, I googled and find some articles:
Managing Wireless Builds with Ant, this article from oracle mentioned that J2ME build requires target value to be 1.1:

<javac bootclasspath="${midp_lib}" destdir="build/classes" srcdir="src" target="1.1">

and I found this article in BlackBerry forum:javac -target parameter used by the rapc.exe tool
It shows that you need to use target 1.1 to build blackberry compatible application:

Rapc.exe automatically ensures that these .class files are targeted for BlackBerry® Java® Virtual Machine (BlackBerry JVM) 1.1 or later. This is done to maintain compatibility on the BlackBerry smartphones when the .class files are actually packaged in a .cod file.

If you are using a custom script that calls javac.exe and rapc.exe separately to package your application, you will have to make sure that you are calling javac.exe with -target option so that all generated .class files are compatible. This is a requirement to make sure that the generated .class files are consistent with Connected Limited Device configuration (CLDC) 1.1 BlackBerry JVM implementation.

Conclusion
Based on what I found, I know that Both J2ME and Blackberry Build need target option to 1.1. otherwise the blackberry application might have a weird issue like NoClassDefFoundError.

How I improved my company's build system - build automation

Last 2 weeks I haven't check twitter, did not go to infoQ, all my spare time is focused on one thing: improve the build system, the goal - one button click triggered the build and deploy automation.
This goal is finally achieved last week. Now the build system is scheduled deployment once a day, the automation jobs include following:

1. check out source code from CVS;
2. build client and server artifacts;
3. generate the change log
4. update the wiki and download page
5. upload the artifact into server and downloading repository
6. execute the server deploy script
7. send new release email 

Before only step 1) and 2) are automatic using the ANT script. All other steps are manual, I found this kind of job is boring,painful and quit easy to make mistakes.

I have the idea for build automation for a long time, especially I was inspired by the book "Continuous Delivery" and "Pragmatic Project automation".  My plan to implement the build automation is pretty simple: simulate the human deployment behaviour, implement an ANT target for each manual step. It sound easy, but when I actually did it, it was harder than imagined. But I finally I made it.
Right now everyday we will automatically release a new build, this is a milestone and a great improvement. I feel so happy and so proud of it.

The tools I used
- ANT,  I used scp, filterset, filterchain, sshexec, cvschangelog, I found Ant has lots of gem.
- Groovy, I used AntBuilder to integrate with the existing ANT task, parsing XML, generating text file, quite neat.
- Jenkins,  everyone recommend Hudson/Jenkins CI server, when I used it, it is so easy to use, and has lots of useful plugins. I use Jenkins to schedule and trigger the build.
- ViewVC, a very good tool to browse CVS source code from web, I configure the ViewVC in Jenkins, Jenkins will provide the actually source code link in the changes page.

What I learned

• Pay more attention to your feeling. If you feel boring and painful, don't just complain, try to find a way to improve it. This is a good chance to improve your self, my previous blog describe this.
• A good developer must be able to delegate the dirty jobs to the computers, it would be shame for a good developer to do the repetitive manual jobs over and over.
• I implemented the automatic build script based on the manual process, so I learned: if you can define a repeatable step by step manual process, then you can automate it by program.
• I finished this job usually in down time, I finished one small piece each time, so the improvement for the legacy system has to be emergent.

Next Actions
This is only the first step, there are way more things needs to be done, my next steps will be:
- Add Unit Test and Regression test automation
- Add the code quality analysis report
- Use Ivy for dependency management
- Use Puppet to manage the configuration

Ant mail task issue in Mac

Last week I get stuck again when using the Ant mail task, based on the document from Ant site, the mail task requires the following 2 jar files:
 - mail.jar from JavaMail API 
 - activation.jar from  JAF API

I already downloaded and copied them into /usr/share/ant/lib folder, but still I got the error.
After google a while, I realized that Mac missed another jar file in its ant package, this time it is ant-javamail.jar! So when I just copied the ant-javamail.jar from the downloaded ant package into /usr/share/ant/lib folder, then every thing worked perfectly.

In summary, to make the mail task work in Mac, you need 3 jar files:
 - mail.jar
 - activation.jar
 - ant-javamail.jar

Ant target to send email example (using gmail smtp server):

<target name="sendmail" description="send email">
        <mail mailhost="smtp.gmail.com"
                  mailport="25"
                  user="${your.gmail.account}"
                  password="${your.gmail.password}"
                  ssl="true"
                from="${your.email.address}"
                tolist="${mail.recipient}"
              subject="Results of nightly build">
          <message>This is a test email</message>
       </mail>
</target>

Ant scp and sshexec task issues in Mac

Last week when I tried to run my ant target using scp and sshexec task in my macbook pro, I got the following issues:

1) scp task issue
Problem: failed to create task or type scp
Cause: the class org.apache.tools.ant.taskdefs.optional.ssh.Scp was not found.
        This looks like one of Ant's optional components.
Action: Check that the appropriate optional JAR exists in
        -/usr/share/ant/lib
        -/Users/steve/.ant/lib
        -a directory added on the command line with the -lib argument

Do not panic, this is a common problem.
The commonest cause is a missing JAR.

2) sshexec task issue
Problem: failed to create task or type sshexec
Cause: the class org.apache.tools.ant.taskdefs.optional.ssh.SSHExec was not found.
        This looks like one of Ant's optional components.
Action: Check that the appropriate optional JAR exists in
        -/usr/share/ant/lib
        -/Users/steve/.ant/lib
        -a directory added on the command line with the -lib argument

Do not panic, this is a common problem.
The commonest cause is a missing JAR.

I already download the jsch-0.1.44.jar from Jcraft,  and copied the jar file into folder /usr/share/ant/lib, and I also tried to put the jar file into different folder as the error suggested, but no help.
Finally I found this article:, it mentioned that in Mac, the ant-jsch.jar is missing! 
and when I look at the /usr/share/ant/lib folder, yes there is no ant-jsch.jar file!  So I downloaded the ant 1.8.2 from ant website, unpack it, and copy it into the /usr/share/ant/lib folder, it just works!

In summary, to make the ant scp and sshexec task work in Mac, you need to do 2 things:

1. Install ant-jsch.jar file: download the ant package from ant website, then copy the ant-jsch.jar into /usr/shar/ant/lib folder
2. Go to Jcraft website, download the latest version of jsch jar file, copy it into /usr/share/ant/lib

Handling incoming call interruption in BlackBerry audio player

Last week I spent two days to fix an issue of blackberry audio player: the audio stream need to pause the player during incoming call interruption, and resume when call is finished.
At the beginning, I feel it is easy to fix, because I fixed this kind of issue many times before for blackberry platform.
So I just program based on my old experiences, but it did not working.

My Mistake

At first I use BlackBerry API PhoneListener to register the call interruption event. But it did not work, I found some issues:

- If you use PhoneListener, the application will prompt use to agree to use Phone API, which is annoying.

- When you delete the application, you have to restart the phone.

- The application can get the incoming call event via callIncoming() method, but some time the application get exception;

- The application get the call end event via callDisconnected() method, but when you call MMAPI player play() method, it always throws MediaException.

  It seems the application the the event earlier than the voice call app release the resource.

- I found the Phone already stop the media player before it send the incoming call event the the application, so you have to use a workaround to deal with this issue.

I tried the different workarounds, but none of them working, it seems the PhoneListener solution is not reliable, and even not working. I don't remember why it works before.

maybe because of current phone is 5.0 while before are 4.3 or 4.6? Actually I don't know.

New solution

After I googled and I found a really simple solution: handle the DEVICE_AVAILABLE and DEVICE_UNAVAILABLE event in the playerListener. Based on this article:

The DEVICE_UNAVAILABLE event occurs when there is an incoming call. The DEVICE_AVAILABLE event occurs when the call is ended.

So all you need to do is pause the player in DEVICE_UNAVAILABLE, and resume when DEVICE_AVAILABLE event comes.

Code snippet like this:

playerUpdate(Player player, String event, Object eventData) 
{
    ...

    if(event == PlayerListener.DEVICE_UNAVAILABLE)
    {
        if(player.getState() == Player.STARTED)
        {
              player.stop();
              bPausedByIncomingCall = true;
        } 
    }
    else if(event == PlayerListener.DEVICE_AVAILABLE)
    { 
        if(bPausedByIncomingCall)
        {
            player.start();
            bPausedByIncomingCall = false;
        }

    }

   ...
}

The solution is very easy and it really works. I tested it when the DEVICE_UNAVAILABLE event comes, the player is still playing, which means the event is sent before the incoming call interruption.

Retrospection

I checked the MMAPI API and Blackberry API document, I found this event is not new (Blackberry support this since 4.0!), but I did not notice this until now. Why I did not notice these two events before? I did not find them in the books, nor I did not heard from them from previous colleagues. This is my another blind spot and it exposed my ignorance of MMAPI. This issue tells me: sometime your old experience is unreliable; you still need to empty your mind, like Andy Hunt said: always have beginner's mind!

Functional programming in OO languages

Yesterday I watch the presentation: Functional Programming: A Pragmatic Introduction in Infoq.com. Jim Duey explains functional programming through Java code samples. He covers 3 parts:

•  high order functions
•  laziness
•  composing

I am more interested in the first part - high order functions. Jim shows 3 important functions:
      reduce, filter and map.
Comparing with the java code, Jim describes how these 3 methods can help us reduce the duplicated boilerplate java code: like loop, iteration. He even emphasizes that these 3 methods - reduce, filter and map can reduce 90% of the loop code!  
Copied from his slide, the benefits of the high order functions are:

•   Allows common code structures to reused 
•   Abstracts away details 
•   Isolates the essential parts of the code 
•   Allows easier unit testing
•   Lets you think at a higher level 
•   Allows easy parallelism

If you don't want to watch the video, then I found this article is really great to explain the map, filter and reduce method, also gives Java implementation.
As a java developer, the implementation code is not new to me, mostly the template method pattern is used. Actually I already saw the similar benefit in Spring framework: it is so amazing the JdbcTemplate can removed lots of the JDBC boilerplate code! But I did not aware that it shares the same idea with the function programming.
Since I learned that high order functions are so important, so I decide to investigate more and see how to implement them in other OO languages beside Java.

Javascript
Joel has a nice article about function programming using javascript: Can Your Programming Language Do This?

Groovy
I checked the Groovy API and recalled what I learned before, so I found the equivalent methods for reduce, filter and map in Collection API are:
   reduce  => inject()
   filter  => findAll()
   map     => collect()

Groovy functional programming link

Ruby
Groovy is very similar with Ruby, especially when I learned Groovy, I suspect if Groovy copies some concept from Ruby, since the collection API is so similar.
      reduce  => reduce(), inject()
      map     => map(), collect()
      filter    => select(), find_all()

This article is really great to explain them more in Ruby

Other resources
Why functional programming matters

Conclusion
- To use functional programming, it would be better to use the language that support functional programming in first class. I have to say Java implementation is too ugly, Groovy and Ruby are much better.
- Changing mindset to functional programming is hard but really important. Functional programming is another way of programming to elevate your abstraction layer.

Next Action
To pursue the way of Functional Programming, you need to pick up a new Functional Programming language ASAP, since I learned LISP through SICP, so Clojure would be my next choice.

Rediscover the power of Ant

My last post talks about using groovy template to generate a text file, but I did not realize it can be done in Ant until reading the book- "Ant in action".
Recently when I refactored the ant build script of our comany's build system, I realize I need to learn ant systematically to improve my skills in Ant, so I choose this famous book.
When I learned the "filterset" datatype, I realize I can use that to generate the text file from a template, for example the last example can be written in Ant:

<copy file="test2.template" toFile="test2.txt" overwrite="true">
   <filterset begintoken="%" endtoken="%">
     <filter token="firstname" value="Steve"/>
     <filter token="lastname" value="Zhang"/>
     <filter token="city" value="Toronto"/>
   </filterset>
 </copy>

It is even simpler than Groovy in this case, right?
This example exposes my ignorance level of Ant, I realize that there a still lots of Gems in Ant. As Douglas Crockford said most language contains good parts and bad parts, we need to use only good parts and avoid bad parts, I think what he said also can apply to Ant. So don't easily discard a tool or a language, right now in my case, I love Groovy, I love Gant, they look so cool, but I am not 100% precent for sure Groovy/Gant can fully replace our build script. I have to rely on Ant now.
I start understand the reason why our build script is a kind of ugly: probably we did not fully use the good parts of ant and too much bad parts instead.
So my next step is start reading Ant in Action!