Sunday, September 5, 2010

Be careful to use "Class.forName()" in J2ME

A couple weeks ago, I made another mistake: I tried to use Class.forName() method to solve the multiple platform issue, but unfortunately it did not work on the actual devices due to the issue caused by the obfuscation. This is my another blind spot, because of my ignorance of J2ME obfuscation.

Scenario
I need to provide an API to support multiple mobile platform, for example: Nokia, Blackberry, and simulator, etc.
What I implement is pretty easy:
  • define an interface: Service
  • Implement the Service interface for each mobile platform, for example: NokiaService, BlackBerryService, SimualtorService
  • At the build time, inject different Service class file in the package.
  • At the run time, the application use the Class.forName() to create the Service objects.
The snippet code looks like this:
public Service createService(int device)
 {
    Service service = null;

    if( device == NOKIA)
    {
        service = Class.forName("NokiaService").newInstance();
    }
    else if( device == BLACKBERRY)
    {
        service = Class.forName("BlackBerryService").newInstance();
    }
    else if(device == SIMULATOR)
    {
        service = Class.forName("SimulatorService").newInstance();
    }
    else 
        service = Class.forName("GenericService").newInstance();

     return service;
 }

Here I use a factory pattern to support multiple platform, no need to use preprocess, sound pretty simple, right?
But I found the above solution only works on BlackBerry platform, on other J2ME platform, the application will throw the ClassNotFound Exception.

The reason of failure
The problem is in obfuscation phase, we usually use Proguard to do preverification and obfuscation for the J2ME application. The proguard will remove the unused class file. (for detail please check this link ) In my solution above use class.forName() by passing different string values, which is not recognized in Proguard. For example: in Nokia platform, the NokiaService.class is used, but Proguard did not find the NokiaService.class is explicitly reference in the java code, so it think it is unused class file, then remove it from the final package. Then it cause the ClassNotFoundException.If you don't use obfuscation, the application works, but it is really rarely the commercial application does not have obfuscated.

But why it works on BlackBerry platform?
Because Blackberry is quite different with other J2ME platforms, it requires it own preverification and obfuscation using its own RAPC compiler. And I found that in Blackberry application package it usually keep all the class files in the package, which is different with the Proguard. That is the reason why it works in BlackBerry platform

Solution
I have to rewrite the above code, remove all the Class.forName(), replaced with the explicit method call.
To support multiple platform, I use javaassit to adjust the createService() method for different platform, while you can use pre-process if you want, even I really don't like the J2ME pre-process, because it will bring other more complicated issue in your application.

Conclusion
Class.forName() seems the only reflection that is support in the J2ME platform, but actually it will cause the problem during obfuscation phase, try not to use the Class.forName() in your J2ME application.

Thursday, August 19, 2010

Aspire to be agile missionary

In an evening of earlier this week, when I went out for a walk near my house, I met 2 young Christian missionaries. From their eyes, I can feel their passion and ideal. They want to help people, and enjoy seeing how they can make changes to them. They did that all for free, they have a mission to transfer what the God said to other people. I was totally impressed by their passion, although I don't believe in God.

I was inspired by these 2 young missionary: we might need missionaries in the software industry. I love to be a missionary if there is a similar group about software development which share a unique vision about software developing, like software craftsmanship for example.  I would love to share my knowledge and experience; I would love to transfer to other developer that what I believe the right thing, right principle; I love to pursue the way of software development, find the philosophy behind the principles. I love to do that for free, and I am sure I will enjoy what I am doing and seeing the changes I made.

What is the reason that I want to be agile missionary? During year 2002, at that time I had been a software engineer for about 10 years, I became confused about my career as software developer: should I continue my career as developer or change to a manager? Why programmer's job is so hard, so boring, so messy? It doesn't make sense! And in that year, occasionally I began to know about Agile movement, it totally changed my mindset of software developing. Agile is like philosophy: it tells you the right way beyond the detailed technology, computer language and tools. Since then I read lots of books from Kent beck, Martin Fowler, Uncle Bob, Dave Thomas ... etc. Since then I found the answer of my career path: continue my career as software developer, because there so many good stuff you need to know, there are lots of potentials for to improve yourself. Then I started learning Design patterns, doing refactoring, investigate new tools. Gradually I realized that I can feel the difference between me and my colleagues: not because of my experience, but because I have the deeper knowledge of agile and the right way of programming. I really love to share my experiences, because I learned a lot from agile. Before I just unconsciously shared my experiences and knowledge, from now I need to do that consciously and deliberately.

Being agile missionary is not easy. First, you need to keep improving your skills, then you can lead by example; second, you need passion and strong vision; third, you need to know some knowledge of psychology, because different person has different background and different mindset, you need to aware that and use different strategy; fourth, the communication skills is definitely important, you need to keep improving it.

I already set my goal: pursuing the right way of software of development, finding the true principles, strategy and philosophy, attain mastery. Right now I feel I just enter the gate to that road, there is till a long journey to go. I hope at this journey, when I work as a missionary by help other developers, I can improve myself and find other missionaries.

Sunday, August 8, 2010

Is it possible for a developer to become an artist?

I am still reading the book: Linchpin.  I am interested with the author's definition about art:
Art is a personal gift that changes the recipient. Art is something creative, passionate and personal, which can bring some change and make the difference.
The author expanded the  definition of art , it focuses on "making changes".This idea share the similar idea of software crafsmanship.

Where is the art in software developing?
The developer's daily job can be boring, repetitive instead of creative. For example:
you can just copy and paste from the old code; or just use google check some API usage;
You don't need to care the code quality, don't need to care about the maintainability.
you can also finsih the job. You can do like that everyday.
But this job is not art, you will not make the difference, any one with the similar knowledge can do it.
then your job are easily been replaced.

But the developer's job can be a kind of art if you choose to do in different way:
   there is the art of making simple design;
   there is the art of wrting clean code;
   there is the art of refactoring;
   there is the art of maintaining;
   there is the art of debugging;
   there is the art of communciation and interaction;

Anything which can make things better and make some difference is the art of programming.
if you bring your passion, and make some difference, then that is art;
if you do that day by day, keep improving yourself, then you will become a master, which is also an artist of software.

The road to become artist
I like the concept of Shu-Ha-Ri, it can be treated as a path from an average developer to a master developer:
    In Shu Level, the point is "following, copying", all you need to do is learning, so there is no art behaviour at this level;
  In Ha Level,  the key point is "detaching, breaking", you try to bring some change, bring what you learned into different context, you will have some art behavior in this stage;
  In Ri Level, the key point is "transcending", you becomes master, everything you do is from your intuition, you are freely to express your idea,which is the artists behaviour.

So I believe the truly artist is only existed in Ri level, which needs years of the deliberate practice, it should be an ultimate goal for any serious software developer. You will become master and artist at the same time.

Monday, August 2, 2010

Linchpin: to be or not to be?

Are you a cog or lincphin? Every employee has to answer this question.

In your team, you work hard, show up on time, folllow the order, try to fit in the team, compliance to your boss, always say "yes", finished what you've been told.If you are lucky, you might be given some bonus. But if you only just did that, then you are just an average developer, will be no difference with other colleague, you are just an interchangable component, like a cog in a machine, which are easily to be replaced.
The problem in IT industry, average and mediocrity can not survive, whether a company or a developer.  If you are just an average cog, then you are easily to be replaced, you have to compete with other lower priced developer; and even more finally your job might be outsourced to India or China.

So if you want to survive, secure your job,  you have to be lincpin: you are indespnesible. Make sure you stand out, remarkable, quite different with others average people; you have passion, you are emotional labour. If you are linchpin, you will become independent and get freedom, you don't need to worry about job, job is always looking for you. Since you are linchpin, you deserve the high pay because you bring much more value to the company.The difference productivity between a great software developer and average one is 10:1, think about how much you can earn if you are linchpin. Think about Apple, how the iPhone makes such difference with other regular phones.

You have to make a choice between these two: to be linchpin or not to be. It is up to you.
If you want more details about linchpin, please read the book: Linchpin: Are you indispensable?



Also check this article: The indispensable Developer
And this: You are not your role

Sunday, July 25, 2010

What drive you work? - The review of book Drive

Today I finish reading the book Drive-The surprising Truth About What Motivates Us". I knew this book from this 10 minutes cartoon on twitter, and the author's video about drive.The topic sound interesting to me, so I started to read the book. To my surprise, the book is quite easy to read, I was so absorbed in it,  most of the book was finished on the flight trip to London.This is very unusual to me, since I seldom finishing a English book in such a short time, except Agatha Christie's  novel.
The core concept of the book: The traditional external carrot & stick motivator does not work on 21 century; we need to upgrade to intrinsic motivator: autonomy, mastery, & purpose.
This concept is not new to me, it reminds me some other books which share the similar theory behind:
  • Maslow's self realization theory;
  • Lean Technology and Deming's theory;
  • Seven Habits
  • Maverick
  • The passionate programmer
  • The fifth discipline

This book drives me into deep thinking about the 3 factors: autonomy, mastery and purpose.

What I learned from the book
 - Autonomy, mastery and purpose are correlated, and each of them supports each other.
      Good example: Since I have the mastery in my mind, I have the passion to do things better; if I get the autonomy, I get trust, I am allowed to do my job on my way, then I will focus entirely on the job, and try to doing my job better, since I know the purpose of my job, I know the whole picture, and I enjoy the whole process.
      Bad example: In the command & control hierarchy, I am just a interchangable resource, I can not control what I am going to do; I have to follow the order from my boss; my job is make my boss happy, I don't care job itself. I don't know the my job related to any purpose, I just know get my job done, make my boss happy and get my money. It is impossible to get highly motivated.
  - The road to "go independent" is the right way to me, I get this idea from the book of "the Passionate programmer".This is the only thing that can guarantee you to get autonomy. You can not control your company, you can not decide who is your boss, but at least you can control your self if you are independent software developer.
 -  Reinforce my goal to mastery.
  I feel lucky that I am still a programmer instead of manager, this give me a chance to become a master in future.Keep improve yourself, love your job, treat your career as an infinite game, keep challenge your self.
     
Next step thinking:
- Compare drive with Lean technology
- Investigate the "flow" concept
- If you are a Type I person, how can you do in a Type X organization, how to deal with your Type X boss?
- What kind of Type I organization should be? How to migrate from Type X to Type I organization?
- Why we should focus Type I behavior in software industry?
- Why manager need to give up control?
- use the theory in "drive" to explain the "three stone cutter stories"
- Read the book of "the fifh discipline".

Resources:
1. 10 minutes video link
2. author Daneil Pink's presentation
3. Dan pink's website
4. The reading list of drive

Sunday, July 4, 2010

Time Machine Wireless backup using your own network drive

Last Friday My Macbook is broken again, this is the third time since this year, I don't know why I am so unlucky and I love my macbook so much! I really worried about my hard drive because the computer could not recognize the hard drive, and I never backup my hard drive data before! Finally the issue was resolved in the Apple Store, the problem is in the harddrive cable.
This time it gave me a lesson: I need to backup my hard drive ASAP.
But when I started to use the time machine for the first time, I got stuck: the Time Machine requires the Time Capusle and Apple airport, while I want to backup into my own NAS drive. To solve this issue, I spent  almost half a day on google and trying out back and forth. Finally I made it, I can use time machine to backup my harddrive into my own network drive.
Here I would like to share my experience about how I solve it. Basically I referred this blog, it is so great, I could not able to finish it without this article.

The Goal
Backup my mac hard drive into my own NAS via wireless network.

Environment
Computer: MacBook Pro
OS: Snow Leopard
Wireless Router: Netgear WNDR3700
Network drive: West Digit MyBook Essential

Steps
1. Setup network drive, create a network shared folder for time machine. For example, I called the folder "\mac".
2. Connect the NAS storage, for example in my netgeare wireless network, I type following in Finder:
    smb://readyshare/mac
3. Enable time machine to use your network drive, run following command in the terminal:

defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1

4. Find out your computer name and Ethernet ID.
   To get the computer name, go to "System Preferences" - "Sharing" , then you will see the computer name at the top, and you can change to any name without restarting the mac computer.
   For example, my computer name is: steve_zhang_mac.  The name contains space and other special characters should be fine, but I prefer to make it simple.
   For the Ether ID, go to "System Preferences" - "Network" - "Ethernet" - "Advanced" - "Ethernet". The ethernet ID in my computer is: d4:9a:20:04:65:34
   Or you can try to type following command in the terminal:
      ifconfig en0 | grep ether | sed s/://g | sed s/ether//
 Please remeber: it requires the computer's MAC address regardless if you use wireless or wired connection.
   If you use Wi-Fi, you will find another different ethernet ID under the  "Airport" configuration, don't use this value.
5. Create the disk image file
   Create the disk image file locally, the file name is like computername_MACaddrss.sparsebundle
   based on the information from step 4), run following in the terminal:
  
    DISK_IMAGE_NAME="steve_zhang_macbook_d49a20046534.sparsebundle"
    DISK_SIZE=200000
    hdiutil create -library SPUD -megabytes $DISK_SIZE -fs HFS+J -type SPARSEBUNDLE -volname "$DISK_IMAGE_NAME" "$DISK_IMAGE_NAME"
  

Copy the created image file into the NAS storage folder.
6.  Run Time Machine, choose your NAS box,  if everything is OK, then the time machine will start working.

Some Erorrs I met
- Error 45
  This is because the time machine could not create the .sparsebundle file, in your NAS storage folder, you might find a file name and folder called like coputerName_tmp.sparsebundle, please follow step 4 and 5,  copy the generated file into the specific shared folder.
  I met this issue twice, one is I did not create the image file, the other is since I use the wrong ehternet ID which I got from the "airport" configuration. After I used the correct ehternet ID,this error does not happened again.

- Error of" .....sparsebundle is already been used."
  The .spasebundle file is used by another process, please restart the computer to run the Time machine again, this issue will solved.

Other tricks
1. For the first time backup, it is better to use LAN instead of Wi-Fi, for example, it took me about 14 hours to backup 113G data using LAN mode.
2. How to check the connected NAS in terminal:
   All the harddrives are under folder /Volumes, so if you type cd /Volumes in terminal, you will see your hard drive and your network drive.
3. How to check the system log:
   System log is in /var/log/system.log
4. If you don't like the time machine to backup every hour, please try the Time Machine Editor, you can customize your own backup schedule with this free app.
5. If you use Parallels virtual machine, then it is better to turn off the backup for parallels virtual machine file,  go to Parallels Desktop - Virtual Machine - Configure... , choose "General" tab, check the "Do not back up with time machine." For example, before my computer needs about 120G, right now it only needs to backup 59G data.  Or you can open the Time Machine Preferences, choose options,  add exclude the Virtual machine file: it is in /Users/Shared/Parallels/Windows XP.pvm


References
1.  Getting Time Machine to work on an Unsupported NAS
2. Time Machine Wireless Backup without Time Capsule
3. Time Machine Editor

Sunday, June 27, 2010

Debug BlackBerry Application Inside NetBeans

This post continues the my last post  about blackberry application development.

I added the ANT target to launch the blackberry JDWP debugger from these two articles: 1) and 2). I can launch the BlackBerry JDWP and simulator, but unfortunately the app did not stopped at the break point.
I googled a lot, but it seems none of the post tells you that debugging blackberry successfully inside the NetBeans IDE, I did not find any tutorial about using JDWP in blackberry development document either.
So I have to figure out by myself by using reverse engineering again.

First I set up the BlackBerry project in the JDE environment, and it launched the debugger successfully, can stop at the break point, and can single step;
then I go to the simulator folder of the BlackBerry ( in my pc is C:\Program Files\Research In Motion\BlackBerry JDE 4.5.0\simulator), I found there not only .cod file and .jad file, but also found
 .debug, .jar, and .alx file. I realize this might be the reason: you need to copy .debug files in the simulator! In my old build script, I only copied jad and cod files, did not copy .debug files into the simulator.

So I modified my ant build script, make sure it will copy all the rapc generated files into the simulator folder, which includes:
  - .jad and .jar file, I don't know why JDWP needs jar file, but it works any way;
  - debug file, like .debug and xxx-1.debug, xxx-2.debug, etc;
  - .cod file
  - .alx file

The ant script snippet look like this:
1.  ANT target: copy files.  Since I used bb-ant, this target means copy all the file contains the project name under \bbant folder to the simulator folder.

<target name="copyBBSimulator" if="do.rapc">
    <copy todir="${platform.home}/simulator" verbose="true">
             <fileset dir="${dist.dir}/bbant">
                <include name="**/${name}*"/>
            </fileset>
    </copy>
</target>


2. ANT target: launch blackerry debugger, this target will launch the blackberry JDWP remote server and blackberry simulator.

<target name="bb-debug" depends="copyBBSimulator" if="do.rapc">
    <delete file="${preprocessed.dir}/.timestamp"/>
    <parallel>
          <java jar="${platform.home}/bin/JDWP.jar" fork="true" dir="${platform.home}/bin">
            <jvmarg value="-Xmx128M"/>
          </java>

          <sequential>
              <sleep seconds="5"/>
              <antcall target="nbdebug"/>
         </sequential>
   </parallel>
</target>


At this time I use the new ant script and press the debu button in NetBeans, this time it works!!! It takes a longer time to launch the debugger and simualtor, but finally the app stops at the break point, and it allows me to step in and step out.
Another thing I like to mention is for the system outputs, you can only see them from the BlackBerry JDWP output window, you can not see them from the NetBeans output window.


For your reference, I post the whole build script and source code:
1. bb-build.xml

<target name="post-init">
          <available file="${platform.home}/bin/rapc.exe" property="do.rapc" />
          <condition property="jpda.port" value="8000">
              <isset property="do.rapc">
              </isset>
         </condition>
        
  </target>


   <target name="post-jar" if="do.rapc">
     <antcall target="bbbuild" />
   </target>

   <target name="run" depends="init,jar,bb-run,cldc-run" />
   <target name="debug" depends="init,jar, bb-debug, cldc-debug" />


   <target name="cldc-run" unless="do.rapc" >
       <nb-run jadfile="${dist.dir}/${dist.jad}" jarfile="${dist.dir}/${dist.jar}" jadurl="${dist.jad.url}" device="${platform.device}" platformhome="${platform.home}" platformtype="${platform.type}" execmethod="${run.method}" securitydomain="${evaluated.run.security.domain}" commandline="${platform.runcommandline}" classpath="${platform.bootclasspath}:${dist.dir}/${dist.jar}" cmdoptions="${run.cmd.options}"/>
   </target>

   <target name="cldc-debug" unless="do.rapc">
       <delete file="${build.dir}/.timestamp"/>
                <parallel>
                    <nb-run debug="true" debugsuspend="true" debugserver="true" debuggeraddressproperty="jpda.port" platformtype="${platform.type}" platformhome="${platform.home}" device="${platform.device}" jadfile="${dist.dir}/${dist.jad}" jarfile="${dist.dir}/${dist.jar}" execmethod="${run.method}" securitydomain="${evaluated.run.security.domain}" commandline="${platform.debugcommandline}" classpath="${platform.bootclasspath}:${dist.dir}/${dist.jar}" cmdoptions="${run.cmd.options}"/>
                    <sequential>
                        <sleep seconds="5"/>
                        <antcall target="nbdebug"/>
                    </sequential>
                </parallel>
   </target>

<target name="post-clean">
   <delete failonerror="false">
      <fileset dir="${platform.home}/simulator">
       <include name="**/${name}*">
       </include>
      </fileset>
   </delete>
</target>

<typedef resource="bb-ant-defs.xml" classpath="bb-ant-tools.jar" />
  <target name="bbbuild" description="blackberry build" depends="init" if="do.rapc">
          <echo message="rapc build, dir=${dist.dir}"></echo>
          <mkdir dir="${dist.dir}/bbant" />
          <rapc verbose="true" output="${name}" jdehome="${platform.home}" import="${platform.bootclasspath}" destdir="${dist.dir}/bbant/" noconvert="true">
          <src>
             <fileset file="${dist.dir}/${dist.jar}" />
          </src>
          <jdp title="${app.title}" vendor="${app.vendor}" version="${app.version}" type="cldc" icon="${app.icon}" runonstartup="false" >
          </jdp>
          </rapc>

        <!-- sigtool jdehome="C:Program FilesResearch In MotionBlackBerry JDE 4.7.0" codfile="${dist.dir}/bbant/${name}.cod" password="" / -->
        <alx destdir="${dist.dir}/bbant" filename="${name}.alx">
            <application id="${app.title}">
               <codset>
                    <fileset dir="${dist.dir}/bbant" includes="*.cod">
                   </fileset>
               </codset>
           </application>
       </alx>
       <mkdir dir="${dist.dir}/bbant-final" />
       <jadtool input="${dist.dir}/bbant/${dist.jad}" destdir="${dist.dir}/bbant-final">
        <fileset dir="${dist.dir}/bbant" includes="*.cod">
        </fileset>
      
       </jadtool>
 </target>

<target name="copyBBSimulator" if="do.rapc">
    <copy todir="${platform.home}/simulator" verbose="true">
             <fileset dir="${dist.dir}/bbant">
                <include name="**/${name}*"/>
            </fileset>

    </copy>

</target>

<target name="bb-run" depends="copyBBSimulator" if="do.rapc">
    <exec os="Windows NT Windows 95 Windows 98 Windows 2000 Windows XP"
             dir="${platform.home}/simulator"
            executable="${platform.home}/simulator/${platform.device}.bat"
        failonerror="true" resolveExecutable="true"/>
</target>

<target name="bb-debug" depends="copyBBSimulator" if="do.rapc">
    <delete file="${preprocessed.dir}/.timestamp"/>
    <parallel>
          <java jar="${platform.home}/bin/JDWP.jar" fork="true" dir="${platform.home}/bin">
            <jvmarg value="-Xmx128M"/>
          </java>

          <sequential>
              <sleep seconds="5"/>
              <antcall target="nbdebug"/>
         </sequential>
   </parallel>
</target>

2. build.xml


<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE project [
       <!ENTITY bb-common SYSTEM "bb-build.xml">
]>
<project name="helloLWUIT" default="jar" basedir=".">
    <description>Builds, tests, and runs the project .</description>
    <import file="nbproject/build-impl.xml"/>
    <property name="app.title" value="helloLWUIT" />
    <property name="app.version" value="1" />
    <property name="app.icon"   value="/icon.png" />
    <property name="app.vendor" value="foo" />

     &bb-common;
</project>



3. source code project: helloLWUIT:
  Here
  Environment requirements: please refer Shai's blog for blackberry to set up the project environment


Next Steps:
  1. Figure out how to debug blackberry application on the actual device inside NetBeans;
  2. Make sure how to read attribute in the jad file in the BlackBerry CLDC application;
  3. How to easier copy the customized attribute in the midlet jade file to the BlackBerry CLDC jad file.

References:
  1. http://www.xenglobaltech.com/blackberry/index.php?option=com_content&view=article&id=45&Itemid=40
  2. http://supportforums.blackberry.com/t5/Java-Development/Building-Blackberry-apps-with-Netbeans/td-p/431612