PURSUING THE TAO OF PROGRAMMING

Why I decided to take Scrum training?

2011-12-22T21:45:00.002-05:00

Yesterday I registered an Certified Scrum Master training. I choose the seminar on Jan 12, 2012 from Berteig Consulting. Actually I am not so interested in SCRUM, rather I am super fan of Lean/Kanban, XP and software craftsmanship.
I have a bias on SCRUM because I feel its process is oversimplified, and focus on manager side only, ignores developer side; and it lack the whole picture of Lean and flexibility of Kanban. Another reason is so far I did not find a good SCRUM example in my past experience, so I really doubt the its performance.
The third reason is recently I found many articles which pointed out the issues of SCRUM.

As a many years of working experience as a software developer, I really doubt a certificate is useful. I don't think that you will become an SCRUM master just for 2 days training, which means master Agile philosophy is too easy.

But recently I changed my mind, and decided to get a SCRUM Master certificate. You may wonder why I changed my mind so quickly.

Why learn Scrum?
1. I realized that the current IT situation is even though Lean/Kanban is really hot inside the Agile community, but outside of the community, Scrum is still the most popular one. I will say 90% of the IT managers does not know Kanban. They are more familiar with SCRUM. If you have a Scrum training or get a certificate, it will be helpful for your career path. My long term goal is Agile coach, so I hope I can use CSM as a business card to help me find better jobs.

2. I hope the CSM certificate can help me make changes in my company. 6 month earlier, I tried to suggest to my managers thinking of using Lean/Kanban, they are familiar with Scrum, but not Kanban. I found I get stuck, because I don't know how to persuade them, since neither I have Kanban experience, nor I have Scrum certificate. Now there are 3 Scrum Master in our company. I hope if I have a Scrum certificate, then I could use their familiar "Scrum" language to communicate, then it would be easier for me help the company to transform from Scrum to Lean/Kanban.

What do I want to know about Scrum?

Clear the misunderstandings about Scrum;
Focus on the principles and philosophy behind Scrum;
Figure out the fundamental assumption which support Scrum;
Find out the good and bad parts of Scrum;

Cynefin framework and software code

2011-12-12T23:15:00.003-05:00

Recently I read some article about Cynefin framework, it is really inspiring. As the diagram shows, cynefin framework categorize the problems in to 4 domains:

simple
complicated
complex
chaotic

For different domain, the approach and strategy are quite different.
Today I would like to analyze the software code using cynefin framework.

Which domain does software belong to?
In General I will say software belongs to complex domain. One reason is software product environment changes so fast, the other reason is its entropy nature: always grow from simple to complex. Software product is so hard to predict and plan. That is the reason why we choose agile methodology: we follow TDD, keep refactoring, iterative developing and continuous integration. These methods are to deal with rapid the software's rapid changing context.
Let's compare the complex domain with agile practices:

complex, in which the relationship between cause and effect can only be perceived in retrospect, but not in advance, the approach is to Probe - Sense - Respond and we can sense emergent practice.

Here The probe - sense - Respond is same as TDD and iterative development, since we don't understand the domain, so we have to rely on immediate feedback.
And the emergent practice and retrospection, which is the same as refactoring, refactoring itself is emergent practice.

We can also conclude that the reason why waterfall does not working is because they assume the problem domain in software is always in simple domain.

How about other domains?
If we don't care code base and let it grows, then code base will becomes more and more complex until it becomes chaotic. As a developer we all have the experience of working with messy legacy code base: the messy code is hared to understand, hard to maintain; you fix one issue, then will generate another issue unexpectedly; the software is unstable, a small change will cause a big issue; the issue is always unpredictable;nobody understand the system; we spent all the time putting fires. Which is totally unstable and uncontrollable. These symptoms fit nicely with the chaotic domain model.

But if we keep writing clean code, the code are easy to understand, easy to maintain, easy to change. In agile world writing clean code also means simple design. So this will fit nicely in simple or complicated domain. Since they are both ordered. so they are manageable and controllable. That is software's ultimate goal - flexibility, maintainability and reusability. So we will say clean code will help the software reduce the complexity.

Why clean code will help reduce the complexity?
The following tools and method help us writing clean code:

SOLID principles
high cohesion/loose couple
Design patterns
DRY principle
LoD principle
Refactoring

Lets' dig them deeper:
- Break the unnecessary dependencies

Uncle Bob once said writing software is all about dependency management. SOLID principle, high cohesion/loose couple means reducing dependencies. We all know that the more interactions between objects, the more complex the software will be, so reducing the dependencies means making the software simpler.
- Set Rules and Constraints
Design patterns basically is describing the rules of the relationship between objects in a specific context. To make software reusable, stable and manageable, we do not allow objects interact each other randomly, they have to follow certain rules. This is quite similar in human society, we have to maintain the orders via laws, rules or regulations.

- Encapsulate changes

One of the advantage of design patterns is how it deals with change: encapsulate changes. Which means when we identify a concept which it changes, then we encapsulate it by putting it into a class. we don't want them randomly change. we put them in a box, isolate them. Once we put those details evil in the box, then we can understand or control the system by using abstraction. Then the system becomes stable, manageable and controllable

Conclusion

In General software belongs to complex domain. The agile practices are the right practices to solve the complex domain problems;
Writing clean code will help us reduce the complexity, will move from complex to simple/complicated domain;
Writing messy code will increase the complexity, will move from complex into chaotic domain, this is what we are trying to avoid.

Strategy
So our strategy will be constantly refactoring our code base, keep code clean, avoid messy code.

Resources

How javascript call java in Phonegap android

2011-12-03T23:10:00.001-05:00

Phonegap is a great mobile solution, since I am working in Grapple mobile now, our platform use quite similar solution with Phonegap, so I really want to learn Phonegap to see if we can steal some ideas from it.
What troubled me a long time is in its android platform: I wonder how does its javascript code calls java native code? Before I tried to read its android source code, but did not find the answer. This week I spent some time on its source code, finally I find the answer. I would like to share this result with you, this might be valuable especially if you want to dig more to phonegap, since I seldom find the answer from google or books about phonegaps, none of them talks about this.

The phonegap source code I checked out is from github, the current version is 1.2:
1. phonegap android source code: https://github.com/callback/callback-android
2. phonegap platform sample code: https://github.com/callback/phonegap

Digging the source code
From phonegap-1.2.0.js

For example looking at accelerometer: in javascript layer, the accelerometer is implemented by calling Phonegap.exec() method:
The PhoneGap.exec() method will be like this:So the exec method will invoke a javascript prompt()- which is popping up an dialog, huh, it sounds weird.

Let's look at the java code:
Inside com.phonegap.DroidGap.java, find the inner class GapClient, which is a subclass of WebChromeClient, it override the following 3 methods: onJsAlert(), onJsConfirm() and onJsPrompt()
Here we only look at the onJsPrompt(), look at the android API for onJsPrompt():

Tell the client to display a prompt dialog to the user. If the client returns true, WebView will assume that the client will handle the prompt dialog and call the appropriate JsPromptResult method. If the client returns false, a default value of false will be returned to to javascript. The default behavior is to return false.

So it is quite clear that: if prompt() is called in javascript layer, then onJsPrompt() method will get invoked in java layer. Phonegap just uses this way to handle the javascript to java communication.

Let's look at the onJsPrompt() method in details: So here the phoneGap PluginManager will delegate the the javascript call to different native plugins.

From the article of http://stackoverflow.com/questions/7251557/phonegap-and-prompt
and phonegap release note: http://phonegap.com/2011/04/28/phonegap-0-9-5-released/, it mentioned that

Worked around JavaScript bridge exception for Android 2.3. Use “prompt” instead of calling objects directly.

So it seems that this is a workaround for an issue in Android 2.3.

Summary
To recap, this is the flow of javascript call java in phonegap android is like this:
1. in javascript layer, all the javascript method will finally delegate to Phonegap.exec() method
2. PhoneGap.exec() will call prompt()
3. In java layer, DriodGap.GapClient override onJsPrompt(), which will capture the prompt() method call in java, which then delegate to PluginManager to invoke the native java method call.

3 types of work - yesterday, today and tomorrow

2011-11-25T21:39:00.001-05:00

I found it is useful to describe software developer's daily job into 3 different types: yesterday, today and tomorrow.
Yesterday: bug fix, maintenance job. since this kind of job is based on the product which is supposed to be finished, so I describe it as yesterday.
Today: developing phase, the work on the product is under development,
Tomorrow: some research and R&D work.
These 3 different type has different nature, and they need different mindset and strategy to deal with them.
Let me dig them into deeper.

Yesterday's work
Yesterday's work are bug fixing and maintenance job. This kind of job is putting off fire, they are urgent, and undeniable. We need to fix them ASAP.

Yesterday's work has no value or negative value, because customer already paid us up front, this work is support and service. This kind of job is not adding value, but removing the negative value.
The more time and effort we spend on the yesterday's work, the less value remained for us. The cost of fixing a bug in production phase is much expensive than fixing it in developing phase( today's work).

Yesterday's work is quite stressful. A minor issue delayed in yesterday will cause higher priority work. Yesterday's job is quite critical for time. we just need the job get done, there is no time for refactoring, no time to focus on better design. There is no room for developer to improve the system (code base, design, code quality etc) at this stage.

If there are too many yesterday's work, then this is not a good signal: It means we have a large amount of technical debts from today's work, it means bad design and careless decision.
So yesterday's job is highly dependent on today's work.
Ideally we should keep the number of yesterday's work as low as possible.

Today's work
Today's work is about current realistic issues. Today's work is the tasks which is belong to developing phase. We develop a new feature which is our customer wanted, so this job will bring values. They are urgent and important, but some features can be negotiable. Customer will focus on feature than time, so the time is not so critical as Yesterday's work.
Developer has lots of room to improve the system, he can have time to improve the design, refactoring.

The strategy of today's work will be - Build quality in. Focus on quality over time. If we focus on quality at this phase, then time must be shorten on maintain phase.
Quality means more than functionality, also means maintainability and readability. We should focus on simple design, write clean code, keep refactoring. and do not easily leak the bug to maintenance phase.

Tomorrow's work
Tomorrow's work is focused on research and innovation. It will bring us potential value in future, so I call it tomorrow's work, means work for tomorrow.

It for a long term goal, we might not realize its value at present. it's important but not so urgent.

Tomorrow's job is like investment. there is a risk, we don't know how much value it will bring us.

Usually the priority of tomorrow's work is less than yesterday and today, if there are too many work spending on yesterday and today, then there is no time for us on investment. But if we did a good job on today's work, then we need less time on yesterday, then we have more time spending on tomorrow.

Mapping to First thing first
I found these 3 types of work map perfectly with Stephen Covey's first thing first :
Yesterday - Urgent but not important
today - Urgent and important
tomorrow - Not urgent but important

Conclusion
Yesterday is your debt, today is your earning, tomorrow is your investment.
We need to focus on today's work, keep improving the quality of today's work.
If we want to be productive, we need to balance these 3 types of work wisely.

Be careful "this" in Javascript Object

2011-11-06T21:09:00.000-05:00

Consider the following javascript code:
What if I remove the this in method increment()?, like this:

I must admit at first I wrote the code like the second, But it was wrong.
As a Java programmer, I thought keyword this in javascript will like java this always refer to the object owner, so this can be omitted. But I was wrong, the code above is wrong, because javascript requires you to use this keyword explicitly, otherwise it will treat variable value as undefined. It seems javascript will treat variable value as local variable, except you specify this explicitly.

From this mistake, I understand that the experience in java is not working in javascript, if you want to access the class level variable in javascript method, you have to explicitly using this. Be careful for "this"!

The function definition in Javascript

2011-10-22T22:48:00.003-04:00

I highly recommended Douglas Crockford's book "Javascript:The good parts". In the page 133 of the book, the author mentioned 2 ways of defining a function:

use function statement,
use function expression

Their results are exactly the same. For example,
Function statement:

Function expression:

The above two have the exactly same result. they will both define a function foo. But I found there are still one difference: scope.
The function statement is globally scoped, it can be put anywhere in the javascript, and you can call the function before or after it is defined.
For example:

But in function expression is different, the function call statement must be after the function is defined, otherwise the function call will be failed.
For example:

When you think a little deeper, the error of function expression make sense, because the function expression means first declare a variable, then assign it with a function object. Why it failed before the function defined is because the variable is not declared and assigned yet.

So if you use the function expression to define a javascript function, you need to aware this issue.

Learning javascript: understanding prototype property

2011-10-16T23:32:00.002-04:00

Prototype in javascript is so important, because javascript implements the object inheritance via prototype delegation. Here I tried to summarize what I found for the javascript property. The examples I used are tested on Firefox Javscript shell .
Please note that in Javascript function is also an object, so here function and function object are interchangeable.

1. Every function object is automatically created with its own prototype property, the prototype property itself is an javascript object.You can add property or functions to the prototype property via function object.
Example:
2. When a object is created by using new keyword, it will get the prototype object from the constructor function; created object can not access prototype property directly like function object, but it can access the property of the prototype object:
example:

3. When the created object instance read a property of the prototype, which is directly linked to the prototype object of its function constructor; while you want to modify the property, you end up by creating a new property in the object itself, which will not impact the same property name in the original function prototype. I believe javascript in this way implements the OO inheritance and overriding.
Example:

4. The function object prototype property, which itself is an object, its default value contains only one hidden property: constructor, whose value is the same as the function itself.
Example:

Code is constraint

2011-10-10T23:04:00.003-04:00

Last week I watched Michael Feathers presentation: Code blindness.This is a great presentation, and it inspired me to think deeper.

Before I think code is important from the programmer's perspective: craftsmanship, and professionalism. But it seems quite far to the managers. They don't know understand it and don't know how to connect this with their daily management.

Today I would try to think from manager's perspective: borrowing the idea from theory of constraints, I will say code is just another constraint, like other constraints manager usually considering: time, budget, people. More specifically code quality will impact developers productivity and on time delivery.

We know that the clean code base will easier to understand, easy to make change, and easy to maintain,but bad code base will hard to understand, hard to change, and hard to maintain, which greatly slow down developer's productivity.To avoid this happens, we have to deliberately keep code lean.

There are two main metaphor to support the clean code:

Metaphor of technical debt: code base is like technical debt, we have to pay off first by taking effort to refactoring, otherwise it will become harder and harder to change and maintain in future, so refactoring is the interest we pay for the debt.
Metaphor of software entropy: code base is like entropy, by nature it will gradually go to messy, we have to constantly refactoring and keep our codebase clean.

So we can get the conclusion: code can be asset if it is clean; and can be debt as well if it is messy.

So far we know that code is very important to developer's productivity, it should be consider as a constraint by the manager. But I found many managers are totally unaware of this issue:
They just assume developers are all professional and their code will always good enough;
They only care the schedule, only care something get done, but does not care if the code is well maintained or not.
And since managers unaware or ignore this issue, they might make decisions to make code base even messier, for example,
- they will pressure developer to push them make quick-and-dirty fix, which cause the long term issue;
- They do not care for the code, so they measure the performance only by task been finished, not by how good the code is written, so we can imagine developer will not actively care for the code.

Code is important, but if you unaware or ignore it, then you will not manage it.
If you don't manage it, then you will finally lost the control and then we will get real problems.
When you have the problems, but you don't understand the root cause is code quality, then you probably trying to solve the issue in a wrong way.
I heard so many times from managers said something like "we need to work harder", "we should work more careful", which are totally useless slogans.

Suggestions

Manager should aware the code quality's importance, bring the connection between good code quality to developer's productivity and developing schedule.
Manager should cover code quality into their daily management, right now there are so many different tools to do code analysis and measurement.
Manager should create a culture to encourage software craftsmanship, clean code is a foundation of good software product, clean code fit nicely the "build quality in" lean principle.

Making developer multitasking is a bad idea

2011-10-01T23:30:00.002-04:00

Following is copied from a job description for a mobile software developer from my previous employer QuickPlay Media:

"Well organized with the ability to effectively manage multiple projects simultaneously. Extensive experience in a fast-paced deadline-oriented environment".

When I first look at it, I was really surprised by the word "multiple" and "simultaneously", it is pretty obvious that they requires developers to be able to do multitasking jobs. Today I would like to dig this topic a little bit.

What does "manage multiple projects simultaneously" exactly mean?
- Working project A for 5 minutes, then working on prjoect B another 5 minutes, and project C for another 5 minutes?
- Or when you are working on project A, then interrupt by project B, you have to suspend the project A, then switch to project B;
- Does it mean the developer has more project working simultaneously, the better for his performance review?
- Does multitasking really improve the productivity? get faster delivery?

This idea is totally wrong
In the last 2 years I read a lot books about agile and software craftsmanship, so I am quite confident to say this is totally wrong, it will only make things worst!

Fist it is against human nature. Human is not machine, he is non-linear, a person can only focus on one thing at a time. Interrupt and context switching is very expensive. The book Pragmatic Thinking & learning and peopleware cover this topic, and explain why managing focus is so important to maintain a higher productivity. and that is the reason why the Pomodoro technique will force you only focus on one task in each 25 minute session. Managing people's focus is the key of improving his productivity.

Second it is against the agile principles and and other proven theories. The goal of agile is to deliver the product as fast as possible, but the interesting thing is the agile trying to reach this goal by limiting the Working-In-Progress items (WIP). Scrum is trying to limit WIP in one iteration, WIP is called inventory in Lean, which is a waste and should be limited. Limiting WIP is the key concept of Kanban technology. and in theory of constraints also proved that putting to many task in the queue will only slow down the productivity.The reason is pretty obvious, think about this scenario, if you have 3 projects, each will take one week. If you work sequentially, you will deliver one project after 1 week, 2 projects after 2 weeks, and you will get 3 projects after 3 weeks. But if you work simultaneously, you will get any project done until after 3 weeks, but considering the context switch cost, it will take you longer time to finish all 3 projects. So based on Scrum, or Lean/Kanban, or Theory of Contraints, working simultaneously will not deliver fast, it will only delay the delivery time.

Third is it is a shift burden to from mangers to developers. Managing backlog is manager's responsibility, but requiring developer multitasking means putting some more backlogs into developers. It might looks better for managers, but actually it is bad, since those extra backlogs are either under working or putting into developer's own backlog. This is actually hide the visibility, which is even harder for manager to track the progress. I can feel that the multitasking is a solution which managers are trying to solve too many backlogs, but it is a bad idea, since it only move the backlog from manager layer to developer layer, which making things even worse; the manager should figure out the root cause and find a true solution to increase the burden down rate or increasing the productivity. I like to call this issue is leaky management issue(copied the idea from leaky abstraction from OO design), because it is supposed to be solved in the management layer, but it leaks to the developer layer.

Based on the above analysis, I can feel multitasking will generate following result:
developer will have low productivity, projects will get more delayed, developer have to work harder and over time; since more projects get delayed, so manger put more projects to developer to work simultaneously, then becomes a positive feedback loop, which will make things even worse...

Summary
I feel very sad for my old colleagues, because they are not treated well; and I feel sorry for my ex-managers, because they are unaware of the basic principle of agile. To make a developer working efficiently, the only thing is trying to make him to work on one thing only. I strongly suggest QuickPlay to remove multitasking in its job requirements, but removing that from managers minds are much harder.

learning clojure: how to compile clojure source code

2011-09-13T22:13:00.002-04:00

Currently I am learning clojure now, I focused on the book "The joy of Clojure", but actually I did not enjoy the joy of clojure so far. For me the book is still difficult. As the author mentioned, this book is not for beginner. Most of my time I have to refer on the Clojure API document. Today I want to share my experience on learning chapter 10.2.1, how to compile a clojure source code.

The source code is in page 212, listing 10.3, called DynaFrame.clj:

The author mentioned how to ompile the source code:

You can compile this namespace by saving it in a directory joy/gui, located on the classpath, in a file named DynaFrame.clj and executing the function (compile 'joy.gui.DynaFrame) in a fresh REPL.

As a clojure beginner, I found it is described not detailed enough, it took me almost a night to figure out how to build it successfully.
After fixing different weird problems, finally I got it working. Here is what I found:

1. Be careful clojure version: at first I use clojure 1.0 to launch the REPL to compile it, but I got Exception to complain not found java.lang. Later I realize the problem is in ^{:static true}, you have to use clojure 1.2 to compile it; if you still want to use clojure 1.0, then use #^{:static true} instead.

2. Use cljr tool, I found it is really handy to add classpath: just run cljr add-classpath <your folder>; run cljr list-classpath will display your classpath.

3. Add source code folder in the classpath
put DynaFrame.clj under joy/gui, and
run cljr add-classpath ./

4. Add *compile-path*
create an directory called "classes", and
run cljr add-classpath classes/
Based on the clojure API document:*compile-path* defaults to "classes", the directory must be in the classpath for 'compile' to work.

5. run cljr repl to launch the REPL, type
(compile 'joy.gui.DynaFrame)
and
(joy.gui.DynaFrame. "1st")

In summary, to compile the clojure code successfully, you need:
- Add source folder in the classpath;
- Create and add "classes" folder in the classpath

Developing Blackberry application using NetBeans Plugin - part 2, fixing debug issue

2011-09-06T22:14:00.000-04:00

On my previous post, I reviewed the new blackberry Netbeans Plugin. Latre I found Shai also published his post about how to use the new plugin in developing the LWUIT blackberry application.
But I found the new plugin could not debug the blackberry application.

Reason
The reason is if you want to debug your blackberry application, you have to copy all the .debug files into the blackberry simulator folder, include xxx.debug, xxx-1.debug, xxx-2.debug, ... etc. From the generated build-impl.xml script file, we know that blackberry plugin only copied the xxx.debug file and missing all other debug files.

Solution
Add the following targets in the build.xml, which will override the same target in the build-impl.xml:
1. Update target blackberry-pre-debug, it will copy all the debug files into the simulator folder.

2. Update target clean-blackberry, it will delete all the files of the application in the simulator folder.

3. If you want to build BlackBerry CLDC application, you need to add the target create-cod:

The only difference with build MIDlet application is in <arg value="-cldc"/>, by default, the Blackberry plugin already choose MIDlet by using <arg value="-midlet"/>.

Developing Blackberry application using NetBeans Plugin - part 1, first impression

2011-08-10T22:31:00.005-04:00

Updates: I got some feedbacks that the download link in Netbeans is not working anymore, I uploaded the plugin binary into my google doc, please try my link if you have problem to download.

Recently when I checked out the latest version of the source code of LWUIT, I realized that LWUIT used a new solution of developing blackberry application - using Netbeans Blackberry plugin. So today I tried to give an review about this new tool.

Plugin Installation
1. go to this link (or my link)to download the blackberry netbeans plugin, save it in your local drive.
2. In your Netbeans IDE, choose Tools - Plugins, click Downloaded tab, press "Add Plugins..." button, choose the downloaded blackberry plugin .nbm file,
3. Restart your Netbeans IDE, the thing is it seems the blackberry plugin has an issue, you can not see it under the Installed tab of the Plugins UI Form. I was confused it at the beginning as well.

Add BlackBerry SDK platforms
The plugin download link (or here)gives some screen shots about how to add blackberry SDK platform into your NetBeans IDE environment, here I just give an rough description, if you are familiar with the steps of adding other J2ME SDK into the NetBeans IDE, you will know it is the exactly the same steps.
Compare this step with the older solution, you will feel how easy it is.

After Adding the BlackBerry SDKs in your NetBeans IDE, you can use the same way as J2ME Mobility package for blackberry development. Really Nice!!! Good job for the author!

Test
To test how the Plugin works, I choose 2 tests, one is for MIDlet and the other is for CLDC application (RIMlet).

Test1 - build an BlackBerry MIDlet application
I imported an JavaME SDK sample application - SunSamples-Demos into my NetBeans IDE, then add another configuration, choose an BlackBerry platform, I just choose JDE 4.5, then everything else is copied from the default configuration.
1. Test Build
Click "Clean and Build", the IDE build successfully and generate .cod file and bunch of .debug files.
So Build test passes.
2. Test Run
When I click "Run", the IDE launches BlackBerry simulator, then I found the several applications (since SunSamples-Demos contains multiple MIDlets), I choose some apps, they work perfectly.
Application Running also passed.
3. Test Debug
I set several break points in the IDE, then I click "Debug", the IDE launches the BlackBerry JDWP application and the blackberry simulator, when choose an application, the application did not stop at a specific break point.
The debug test failed.

Test2 - build an BlackBerry CLDC application
This time I choose a LWUIT blackberry port demo application - LWUITDemoRIM, just check out the LWUIT source code from its SVN, open the LWUITDemoRIM project. If you installed the plugin and added the blackBerry SDK properly, it is very easy to configure the project in the NetBeans.
Here I would like to mention, for native CLDC application, you need to do a little extra work in your build script. Check this link for details: http://forums.netbeans.org/topic24475.html

1. Test Build
Click "Clean and Build", the IDE build successfully and generate .jad file, .cod file and bunch of .debug files. If you open the jad file, it will tell you the application is CLDC application.
So Build test passes.
2. Test Run
When I click "Run", the IDE launches BlackBerry simulator, you will find the familiar LWUIT demo icon, the application works perfectly as I saw before.
So running test passed.
3. Test Debug
When I click "Debug", I saw the same behaviour as the MIDlet application test, the application never stops at the break point.
Debug test failed as well.

Conclusion
Pros:
- The Blackberry Plugin for NetBeans is pretty easy to install and use. The developer will feel it looks pretty same as other J2ME SDK.
- The Build process is really simple and almost same as regular J2ME application, it will convert jar file into blackberry .cod file.
- The IDE can launch Blackberry native simulator and can run the application successfully.
- The plugin built-in support MIDlet build, need a little work for RIMlet.

Cons:
- Debug is not working, after I read the build script, I know why: based on my experience, you need to copy all the .debug files into the simulator folder, but in build-impl.xml, the target "blackberry-pre-debug" only copied 1 .debug file but missing others. I will post the detailed solution in future.

Next Action
In future, I will continue post my investigation about this plugin, includes:
- How to build the MIDlet and RIMlet
- Dig deeper on how it implement the blackberry build
- How to fix the debug issue

Becoming generalist - dealing with constraints in software development

2011-07-17T00:20:00.007-04:00

Recently I read the fascinating book - the goal, it is really an eye opener for me. I learned the theory of constraints from the book. Everything of TOC looks like common sense, but the conclusion is really striking.

TOC tells us the importance of bottle neck (constraint). We need to identify it, and protect it. In the example of manufacturing assembly line, to make sure the task in bottle neck is the highest priority, other jobs even have to be idle instead of working, otherwise it will make the whole productivity even worse!

I believe TOC can apply to software development as well: because the software developing also needs a series dependent activities. And similar to manufacturing, software development also has constraints(bottlenecks), it might be in: test,coding,deploy,maintenance. On the other side, compared to manufacturing, software development is people centered vs. machine centered; most of the constraints is about people; software process is lot easier to change compare to assembly line.
In software industry, we could have a big advantage over manufacturing: to deal with bottleneck, we don't need put other people to sit in idle, we can put more people working on the bottle neck! But this advantage depends on a requirement - if our developers are generalists.

If every developer in a team is a generalist, which would be a great help to maintain a high productivity, then we can deal with almost any constraint, since when there is a bottleneck, we can put more people on it. So the developing flow will be well controlled in a high productivity speed.

Becoming a generalist raise the bar for developers: we are not just a geek who focus on a single area, we are not only J2EE developers, but we also need to fix the bug of a J2ME application; we might need to mange data base as DBA; or improve the build system by writing Ant or shell script.

Becoming a generalist is really challenging, and much harder than focusing on one single area. But based on my experience, software developing job is not a rocket science, every technology can be learned if given an enough time. Technology is not a problem for becoming a generalist, the problem is attitude and mindset. We need to be open minded, be flexible, and ready to wear different hat at different context.

A generalist has following benefits:

Doing different jobs, give developer a chance to learn different new technology and tools, learn by doing will make you learn faster.
Have chance to know the big picture of your work.
Better for team work and collaboration
Improve your competence and marketability, since you can put more stuff on your resume.
More changes to become linchpin and less likely being laid off, who wants to fire a developer who can do multiple jobs?

I will suggest our managers should encourage developers to become generalist. For example, he can deliberately assign different jobs for one developer; rotate developer to another team; foster pair programming and mentoring, etc.

Can software be estimated?

2011-07-10T23:09:00.003-04:00

As a software developer, I am always asked by project managers:How long do you think you need to finish project X or fix issue Y? Most of the time I feel annoyed and guilty, because usually I don't really know the answer but I have to give them a number - which is far from accurate.

I believe how to get a good estimation is one of the major issue for all the developer and managers. An accurate estimation help manager control and plan the project, which make sense. But it is based on an assumption: the software can be estimated. Is this assumption really true? Let's investigate it.
First, if we assume software is estimatable, => then we can see software is predictable, => then we can plan ahead, => waterfall model. We all know that waterfall and CMM is not workable, try to harden the software project is myth.
Next, let's check the software development model. From Lean/Agile software development we know that software is a discovery process, an explorative process, a theory building process, an process of reducing ignorance level. In summary, it is a learning process, the problem solving process itself is also a learning process. That is why agile uses iterative, feedback process. Which means at the beginning you know very little, you only understand the problem when you solve the problem at the end. So how could you estimate or predict at the beginning? How could you give an accurate estimation at the beginning? You can predict only if you know all the variables, but in software development, there are so many unknown variables: third party library issues, platform compatible issues, IDE, target constraints, etc and other issues you might not aware(you don't know you don't know). So based on analysis the above , we can say estimation in software is impossible.

Let's look at another question: can we estimate bug fix? Bug fix is also a learning process, especially when you are assigned a bug which code base is not written by you. You need to take time to understand the bug, reproduce the issue, understand the code base, familiar with the context, all this stuff are learning process and takes time. How can we estimate this time, because this is really case by case. And we all know that bug fix is actually we spend 90% of time on finding the bug, another 10% time on fixing the bug. Once we found the bug, we know how to fix it. So we can only see we can estimate the bug only after we found the bug, but at this moment it is pointless to give an estimation since we already solve the 90% of the problem.

I would like to share my own experience:In an iterative development process, there is no need to estimate, just do it. Before I start fixing a bug, I knew nothing; the time I know the estimation answer is when I solve the issue, so the process of finding the answer of estimation is the same process of solving the problem. So at this case why we need the estimation?

So we know that software nature is impossible to estimate and needless to estimate. But why project managers need it? Maybe they don't have technical background and still trying to use the concept of waterfall methodology; or maybe the psychological reason: human usually not comfortable with uncertainty and needs for closure .

Forcing estimation cause some worse consequences:

Force developer to plan up front, which will fallback to waterfall;
Developer faces the pressure to lie, and give the wrong number, which cause,
Wrong measurement is worse than no measurement. Like weather forecast.

What can we do when the project manager ask you estimation next time:

Live with the uncertainty, that is the nature of the software development;
To be honest, tell them you don't know yet, you only estimate what you can deliver after the current iteration;
Timebox the estimation, spend 30 minutes to investigate the problem, you should know if you can finish it by the end of the day or not. Avoid yak-shaving.

Learning the theory of Constraint - "The goal" Review

2011-07-03T23:18:00.001-04:00

Yesterday I finished the book:The Goal: A Process of Ongoing Improvement. The book is so fascinating, when I read it, I can not put it down, just like reading one of Agatha Christine's books. I never imagine that a business book can be written like this.
What I am most interested is the principle described in book: the theory of constrains. Here I copied some key points from the TOC WIKI:

Key assumption
The underlying premise of Theory of Constraints is that organizations can be measured and controlled by variations on three measures: throughput, operational expense, and inventory.
Throughput is the rate at which the system generates money through sales.
Inventory is all the money that the system has invested in purchasing things which it intends to sell.
Operational expense is all the money the system spends in order to turn inventory into throughput.
"The Goal" itself is to "make money". All other benefits are derived, in one way or another, from that single primary goal.

The five focusing steps
Theory of Constraints is based on the premise that the rate of goal achievement is limited by at least one constraining process. Only by increasing flow through the constraint can overall throughput be increased.Assuming the goal of the organization has been articulated (e.g., "Make money now and in the future") the steps are:

Identify the constraint (the resource or policy that prevents the organization from obtaining more of the goal)
Decide how to exploit the constraint (get the most capacity out of the constrained process)
Subordinate all other processes to above decision (align the whole system or organization to support the decision made above)
Elevate the constraint (make other major changes needed to break the constraint)
If, as a result of these steps, the constraint has moved, return to Step 1. Don't let inertia become the constraint.

Applying to software development
This book is about manufacturing, but I found it can be easily applied to the software development. And I found it is very similar with Kanban method: Kanban help us visualize the bottleneck, improve the productivity by limiting the Work-In-Progress. Which is exactly the same as the theory of constraints. I feel that the TOC can be regarded as the scientific theory behind the Kanban system. Before when I learn Kanaban, I just accept the principles and conclusion from the Kanban, but I never thought of why it is like this or that, Since this book uses lots of examples and step by step from question to conclusion. Now I understand more about Kanban. So I can say Kanban is an implementation of TOC.
If you knows Lean/Agile, you will find there are lots of concepts shared between lean and TOC:

lead time
Inventory is bad
WIP (Work-In-Progress)
small batch increase the productivity
local optimization is not working only optimize the whole works.
The dependent event queue can be mapped to the value stream map, or Kanban Queue.

Even the subtitle of the book - "A Process of Ongoing Improvement", which is perfectly match one of the lean philosophy - Kaizen, which means continuous improvement.

I just found David J. Anderson's book: Agile Management for Software Engineering: Applying the Theory of Constraints for Business Results. I haven't read that, but based on the title, it must be about applying the TOC into software development. In his recent blog post: REFLECTIONS ON THE PASSING OF ELI GOLDRATT, he describe how TOC inspired him for his book and Kanban system.
Actually when I read this article from twitter last week, and this is the reason to let me read the book of "The goal".

And I found this youtube video is interesting: Standing on the Shoulder of Giants by Eliyahu M. Goldratt, Godratt talks about the TOC, Kanban and Toyota Production System.

I realized I need to read this book one more time. the TOC is so powerful, but I still have lots of questions, like:
- How can we apply it into our daily software development?
- How to find the bottleneck?
- What are the measurements we need?

Hope I can find the answers from the goal, kanban and Lean.

NoClassDefFoundError in Blackberry

2011-06-20T16:19:00.005-04:00

Background
Since lastweek I spent couple of days trying to fix an issue: The application keep throwing NoClassDefFoundError Exception in a callback method. Through JDE debugger, I can figure out exactly where the exception is threw and which object cause the exception, but when I check the code back and forth, I could not find the problem, since the class file is there. Finally I realized that I also changed the build script: before I use antenna wtkbuild task to generate the library, the new build I use regular javac task, since I used the blackberry native API. Will it cause the problem? It does not make any sense to me. I tried to switch back the build script using twkbuild task and adding the rim api in the class path, this time everything works! No more NoClassDefFoundError any more!

Finding the root cause
I compared the wtktask and the javac task with blackberry build script:

build target(with issue):

<target name="compile_rim" depends="init">
 
  <javac destdir="${outputclasses}" encoding="${src.encoding}" source="1.3" srcdir="${src.dir}:${rim.src.dir}">
    <classpath location="${rimlib.loc}"/>
   </javac>
</target>

Created by Pretty R at inside-R.org

wtk build ( no issue)

Form the above, I can not tell the difference, and can not figure out why there is an issue. So I have to look at the antenna source code. When I looked at the WtkBuild.java, I noticed that antenna sets the javac target to 1.1 by default! This is interesting.

Check the ant document for javac task, for target attribute, it says:

Generate class files for specific VM version (e.g., 1.1 or 1.2). Note that the default value depends on the JVM that is running Ant. In particular, if you use JDK 1.4+ the generated classes will not be usable for a 1.1 Java VM unless you explicitly set this attribute to the value 1.1 (which is the default value for JDK 1.1 to 1.3). We highly recommend to always specify this attribute.

So based on the above statement, I suspect that the Ant require us to specify the target version, if we don't specify ant will use the default version.and the default target version cause the blackberry device throws the exception.

What is the target version for J2ME and Blackberry
In Antenna, the target is set to 1.1, is this the correct value for J2ME and Blackberry? To prove this, I googled and find some articles:
Managing Wireless Builds with Ant, this article from oracle mentioned that J2ME build requires target value to be 1.1:

<javac bootclasspath="${midp_lib}" destdir="build/classes" srcdir="src" target="1.1">

and I found this article in BlackBerry forum:javac -target parameter used by the rapc.exe tool
It shows that you need to use target 1.1 to build blackberry compatible application:

Rapc.exe automatically ensures that these .class files are targeted for BlackBerry® Java® Virtual Machine (BlackBerry JVM) 1.1 or later. This is done to maintain compatibility on the BlackBerry smartphones when the .class files are actually packaged in a .cod file.

If you are using a custom script that calls javac.exe and rapc.exe separately to package your application, you will have to make sure that you are calling javac.exe with -target option so that all generated .class files are compatible. This is a requirement to make sure that the generated .class files are consistent with Connected Limited Device configuration (CLDC) 1.1 BlackBerry JVM implementation.

Conclusion
Based on what I found, I know that Both J2ME and Blackberry Build need target option to 1.1. otherwise the blackberry application might have a weird issue like NoClassDefFoundError.

How I improved my company's build system - build automation

2011-06-11T23:50:00.003-04:00

Last 2 weeks I haven't check twitter, did not go to infoQ, all my spare time is focused on one thing: improve the build system, the goal - one button click triggered the build and deploy automation.
This goal is finally achieved last week. Now the build system is scheduled deployment once a day, the automation jobs include following:

check out source code from CVS;
build client and server artifacts;
generate the change log
update the wiki and download page
upload the artifact into server and downloading repository
execute the server deploy script
send new release email

Before only step 1) and 2) are automatic using the ANT script. All other steps are manual, I found this kind of job is boring,painful and quit easy to make mistakes.

I have the idea for build automation for a long time, especially I was inspired by the book "Continuous Delivery" and "Pragmatic Project automation". My plan to implement the build automation is pretty simple: simulate the human deployment behaviour, implement an ANT target for each manual step. It sound easy, but when I actually did it, it was harder than imagined. But I finally I made it.
Right now everyday we will automatically release a new build, this is a milestone and a great improvement. I feel so happy and so proud of it.

The tools I used
- ANT, I used scp, filterset, filterchain, sshexec, cvschangelog, I found Ant has lots of gem.
- Groovy, I used AntBuilder to integrate with the existing ANT task, parsing XML, generating text file, quite neat.
- Jenkins, everyone recommend Hudson/Jenkins CI server, when I used it, it is so easy to use, and has lots of useful plugins. I use Jenkins to schedule and trigger the build.
- ViewVC, a very good tool to browse CVS source code from web, I configure the ViewVC in Jenkins, Jenkins will provide the actually source code link in the changes page.

What I learned

Pay more attention to your feeling. If you feel boring and painful, don't just complain, try to find a way to improve it. This is a good chance to improve your self, my previous blog describe this.
A good developer must be able to delegate the dirty jobs to the computers, it would be shame for a good developer to do the repetitive manual jobs over and over.
I implemented the automatic build script based on the manual process, so I learned: if you can define a repeatable step by step manual process, then you can automate it by program.
I finished this job usually in down time, I finished one small piece each time, so the improvement for the legacy system has to be emergent.

Next Actions
This is only the first step, there are way more things needs to be done, my next steps will be:
- Add Unit Test and Regression test automation
- Add the code quality analysis report
- Use Ivy for dependency management
- Use Puppet to manage the configuration

Ant mail task issue in Mac

2011-05-28T23:35:00.002-04:00

Last week I get stuck again when using the Ant mail task, based on the document from Ant site, the mail task requires the following 2 jar files:
- mail.jar from JavaMail API
- activation.jar from JAF API

I already downloaded and copied them into /usr/share/ant/lib folder, but still I got the error.
After google a while, I realized that Mac missed another jar file in its ant package, this time it is ant-javamail.jar! So when I just copied the ant-javamail.jar from the downloaded ant package into /usr/share/ant/lib folder, then every thing worked perfectly.

In summary, to make the mail task work in Mac, you need 3 jar files:
- mail.jar
- activation.jar
- ant-javamail.jar

Ant target to send email example (using gmail smtp server):

<target name="sendmail" description="send email">
        <mail mailhost="smtp.gmail.com"
                  mailport="25"
                  user="${your.gmail.account}"
                  password="${your.gmail.password}"
                  ssl="true"
                from="${your.email.address}"
                tolist="${mail.recipient}"
              subject="Results of nightly build">
          <message>This is a test email</message>
       </mail>
</target>

Ant scp and sshexec task issues in Mac

2011-05-28T23:10:00.000-04:00

Last week when I tried to run my ant target using scp and sshexec task in my macbook pro, I got the following issues:

1) scp task issue
Problem: failed to create task or type scp
Cause: the class org.apache.tools.ant.taskdefs.optional.ssh.Scp was not found.
   This looks like one of Ant's optional components.
Action: Check that the appropriate optional JAR exists in
   -/usr/share/ant/lib
   -/Users/steve/.ant/lib
   -a directory added on the command line with the -lib argument

Do not panic, this is a common problem.
The commonest cause is a missing JAR.

2) sshexec task issue
Problem: failed to create task or type sshexec
Cause: the class org.apache.tools.ant.taskdefs.optional.ssh.SSHExec was not found.
   This looks like one of Ant's optional components.
Action: Check that the appropriate optional JAR exists in
   -/usr/share/ant/lib
   -/Users/steve/.ant/lib
   -a directory added on the command line with the -lib argument

Do not panic, this is a common problem.
The commonest cause is a missing JAR.

I already download the jsch-0.1.44.jar from Jcraft,  and copied the jar file into folder /usr/share/ant/lib, and I also tried to put the jar file into different folder as the error suggested, but no help.
Finally I found this article:, it mentioned that in Mac, the ant-jsch.jar is missing!
and when I look at the /usr/share/ant/lib folder, yes there is no ant-jsch.jar file! So I downloaded the ant 1.8.2 from ant website, unpack it, and copy it into the /usr/share/ant/lib folder, it just works!

In summary, to make the ant scp and sshexec task work in Mac, you need to do 2 things:

Install ant-jsch.jar file: download the ant package from ant website, then copy the ant-jsch.jar into /usr/shar/ant/lib folder
Go to Jcraft website, download the latest version of jsch jar file, copy it into /usr/share/ant/lib

Handling incoming call interruption in BlackBerry audio player

2011-05-21T21:28:00.002-04:00

Last week I spent two days to fix an issue of blackberry audio player: the audio stream need to pause the player during incoming call interruption, and resume when call is finished.
At the beginning, I feel it is easy to fix, because I fixed this kind of issue many times before for blackberry platform.
So I just program based on my old experiences, but it did not working.

My Mistake

At first I use BlackBerry API PhoneListener to register the call interruption event. But it did not work, I found some issues:

- If you use PhoneListener, the application will prompt use to agree to use Phone API, which is annoying.

- When you delete the application, you have to restart the phone.

- The application can get the incoming call event via callIncoming() method, but some time the application get exception;

- The application get the call end event via callDisconnected() method, but when you call MMAPI player play() method, it always throws MediaException.

It seems the application the the event earlier than the voice call app release the resource.

- I found the Phone already stop the media player before it send the incoming call event the the application, so you have to use a workaround to deal with this issue.

I tried the different workarounds, but none of them working, it seems the PhoneListener solution is not reliable, and even not working. I don't remember why it works before.

maybe because of current phone is 5.0 while before are 4.3 or 4.6? Actually I don't know.

New solution

After I googled and I found a really simple solution: handle the DEVICE_AVAILABLE and DEVICE_UNAVAILABLE event in the playerListener. Based on this article:

The DEVICE_UNAVAILABLE event occurs when there is an incoming call. The DEVICE_AVAILABLE event occurs when the call is ended.

So all you need to do is pause the player in DEVICE_UNAVAILABLE, and resume when DEVICE_AVAILABLE event comes.

Code snippet like this:

playerUpdate(Player player, String event, Object eventData) 
{
    ...

    if(event == PlayerListener.DEVICE_UNAVAILABLE)
    {
        if(player.getState() == Player.STARTED)
        {
              player.stop();
              bPausedByIncomingCall = true;
        } 
    }
    else if(event == PlayerListener.DEVICE_AVAILABLE)
    { 
        if(bPausedByIncomingCall)
        {
            player.start();
            bPausedByIncomingCall = false;
        }

    }

   ...
}

The solution is very easy and it really works. I tested it when the DEVICE_UNAVAILABLE event comes, the player is still playing, which means the event is sent before the incoming call interruption.

Retrospection

I checked the MMAPI API and Blackberry API document, I found this event is not new (Blackberry support this since 4.0!), but I did not notice this until now. Why I did not notice these two events before? I did not find them in the books, nor I did not heard from them from previous colleagues. This is my another blind spot and it exposed my ignorance of MMAPI. This issue tells me: sometime your old experience is unreliable; you still need to empty your mind, like Andy Hunt said: always have beginner's mind!

Functional programming in OO languages

2011-05-07T23:02:00.003-04:00

Yesterday I watch the presentation: Functional Programming: A Pragmatic Introduction in Infoq.com. Jim Duey explains functional programming through Java code samples. He covers 3 parts:

high order functions
laziness
composing

I am more interested in the first part - high order functions. Jim shows 3 important functions:
reduce, filter and map.
Comparing with the java code, Jim describes how these 3 methods can help us reduce the duplicated boilerplate java code: like loop, iteration. He even emphasizes that these 3 methods - reduce, filter and map can reduce 90% of the loop code!
Copied from his slide, the benefits of the high order functions are:

Allows common code structures to reused
Abstracts away details
Isolates the essential parts of the code
Allows easier unit testing
Lets you think at a higher level
Allows easy parallelism

If you don't want to watch the video, then I found this article is really great to explain the map, filter and reduce method, also gives Java implementation.
As a java developer, the implementation code is not new to me, mostly the template method pattern is used. Actually I already saw the similar benefit in Spring framework: it is so amazing the JdbcTemplate can removed lots of the JDBC boilerplate code! But I did not aware that it shares the same idea with the function programming.
Since I learned that high order functions are so important, so I decide to investigate more and see how to implement them in other OO languages beside Java.

Javascript
Joel has a nice article about function programming using javascript: Can Your Programming Language Do This?

Groovy
I checked the Groovy API and recalled what I learned before, so I found the equivalent methods for reduce, filter and map in Collection API are:
   reduce => inject()
   filter => findAll()
   map => collect()

Groovy functional programming link

Ruby
Groovy is very similar with Ruby, especially when I learned Groovy, I suspect if Groovy copies some concept from Ruby, since the collection API is so similar.
   reduce => reduce(), inject()
   map => map(), collect()
   filter => select(), find_all()

This article is really great to explain them more in Ruby

Other resources
Why functional programming matters

Conclusion
- To use functional programming, it would be better to use the language that support functional programming in first class. I have to say Java implementation is too ugly, Groovy and Ruby are much better.
- Changing mindset to functional programming is hard but really important. Functional programming is another way of programming to elevate your abstraction layer.

Next Action
To pursue the way of Functional Programming, you need to pick up a new Functional Programming language ASAP, since I learned LISP through SICP, so Clojure would be my next choice.

Rediscover the power of Ant

2011-04-30T21:11:00.004-04:00

My last post talks about using groovy template to generate a text file, but I did not realize it can be done in Ant until reading the book- "Ant in action".
Recently when I refactored the ant build script of our comany's build system, I realize I need to learn ant systematically to improve my skills in Ant, so I choose this famous book.
When I learned the "filterset" datatype, I realize I can use that to generate the text file from a template, for example the last example can be written in Ant:

<copy file="test2.template" toFile="test2.txt" overwrite="true">
   <filterset begintoken="%" endtoken="%">
     <filter token="firstname" value="Steve"/>
     <filter token="lastname" value="Zhang"/>
     <filter token="city" value="Toronto"/>
   </filterset>
 </copy>

It is even simpler than Groovy in this case, right?
This example exposes my ignorance level of Ant, I realize that there a still lots of Gems in Ant. As Douglas Crockford said most language contains good parts and bad parts, we need to use only good parts and avoid bad parts, I think what he said also can apply to Ant. So don't easily discard a tool or a language, right now in my case, I love Groovy, I love Gant, they look so cool, but I am not 100% precent for sure Groovy/Gant can fully replace our build script. I have to rely on Ant now.
I start understand the reason why our build script is a kind of ugly: probably we did not fully use the good parts of ant and too much bad parts instead.
So my next step is start reading Ant in Action!

Invoke Groovy script from Ant or Gant

2011-04-22T23:06:00.002-04:00

Recently I tried to use groovy script to improve our build system. I found Groovy TemplateEngine is so powerful and very easy to use, quite similar to erb in Ruby. I want to use it to generate a text file. The following example shows how to generate a text file based on a template.

file test.template

Dear "$firstname $lastname",
So nice to meet you in <% out << (city == "New York" ? "\"The Big Apple\"" : city) %>.

file test.groovy

import groovy.text.SimpleTemplateEngine
import groovy.text.GStringTemplateEngine

def f = new File(this.args[0])
def binding = ["firstname":args[1], "lastname":args[2], "city":args[3]]
engine = new GStringTemplateEngine()
template = engine.createTemplate(f).make(binding)
println template.toString()

If in a terminal if I type: groovy test.groovy Steve Zhang Toronto, and I will get output

Dear "Steve Zhang",
So nice to meet you in Toronto.

Invoke Groovy from Ant
I followed this article from Groovy site
Following is the ant script:
build.xml

<project>
 <taskdef name="groovy"
          classname="org.codehaus.groovy.ant.Groovy"
          classpath="lib/groovy-all-1.7.10.jar"/>
 
 <target name="helloGroovy">
    <groovy>
     println "hello world"
    </groovy>
 </target>
   
    <target name="testTemplate">
   <groovy src="test.groovy">
    <arg line="test.template"/>
    <arg line="Steve" />
    <arg line="Zhang"/>
    <arg line="Toronto"/>
    </groovy>
 </target> 
</project>

It pretty straight forward, first declare <taskdef>, you need to put groovy-all-version.jar in the class path, but it took me a while to figure out where the groovy-all-xxx.jar is, finally I got it: when you download the binary zip file, unzip it, find embeddable folder, inside this folder you will find the groovy-all-1.7.x.jar!
Then you can put any kind of groovy code inside <groovy>.

Invoke Groovy Script from Gant
Gant is a great tool to replace Ant, I can definitely convert the above build.xml to build.gant, but I realize it is a kind of weird, since Gant is already based on Groovy, it already depends on Groovy jars, why we still need to declare the dependency in groovy-all-xxx.jar? There must be a better solution. Through my exploration, I found two ways to call Groovy script from Gant:

A) Use GroovyShell

target( runGroovy:"run a groovy script") {
 def shell = new GroovyShell()
    shell.run(new File("test.groovy"),
               ["test.template","Steve","Zhang","Toronto"] as String[])
    
}

The above gant script also tells you how to invoke a groovy script from another Groovy script file. GroovyShell can do this for us.

B) Embed the groovy source code directly in the Gant script file
During the exploration, I have some feeling that Gant script might can be thought of Groovy code, what if we add the groovy source code directly into the Gant script, so I did this:

import groovy.text.GStringTemplateEngine

target( hello: "hello") {
  println "hello"
}

target( runGroovy:"run a groovy script") {
 def shell = new GroovyShell()
    shell.run(new File("test.groovy"),
               ["test.template","Steve","Zhang","Toronto"] as String[])
    
}

target(runTemplate :"run a groovy template") {
 def f = new File('test.template')
 def binding = ["firstname":"Steve", "lastname":"Zhang", "city":"Toronto"]
 engine = new GStringTemplateEngine()
 template = engine.createTemplate(f).make(binding)
    println template.toString()
}

The target runTemplate will have the same result output, when you look at the import statement at the top, and the groovy code inside the target, I realize that Gant script is actually Groovy code!, the Gant target can be regarded as Groovy method. This conclusion is so important, because it means you can easily integrate the Groovy(or Java) code in the build script, I am sure it will definitely improve developer's productivity.

Foscam FI8918W wireless IP camera review

2011-04-17T18:06:00.001-04:00

I am looking for a video camera as baby monitor for a while. Currently all the baby video cameras in the market are too expensive, over $200 - $300, even that the features are quite limited. So I start looking for the replacement: the IP camera, and finally in Amazon I found FOSCAM FI8918W, it has almost all the features I need: pan/tilt remote control, wireless-IP, info-red, built-in micro phone and speaker. The most feature attract me is the wireless-IP, you can watch your camera anywhere over the internet through your browser or iPhone!

Yesterday I finally got my Foscam wireless IP camera from amazon.ca. I can't wait to unpack it start setting up. I just followed the manual step by step, it did not take me any longer to watch the video over the IE browser; but the hard part is making it work from the internet:
- At first I choose UPnP, but it did not work, this took me a while;
- Then I choose forward port as the manual suggested, configure the camera address and port number in my router, tried several times, finally get it working;
- Last I went to dyndns.org to create an free domain, then configure the parameters in the camera. finally I got it working.

There is another issue: as the manual mentioned, the camera only support IE to get the audio working, but there is no audio on my windows 7, after I followed this article, finally I got it working. Since the camera relies on ActiveX to make audio working, so currently audio does not support on other browsers: Firefox, Chrome, Safari.

Overall the camera does exactly what it mentioned. I like this camera:
Pros:
- info-red feature is awesome, I can see the picture clearly at night when I turned on the info-red;
- remote control is very handy, you can control the camera in 8 direction via browser or iPhone application.
- I just found the motion detection feature, when it detects the motion, the computer will beep and automatically record the video.

Cons:
- No audio output on non IE browser or iPhone application, I heard it due to the video output is Microsoft ASF format.
- No Zoom support
- Only support Wireless b/g, not support wireless-N network
- Picture quality is OK, you can see the pause, quite similar to the web cam

Comparing its feature and the price $100 in amazon.ca, less than the half of the regular video monitor, I think it is a good choice for a baby monitor.

The iPhone application
I downloaded the foscam surveillance pro application in my iPhone, iPod touch and iPad, it costs me $5, the app is really good, except no audio output. And at first I found the left/right control is reversed, after searching the support forum, the solution is choose FI8918W-reversed, then every thing is perfect.

Other resources
watch video with audio (works in vlc player):
http://[your ip]/videostream.asf?user=[your_user]d&pwd=[your_pass]

watch video without audio(works in the browser):
http://[your ip]/videostream.cgi?user=[your_user]&pwd=[your_pass]

watch video with camera in vertical stroll mode:
http://[your ip]/decoder_control.cgi?user=[your_user]&pwd=[your_pass]&command=25

Foscam surveillance application support forum:
Faq
forum

foscam in gadget victims

Deming's philosophy - the root of Lean

2011-04-03T22:05:00.004-04:00

In the Agile community, almost every one talking about Toyota's Lean technology. When we talking about lean, we can not ignore Deming, since Japanese learned a lot from Deming and goes further and have today's Lean.
Yesterday I spent some time watch Deming's videos on youtube:
W. Edwards Deming - Part 1, Part 2, Part 3
Dr. Deming - The 5 Deadly Diseases 1984

From these videos, you will easily find some of the lean principles do get from Deming, like: continuous improvement, eliminate wast, build quality in, respect people, optimize whole, focus on long-term goal, etc.I realize that the core of his philosophy is: respecting people, people first.

Here I copied his 14 points and 5 deadly diseases about management.
Deming's 14 points:

Create constancy of purpose.
Adopt the new philosophy.
Cease dependence on mass inspection.Eliminate the need for massive inspection by building quality into the product in the first place.
Minimize total cost when dealing with suppliers.
Improve continuously products, services, and processes.
Institute training on the job.
Adopt and institute leadership.
Drive out fear.
Break down barriers between departments.
Eliminate management by slogan and exhortation.
Eliminate numerical quotas and goals.
Eliminate barriers to joy in work.
Institute a program of education and self-improvement for everyone.
Put everyone to work on the transformation of the organization.

Deming's Five Deadly Diseases

1) Lack of Constancy of Purpose
- No planning for the future
- Lack of short term definition and goals
2) Emphasis on Short Term Profits
- Worship of the quarterly dividends
- Sacrificing long term growth of the organization
3) Annual Rating of Performance
- Arbitrary and unjust system
- Demoralizing to employees
- Nourishes short term performance
- Annihilates team work, encourages fear
4) Mobility of Management
- No roots in the company
- No Knowledge of the company
- No understanding of problems of company
5) Use of Visible Figures Only
- No use of figures that are unknown or unknowable, i.e., customer or employee satisfaction
- Encouraged by business schools

From the above, you will find these problems are quite common today, we need a long time to go to adopted his philosophy, this is why I am pessimistic about lean, because I feel the largest problem for lean is coming from managers, not from software developers.

Why learning philosophy is so important?
Some one might ask: we can just follow the practices or process, why we need to learn the philosophy?
I think if technology is good and worth doing, it must have some good principles or philosophy behind the concrete technologies. Once you understand the behind philosophy, you can master the technology, you will not be confused by the different variation or context. And also the detailed technologies or practices are context based, they are easily changed or deprecated with the time and context change; but principle/philosophy is different, they are abstract and can transcend the detailed context, the philosophy can be so powerful that they can apply to different context without change. That is why the art of war and the book of five ring is still useful today, because their principles are general and timeless. This is the meaning of Tao, that is why I am so obsessed by the principle and philosophy. The philosophy is hard to mastered, I remembered the author of The Toyota Way once mentioned, many companies adopted the Toyota production System, but Toyota find few companies really understand the behind philosophy. That is my suggestion, when you learn lean, you still need to understand the behind philosophy. If you just do the lean practices without knowing the philosophy, I am sure you will not do Lean correctly; you can do lean correctly only if you fully understand the lean philosophy.
Here I would like Quote What Deming said:(from his wiki)

"Once the individual understands the system of profound knowledge, he will apply its principles in every kind of relationship with other people. He will have a basis for judgment of his own decisions and for transformation of the organizations that he belongs to. The individual, once transformed, will:
Set an example;
Be a good listener, but will not compromise;
Continually teach other people; and
Help people to pull away from their current practices and beliefs and move into the new philosophy without a feeling of guilt about the past."

Blackberry trick: retrieving the MNC correctly

2011-03-11T15:02:00.003-05:00

Recently I need to retrieve the MNC value from the blackberry device, but though the real network testing, I realize that the value might not be correct for some cases.

The 3 different solutions of retrieving the MNC

via GPRSInfo

int mnc = GPRSInfo.getHomeMNC();

via RadioInfo

int currentIndex = RadioInfo.getCurrentNetworkIndex();  
      int mnc = RadioInfo.getMNC(currentIndex);

via GPRSCellInfo

GPRSInfo.GPRSCellInfo cellInfo = GPRSInfo.getCellInfo();  
       int mnc = cellInfo.getMNC();

Note:in blackberry the returned MNC value is decimal, which is different with the MNC standard, which is hexadecimal.

I found that solution 1-2 are always consistent, they are always same if you are not in the roaming network. But in solution 3, in some cases you will get the wrong value. For example I tested the following UK carriers, the retrieved mnc values are all wrong:

Carrier Name	MNC(Hex)	retrieved(dec)	retrieved(hex)
3 UK	20	527	20F
Orange UK	33	831	33F
Vodafone UK	15	351	15F
T-Mobile	30	783	30F
O2	10	271	10F

From the above table, we can tell that all the mnc values are appended letter F at the end, if we removed the F, then the value is correct. This is the first time I have ever met. Actually I found this article mentioned this issue a little bit:

...therefore the corresponding hexadecimal value returned by getMCC()or getMNC()may contain the letter F as padding. For example, for T-Mobile®, the MCCs can be identified as either 260 or 26F.

If you want to use cellInfo ( ex. MCC, MNC, LAC, cellID) to get the current location via openCellID for example, then you need to be careful for this issue.

Conclusion
So the solution to get the correct MNC value from blackberry API should be: first check the last 4 bits value, if it is F (or more than 9), then right shift 4 bits, that is it!

Learning SICP: resources and tool

2011-02-16T01:51:00.001-05:00

I started learning SICP since the beginning this year. The reason why I learn SICP is because I was inspired by Michael Feathers and Uncle Bob. In 2009, I watched the Michael Feathers presentation "Self-education and the craftsman" for SCNA 2009, he mentioned the SICP book, I did look at it at first, but LISP's lots of parenthesis scared me. Then last year when I read Uncle Bob's blog post "Why Clojure?", he recommend this book, and I also listened his podcast and watched his video about SICP, finally make to decide to start learning it.

Here I would like to share What I found the useful resources and tools to learn SICP.

SICP text book
Official Web site: http://mitpress.mit.edu/sicp/
You can also get PDF file from here.

SICP Video
MIT open course ware

IDE
I tried several LISP IDE, but finally I found DrScheme is the best one. DrScheme is included in the software called Racket, it is formally called PLT scheme. You can download Racket software from this site. It works on my Mac Pro, I haven't tried it on the Windows environment.

To run the program and exercises on the book, you need to download the package for SICP,
SICP support in PLT scheme. Just follows the installation instruction step by step. The latest version is SICP (PLaneT 1.13)

To use the SICP language in DrScheme, first choose Language|Choose Lanuage..., in Teaching language, choose SICP(PLneT 1.13),
and another most important thing: click "Show Details" Button, in "Output Syntax" group, under "Output Style", choose "write" radio button, then select OK.
I found if you use default selection, which "print" is selected, the output for list is quite annoying.

SICP exercise solutions
I found following two sites are pretty good:
http://community.schemewiki.org/?SICP-Solutions
http://wqzhang.wordpress.com/sicp-solutions/

Uncle bob's blog post about SICP
why clojure?
http://thecleancoder.blogspot.com/2010/08/why-clojure.html

STUB1: SICP & Assignment
http://cleancoder.posterous.com/stub-1-sicp-assignment

Robert Matin interview in The Pragmatic BookShelf
http://pragprog.com/podcasts/show/32

michael feathers
SCNA 2009 presentation:Self-education and the craftsman
http://scna.softwarecraftsmanship.org/media/michaelfeathers

About DrScheme

Learning to Program with DrScheme

I am interested in the chaos graphs in this article.

The Facebook login & logout flow for Android platform

2011-01-17T19:54:00.002-05:00

Recently I am working on a mobile application project using the FaceBook API. I never touched Facebook API before, for me the first thing is to understand how the Facebook login/logout works on the mobile apps, so I spent some time on figure out how does it work on the Android platform.

I just summarized the login and logout flow based on the Facebook android SDK source code and its sample app log output.

First you need to register your application and get your application ID from the facebook website.

Login Flow

Request format:

https://m.facebook.com/dialog/oauth?client_id=xxxx&redirect_uri=fbconnect://success&display=touch&type=user_agent

example:

https://m.facebook.com/dialog/oauth?client_id=175729095772478&redirect_uri=fbconnect://success&display=touch&type=user_agent

Here,
client_id is your application id.
display=touch means the device smart phone (android or iphone)

response:

fbconnect://success/#access_token=xxxxx&expires_in=xxxxx

example:

fbconnect://success/#access_token=175729095772478%7C2.vQIILuM8Bu_9nLKrl4yIew__.3600.1294938000-100000460660639%7Ch9OreeWmU4iUMB47UKwl20H64Gg&expires_in=4593

You need to launch a browser inside your app to send the login the request, browser will display a login dialog let input the facebook account and password, then if login successful, the browser will return the redirected URL(fbconnect://success) with the access_token and expired value.

To retrieve these values, you need to intercept the redirected URL: check if it starts with fbconnect://success, then parse the response, get the access_token and expire value.

Logout Flow
format:

https://api.facebook.com/restserver.php?access_token=[your access_token]
                                       &method=auth.expireSession&format=json

example:

https://api.facebook.com/restserver.php?access_token=175729095772478%7C2.mJlReipn5lu6jEJrKU2D6w__.3600.1295024400-100000460660639%7CE7f8hi-RkJtxOoJmGTyaWFgXHfI&method=auth.expireSession&format=json

If logout successfully, the response will return text "true",
otherwise it will return a JSON text contains the detailed error message.

The power of Groovy - notes of "Transforming to Groovy"

2010-12-28T21:36:00.002-05:00

Yesterday I watched a video presentation from infoQ:"Transforming to Groovy", presented by Venkat Subramaniam, I read his book: Programming Groovy. The presentation is really great, the author used lots of code examples to explain the power of groovy: first he provided the java code, then refactored it into groovy code, so you will find out how concise the Groovy is compare to Java. The whole presentation is 1.5 hour, but I did not feel it is quite long. I remembered the author describe the Groovy using the term "less ceremony", Which reminds me the how to describe the power of Groovy: "Accomplish more by doing less". And when he refactored java code to Groovy code, he described it as "reducing the noise". I like the word "noise" to describe Java code, since Groovy is quite expressive, concise, and its dynamic nature allow developer to focus more on their core business problems ( essential problem), and spend less time to deal with the language issues (accidental problem), which is noise compare to the business problem.

What I learned
AST transformation
Before I did not aware the feature of AST transformation in Groovy, probably this is the new feature, because I did not find this topic in the book.
The author covered @Immutable, @Newify, and @Delegate

@Immutable

@Immutable class Person {
   String firstName;
   String lastName;
   int age;
}

@Immutable annotation applies to class level only, it does not support on the field level.

@Newify

@Newify def createPerson {
    Person.new("John", "Smith", 28)
}

@Newify(Person)
def createPerson {
   Person("John", "Smith", 28)
}

@Delegate

class TirelessWorker {
   public void work() {}
   public void report() {}
}

class Manager {
   @Delegate employee = new TirelessWorker();
   public void schedule() {}
}

Here the Manager class can use all the methods of TirelessWorker class, if you add/modify method of class Tirelessworker, you don't have to modify the Manager class, it follows the OCP principle.

Multimethods
Multimethods is one of the major difference between Groovy and Java. Multimethod means the method call depends on the target and the parameter type, which reminds me the multiple dispatch, Java does not support mutlimethod, it only support single dispatch, I remember when I learn the Visitor pattern, I learned the multiple dispatch term, I am sure the Visitor pattern will be much easier to understand if the language support multiple dispatch, cool.

ArrayList
use spread-dot operator to access ArrayList
*. is called spread-dot operator, for example:

list*.member  =  list.collect { item -> item?.member }

It means given a list, iterate each item's specific method, form the return value as a collection.
get each item's first name: list*.firstName

Groovy follows the Kanban principle
I think Groovy is a good example of evolutionary and emergent design, since you can begin with purely java code, then migrate with Groovy, and even more you can mix with java and Groovy code in your project. Let's look at the Kanban's 2 foundational principles: Start with what you do now; Agree to pursue incremental, evolutionary change. Pretty similar.

Based on this, I think Groovy is an important language that worth every Java developer to looking at.

Try Pomodoro

2010-12-15T23:06:00.001-05:00

Today I finished the book of "Pomodoro Technique Illustrated ". Actually I heard of the Pomodoro in last year, but I did not think it seriously, until recently Dan North recommend it in his last two talks, so I decided to look at it. Through over a week's learning and practicing, I like it, it is simple, straight forward, and powerful.

The Pomodoro technique is pretty simple:
- Choose one single activity from your TO DO List;
- Set clock to 25 minutes focused on a single activity for 25 minutes, during this time-boxed 25 minutes, you only have one activity, and one goal: finish it.
- After 25 minutes, take 5 minutes break;
- Choose the next most important activity, start again;

The Pomodoro is timeboxed, focus on single activity and single goal. It helps you prioritize your task, help you keep focused and remain high productivity.
It emphasize on execution, getting things done, and NOW. I found it is similar as Getting Things Done, but comparing with GTD, I like Pomodoro because it is quite hands on and easy to implement, actually I have not finished the GTD book yet. It focus on only one thing at a time, and its emphasize on NOW remind me the ZEN philosophy.

From Agile perspective, Pomodoro like a personal agile methodology, or specifically it very similar as SCRUM, I would rather say it as personal SCRUM, because you can treat each Pomodoro as SCRUM sprint, and the inventory as SCRUM backlog. And quite similar, in SCRUM, you can not change your task during each sprint; while you can not change your task during each Pomodoro. Pomodoro also focus on planning,processing, tracking,retrospective, so it is really agile.

I only started Pomodoro for only a week, I like it because it provide you the awareness the time,prioritizing and single minded; before it is quite easy to yak shaving, or get distracted. The Pomodoro provides a mechanism to let you get focused and remain higher rhythm, then improve your productivity.

Some notes from the book:

- time-boxed, single activity,single goal
- 5 stages: Planing, Tracking, Processing, Visualizing - retrospective
- Deming-Shewart cycle: plan, do, check, act
- Tools: TODO Today Sheet, Activity Inventory,Record Sheet
- Pomodoro will first Prioritize, then focus on the most important activity (like scrum)
- Pomodoro need to see the big picture before you decide what to do
- it is goal oriented
- it has immediate feedback

Interruptions
- principle: neither switch activities nor stop an activity in the middle of a Pomodoro.
- avoid LIFO
- avoid BPUF

Deal with internal interruptions
- Strategy: Accept, Record and continue
- Write it into Unplanned & Urgent
- never switch activities in the middle of Pomodoro: "Once a Pomodoro begins, it has to ring."

Deal with external interruptions
- Strategy: inform, negotiate,schedule,call back

Resources:
I use my iphone as a timer, but I want to use it as vibrate only, but I could not find a vibrate only ringtone, try this link, it tells you how to set the silence only ringtone

Some thoughts about Conway's law

2010-12-12T23:02:00.001-05:00

One of my best learned from QCon San Francisco is Conway's Law, I heard it at least 3 times from different talks, especially from Michael Feather's talk. It is so striking, I never aware this before. Here is the definition of the Conway's law:

organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations

In short, you can understand Conway's law like this:

The software design structure is constrained or shaped by the company's physical structure;
Your code structure mirrors your organization's structure.

This concept is quite interesting and so striking, it forced me into further thinking:

Constraints
Based on Conway's law, we can conclude:

Good organization's structure generate good software architecture, bad organization's structure generates bad software architecture;
The messy code base reflect its organization's messy management, the clean code base reflects the organization's clear structure;
Management structure is quite similar as the code structure, for example, some manager like to control everything and focused on details, he does not know how to delegate, just like the "GOD Object" anti-pattern in the software development.
The software manager's style, like/dislike and skills may also constraints the software product;
Developer's own skills, knowledge, communication skills and ignorance level, etc, may also constraint the software product. We need to be aware of these constraints.

Conway's law in the other way
Can we good developer influence/improve organization by applying the good practices/patterns/principles? For example, As a developer we knows how powerful the design patterns and SOLID principles to the OO design, and manager can apply those patterns to improve/refactoring the organization structure. Actually I have the idea long time ago: manage a company is like managing software, if you are a good developer you can be a good CEO or manager. We can use the software technologies to manage the company team. Although the context is quite different, they both solve the similar problems: How to manage dependency, how to make the team or software product more adaptable.
This seems has a higher requirement for the manager, since I seldom see a manager who has a strong developer background. I really experienced how difficult it is to communicate with a non-technical background manager, it is really frustrating, and so hard to explain the issue, some time you have to translate it into an real life example to make it easy to understand. Think about it, it is so hard to let the manager to understand an issue, how could he actively find out the issue that developer himself does not aware? This is definitely an example of the Conway's law.

Use your pain point to measure your incompetence

2010-12-02T23:57:00.001-05:00

Corey Haines's presentation Software Craftsmanship, Beyond the Hype is interesting.
What I get most impressed is what he mentioned a simple way to measure your level of incompetence:
Picture your mind what's your ideal of programming, then picture what you do when deadline comes, the difference is how much you suck is.
In short, the difference between your ideal and your reality is the measuring of your suck.

I totally agree with what he said. Because right now it really applies my situation: I am thinking a long time to improve our build system, I took the training of "Continuous Delivery" in Qcon, I know what is the ideal, but in reality it is really painful, and I don't know where to start, use what tools, it really measures my skills and knowledge for the build process is really low. I have to acknowledge this.

Here I would like to share my experience about measuring my incompetence:
Yesterday I tried to follow this article to improve our build script, we have 3 different environments: local test, stage box and product box. The first thing I need to do is put the parameters in the build script into 3 different property file, like: local.properties, stage.properties, and prod.properties. then in ant build.xml will read different property file based on the different server environment variables; what we need to do is in different build script, passing its own server environment variable, for example:
inside local.sh, call ant -Dbuild.env=local
inside stage.sh, call ant -Dbuild.env=stage

This's change is not hard, it did not take me a long time to do that. Then I noticed that in 3 different build script are quite duplicated: the major difference is the build.env value, others are pretty similar, can we refactor the shell script like refactoring the java code? Or can we apply the OO concept to the shell script? Till now I realized that I actually I did not do too much shell script programming: most of the time I just read and copy & past. Then when I googled, I just know that unix shell also support function, you can pass arguments, and even it can return values, this is cool! I did some test, it works!
But I need another feature: inside one script, call the methods in another script file. Basically I want to put the common methods into another file, and so that 3 different script files call the methods in this common file; just like parent class and child class in java, which can make the script file much simple and reusable. After googling, I found the solution again, here I would like to post the sample code and show how it works.

In build-common.sh, define 3 functions:
process_arguments()
set_ant_env()
call_ant()

buildlocal.sh

  . ./build-common.sh # this is how to import another file
  set_ant_env
  process_arguments $*
  call_ant local

buildstage.sh
   . ./build-common.sh
  set_ant_env
  process_arguments $*
  call_ant stage

buildprod.sh
  . ./build-common.sh
  set_ant_env
  process_arguments $*
  call_ant local

Actually I still can improve the build script by passing an extra argument, but I am afraid we get used to use different shell to manage different environment, so I just leave it now.

The most important thing is how to import another shell into your shell script:

1.
. ./buld-common.sh

2. then you can the function, like calling the local function:
   set_ant_env

From this small improvement, I learned some thing about the unix shell, not bad. And more important what I learned is whenever you feel the pain, especially it will be painful over and over, do not just let it control you, you should to face it and think about how to solve it. Because it is good chance to measure you incompetence and a good chance for you to improve your skills.

Hacking the Wireless Toolkit for Mac

2010-11-30T21:55:00.002-05:00

Today I spend a whole day trying to fix an J2ME build issue: the build script works on windows, but failed on Mac. My working environment is:
WTK
   - Windows PC: WTK 2.5.2
   - Mac: WTK 2.5.2 Linux version, copied preverify from JavaME SDK 3.0 Mac
Build tool is Antenna
   - Windows: antenna 1.1.0
   - Mac: antenna 1.2.1 ( the earlier versions have preverify issue on Mac )

Actually we usually build J2ME application on windows or Linux box, this is the first time to build on Mac. I checked the build log, the log shows the preverify error.
Since I used Sun's latest version of preverify file for Mac, I never suspect it might have issue. I thought issue might due to Mac is picky with the Antenna build script, so I spent lots of time trying to tweaking the build script, but no luck. So I started to thinking the problem might be the preverify file I used for Mac.

Finally I have to google and I was so lucky that I found Sam's blog post: J2ME Development on OS X, revisited. This is a fantastic article, the most important for me is he tells me how to get the Mac version of preverify: Download the phoneME project source code, and the preverify for mac is under phoneme_feature/cldc/build/share/bin/darwin_powerpc/, as the author suggested, I downloaded the PhoneME feature M2 source code, and I did found the preverify, and after I copied the preverify in my WTK /bin folder of my mac machine, then when I run the build script, it works, awesome. I really admire this guy, how could he find out this tool!

Now let me describe how to install WTK 2.5.2 on Mac in step by step, please note this only works if you still use WTK 2.5.x, and it is only useful for building the application, the emulator and other tools are not working, you can use JavaME SDK 3.0 for Mac, but based on my today's experience, its preverify still has issues:

Download and Install the WTK 2.5.2

Goto here to download the Sun's WTK 2.5.2 for Linux, file is called sun_java_wireless_toolkit-2_5_2-linux.bin
Execute the downloaded file (sh sun_java_wireless_toolkit-2_5_2-linux.bin)
When asked for a path to JDK, enter “/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home/bin/”
Follow the instructions, probably best to install into /Applications/WTK2.5.2

Get the preverify binary

Download the PhoneME project feature source code, choose the M2 source bundle
Extract phoneME build, it creates a directory called “phoneme_feature”
Copy /phoneme_feature/cldc/build/share/bin/darwin_powerpc/preverify and overwrite the preverify binary in your WTK bin folder.
You might need to change the permission to w+x of preverify.

You might need to install Rosetta before running preverify
Today one of my colleague found this issue on this macbook, when I give him a copy of preverify binary file, when he run it on his machine, it prompt need to install Rosetta to run it. it said the Rosetta comes with the MacOS installation CD, so if you have the similar issue, then install it first.

Other tools:
Antenna: please use the latest version for Mac (1.2.1).
Gant: the groovy based DSL for build script, it built-in support all the ant tasks, but you don't need to write XML any more, I just started using it, and I love it.

Review of BlackBerry Eclipse Plugin for Mac OS X

2010-11-17T21:52:00.000-05:00

Today I have a chance to look at BlackBerry's Eclipse Plugin for Mac OS X. Since it is only a preview version, I did not expect too much. After install and play a while, my excitement vs. disappointment is half and half.

The software contains the full Eclipse 3.5 and it's Eclipse Plugin, like the windows version of earlier this year. It requires to install the BlackBerry Desktop manager for Mac.

What I don't like:

Does not support simulator yet. From the IDE menu, I could not find the where I can launch the BlackBerry simulator, then when I read the manual document, I realize this feature is not supported, it only support running/debug on the actual device via USB cable.
On-device debugging is kind of working. It takes a long time to connect the device, it can stop at the break point, but the variable value is wrong, always 0 or null.
Support BlackBerry OS 6 only.
Signing is more restricted, in windows version you can hack the signing tool by putting the file sigtool.csk and sigtool.db in the \bin folder of blackberry SDK component folder. It does not working on Mac version now.

Now I understand why it is called a preview version.

What I like most is the blackberry tool "javaloader" is now supported in Mac, I tested it works on Mac, you can install/remove an application using javaloader via USB cable. Really nice!

The javaloader file is under the folder like:

BBEclipse/plugins/net.rim.ejde.componentpack6.0.0_6.0.0.29/components/bin

If you are familiar with the Blackberry Windows version of Eclipse Plugin, you will find they have the same folder structure. Just remember you have to manually change the permission to make sure it is executable. You can also find the preverify tool in the same folder, and fledge tool in the simulator folder, which is supposed to launch the simulator, unfortunately it does not working right now.

Another interesting thing is I wonder how it build the application in the IDE. From the console window I got following output:

Compiling CLDC Application:

java -jar /Users/steve/Desktop/BBEclipse/plugins/net.rim.ejde.componentpack6.0.0_6.0.0.29/components/bin/rapc.jar -convertpng -quiet codename=deliverables/Standard/6.0.0/HelloWorldDemo deliverables/Standard/6.0.0/HelloWorldDemo.rapc -exepath=/Users/steve/Desktop/BBEclipse/plugins/net.rim.ejde.componentpack6.0.0_6.0.0.29/components/bin/ -sourceroot=/Users/steve/dev/workspace/HelloWorldDemo/src:/Users/steve/dev/workspace/HelloWorldDemo/res -import=/Users/steve/Desktop/BBEclipse/plugins/net.rim.ejde.componentpack6.0.0_6.0.0.29/components/lib/net_rim_api.jar /Users/steve/dev/workspace/HelloWorldDemo/bin

Compiling MIDlet application:

java -jar /Users/steve/Desktop/BBEclipse/plugins/net.rim.ejde.componentpack6.0.0_6.0.0.29/components/bin/rapc.jar -convertpng -quiet codename=deliverables/Standard/6.0.0/MIDletDemo -midlet deliverables/Standard/6.0.0/MIDletDemo.rapc -exepath=/Users/steve/Desktop/BBEclipse/plugins/net.rim.ejde.componentpack6.0.0_6.0.0.29/components/bin/ -sourceroot=/Users/steve/dev/workspace/MIDletDemo/src:/Users/steve/dev/workspace/MIDletDemo/res -import=/Users/steve/Desktop/BBEclipse/plugins/net.rim.ejde.componentpack6.0.0_6.0.0.29/components/lib/net_rim_api.jar /Users/steve/dev/workspace/MIDletDemo/bin

Now we understand that in Mac, it uses java to launch rap compiler, and it has some option I never seen before: like -convertpng, -exepath, etc. I wonder RIM might upgrade it new version of RAPC compiler.

More about "Drive"

2010-11-14T14:01:00.001-05:00

I just by accident read Martin Fowler's article: Can Not Measure Productivity. The author concluded that you can not measure developer's productivity, because the output is not measurable. Even if you try to measure the performance by using the wrong metrics, things even become worse. In the article, he recommend a book called Measuring and Managing Performance in Organizations, written by Robert Austin. but it seems it is not published any more, I just found a slide of the book., and a brief introduction of the book.

Quotes from the book introduction:

Because people often react with unanticipated sophistication when they are being measured,measurement-based management systems can become dysfunctional,interfering with achievement of intended results. Fortunately, as the author shows, measurement dysfunction follows a pattern that can be identified and avoided.

Here I got some quotes from the slide:

"A system of incentives is dysfunctional if the resulting effort allocation provides less value to the customer than when there is no supervision.” (Austin)

Measurement versus Delegation

• Full Supervision: incentives via measurement

– Relies on extrinsic motivators

• Money

• Continued employment

• “Forcing Contract”

• No Supervision: incentives via “delegation”

– Relies on intrinsic motivators

• Pride of workmanship, joy in being part of the team

• Identification with the company’s values and goals

• Desire to please the customer

Intrinsic versus Extrinsic Motivators

• Intrinsic motivators are much more powerful

• But appear to be overridden by extrinsic motivators

• Japanese versus American management styles

– Japanese rely heavily on delegation

– But it is cheaper for them

– It is both a strength and a weakness

• Delegation may not be cost effective either

• Sometimes neither style works

– “Large software development firm”

Summary

• You may think you have full supervision

• But you are almost certainly wrong

• The ship’s wheel is not directly connected to the rudder

• Austin versus Deming

– W. Edwards Deming, Out of the Crisis, MIT Press, 2000

– Former BTL lead Japanese quality movement in 1980s

– Methodology summarized in his “Fourteen Points”

So based on the text above, we can conclude that the book of "Measuring and Managing Performance in Organizations" share the same concept with the book of "Drive", use different perspective, it proves that people's performance and productivity is unmeasurable, the manager should not rely on external performance incentive, but should rely on people's intrinsic motive. It also proved that this idea share with Deming's 14 point. Interesting.

Another useful link:

http://www.slideshare.net/RobMyers64/metrics-in-an-agile-world

Manage Change by Managing Unchanged

2010-11-13T23:22:00.003-05:00

As a software developer, we have to deal with different changes:
- The changing requirements, which means change code, bug fix, revers engineering...
- The changing technologies, you have to deal with new buzz, new API, new languages...

How can we deal with these changing? because chasing change is really hard, because:

New stuff in software industry are too much, they are out of control, every year there are so many buzz, languages and SDKs waiting for you;
Some new stuffs are quite easy to obsolete, some time even when you try to start learning it, it is just gone.
Understand too many details in short time is really stressful
They are quite context specific, they will be useless in different area.

Since human's time and energy is limited, how can we manage these changes properly? Why not focus on unchanged stuff instead of changing stuff? What I mean the unchanged is something behind the detailed technologies. For example, like principles, rules, patterns, philosophy etc. They are more general which means last longer, they are also the high level abstract knowledge which easier to transfer another area, so based on these 2 characteristics, they are manageable.

Another interesting characteristic about unchanged is they are meta level of details. For example, measure an object's change of position, then we use velocity, which is can be thought of meta position: the change of position; if we want to measure the change of velocity, then we use accelerator. This example means the higher meta level, the less likely to change and easier to manage. Another example is in software developing, in object oriented developing, the object's meta level is class, the class's meta level is meta class, we manage the class to make the code more reusable, we manage the meta class to make the code more dynamic and more adaptable.

So how does the concept of "Manage change by manage unchanged" help in the software development?

We should focus more on classics, like algorithm, data structure, fundamental computer theory, OO design patterns, OO SOLID principles, etc. because they are fundamental and general, they are powerful, they are going to still valid for a long time;
When you learn a new technology, or a new SDK or even a new language, don't try to remember every details, which will overkill you, but you should focus more on the principles, rules or philosophy, since detailed technologies are easy to forget, but when you understand the principles behind the details, it is much easier to memorize, and much easier to transfer to another area;
It help you to identify which technology is worth learning by analyzing its unchanged properties
In software design, when we design an adaptive software system, first we need to identify the changed stuff and unchanged, then design the structure based on unchanged, then isolate and encapsulate the changed stuff; make sure the system depends on unchanged, not on changed, this is the famous Dependency Inversion Principle.

Mind map: Continuous Delivery Notes

2010-11-11T23:12:00.003-05:00

In QCon San Francisco, I attended the tutorial -"Continuos Delivery", it is a great training. I learned a lot, then I realized I have to record and organize what I learned - which is the part of the GTD principle. Also at the meeting I noticed that Dan North used a Mind Map for his presentation, which is awesome. The software he used is called FreeMind, so I just tried to use this tool to take my notes in the mind map format. The notes like this:

Since this is my first time to use it, it is pretty basic, but it is good enough for me right now. It is a good starting point. FreeMind provide lots of nice features, please check the pdf manual for details.

I realize that using Mind map to take notes and posted on the website is a great idea. I already use mind map to take notes, but just manually drawing, this is a great tool.

The next step I need to figure out how to embed the actual map on the website. This article tells you how to install and use the FreeMind Applet on your website, then it seems I need to have my own website, so right now it is time for me to build my own website.

Impressions of Qcon San Francisco 2010 conference

2010-11-06T22:15:00.002-04:00

My journey of 5-day Qconsf 2010 is just finished, now I am on the plane flying back to Toronto. This my first time to attend Qcon conference, the whole conference is so nice, I really enjoyed the whole conference. Now I am trying to write down what I thought about this conference. It will be just an overview, I will give more detailed post for each interesting talks in future.

2 Days tutorial: fantastic
1. Joseph Yoder and Rebecca Wirfs-Brock's Adaptive Object Model is really nice, actually I supposed to take Dan North's architect course, but I changed my mind in the last minute. I really missed Dan North's course, but AOM is really good. because it provide several patterns to make the application architecture more flexible, which is really important for using static language like Java. I think it is really useful to use this model in my current java application development.

2. Continuous Delivery
I am Martin Fowler's follower, finally I have a chance to listen to Martin Fowler training. My current build process is really painful, I think I have an idea to improve my current build system, including:

break down one big build script to several pieces,
create dependency
use code analysis
use build server like Hudson
use Junit test
set up build artifact
unify configuration of developing environment and deploy environment
decouple the dependency of developing build script from deploy build script.
use puppet and cucumber
simplify the ant build script: use gradle or gant

My goal: shorten the deploy build time, shorten the developing unit test build time, try to automate every thing, shorten feedback, improve productivity

Martin's Friday's Keynote is also fantastic, I bought his latest book: Domain Specific Language, my suggestion is just to buy any of his book, you will not be disappointed.

overview of 3-day talks
The 3 days talks are really good, every day I feel struggle to choosing which one I should choose. I choose Java track (wednesday), NoSQL(Thursday), Parallelism programming(Friday). I will say most talks are really good, some talks you will find which is not you expected. My most favourite is Michael Feather's talk:the most important part is about conway's law: your code structure mirror your organization structure, wow!, totally striking, before I have some kind of impression, but I did not aware this rule, fantastic! Michael's talk is so important because it gives your insight to deeper thinking. like zen medication, action of awaken and awareness. Thank you Michael!

My favorite speakers
I am so happy that I have chance to meet following famous speakers:

Martin Fowler
Dan North
Ralph Johnson
Dan Ingalls
Erik Meijer
Michael Feathers

Some speakers I am not familiar, they give really wonderful talking:
   Joe and rebecca
   Stuart Halloway

What I learned
NoSQL, Concurrency & parallel programming and functional programming are my blind spots. Thorough 3 ays conference, I just get some concepts, since they are all very good speakers, I know what I am going to learn next steps.
Especially when I attended two talks by Stuart Halloway about Clojure, I learned that clojure is such a elegant beautiful language, and the more important is closure defines/redefines the concepts about value, time, perception, which are totally different with traditional object oriented model. OO is not the only thing to observe the world. This is big mindset change. Stuart's second talk about time model in clojure is brilliant, pretty philosophy, it will take me longer time to fully understand them.

who I meet
Mark: I talked to him a lot, who is a ruby guy, I got lots of information about tools of ruby from him.
Jai: Java guy, I found we read quite similar computer books,
Wei: java guy, I learned some form him about NoSQL.
Daniel: .NET guy, I am so impressed by his productivity of blogging, I noticed he just he keep writing in a Martin Fowler's keynotes talking, I thought he might do something else,like checking email, but when after the talk, he told me he just finished his another blog post about the talk, oh my god! Compare to him, usually I take notes on my notebook, then later type them on the computer; it always take a longer time to write a blog post, I need to learn this skill. how to blog your thought easier and faster.

And I talked many other guys I forget their names, I found we almost read same books, shared same vision, like "learn a new language every year", it is a very good chance to learn, get feedback, you will aware your ignorance, your blind spot, you need to learn more !

It is really interesting that a group of people come from different country, with different back ground, share the same philosophy.

Next actions
  Am I going to come QCon next year? Maybe, it depends on my own budget.
  Get more involved in infoq.com? I talked to Floyd about submitting articles to infoq, which is a good idea, but you have to commit to submit at least one article each week, which is sound stressful to me right now, it is challenge you have to overcome. Do it!

Gant Script: print ant default properties

2010-10-16T23:25:00.001-04:00

If you want to know exactly what kind of ant properties in your environment, try the following gant target:

target(properties : "print ant default properties") {
     println "ant properties are:"
     ant.project.properties.each { key, value ->
         println "${key} = ${value}"
     }
}

Here are some of the out put:

ant properties are:
ant.version = Apache Ant version 1.8.1 compiled on April 30 2010
sun.os.patch.level = unknown
java.vm.vendor = Apple Inc.
environment.HOME = /Users/steve
environment.TERM_PROGRAM = Apple_Terminal
file.encoding = MacRoman
user.language = en
java.class.path = /opt/local/share/java/gant/lib/groovy-all-1.7.3.jar
file.separator = /
java.runtime.name = Java(TM) SE Runtime Environment
basedir = /Users/steve/dev/gant
java.vm.specification.version = 1.0
awt.nativeDoubleBuffering = true

...
...

Look how easy it work in the Groovy/Gant script, I am sure it is much harder to write in ant xml build script, and you have to use ant-contrib to implement the for loop! Byebye XML!

Build J2ME app using Gant

2010-10-16T23:11:00.004-04:00

These days I spent some time learning Groovy. Why I learn Groovy? Because I want to use Gant. Gant = Groovy + Ant. Gant is really powerful, since you can use Groovy script and Ant API to write your build script instead of XML. I struggle a long time in Ant, because I have to use Ant-Contrib plugin, and it is so hard to write an condition block and even for loop, it is much easier to do that in Gant script.

Now I am going to share the gant script of building J2ME app, this is just basic, but you can see the power of Gant.

Requirement:
1. You need to install Groovy and Gant, here is the link, I used 1.9.3 in Mac.
2. download the latest version of antenna, here is the download link, I used 1.2.1-beta.
3. install the JavaME SDK, I choose JavaME SDK 3, since it works on Mac.

The J2ME MIDlet class source code:

import javax.microedition.midlet.*;
import javax.microedition.lcdui.*;
 
public class HelloWorld extends MIDlet implements CommandListener {
    private final Command exitCommand;
    private final Form form;
 
    public HelloWorld() {
        exitCommand = new Command("Exit", Command.EXIT, 1);
        form = new Form("Hello World!");
        form.append("from Steve");
        form.addCommand(exitCommand);
        form.setCommandListener(this);
    }
 
    protected void startApp() {
        Display.getDisplay(this).setCurrent(form);
    }
 
    protected void pauseApp() {}
    protected void destroyApp(boolean force) {}
 
    public void commandAction(Command cmd, Displayable disp) {
        if (cmd == exitCommand) {
            destroyApp(false);
            notifyDestroyed();
        }
    }
}

The Ant script looks like:

<project name="antenna-test" default="jar" basedir=".">
 
 <taskdef resource="antenna.properties" classpath="./antenna-bin-1.2.1-beta.jar" />
 
 <property name="wtk.home" value="/Users/Steve/dev/wtk252" />
 <property name="wtk.midp.version" value="2.0" />
 <property name="wtk.cldc.version" value="1.1" />
 
 <property name="src.dir" location="src" />
 <property name="out.dir" location="out" />
 <property name="dist.dir" location="dist" />
 <property name="preverified.dir" location="preverified" />
  
 <!-- Regular targets -->
 <target name="clean" description="cleans the target directory">
  <delete dir="${out.dir}"/>
  <delete dir="${dist.dir}"/>
  <delete dir="${preverified.dir}"/>
 </target>
 
 <target name="compile" depends="init">
  <wtkbuild source="1.3" srcdir="${src.dir}" destdir="${out.dir}">
  </wtkbuild>
 </target>
 
 
 <target name="preverify" depends="compile">
  <wtkpreverify srcdir="${out.dir}" destdir="${preverified.dir}">
  </wtkpreverify>
 </target>
 
 <target name="jar" depends="compile,jad,preverify">
  <wtkpackage jarfile="${dist.dir}/helloworld.jar"
                      jadfile="${dist.dir}/helloworld.jad"
                      obfuscate="false"
                      preverify="false">

       <fileset dir="${out.dir}"/>
        </wtkpackage>
   
 </target>
 
 <target name="jad" depends="compile">
  <wtkjad  version="0.01" name="hello" vendor="foo" jadfile="${dist.dir}/helloworld.jad" jarfile="${dist.dir}/helloworld.jar">
    <midlet name="hello" class="HelloWorld">
    </midlet> 
  </wtkjad>
 </target> 
 <target name="init" description="initialization for properties and paths" depends="clean">
  <buildnumber/>
  <mkdir dir="${out.dir}"/>
  <mkdir dir="${dist.dir}"/>
  <mkdir dir="${preverified.dir}" />
 </target>
 
 
</project>

the build script build.gant:

ant.taskdef(resource:"antenna.properties", classpath:"./antenna-bin-1.2.1-beta.jar")

ant.property(name:"wtk.home", value:"/Applications/Java_ME_SDK_3.0.app/Contents/Resources")
ant.property(name:"wtk.midp.version", value:"2.0")
ant.property(name:"wtk.cldc.version", value:"1.1")

final srcDir = "src"
final outDir = 'out'
final distDir = 'dist'
final preverifiedDir = 'preverified'

//clean target
includeTargets << gant.targets.Clean
cleanPattern << '**/*~'
cleanDirectory << [ outDir, distDir, preverifiedDir]


target(name:"init", description:"initialization") {
 depends(clean)
 /*buildnumber() */
 mkdir(dir:outDir)
 mkdir(dir:distDir)
 mkdir(dir:preverifiedDir)
}

target(name:"compile", description:"compile") {
 depends(init)
 wtkbuild( source:"1.3", srcdir:srcDir, destdir:outDir)
}

target(name:"preverify", description:"preverify") {
 depends(compile)
 wtkpreverify(srcdir:outDir, destdir:preverifiedDir)
}

//target jad file

target(name:"jad", description:"generate jad file") {
    depends(compile)
 wtkjad( version:"0.01", name:'hello', vendor:"foo", jadfile:"${distDir}/helloworld.jad", jarfile:"${distDir}/helloworld.jar" ) {
  midlet(name:"hello", class:"HelloWorld")
 }
}

//target jar
target(name:"jar", description:"generating jar file") {
 depends(compile,jad)
 wtkpackage( jadfile:"${distDir}/helloworld.jad", jarfile:"${distDir}/helloworld.jar", obfuscate:"false", preverify:"false")  {
  fileset(dir:outDir)
 }
}

//target obfuscate

//target all
target(name:"all", description: "build the whole app") {
 depends(jar)
}

setDefaultTarget(all)

From the above script, you can see how easy to write gant build script.
This is only the basic implementation, I am going to post more Gant based script in future.

In future I am going to consider:
1. How to integrate with ANT script;
2. How import multiple Gant script file.

Use the BlackBerry Issue Tracker to find bugs

2010-09-26T22:57:00.002-04:00

I don't know when BlackBerry start opening their issue list for the developers, recently I just know this service. This is a great service, you can browse, search and submit the Blackberry issues.

Here I would like to share my experience how to use the Issue tracker:

First go to BlackBerry Developer Zone, register an account, use this link:
https://www.blackberry.com/bdsc/?lang=na#login

After login in, in the left part of the portal, click Resources tab, it will expand several sub links, then select "Developer Issue Tracker", then click link "Submit an Issue", finally your will enter the BlackBerry Issue Tracker portal.

For example, to identify an BlackBerry OS 5 SMS issue, I found a bug which reported the exact same issue, which is very helpful.
JAVAAPI-553:
Java APIs
invokeApplication() Ignores PayloadText for Populating SMS Text Message
Created: 19/Oct/09 10:44 PM Updated: 07/Jun/10 05:50 AM
DevTrack ID: 360769
Fix Version/s: None
App Build Number: 5.0.0.224
Device: Bold 9000
Component: --
Description: When calling Invoke.invokeApplication() and passing TextMessage arguments, the PayloadText is ignore when it should be used to populate the body of a new SMS message.

You will be surprised that there are some many open issues for different blackberry OS.
So I will suggest you to go to the Issue Tracker if you want to confirm a BlackBerry device issue.

The incompatible issues of BlackBerry OS 5

2010-09-26T22:18:00.001-04:00

Recently I realize that BlackBerry OS 5 has more and more incompatible issues with 4.x, usually a feature works on pearl, curve, bold (4.6), but does not working on bold 5.0, which is very disappointing.

Here I just make a list of incompatible issues that I found recently:

1. BACK key issue
In 4.x, the BACK key value mapped in MIDlet is 27, I tested on 8100/8300/8900, this key value is the same: 27. But I just found in Bold with OS 5.0, the BACK key is not mapped into MIDlet at all, the MIDlet can not get the BACK key event.

2. SMS send issue. When you launch the BlackBerry native Message application to send SMS using Invoke class, you can pre-populate with phone number and text message, it works in 4.x, but does not work on 5.0, it is a know issue of blackberry 5.0, the message body is empty. Please check this link for details.

3. Retrieving CellID issue. We know that using GPRSInfo we can get phone's Cell-ID, but in 5.0 you will get the wrong id if you use the same method, this is because for 3G network, you need to use different method, how complicated it is!

4. Icon issue, I found in OS 5.0, you can set the rollover icon the jad file, and blackberry will recognize it even if your application is MIDlet, but I found you need to use JDE 5.0 to build your app; if you use JDE 4.x, it does not work.

I feel disappointed about BlackBerry platform, since I feel 4.2 - 4.6 even 4.7, the OS are quite consistent with each version, but in OS 5.0, it seems this is a big difference with previous ones. If OS 5 look like this, how can we trust OS 6?

Be careful to use "Class.forName()" in J2ME

2010-09-05T23:50:00.003-04:00

A couple weeks ago, I made another mistake: I tried to use Class.forName() method to solve the multiple platform issue, but unfortunately it did not work on the actual devices due to the issue caused by the obfuscation. This is my another blind spot, because of my ignorance of J2ME obfuscation.

Scenario
I need to provide an API to support multiple mobile platform, for example: Nokia, Blackberry, and simulator, etc.
What I implement is pretty easy:

define an interface: Service
Implement the Service interface for each mobile platform, for example: NokiaService, BlackBerryService, SimualtorService
At the build time, inject different Service class file in the package.
At the run time, the application use the Class.forName() to create the Service objects.

The snippet code looks like this:

public Service createService(int device)
 {
    Service service = null;

    if( device == NOKIA)
    {
        service = Class.forName("NokiaService").newInstance();
    }
    else if( device == BLACKBERRY)
    {
        service = Class.forName("BlackBerryService").newInstance();
    }
    else if(device == SIMULATOR)
    {
        service = Class.forName("SimulatorService").newInstance();
    }
    else 
        service = Class.forName("GenericService").newInstance();

     return service;
 }

Here I use a factory pattern to support multiple platform, no need to use preprocess, sound pretty simple, right?
But I found the above solution only works on BlackBerry platform, on other J2ME platform, the application will throw the ClassNotFound Exception.

The reason of failure
The problem is in obfuscation phase, we usually use Proguard to do preverification and obfuscation for the J2ME application. The proguard will remove the unused class file. (for detail please check this link ) In my solution above use class.forName() by passing different string values, which is not recognized in Proguard. For example: in Nokia platform, the NokiaService.class is used, but Proguard did not find the NokiaService.class is explicitly reference in the java code, so it think it is unused class file, then remove it from the final package. Then it cause the ClassNotFoundException.If you don't use obfuscation, the application works, but it is really rarely the commercial application does not have obfuscated.

But why it works on BlackBerry platform?
Because Blackberry is quite different with other J2ME platforms, it requires it own preverification and obfuscation using its own RAPC compiler. And I found that in Blackberry application package it usually keep all the class files in the package, which is different with the Proguard. That is the reason why it works in BlackBerry platform

Solution
I have to rewrite the above code, remove all the Class.forName(), replaced with the explicit method call.
To support multiple platform, I use javaassit to adjust the createService() method for different platform, while you can use pre-process if you want, even I really don't like the J2ME pre-process, because it will bring other more complicated issue in your application.

Conclusion
Class.forName() seems the only reflection that is support in the J2ME platform, but actually it will cause the problem during obfuscation phase, try not to use the Class.forName() in your J2ME application.

Aspire to be agile missionary

2010-08-19T23:41:00.001-04:00

In an evening of earlier this week, when I went out for a walk near my house, I met 2 young Christian missionaries. From their eyes, I can feel their passion and ideal. They want to help people, and enjoy seeing how they can make changes to them. They did that all for free, they have a mission to transfer what the God said to other people. I was totally impressed by their passion, although I don't believe in God.

I was inspired by these 2 young missionary: we might need missionaries in the software industry. I love to be a missionary if there is a similar group about software development which share a unique vision about software developing, like software craftsmanship for example. I would love to share my knowledge and experience; I would love to transfer to other developer that what I believe the right thing, right principle; I love to pursue the way of software development, find the philosophy behind the principles. I love to do that for free, and I am sure I will enjoy what I am doing and seeing the changes I made.

What is the reason that I want to be agile missionary? During year 2002, at that time I had been a software engineer for about 10 years, I became confused about my career as software developer: should I continue my career as developer or change to a manager? Why programmer's job is so hard, so boring, so messy? It doesn't make sense! And in that year, occasionally I began to know about Agile movement, it totally changed my mindset of software developing. Agile is like philosophy: it tells you the right way beyond the detailed technology, computer language and tools. Since then I read lots of books from Kent beck, Martin Fowler, Uncle Bob, Dave Thomas ... etc. Since then I found the answer of my career path: continue my career as software developer, because there so many good stuff you need to know, there are lots of potentials for to improve yourself. Then I started learning Design patterns, doing refactoring, investigate new tools. Gradually I realized that I can feel the difference between me and my colleagues: not because of my experience, but because I have the deeper knowledge of agile and the right way of programming. I really love to share my experiences, because I learned a lot from agile. Before I just unconsciously shared my experiences and knowledge, from now I need to do that consciously and deliberately.

Being agile missionary is not easy. First, you need to keep improving your skills, then you can lead by example; second, you need passion and strong vision; third, you need to know some knowledge of psychology, because different person has different background and different mindset, you need to aware that and use different strategy; fourth, the communication skills is definitely important, you need to keep improving it.

I already set my goal: pursuing the right way of software of development, finding the true principles, strategy and philosophy, attain mastery. Right now I feel I just enter the gate to that road, there is till a long journey to go. I hope at this journey, when I work as a missionary by help other developers, I can improve myself and find other missionaries.

Is it possible for a developer to become an artist?

2010-08-08T22:57:00.001-04:00

I am still reading the book: Linchpin. I am interested with the author's definition about art:
Art is a personal gift that changes the recipient. Art is something creative, passionate and personal, which can bring some change and make the difference.
The author expanded the definition of art , it focuses on "making changes".This idea share the similar idea of software crafsmanship.

Where is the art in software developing?
The developer's daily job can be boring, repetitive instead of creative. For example:
you can just copy and paste from the old code; or just use google check some API usage;
You don't need to care the code quality, don't need to care about the maintainability.
you can also finsih the job. You can do like that everyday.
But this job is not art, you will not make the difference, any one with the similar knowledge can do it.
then your job are easily been replaced.

But the developer's job can be a kind of art if you choose to do in different way:
   there is the art of making simple design;
   there is the art of wrting clean code;
   there is the art of refactoring;
   there is the art of maintaining;
   there is the art of debugging;
   there is the art of communciation and interaction;

Anything which can make things better and make some difference is the art of programming.
if you bring your passion, and make some difference, then that is art;
if you do that day by day, keep improving yourself, then you will become a master, which is also an artist of software.

The road to become artist
I like the concept of Shu-Ha-Ri, it can be treated as a path from an average developer to a master developer:
   In Shu Level, the point is "following, copying", all you need to do is learning, so there is no art behaviour at this level;
  In Ha Level, the key point is "detaching, breaking", you try to bring some change, bring what you learned into different context, you will have some art behavior in this stage;
  In Ri Level, the key point is "transcending", you becomes master, everything you do is from your intuition, you are freely to express your idea,which is the artists behaviour.

So I believe the truly artist is only existed in Ri level, which needs years of the deliberate practice, it should be an ultimate goal for any serious software developer. You will become master and artist at the same time.

Linchpin: to be or not to be?

2010-08-02T22:59:00.003-04:00

Are you a cog or lincphin? Every employee has to answer this question.

In your team, you work hard, show up on time, folllow the order, try to fit in the team, compliance to your boss, always say "yes", finished what you've been told.If you are lucky, you might be given some bonus. But if you only just did that, then you are just an average developer, will be no difference with other colleague, you are just an interchangable component, like a cog in a machine, which are easily to be replaced.
The problem in IT industry, average and mediocrity can not survive, whether a company or a developer. If you are just an average cog, then you are easily to be replaced, you have to compete with other lower priced developer; and even more finally your job might be outsourced to India or China.

So if you want to survive, secure your job, you have to be lincpin: you are indespnesible. Make sure you stand out, remarkable, quite different with others average people; you have passion, you are emotional labour. If you are linchpin, you will become independent and get freedom, you don't need to worry about job, job is always looking for you. Since you are linchpin, you deserve the high pay because you bring much more value to the company.The difference productivity between a great software developer and average one is 10:1, think about how much you can earn if you are linchpin. Think about Apple, how the iPhone makes such difference with other regular phones.

You have to make a choice between these two: to be linchpin or not to be. It is up to you.
If you want more details about linchpin, please read the book: Linchpin: Are you indispensable?

Also check this article: The indispensable Developer
And this: You are not your role

What drive you work? - The review of book Drive

2010-07-25T01:45:00.009-04:00

Today I finish reading the book Drive-The surprising Truth About What Motivates Us". I knew this book from this 10 minutes cartoon on twitter, and the author's video about drive.The topic sound interesting to me, so I started to read the book. To my surprise, the book is quite easy to read, I was so absorbed in it, most of the book was finished on the flight trip to London.This is very unusual to me, since I seldom finishing a English book in such a short time, except Agatha Christie's novel.
The core concept of the book: The traditional external carrot & stick motivator does not work on 21 century; we need to upgrade to intrinsic motivator: autonomy, mastery, & purpose.
This concept is not new to me, it reminds me some other books which share the similar theory behind:

Maslow's self realization theory;
Lean Technology and Deming's theory;
Seven Habits
Maverick
The passionate programmer
The fifth discipline

This book drives me into deep thinking about the 3 factors: autonomy, mastery and purpose.

What I learned from the book
- Autonomy, mastery and purpose are correlated, and each of them supports each other.
      Good example: Since I have the mastery in my mind, I have the passion to do things better; if I get the autonomy, I get trust, I am allowed to do my job on my way, then I will focus entirely on the job, and try to doing my job better, since I know the purpose of my job, I know the whole picture, and I enjoy the whole process.
      Bad example: In the command & control hierarchy, I am just a interchangable resource, I can not control what I am going to do; I have to follow the order from my boss; my job is make my boss happy, I don't care job itself. I don't know the my job related to any purpose, I just know get my job done, make my boss happy and get my money. It is impossible to get highly motivated.
- The road to "go independent" is the right way to me, I get this idea from the book of "the Passionate programmer".This is the only thing that can guarantee you to get autonomy. You can not control your company, you can not decide who is your boss, but at least you can control your self if you are independent software developer.
- Reinforce my goal to mastery.
I feel lucky that I am still a programmer instead of manager, this give me a chance to become a master in future.Keep improve yourself, love your job, treat your career as an infinite game, keep challenge your self.

Next step thinking:
- Compare drive with Lean technology
- Investigate the "flow" concept
- If you are a Type I person, how can you do in a Type X organization, how to deal with your Type X boss?
- What kind of Type I organization should be? How to migrate from Type X to Type I organization?
- Why we should focus Type I behavior in software industry?
- Why manager need to give up control?
- use the theory in "drive" to explain the "three stone cutter stories"
- Read the book of "the fifh discipline".

Resources:
1. 10 minutes video link
2. author Daneil Pink's presentation
3. Dan pink's website
4. The reading list of drive

Time Machine Wireless backup using your own network drive

2010-07-04T20:47:00.009-04:00

Last Friday My Macbook is broken again, this is the third time since this year, I don't know why I am so unlucky and I love my macbook so much! I really worried about my hard drive because the computer could not recognize the hard drive, and I never backup my hard drive data before! Finally the issue was resolved in the Apple Store, the problem is in the harddrive cable.
This time it gave me a lesson: I need to backup my hard drive ASAP.
But when I started to use the time machine for the first time, I got stuck: the Time Machine requires the Time Capusle and Apple airport, while I want to backup into my own NAS drive. To solve this issue, I spent almost half a day on google and trying out back and forth. Finally I made it, I can use time machine to backup my harddrive into my own network drive.
Here I would like to share my experience about how I solve it. Basically I referred this blog, it is so great, I could not able to finish it without this article.

The Goal
Backup my mac hard drive into my own NAS via wireless network.

Environment
Computer: MacBook Pro
OS: Snow Leopard
Wireless Router: Netgear WNDR3700
Network drive: West Digit MyBook Essential

Steps
1. Setup network drive, create a network shared folder for time machine. For example, I called the folder "\mac".
2. Connect the NAS storage, for example in my netgeare wireless network, I type following in Finder:
smb://readyshare/mac
3. Enable time machine to use your network drive, run following command in the terminal:

defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1

4. Find out your computer name and Ethernet ID.
   To get the computer name, go to "System Preferences" - "Sharing" , then you will see the computer name at the top, and you can change to any name without restarting the mac computer.
   For example, my computer name is: steve_zhang_mac. The name contains space and other special characters should be fine, but I prefer to make it simple.
   For the Ether ID, go to "System Preferences" - "Network" - "Ethernet" - "Advanced" - "Ethernet". The ethernet ID in my computer is: d4:9a:20:04:65:34
   Or you can try to type following command in the terminal:
   ifconfig en0 | grep ether | sed s/://g | sed s/ether//
Please remeber: it requires the computer's MAC address regardless if you use wireless or wired connection.
   If you use Wi-Fi, you will find another different ethernet ID under the "Airport" configuration, don't use this value.
5. Create the disk image file
   Create the disk image file locally, the file name is like computername_MACaddrss.sparsebundle
   based on the information from step 4), run following in the terminal:

    DISK_IMAGE_NAME="steve_zhang_macbook_d49a20046534.sparsebundle"
    DISK_SIZE=200000
    hdiutil create -library SPUD -megabytes $DISK_SIZE -fs HFS+J -type SPARSEBUNDLE -volname "$DISK_IMAGE_NAME" "$DISK_IMAGE_NAME"

Copy the created image file into the NAS storage folder.
6. Run Time Machine, choose your NAS box, if everything is OK, then the time machine will start working.

Some Erorrs I met
- Error 45
  This is because the time machine could not create the .sparsebundle file, in your NAS storage folder, you might find a file name and folder called like coputerName_tmp.sparsebundle, please follow step 4 and 5, copy the generated file into the specific shared folder.
  I met this issue twice, one is I did not create the image file, the other is since I use the wrong ehternet ID which I got from the "airport" configuration. After I used the correct ehternet ID,this error does not happened again.

- Error of" .....sparsebundle is already been used."
  The .spasebundle file is used by another process, please restart the computer to run the Time machine again, this issue will solved.

Other tricks
1. For the first time backup, it is better to use LAN instead of Wi-Fi, for example, it took me about 14 hours to backup 113G data using LAN mode.
2. How to check the connected NAS in terminal:
   All the harddrives are under folder /Volumes, so if you type cd /Volumes in terminal, you will see your hard drive and your network drive.
3. How to check the system log:
   System log is in /var/log/system.log
4. If you don't like the time machine to backup every hour, please try the Time Machine Editor, you can customize your own backup schedule with this free app.
5. If you use Parallels virtual machine, then it is better to turn off the backup for parallels virtual machine file, go to Parallels Desktop - Virtual Machine - Configure... , choose "General" tab, check the "Do not back up with time machine." For example, before my computer needs about 120G, right now it only needs to backup 59G data. Or you can open the Time Machine Preferences, choose options, add exclude the Virtual machine file: it is in /Users/Shared/Parallels/Windows XP.pvm

References
1. Getting Time Machine to work on an Unsupported NAS
2. Time Machine Wireless Backup without Time Capsule
3. Time Machine Editor

Debug BlackBerry Application Inside NetBeans

2010-06-27T20:49:00.003-04:00

This post continues the my last post  about blackberry application development.

I added the ANT target to launch the blackberry JDWP debugger from these two articles: 1) and 2). I can launch the BlackBerry JDWP and simulator, but unfortunately the app did not stopped at the break point.
I googled a lot, but it seems none of the post tells you that debugging blackberry successfully inside the NetBeans IDE, I did not find any tutorial about using JDWP in blackberry development document either.
So I have to figure out by myself by using reverse engineering again.

First I set up the BlackBerry project in the JDE environment, and it launched the debugger successfully, can stop at the break point, and can single step;
then I go to the simulator folder of the BlackBerry ( in my pc is C:\Program Files\Research In Motion\BlackBerry JDE 4.5.0\simulator), I found there not only .cod file and .jad file, but also found
.debug, .jar, and .alx file. I realize this might be the reason: you need to copy .debug files in the simulator! In my old build script, I only copied jad and cod files, did not copy .debug files into the simulator.

So I modified my ant build script, make sure it will copy all the rapc generated files into the simulator folder, which includes:
  - .jad and .jar file, I don't know why JDWP needs jar file, but it works any way;
  - debug file, like .debug and xxx-1.debug, xxx-2.debug, etc;
  - .cod file
  - .alx file

The ant script snippet look like this:
1.  ANT target: copy files.  Since I used bb-ant, this target means copy all the file contains the project name under \bbant folder to the simulator folder.

<target name="copyBBSimulator" if="do.rapc">
    <copy todir="${platform.home}/simulator" verbose="true">
             <fileset dir="${dist.dir}/bbant">
                <include name="**/${name}*"/>
            </fileset>
    </copy>
</target>

2. ANT target: launch blackerry debugger, this target will launch the blackberry JDWP remote server and blackberry simulator.

<target name="bb-debug" depends="copyBBSimulator" if="do.rapc">
    <delete file="${preprocessed.dir}/.timestamp"/>
    <parallel>
          <java jar="${platform.home}/bin/JDWP.jar" fork="true" dir="${platform.home}/bin">
            <jvmarg value="-Xmx128M"/>
          </java>

          <sequential>
              <sleep seconds="5"/>
              <antcall target="nbdebug"/>
         </sequential>
   </parallel>
</target>

At this time I use the new ant script and press the debu button in NetBeans, this time it works!!! It takes a longer time to launch the debugger and simualtor, but finally the app stops at the break point, and it allows me to step in and step out.
Another thing I like to mention is for the system outputs, you can only see them from the BlackBerry JDWP output window, you can not see them from the NetBeans output window.

For your reference, I post the whole build script and source code:
1. bb-build.xml

<target name="post-init">
          <available file="${platform.home}/bin/rapc.exe" property="do.rapc" />
          <condition property="jpda.port" value="8000">
              <isset property="do.rapc">
              </isset>
         </condition>
        
  </target>


   <target name="post-jar" if="do.rapc">
     <antcall target="bbbuild" />
   </target>

   <target name="run" depends="init,jar,bb-run,cldc-run" />
   <target name="debug" depends="init,jar, bb-debug, cldc-debug" />


   <target name="cldc-run" unless="do.rapc" >
       <nb-run jadfile="${dist.dir}/${dist.jad}" jarfile="${dist.dir}/${dist.jar}" jadurl="${dist.jad.url}" device="${platform.device}" platformhome="${platform.home}" platformtype="${platform.type}" execmethod="${run.method}" securitydomain="${evaluated.run.security.domain}" commandline="${platform.runcommandline}" classpath="${platform.bootclasspath}:${dist.dir}/${dist.jar}" cmdoptions="${run.cmd.options}"/>
   </target>

   <target name="cldc-debug" unless="do.rapc">
       <delete file="${build.dir}/.timestamp"/>
                <parallel>
                    <nb-run debug="true" debugsuspend="true" debugserver="true" debuggeraddressproperty="jpda.port" platformtype="${platform.type}" platformhome="${platform.home}" device="${platform.device}" jadfile="${dist.dir}/${dist.jad}" jarfile="${dist.dir}/${dist.jar}" execmethod="${run.method}" securitydomain="${evaluated.run.security.domain}" commandline="${platform.debugcommandline}" classpath="${platform.bootclasspath}:${dist.dir}/${dist.jar}" cmdoptions="${run.cmd.options}"/>
                    <sequential>
                        <sleep seconds="5"/>
                        <antcall target="nbdebug"/>
                    </sequential>
                </parallel>
   </target>

<target name="post-clean">
   <delete failonerror="false">
      <fileset dir="${platform.home}/simulator">
       <include name="**/${name}*">
       </include>
      </fileset>
   </delete>
</target>

<typedef resource="bb-ant-defs.xml" classpath="bb-ant-tools.jar" />
  <target name="bbbuild" description="blackberry build" depends="init" if="do.rapc">
          <echo message="rapc build, dir=${dist.dir}"></echo>
          <mkdir dir="${dist.dir}/bbant" />
          <rapc verbose="true" output="${name}" jdehome="${platform.home}" import="${platform.bootclasspath}" destdir="${dist.dir}/bbant/" noconvert="true">
          <src>
             <fileset file="${dist.dir}/${dist.jar}" />
          </src>
          <jdp title="${app.title}" vendor="${app.vendor}" version="${app.version}" type="cldc" icon="${app.icon}" runonstartup="false" >
          </jdp>
          </rapc>

        <!-- sigtool jdehome="C:Program FilesResearch In MotionBlackBerry JDE 4.7.0" codfile="${dist.dir}/bbant/${name}.cod" password="" / -->
        <alx destdir="${dist.dir}/bbant" filename="${name}.alx">
            <application id="${app.title}">
               <codset>
                    <fileset dir="${dist.dir}/bbant" includes="*.cod">
                   </fileset>
               </codset>
           </application>
       </alx>
       <mkdir dir="${dist.dir}/bbant-final" />
       <jadtool input="${dist.dir}/bbant/${dist.jad}" destdir="${dist.dir}/bbant-final">
        <fileset dir="${dist.dir}/bbant" includes="*.cod">
        </fileset>
      
       </jadtool>
 </target>

<target name="copyBBSimulator" if="do.rapc">
    <copy todir="${platform.home}/simulator" verbose="true">
             <fileset dir="${dist.dir}/bbant">
                <include name="**/${name}*"/>
            </fileset>

    </copy>

</target>

<target name="bb-run" depends="copyBBSimulator" if="do.rapc">
    <exec os="Windows NT Windows 95 Windows 98 Windows 2000 Windows XP"
             dir="${platform.home}/simulator"
            executable="${platform.home}/simulator/${platform.device}.bat"
        failonerror="true" resolveExecutable="true"/>
</target>

<target name="bb-debug" depends="copyBBSimulator" if="do.rapc">
    <delete file="${preprocessed.dir}/.timestamp"/>
    <parallel>
          <java jar="${platform.home}/bin/JDWP.jar" fork="true" dir="${platform.home}/bin">
            <jvmarg value="-Xmx128M"/>
          </java>

          <sequential>
              <sleep seconds="5"/>
              <antcall target="nbdebug"/>
         </sequential>
   </parallel>
</target>

2. build.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE project [
       <!ENTITY bb-common SYSTEM "bb-build.xml">
]>
<project name="helloLWUIT" default="jar" basedir=".">
    <description>Builds, tests, and runs the project .</description>
    <import file="nbproject/build-impl.xml"/>
    <property name="app.title" value="helloLWUIT" />
    <property name="app.version" value="1" />
    <property name="app.icon"   value="/icon.png" />
    <property name="app.vendor" value="foo" />

     &bb-common;
</project>

3. source code project: helloLWUIT:
Here
Environment requirements: please refer Shai's blog for blackberry to set up the project environment

Next Steps:

Figure out how to debug blackberry application on the actual device inside NetBeans;
Make sure how to read attribute in the jad file in the BlackBerry CLDC application;
How to easier copy the customized attribute in the midlet jade file to the BlackBerry CLDC jad file.

References:

http://www.xenglobaltech.com/blackberry/index.php?option=com_content&view=article&id=45&Itemid=40
http://supportforums.blackberry.com/t5/Java-Development/Building-Blackberry-apps-with-Netbeans/td-p/431612

Professional Devlelopers require professional managers

2010-06-21T22:52:00.002-04:00

I realize that there are similarities between software developers and software managers:
- Developers manage codes, managers manage people;
- Developers resolve bugs in the codes, managers resolve bugs among the people;
- A developer has to make decisions all the time, compromise his solution with different constraints; A manager has to make decision all the time based on people, schedule and budget;
- A professional developer always choose the right framework and right tool for the task, a professional manager choose the right person to do the right thing;
- A professional developer improves his code by keep constantly refactoring and reflection, a professional manager keep improving his team;
- A professional developer always follows the software principles and design patterns to guide his work, deliberately improve his skills, keep finding new tools; A professional manager will follow the agile patterns and other principles and philosophy (ex. The Art Of War) to guide his work, his has the clear picture of his vision, and keep finding the new process to improve the productivity of his team.

But in reality, unfortunately in my career so far I haven't found a manager which I can regard as a professional manager. Most of them I call them "event dispatcher". They care more about passing the command down to developers, focus more on his boss instead of developers, they don't care how developer finish it nor be interested in it. I even could not find his strategy of management. I really want he can show me how he solved a difficult issue creatively and beautifully, just like I saw my coworker solve an issue by using a pattern or a tool, but so far I haven't seen once.

Right now I know the reason why software industry is still primitive, why there are so many unprofessional developers, because most of our managers are unprofessional. If a manger dos not know the principles and rules of software, how can he manage well for his team? If a manager does not care his developer, how can his developer has motivation to become better?

A good manager for a software team just as a good coach for a soccer team. Professional developers do require professional managers.

LWUIT blackberry ANT script refactored 2

2010-05-27T23:03:00.006-04:00

Last time I posted the refactored blackberry build script, today I made some other small improvements:

1. Add a "run" target, support running both blackberry and regular J2ME simulator.

<target name="run" depends="init,jar,bb-run,cldc-run">
     
    <target name="cldc-run" unless="do.rapc" >
       <nb-run jadfile="${dist.dir}/${dist.jad}" 
                  jarfile="${dist.dir}/${dist.jar}" jadurl="${dist.jad.url}" 
                  device="${platform.device}" platformhome="${platform.home}" 
                  platformtype="${platform.type}" 
                  execmethod="${run.method}" 
                  securitydomain="${evaluated.run.security.domain}" 
                  commandline="${platform.runcommandline}" 
                  classpath="${platform.bootclasspath}:${dist.dir}/${dist.jar}" 
                  cmdoptions="${run.cmd.options}"/>
    </target>
    
    <target name="bb-run" depends="copyBBSimulator" if="do.rapc">
     <exec os="Windows NT Windows 95 Windows 98 Windows 2000 Windows XP"
             dir="${platform.home}/simulator"
            executable="${platform.home}/simulator/${platform.device}.bat"
        failonerror="true" resolveExecutable="true"/>
   </target>

The above code snipplet means, when user press "run" button in NetBeans IDE, if the current configuration is blackberry, then launch the blackberry simulator, else launch the regular J2ME simulator.

2. Separate the blackberry specific ant targets into aother build script file, make the build file generic and easier to use.
At the beginning I tried to put all the targets in a separate file: bb-build.xml, then import it in the build.xml, but I found the netbeans custom targets like post-init, post-jar are not called, it seems these targets needs to be explicit specified in the build.xml.
I found another solution - embedding the xml snippet in the build.xml using XML entity, based on this link:
for exmaple, like this:

<!DOCTYPE project [
       <!ENTITY bb-common SYSTEM "bb-build.xml">
    ]>

   &bb-common;

The above snippet is equivlent of copying the code from bb-build.xml into build.xml.

Benefits:
The new refactored build script becomes generic, does not specific to lwuit project, can apply to any blackberry CLDC app.

I attach the full build script here, and you can use them as template for your blackberry build script in your NetBeans project.
1) build.xml

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE project [
       <!ENTITY bb-common SYSTEM "bb-build.xml">
]>

<project name="myApp" default="jar" basedir=".">
    <description>Builds, tests, and runs the project .</description>
    <import file="nbproject/build-impl.xml"/>
   
    <property name="app.title" value="myApp" />
    <property name="app.version" value="1" />
    <property name="app.icon"   value="/icon.png" />
    <property name="app.vendor" value="foo" />

    &bb-common;
 </project>

2) bb-build.xml, the blackberry build script snippet

<target name="post-init">
          <available file="${platform.home}/bin/rapc.exe" property="do.rapc" />
          <condition property="jpda.port" value="8000">
              <isset property="do.rapc">
              </isset>
         </condition>
         
  </target>


   <target name="post-jar" if="do.rapc">
     <antcall target="bbbuild" />
   </target>

   <target name="run" depends="init,jar,bb-run,cldc-run">

</target>

   <target name="cldc-run" unless="do.rapc" >
       <nb-run jadfile="${dist.dir}/${dist.jad}" jarfile="${dist.dir}/${dist.jar}" jadurl="${dist.jad.url}" device="${platform.device}" platformhome="${platform.home}" platformtype="${platform.type}" execmethod="${run.method}" securitydomain="${evaluated.run.security.domain}" commandline="${platform.runcommandline}" classpath="${platform.bootclasspath}:${dist.dir}/${dist.jar}" cmdoptions="${run.cmd.options}"/>
   </target>

<target name="post-clean">
   <delete failonerror="false">
      <fileset dir="${platform.home}/simulator">
       <include name="**/${name}.*">
       </include>
      </fileset>
   </delete>
</target>

<typedef resource="bb-ant-defs.xml" classpath="bb-ant-tools.jar" />
  <target name="bbbuild" description="blackberry build" depends="init" if="do.rapc">
          <echo message="rapc build, dir=${dist.dir}"></echo>
          <mkdir dir="${dist.dir}/bbant" />
          <rapc verbose="true" output="${name}" jdehome="${platform.home}" import="${platform.bootclasspath}" destdir="${dist.dir}/bbant/" noconvert="true">
          <src>
             <fileset file="${dist.dir}/${dist.jar}" />
          </src>
          <jdp title="${app.title}" vendor="${app.vendor}" version="${app.version}" type="cldc" icon="${app.icon}" >
          </jdp>
          </rapc>

        <!-- sigtool jdehome="C:Program FilesResearch In MotionBlackBerry JDE 4.7.0" codfile="${dist.dir}/bbant/${name}.cod" password="" / -->
        <alx destdir="${dist.dir}/bbant" filename="${name}.alx">
            <application id="${app.title}">
               <codset>
                    <fileset dir="${dist.dir}/bbant" includes="*.cod">
                   </fileset>
               </codset>
           </application>
       </alx>
       <mkdir dir="${dist.dir}/bbant-final" />
       <jadtool input="${dist.dir}/bbant/${dist.jad}" destdir="${dist.dir}/bbant-final">
        <fileset dir="${dist.dir}/bbant" includes="*.cod">
        </fileset>
       </jadtool>
 </target>

<target name="copyBBSimulator" if="do.rapc">
    <copy todir="${platform.home}/simulator" verbose="true">
            <fileset dir="${dist.dir}/bbant">
                  <include name="*.cod"/>
                  <include name="*.jad" />
            </fileset>
    </copy>

</target>

<target name="bb-run" depends="copyBBSimulator" if="do.rapc">
    <exec os="Windows NT Windows 95 Windows 98 Windows 2000 Windows XP"
             dir="${platform.home}/simulator"
            executable="${platform.home}/simulator/${platform.device}.bat"
        failonerror="true" resolveExecutable="true"/>
</target>

Next Step:
Add a target support debugging blackberry Application inside the NetBeans IDE.

LWUIT BlackBerry ANT script refactored

2010-05-18T17:59:00.006-04:00

My last article about using ANT script to build LWUIT blackberry application was intereted by some people, which is encouraging. I spent some time to review the ant script again and made some improvements. Here I posted my updated build script, it is cleaner and easier to use, you can just use it as template, copy and paste it in your NetBeans project by just configure some parameters.

<property name="app.title" value="helloLWUIT" />
    <property name="app.version" value="1" />
    <property name="app.icon"   value="/icon.png" />
    <property name="app.vendor" value="foo" />

    <target name="post-init">
          <available file="${platform.home}/bin/rapc.exe" property="do.rapc" />
          <condition property="jpda.port" value="8000">
              <isset property="do.rapc">
              </isset>
         </condition>
    </target>

  
   <target name="post-jar" if="do.rapc">
     <antcall target="bbbuild" />
   </target>
  

<target name="post-clean">
   <delete failonerror="false">
      <fileset dir="${platform.home}/simulator">
       <include name="**/${name}.*">
       </include>
      </fileset>
   </delete>
</target>

<typedef resource="bb-ant-defs.xml" classpath="bb-ant-tools.jar" />
  <target name="bbbuild" description="blackberry build" depends="init">
          <echo message="rapc build, dir=${dist.dir}"></echo>
          <mkdir dir="${dist.dir}/bbant" />
          <rapc verbose="true" output="${name}" jdehome="${platform.home}" import="${platform.bootclasspath}" destdir="${dist.dir}/bbant/" noconvert="true">
          <src>
             <fileset file="${dist.dir}/${dist.jar}" />
          </src>
          <jdp title="${app.title}" vendor="${app.vendor}" version="${app.version}" type="cldc" icon="${app.icon}" >
          </jdp>
          </rapc>

        <!-- sigtool jdehome="C:Program FilesResearch In MotionBlackBerry JDE 4.7.0" codfile="${dist.dir}/bbant/${name}.cod" password="" / -->
        <alx destdir="${dist.dir}/bbant" filename="${name}.alx">
            <application id="${app.title}">
               <codset>
                    <fileset dir="${dist.dir}/bbant" includes="*.cod">
                   </fileset>
               </codset>
           </application>
       </alx>
       <mkdir dir="${dist.dir}/bbant-final" />
       <jadtool input="${dist.dir}/bbant/${dist.jad}" destdir="${dist.dir}/bbant-final">
        <fileset dir="${dist.dir}/bbant" includes="*.cod">
        </fileset>
       </jadtool>
 </target>

<target name="copyBBSimulator" depends="bbbuild" if="do.rapc">
    <copy todir="${platform.home}/simulator" verbose="true">
            <fileset dir="${dist.dir}/bbant">
                  <include name="*.cod"/>
                  <include name="*.jad" />
            </fileset>
    </copy>
</target>

<target name="bb-run" depends="copyBBSimulator">
    <exec os="Windows NT Windows 95 Windows 98 Windows 2000 Windows XP"
             dir="${platform.home}/simulator"
            executable="${platform.home}/simulator/${platform.device}.bat"
        failonerror="true" resolveExecutable="true"/>
</target>

It has following improvements:

Separate out the customized parameters in the build scriciptThen entire build script only needs following customized parameters: app.title, app.version, app.icon, app.vendor. You only need to configure them with your own application parameters.
Simplify the build script. In the old build script, you have to use target "copyBBTouchSources" and "copyBBSources" to copy and update the LWUIT library source code into specific folder, then call rapc compiler to build the entire application. But when you investigate a little bit deeper, since NetBeans IDE already generate jar file for you, which already included the preprocessed application and the LWUIT blackbery library class files, why not just pass this jar file to rapc and then generate the cod file? based on this idea, I modified the target "bbbuild" and it works.
One build script supports both BlackBerry touch and non-touch build. since the build script is simplified and only use jar file to generate cod file,the touch or non-touch specific files are already embedded in the jar file, the build script itself becomes generic. To tested it, I installed JDE 4.5 and JDE 4.7 two different JDE in the NetBeans, and set up 4.5 and 4.7 different configurations in my project, and I found both works.
Added the ant target "bb-run", which allow you run the application in the blackberry simulator. To make it work, you had better to check the nbproject\project.properties file, if the simualtor device name is configured properly, In JDE 4.7, the device name is 9500, like this:
```
configs.BlackBerryTouch.platform.device=9500
```
In 4.5, the device name is 8320, like this:
```
configs.BlackBerry.platform.device=8320
```

Update: Sample project
Please click here to download the sample project.

Why should managers embrace agile?

2010-05-04T20:58:00.002-04:00

When you look at the articles about Agile/Scrum/Lean, they are all about self-organizing, decentralized, democracy, etc.Those are welcomed by the developers but conflict with managers benefit, because our conventional management is about hierarchy, command and control, chasing the power, the manager cares more about how his boss think instead of how his subordinates think. For managers, embracing agile means loosing power, weaken their benefit, that is why manager don't like agile.
On the other hand, agile is about changing, changing means risky, but the managers care more about the stability of their position, their power, they don't have much enthusiasm to make changes.

So we have a conclusion that developer and manager like two different classes, developers like agile because it brings them benefits; managers dislike agile because it will lose managers' benefit. The progress of agile movement is quite similar with the progress of human's democracy movement.

This sounds discouraging, but this is the fact, that will explain why agile has not become mainstream after 10 years of its movement.Then what can we do? Try to educate as many developers as possible, gradually if agile principles are accepted by most people, then it is quite possible to force the managers make changes. This will take time, but please remember that the manager won't truely embrace agile,just like no ruler will truely embrace democracy.

ANT build script for LWUIT blackberry application

2010-05-01T22:49:00.003-04:00

Update:
This article is more than a year old, now LWUIT project uses a new tool to build BlackBerry application - NetBeans BlackBerry Plugin, please refer my two latest post:

Developing Blackberry application using NetBeans Plugin - part 1, first impression
Developing Blackberry application using NetBeans Plugin - part 2, fixing debug issue

Shai's article about build script for LWUIT in BlackBerry is great, but it has so many errors,I spend two days trying to fix the errors, and finally make it work.

Here I posted my modified ant buil.xml and application source code. The application is very simple, just display a "Hello World" text. Since this article is based on Shai's article, for details such as project set up, configuration and tools installation etc, please check his article.

application source code
HelloLWUITMidlet.java

import com.sun.lwuit.Display;
import com.sun.lwuit.Form;
import com.sun.lwuit.Label;
import com.sun.lwuit.layouts.BorderLayout;


public class HelloLWUITMidlet extends
//#ifdef RIM
     net.rim.device.api.ui.UiApplication
//#else
//#      javax.microedition.midlet.MIDlet
//#endif
 {
    public void startApp() {

        Display.init(this);
        Form f = new Form();
        f.setTitle("Hello World");
        f.setLayout(new BorderLayout());
        f.addComponent("Center",
        new Label("I am a Label"));
        f.show();

    }
//#ifdef BlackBerry
   public static void main(String[] argv)
   {
       new HelloLWUITMidlet().startApp();
   }

   public void notifyDestroyed()
   {
       System.exit(0);
   }

   public void platformRequest(String s) {
      net.rim.blackberry.api.browser.Browser.getDefaultSession().displayPage(s);
     }
//#endif


    public void pauseApp() {
    }

    public void destroyApp(boolean unconditional) {
    }
}

ANT build script

<?xml version="1.0" encoding="UTF-8"?>
<!-- You may freely edit this file. See commented blocks below for -->
<!-- some examples of how to customize the build. -->
<!-- (If you delete it and reopen the project it will be recreated.) -->
<project name="helloLWUIT" default="jar" basedir=".">
    <description>Builds, tests, and runs the project .</description>
    <import file="nbproject/build-impl.xml"/>
   

   
     <target name="post-init">
        <available file="${platform.home}/bin/rapc.exe" property="do.rapc" />
        <available file="${platform.home}/simulator/9500.bat" property="bbtouch" />
        <condition property="jpda.port" value="8000">
           <isset property="do.rapc">
           </isset>
        </condition>
     </target>


        
        <target name="post-preprocess" depends="copyBBSources,copyBBTouchSources">
        </target>
        

       <typedef resource="bb-ant-defs.xml" classpath="bb-ant-tools.jar" />
       <target name="bbbuild" description="blackberry build" depends="init,copyBBSources,copyBBTouchSources">
          <echo message="Compiling ${preprocessed.dir}"></echo>
          <mkdir dir="${dist.dir}/bbant" />
          <rapc verbose="true" output="${name}" jdehome="${platform.home}" import="${platform.bootclasspath}" destdir="${dist.dir}/bbant/" noconvert="true">
          <src location="${basedir}/${preprocessed.dir}">
          </src>
          <jdp title="helloLWUIT" version="1" type="cldc" icon="/icon.png" >
          </jdp>
          </rapc>
          
        <!-- sigtool jdehome="C:\Program Files\Research In Motion\BlackBerry JDE 4.7.0" codfile="${dist.dir}/bbant/${name}.cod" password="" / -->
        <alx destdir="${dist.dir}/bbant" filename="${name}.alx">
        <application id="helloLWUIT">
       <codset>
        <fileset dir="${dist.dir}/bbant" includes="*.cod">
       </fileset>
       </codset>
       </application>
       </alx>

      
       <mkdir dir="${dist.dir}/bbant-final" />
       <jadtool description="desc" id="appId" input="${dist.dir}/bbant/${dist.jad}" destdir="${dist.dir}/bbant-final">
        <fileset dir="${dist.dir}/bbant" includes="*.cod">
        </fileset>
       </jadtool>
       
      </target>

 <target name="copyBBTouchSources" if="bbtouch">
 <echo message="Copying blackberry touch sources"></echo>
 <copydir forceoverwrite="true" src="${project.BlackberryPort}/build/touch/preprocessed" dest="${basedir}/${preprocessed.dir}" />
 <touch file="${basedir}/${preprocessed.dir}/com/sun/lwuit/M3G.java" />
 <touch file="${basedir}/${preprocessed.dir}/com/sun/lwuit/SVGImage.java" />
 <touch file="${basedir}/${preprocessed.dir}/com/sun/lwuit/animations/Transition3D.java" />
 <touch file="${basedir}/${preprocessed.dir}/com/sun/lwuit/impl/midp/SVGImplementation.java" />
 
 </target>

 <target name="copyBBSources" if="do.rapc">
   <echo message="Copying blackberry sources ${preprocessed.dir}"></echo>
   <copydir src="${project.LWUIT}/src" dest="${basedir}/${preprocessed.dir}" />
   <copydir forceoverwrite="true" src="${project.BlackberryPort}/build/preprocessed" dest="${basedir}/${preprocessed.dir}" />
   <touch file="${basedir}/${preprocessed.dir}/com/sun/lwuit/M3G.java"/>
   <touch file="${basedir}/${preprocessed.dir}/com/sun/lwuit/SVGImage.java"/>
   <touch file="${basedir}/${preprocessed.dir}/com/sun/lwuit/animations/Transition3D.java"/>
   <touch file="${basedir}/${preprocessed.dir}/com/sun/lwuit/impl/midp/SVGImplementation.java"/>
 </target>

 <target name="post-jar" if="do.rapc">
   <rapc verbose="true" output="${name}" jdehome="${platform.home}" import="${platform.bootclasspath}" destdir="${platform.home}/simulator/" noconvert="true">
     <src location="${basedir}/${preprocessed.dir}">
     </src>
     
     <jdp title="helloLWUIT" version="1" type="cldc" icon="/icon.png" >
     </jdp>
   </rapc>
 </target>
 
 <target name="post-clean">
    <delete failonerror="false">
       <fileset dir="${platform.home}/simulator">
        <include name="**/${name}.*">
        </include>
       </fileset>
    </delete>
 </target>

</project>

Note:
This build script is for touch screen build, for non touch sreen build you need to
1.Comment out "bbtouch" property in target "post-init" :
 </available>
2.Remove the "copyBBTouchSources" dependency in target "bbbuild".

Build LWUIT BlackBerry CLDC application

2010-04-21T22:39:00.002-04:00

Last week finally I have time to review the Shai's blog about building LWUIT project on Blackberry platform. At first I tried to follow the step by step in his article, but I am disappointed because of so many errors of his build.xml. After a couple hours fix, I still could not make the build, so I decided to give up Shai's script, and choose an easier way.
My solution is just using NetBeans and BlackBerry JDE, it is much easier to build a demo.
Here is the step by step description:

Build LWUIT BlackBerry library
1. Checkout the LWUIT source code from SVN, (https://lwuit.dev.java.net/svn/lwuit/trunk), the LWUIT source includes the NetBeans BlackBerry port project.
2. Install JDE 4.7 and Configure BlackBerry support in NetBeans, you can follow this link to configure;
3. Start NetBeans, open the "BlackBerryPort" poject under the "BlackBerry" sub folder of lwuit project source, there are two configuration for this project, the default is for non touch screen build, the "touch" configuration is for touch screen device like STORM.for non touch device, just choose DefaultConfiguration, build the project, you will see BlackberryPort.jad and BlackberryPort.jar in \dist folder, they are the LWUIT library for BlackBerry devices;

Build LWUIT Demo into BlackBerry CLDC application
1. Open Blackberry JDE, create a new workspace, called "lwuitdemo.jdw";
2. add a new project under the workspace, name it "lwuit", choose its Application type as "Library", import the BlackberryPort.jad and BlackBerryPort.jar file into the project. then build it, the JDE will generate lwuit.cod and update BlackBerryPort.jad file.
3. download LWUIT 1.3 from sun website, it comes with the famous LWUITDemo source code, open the LWUITDemo project using NetBeans, build it;
4. add another new project in the JDE workspace, name it "lwuitdemo", choose application type as "CLDC Application";
   set up the correct value with title, version and Vendor information;
   next you need to add project dependency to "lwuit" project, right click project, choose "Project Dependencies..", choose "lwuit project";
   copy all the java source code from LWUITDemo src folder, add them into the project, ignore those png files and .res files;
   go to build\preverified folder, copy all the png files and .res files, add them into your JDE project;
   right click icon.png, choose "use as application icon";
5.Modify the UIDemoMIDlet.java
Made following changes:
make sure the UIDemoMIDlet extends from uiApplication, like this:

public class UIDemoMIDlet extends net.rim.device.api.ui.UiApplication  {...}

Add the following methods, copied from Shai's blog:

public static void main(String[] argv)
    {
        new UIDemoMIDlet().startApp();
    }
 
    public void notifyDestroyed()
    {
        System.exit(0);
    }
 
    public void platformRequest(String s) {
       net.rim.blackberry.api.browser.Browser.getDefaultSession().displayPage(s);
    }

In startApp() method, comment out the following statement, because non touch device need to disable the virtual keypad:

// VKBImplementationFactory.init();

6. Build the lwuitdemo project.
7. Sign both lwuitdemo.cod and lwuit.cod fle, load them both into your blackberry device.
8. If you want to provide OTA download, that is a litttle bit tricky:
   You need to download two artifacts, one for lwuit libray, the other for lwuitdemo, which means user need to download two links:
   1) lwuti library:
   BlackBerryPort.jad
   lwuit.cod
   lwuit-1.cod
   ...
   2) lwuitdemo application:
   lwuitdemo.jad
   lwuitdemo.cod
   lwuitdemo-1.cod
   ...

You can download the whole source code of the JDE project from here.

In future I would like to share:
  1. Provide an updated build.xml based on Shai's blog;
  2. provide how to build LWUITDemo MIDlet project for BlackBerry device.

How to send email in BlackBerry Simulator

2010-04-08T22:00:00.050-04:00

Recently I am working on a BlackBerry Email app, I need to setup a test environment so I can test mail send in the BlackBerry simulator.
I did some research, I read the RIM related document it does not help, then I find a solution from a good book about blackberry - Advanced BlackBerry Development, by Chris King.
I just follow the method from the book , and finally get it working, the solution is quite straight forward.

You need BlackBerry ESS ( Email Server Simulator) in order to simulate mail from your simulator, it comes with the JDE SDK and JDE component, currently I used JDK 4.5 , I prefer it because it's simulator is much faster to launch.

There are two mode you can choose - Connected mode or Standalone mode.
I tired the connected mode, it does not working.
The book recommend using the Standalone mode: ESS will listen the localhost port, send out email to your smtp email client. If you set up your email client, choose localhost as server address, and configure the same pop3 and smtp port as ESS configured, your email client will receive the email.
Choose "Standandalone mode", set Pop3 port and Smtp port, I choose default value, which are 995 and 465.
Click Launch, the ESS will start, and you will say a dos command window displaying the server log information.

Then we need to configure your email client, I choose OutLook Express.
Add a new account, you can set up any email address,
Set localhost for both SMTP and POP3 server.
for POP3 port number ,use the same value of ESS: 465
for SMTP port number, use the same value of ESS:995

So far the configuration is finished, all you can do is launching the simulator.
You can start the blackberry email client, type any address, then send. Check your OutLook Express, you will find an incoming email.
And you can write an email from your outlook express, type any address, then send it. You will find the email in your blackberry simulator.
The only drawback is you can only send and receive email from your local machine. which is not real, but it is good enough to test the mail function in the simulator.
It is really good, I tested over 100 email using the solution above.
Finally I want to talk about a little about the book, it covers many good topics, such as BES, BIS which confuses me for a long time. Even I haven't finished it, but I think it is one of the best blackberry book so far.

My mistakes to the J2ME Location API

2010-03-15T23:40:00.000-04:00

Last week when I tried to test the location API for J2ME devices, I made big mistakes.
The 2 devices I tested are: NOKIA N93 and BlackBerry 8320. Actually I don't have any experience for J2ME location API before.Before testing, I checked the spec of these two devices, they both support Location API - JSR 179, so I thought they both have GPS receiver,but when I tested several location applications on theses two devices and even simple demos, none of them works on either devices: N93 keep looking for Blue tooth devices, and then pop up Location exception or pop up timeout exception; while the applications in blackberry keep hanging no response.

I spent the whole afternoon trying to make them works, but no luck. Finally I googled them again, then I found that these two devices do not have internal GPS receiver! They support location API means they need external blue tooth GPS receiver!!! Which explains why N93 keeps looking for blue tooth devices.

So I learned that even if a device supports Location API does not means it has GPS, then it means there is no way to check a device supports GPS or not via programmatic way, because it is impossible to distinguish if a device has built-in GPS with a device without GPS.

A solution for non GPS Blackberry devices
Another reason I made the mistakes is because I saw the Google Map works perfectly in my 8320 devices. Now I understand it might use cell id to get the location information.This will give the developer another hint to enable the loation service to the non GPS blackberry phones.

I found a open source project: OpenCellId, which maintain a cell id database, so you can query your location by providing your device's cell id.
Here is the API.

I checked the BlackBerry API, it has a GPRSInfo class, which provides the GPRSCellInfo class, GPRSCellInfo contains all the cell id information you have.
You can use the following code snippet to get the cell id:
   import net.rim.device.api.system.GPRSInfo;
   ...
   int cellId = GPRSInfo.getCellInfo().getCellId();
You can also get MNC, MCC, LAC, RAC etc.
With these information, you can use OpenCellId to retrieve your location.

This sounds interesting, I haven't try it yet, I am sure I will check it in future.

Software Quality: internal vs. external

2010-02-07T22:25:00.007-05:00

Let's check following questions:
1)How to define the quality of a software product?
2)Is a bug free software a high quality software?
3)Is software quality determined by QA testing?
4)Can QA testing help improve the quality?

The software quality can be divided into two parts: external and internal.

External quality is explicit, transparent and easy for user to experience, easy to test, the bug can be easily found;
Internal quality is implicit, it is hard to describe and visualize by the end user, but it is the true quality of the software, usually related clarity, flexibility, maintainability, readability etc, it is a gene of a software.
The relationship between external and internal quality like a iceberg: external quality is above the water, while most of the stuff under the water are belong to internal quality.
We say that Internal quality determines the external quality:
If internal quality is good, then external quality are also good,
If we found more external bugs, then there must be more internal bugs, we just haven't found them yet.

Now we can answer question 1) and 2):
To determine a software is high quality or not, the most important is how much it can add value to customer, which is based on its internal quality; if it does not bring much value, even if it is no bug, still it can not be treated as a high quality software, because it is useless.

To answer question 3) and 4), Let's check another metaphor:
A human being is healthy is determined by his internal physical condition,or his gene, not by the physical examination. The annual checkup will give you a feedback of your body. The testing itself does not improve your body condition.
So the answer will be: the internal quality determine the software quality; the QA testing just give the feedback of the quality, itself can not improve the quality.

We just pay too much attention on the external quality, but ignore the internal quality; if we want to truly improve the software quality, we need to pay more attention on the internal quality, pay attention on external quality is too late and not efficient.

How to measure the internal quality? How to improve the transparency and visualize the internal quality?
Static analysis tool can tell you how good your code is.

How to improve the internal quality?
Use iterative development process, encourage team reflection, keep constantly refactoring, code review, use design patterns, etc.

Focus on Workflow instead of plans

2010-01-30T23:04:00.002-05:00

Recently I watch the Mary Poppendieck's video "The tyranny of the plan", which is another good video about agile. The key point of the presentation is - Focus on the workflow instead of the detailed plans
I put some of the notes here:

3 main principles:

Design the system to meet the constraints; do not derive constraints from the design.
Decouple workflows; break dependencies!
Workflows are easier to control & more predictable than a schedules.

Mary used the Empire building example is used to support the principles:
   Start with the constraints, then design the system to meet the constraints;
   design based on the schedule (constraints);
   schedule does not depend on the detailed design.

What I learned is that the system design depends on constraints, which might be resource, schedule and cost, etc. while schedule should not depend on design. This is a 180 degree reverse of the mindset change.

The problems of the schedules.
We need schedules because:
- To control when things happen, but
     Detailed schedules are deterministic and do not allow for normal variation.
     Managing a level workflow is a lot easier than following a deterministic schedule.
- To predict when things will happen, but
    Schedules based on experience are reliable; if it is based on wishful thinking are not
    When reality does not match the schedule - the schedule hypothesis was dis-proven!

The workflow is more flexible, always updated based on the situation, "plan is bad, planning is good"

Pull scheduling
Mary also mentioned the pull scheduling, pull scheduling is time-boxed:
don't scope box, don't ask how long will this take, ask what can we done by this date?
I think which is the concept from Lean/Kanban. I will check it in detail later.

Resources:
1. Presentation video
2. PPT, which is different from the presentation, but actual concepts are same.

My Vision of the future mobile platform framework

2010-01-26T22:22:00.004-05:00

Develop a mobile application is painful, develop a portable mobile application is really hard. I've been a mobile developer for many years, and I am struggling many years to make them portable to different platforms. To deal with fragmentation, which is the nature of mobile platform, my vision of the future mobile platform framework would be:

Support multiple platforms
support J2ME, iPhone, android, blackberry, Windows Mobile, etc. One solution supports multiple platform, which will makes our life much easier. I hope new new framework will provide a more abstraction layer, then developer can only write one piece of code can support all.

Separation of concerns
- Separate platform independent logic from platform specific logic
- Separate application logic from specific user interface
The platform specific logic should be encapsulated as component modules, and developer should not care more about their details. Think about the IC and drivers, our mobile application is like a computer mother board which plug different cards and installing different drivers.
The concept of separation of concern reminds me of how GIS works, GIS map is composed of maps in different layer, each layer map focuses on a specific concern, same as the mobile application, the mobile app will be composed of different layers, UI layer, business logic layer, model layer, etc. And each layer can be easily modified and plug-in. Then graphic designer can focus UI, generate the rendering logic; BA focuses on business rules, generate business logic, then developer integrate them all into one application, and each layer can be easily changed and upgraded.

Easy to customization
The mobile framework should takes the advantage of different platforms technologies. we should not do the way of common denominator, like J2ME MIDP widget, which is a bad example.

Open and easy to integrate with other libraries
The new mobile platform should be open to other framework or library, it should be easily to integrate with other libraries. for example, developer can integrate different UI libray easily, like LWUIT, SVG, oepnGL, etc. and the mobile platform does not want to reinvent the wheels, it focuses on integration.

Follow agile
The mobile framework should be agile, which means
- support TDD, provide a mobile unit test framework or support 3rd party test framework
- Encourage short feedback, the mobile framework provides powerful simulator which can simulate different platforms, developer can easily to know what is going on for his application, and it is also a good demo and presentation to our costumer, the simulator can be run inside the browser so that it is a good demo for end users.
- provide build and deploy automation, since different platform has different SDK and build tools, manage them is a nightmare, having a build automation tool will greatly improve the developer's productivity and speed up the application development cycle.

Several possible technical paths:
- DSL/Code generation
Define a mobile Domain Specific Language, then generate the native source code for different platform. we can use Ruby or ANTLR. I like the idea of code generation, because it can meet the needs of different platform, and can easily to implement the AOP

- Use web based script languages
Using html, xml, CSS, java script, then render in the browser or using parser to generate the native source code. This is also a good concept, actually there are already have several open source project doing like this, for example PhoneGap,but what I really worry about are: 1) How to address the fragmentation for each platform; 2) to solve the question 1, may leads to common denominator issue. For this reason, I really doubt the JavaFX Mobile framework, because there is no way to handle the fragmentation issue in script language level.

I hope in the near future, I can do something for the new mobile platform. I have a dream, in future: UI designer, BA, developer and QA will share the same framework, each works on his own part, then developer integrates them by pressing a button, then generate mobile applications for different platforms, hope this won't take long.

How to add a J2ME 3rd party library into your NetBeans project

2010-01-21T21:53:00.008-05:00

Scenario
You need to port our J2ME application to a specific platform, either Nokia, Samsung or LG, etc. You have to use OEM's specific API to implement a feature, but that special API is not included J2ME standard library, nor included in the OEM's J2ME SDK. You got an ZIP or jar file of the API library, but mostly it they are just stub classes, the only real implementations are inside the actual devices. What you need to do is adding the library in your project class path, but without including them into the J2ME jar file.

Solution
In NetBeans IDE, You can add the libray of zip/jar file in your J2ME project without including in the packaged jar file. Here are the steps:

Select your project, right click mouse, choose "Properties"
Select Build - Libraries & Resources, click button "Add Jar/Zip ...", then select the libray file you want to add, press "Open"
You will see the library file is included the file/folder list, pleases make sure uncheck the "Package" checkbox, this is the most important step, it will tell IDE don't include this jar/zip file into your J2ME jar file, or IDE will not build successfully, because it can not find dependencies packages for the libraries. The screen shot is attached at the end of this article.
Select OK, and build your J2ME application

Java ME SDK 3.0 review

2010-01-21T21:43:00.000-05:00

Last week I spent some time on Java ME SDK 3.0. One of my application need to support both landscape mode and portrait mode, currently J2ME wireless toolkit does not support the feature of changing the screen orientation, I heard that Sun's new JavaME SDK V3.0 emulator support changing the screen orientation, so I installed it and hope it will help my work. but unfortunately I feel disappointed.

Issue of integration with NetBeans IDE
After installing it successfully on my windows machine, I tried to add the SDK into My NetBeans 6.1 IDE, and then later I realize that the all the tools and Prefrence setting of the SDK 3.0 are disabled in my NetBeans IDE. which means you can not profile, clear database, change emulator settings inside the NetBeans IDE.
At first I thought maybe due to my NetBeans version is old, so I installed the Netbeans 6.8, which integrates the Java ME SDK 3.0, but I found the same thing; and I also tried the NetBeans 6.5.1, it is still not working. I also found this thread in a forum about the same issue I found in NetBeans 6.7. I don't know when Sun will fix this issue, but I do know that if this issue is not fixed, then few developer will use the new SDK, since so many mobile developers are tied with the NetBeans IDE.

Lack of the options to launch the emulator
For the emulator, I need a scenario that my application launched in a landscape mode, but I did not find there is any setting in the SDK allow you to do that, it always start as portrait mode.While Nokia S60 3rd SDK already supports this feature. Sun should provide this feature both in GUI menu settings and command line.

Failed to debug
I tried to debug my application by setting a break point using the SDK 3.0,I tried twice, but no luck.

No document to describe how to customize the emulator
Compare to WTK 2.5.x, the SDK 3 changed a lot for the emulator file structures, I realize it becomes much more complicated than before. For WTK 2.5.x, it is really easy to customize the emulator, I now how to change screen size, how to enable touch screen; but for SDK 3, I don't where I can start,I hope Sun could provide an official document to describe the architecure of the new emulator. I found this article about customizing the emulator in SDK 3 in Java ME SDK team blog, it might be useful.

Based on the reasons of above, I don't think the current version of the JavaME SDK can help me for my application development, so I have to change back to WTK 2.5, and use Nokia's S60 emulator, even if it is slow,but at least it works. It seems Sun has a long way to improve its new mobile SDK.

This artcile is based on windows version, I haven't touch the Mac version and Linux version yet.

Related links:

Ruby and AOP

2010-01-17T21:33:00.004-05:00

Right now I am learning Ruby, I focus on Dave Thomas' famous pickaxe Ruby book, last week I just finished the chapter 24: "Metaprogramming".Now I learn the power of Ruby Metaprogramming, right now it is still quite difficult for me, I am sure I will read this chapter over and over in future.

Today I want to discuss my impression of AOP concept in Ruby.
I learned AOP from java, in java you have to use AspectJ or Dynamic Proxy implement AOP. But comapre to Java, I realize Ruby support AOP in the lanuage level:

Classes are open: you add new method or attribute for any existing class, or replace the method with new one.
Module and Mixin: you can easily add new functionalities by Mixin the module.
MetaProgramming: you can dynamically define a method on the fly, using hook method as call back, easily implement before, after and around advice

I feel by using the above technologies, it is quite easy using AOP concept in Ruby, for example you can easily implement a Logging Aspect in a Module, then get it mixin in your application code.That is separating of concern, which is the essence of AOP.
So I will say AOP is built in support in Ruby language level.

Today I found a blog called "Does Ruby need AOP?", the author argue that Ruby metaprogramming is not AOP,because he think AOP is all about semantics, while Ruby metaprogramming is too low level. I feel this argument is too academic, from the pragmatic perspective, I think as long as Ruby can do the job of separation of concern, then it is AOP, then who care about the semantics?

Here are some useful links:

Customize Your J2ME Emulator

2010-01-10T19:38:00.013-05:00

Why do we need to customize J2ME emulator ?

As a J2ME mobile developer, you have to port application to different devices, while the J2ME default emulator only provide 4 different profiles, which can not meet our needs, and also some platforms like BlackBerry, Nokia S60 etc, the J2ME emulator they provided are really slow to launch, which is really inconvenient. If you can customize the emulator to support the device you are porting, which will improve your developing productivity.

Where are the emulator device profiles located?

For example, if you installed wireless toolkit 2.5 in c:\WTK25, then the device folder is:

C:\WTK25\wtklib\devices

The mobility package is included in the NetBeans 6.5 or later version, the wireless toolkit folder is like this:

C:\Program Files\NetBeans 6.x\mobility8\Java_ME_platform_SDK_x.x

Create a new emulator device profile

I like to choose Sun Wireless Toolkit 2.5, because its Qwerty Device emulator looks better. For example, if I want to create a device for Black Berry 9000:

1. Copy the entire folder of “QwertyDevice” to a new folder, rename the new folder name to “BlackBerry9000”

2. Go inside the folder, rename the Qwerty.properies file to BlackBerry9000.properties

Enable the touch screen

The touch screen feature is disabled for J2ME default emulators. If you want to enable it, you need to modify the .properties file. In properties file, find following statement:

touch_screen=false

Change false to true will enable the touch screen.

Change the screen size

For example, for BlackBerry 9000, its screen size is 480x320, then go to .prperties file, update the following values:

screenPaintableRegion.width=480

screenPaintableRegion.height=320

and ..
screen.width=480

screen.height=332

Please note that you need to make sure:

screen.height > screenPaintableRegion.height + screenPaintableRegion.y

In the properties file, screenPaintableRegion.y=11, so the screen.height value is 332.

If you don’t put the correct value, the emulator will throw exception once it is launched.

Make your customization available in NetBeans IDE

Once You finished the customization, you need to update your NetBeans IDE environment:

Select Tools – Java Platform, choose Sun wireless toolkit 2.5, click Refresh button
You will see the device names you added in the device folder

Conclusion

You can easily customize J2ME emulator by modifying the .properties file, even it is really simple, it can improve your productivity considerably.

Further steps

This article is only focused on Wireless Toolkit 2.5.x , Sun released the new wireless toolkit 3.0, the device emulators are changed a lot. I need to do more investigation on 3.0 emulator in future..

Book Review: Apprenticeship Patterns

2009-12-30T20:43:00.011-05:00

I finally finished reading the book “Apprenticeship Patterns – Guidance for the aspiring Software Craftsman”. It took me about 2 months to finish it, because I don’t have enough time to read it, usually I read it at waiting time for the Go train.

This book changes my strategy of my career path. If you want to become a master of software craftsman, then you should focus on improving your craft, not on promotion, not on money. This is a long road.

“sustainable motivations”
You need to keep motivate yourself, nurture your passion, beware don’t let yourself trapped into Golden Lock:" I'd like to learn something new ,but what I already know pays too well."

You need to find a better company, better job, team and co-worker. Love your job, do what your love, then money will follow. Money itself is not the goal, it is just the byproduct along the road to master. It gives me an insight that you should focus on finding the job that you truly love and passionate, even if it is low pay, because once you have the passion, you will generate creativity and energy, then it will generate the money for you. Another great example is Steve Jobs, in his commencement address to Standford’s class of 2005, he said: “You've got to find what you love “..., “Don't let the noise of others' opinions drown out your own inner voice. And most important, have the courage to follow your heart and intuition. They somehow already know what you truly want to become. Everything else is secondary.”

This book helps me reinforce the understanding of the patterns, the author draws quite a few maps, connecting different patterns, then I realize the importance of the map, because it reveals the connections and relationship of each pattern. It inspire me that pattern itself is important, but also the connections. We should not only focus on each pattern itself, but also their relationship to other patterns.
Those patterns seems simple, it give ms an insight that yourself can find your own pattern if you think deliberately.

My favorite patterns in the book are:

Be the worst
The white belt
Nurture your passion

My personal goal is finding the Tao of programming, which is the right way of software developing, so go from apprentice to master is a must path for me.

I have another blog about this book: My personal practice map.

The Evil of Java ME preprocess

2009-12-22T23:14:00.006-05:00

Developing Java ME application is painful, because of Java ME's own fragmentation in nature. Java Mobile developers have to deal with different devices, and each device or platform has different implementation. Writing a portable application to support different devices is really hard, and some time it is really a torture.

To solve the fragment issue, NetBeans IDE provides a solution - Preprocess, it is similar to pre-compile in C and C++. Actually preprocess in NetBeans is quite perfect, quite powerful. When I first use it, I feel it is so powerful, so easy to use. Several years later, when I look at our code, which have #if, #else every where, I just feel the code are so ugly, seems to return to the c code era. I really feel preprocess is an evil! Following are my reasons.

Violate OO Principles
Since preprocess is so easy to use, just add something like comments, it gives developer a backdoor to break OO principles easily and brutally. It is so easy to hack the program, so there is no need to care about abstraction, no need to care about design patterns. And the problem is my colleagues like to overuse preprocess, mostly even use preprocess for constants! I saw so many code full of magic numbers, and so many preprocess directive scattered every where to change those values.

It also violates OCP principle, OCP means "Open for extension, closed for modification". But preprocess is modifying the existing source code when you add new preprocess configuration. It is quite common that the change works for the new configuration, but will break the old ones.

I found if we use more design patterns deliberately, like strategy pattern, abstract factory pattern etc, and use constant class instead of magic number directly, we can minimize the preprocess greatly.

Hard to Refactor
Refactoring the preprocessed code is really difficult, for example when you rename a method name or variable name, you can only refactor the current active preprocessed code, because others are commented out. If in your project need 4 different devices type, so every time when you refactor, you have to test 4 times to make sure the change is working. It is really painful!

Hard to Reuse
If you want to reuse a class or a component to another project, but when you see so many preprocess scattered in the code, probably you will not use it. Either you have to add all those preprocess into your new project which might does not make any sense, or you have to manually remove them, which is even harder to test.

Hard to understand
When you read the code has so many #if #else statement, it is ugly to read, more difficult to understand, code is not clean.

Hard to maintain
When you want to port the current application to a new device, you have to understand how the previous preprocess work; when you want to fix a bug for specific device, you have to make sure it won't break the others. Manage so many preprocess configuration is really painful. So preprocess is good for short-term, but really painful for long term.

IDE Lock-in
Because of widely use of preprocess, our projects are locked in NetBeans IDE, we can not use other IDE, like Eclipse. For my experience, NetBeans IDE has some problems for example:

you can't use different source code folder for different configuration, you can easily do that in Eclipse MTJ;
Build automation is difficult, I don't understand the NetBeans generated build xml, I have to build inside the IDE, I never tried to run build file outside the IDE, and I don't know how create a customized ant build file, I think it is really hard;
Difficult for team work. NetBeans project use absolute path, and private project properties. Which means it works in your computer, but might not work in your co-workers environment, or you have to take a while to set up the environment correctly.

Conclusions
Java ME preprocess is easy for short-term, but painful for long term, it solves one issue by generating more issues in future; it delay your pain in a short term, but you will be more painful for long term.

Suggestions for anti-preprocess

Try to minimize the use of preprocess, we can't 100% percent remove the preprocess, but be ware to use it, only use it when no other options.
Have the discipline of writing clean code, readable code and maintainable code, before you use preprocess, think it carefully, if there is better solution;
Use OO design patterns, for example strategy pattern and abstract factory can remove lots of preprocess;
Try to use configuration file instead of preprocess, if we put the constant value into resource property file instead of hardcoded using preprocess, we can also make the code cleaner;
Centralize the preprocess, if you have to use preprocess, don't allow them scattering every where in the code, try to put those preprocess specific logic together, even better to put them into one class, it would be easier to read, understand and maintain;
DSL/Meta programming, if fragmentation is unavoidable, why not use code generation technology to generate source code for each specific platform for us? This needs compiler skills and Domain Specific Language knowledge, it is really an interesting topic.

My Personal Practices Map

2009-12-13T00:04:00.012-05:00

Currently I am reading the book called Apprenticeship Patterns – Guidance for the Aspring Software Craftsman. In page 85, the pattern “Reflect As You Work”, it mentions a technology called “Personal Practices Map”. You can use this technology to analyze and reflect your idea and what you did.

This idea originally comes from London’s Extreme Tuesday Club. The page "Maps of People’s Personal Practices".

This is my Personal Practices Map.

Using map to reflect ideas reminds me the book of GoF design patterns. In that book, the authors draw a map of all 23 patterns and their relationships. The book Apprenticeship patterns draw lots of map to connect each pattern.

This is a great technology. In future I will draw more maps:

“Draw your own map” (Page 47), draw a map of your career path.
Draw the map of reading list

Agile meeting notes at Toronto JUG Dec 1, 2009

2009-12-05T23:40:00.002-05:00

I attended the Toronto JUG meeting on December 1. This time it is about Agile, the presentato is Jack Milunsky from agilebuddy.com. His topic covered agile, SCURM and lean.

I put my notes here:

Adopting agile is hard

Agile requires both bottom up and top down.

Applying agile practices is easy, but the hard part is new mindset and new philosophy, which is quite different with traditional waterfall methodology

Agile/Scrum is pervasive; it involves all different teams to change

Lean software technology

Jack spent lots of time on Lean.

Lean’s two pillars:

1. Just-In-Time Flow

Eliminate inventory,

Do things in batches, which is similar with SCRUM

2. Autonomation:

Stop the line,

Built-in reflex

Just In Time Flow

- inventory is waste => partial done software

- Moving in Just in time

Uncover waste

Focus on overall concept

Focus on value

7 principles of lean software development

1. Eliminate waste

Any partial done software is waste

2. Build Quality in

don’t test it in, prevent defect in the first place

avoid creating defects

stop the line, automation, unit test

3. create knowledge

knowledge create process

don’t lock down

4. Defer commitment

“Planning is good, plans are bad”

5. Deliver fast

Lean focus on time

Don’t equate fast with hacking

Expose issues

Build quality in

Quality is the first priority

6. Respect people

Self organizing team

Trust over command and control

7. Optimize the whole

Mindset is important, not just practice

Transfer ownership

Focus on the whole value stream, the big picture

Focus on value, eliminate waste

Waste

· Extra features

Justify every feature, limit features

Extra features are waste, > 60% are extra features.

· Relearning

Task switching drops the productivity

Undocumented code

No wiki

· Handoff

Transition point are waste point, knowledge degrades

Delays

· Defects

Increasing speed helps the absence of waste

Cycle Time = things in process / average completion rate

How to reduce the cycle time:

- minimize the number of things in process

- minimize the size of the things

- Recommend small batches, which is similar as sprint in SCRUM

80% rule: if the task load is over 80%, the cost, defect rate increase dramatically

Agile in AgileBuddy

Agile buddy spends 20% time for refactoring, (which is very good, they know how important refactoring is, they do it deliberately, very nice!)

Discipline: set the dial to 11( 80% rule)

Sprint length: 2 weeks

My thoughts

After the meeting, I talked to Jack, and we exchanged the idea, I believe that SCRUM mainly focuses on management side, it only focuses on what to do, but does not cover how to do, if you look at the SCRUM, there is no TDD, refactoring, pair programming, etc. I always have the question: do they need to be added in the SCRUM sprint practices? How? So I think SCRUM only is not enough, SCRUM should work with XP. So I asked Jack his idea about SCRUM vs. XP. He replied me that SCRUM only definitely not enough, it has to work with XP, because some of the XP practices are really fundamental, like TDD, refactoring. That what they are doing in agilebuddy.

He also suggest me to organize a lunch and learn meeting, just watch this video by Ken Schwaber about SCRUM..This meeting let think more about agile.Thanks Jack.

You can visit Jack's blog at here.

Code Kata: SuDoKu Solver

2009-11-26T22:41:00.027-05:00

This week I am interested in the Sudoku puzzle, I am thinking how to use program to solve the SuDoKu. I almost spent a week for it. At first I was inspired the a book called “Wicked Cool Ruby Scripts”in chapter 5, it provides a ruby source code for the SuDoKu solver. I spent two days to understand the code. The problem is the book does not describe the algorithm in details, and I don’t know how to debug the ruby code in TextMate. So it took me a longer time to finally understand the whole logic.

Actually it uses the brutal backtrack method. Try fill a number in one cell, then recursively call itself, if it works, continue, then return, try another, until it fills all the 81 cells. I made some changes based on the original code, the major changes once the solver find the solution then exit the loop, while in the original code the program will continue loop until the end, I compare my changes with the original code, the total times executed and execution time both drops about 50%.

Here is my modified code:

The logic is really simple: first reads the puzzle, construct the one dimension array.
The key is in Solver method:
From 81 cells, pick one empty cell, calculate the candidate numbers of this cell,
Pick one number, fill the cell,
Recusively call itself, continue try to fill another empty cell.
If the program could not find the candidate for the cell, or the program iterate all the cell’s candidate number, then erase the number, backtrack;
If the program fills all the 81 cells, return true

The algorithm of getCanidates(j) is easy:
Build an array of 1-9
Find each cell with the same row, erase the number in the cell from the array;
Find each cell with the same column, erase the number in the cell from the array;
Find each cell with the same sub block, erase the number in the cell from the array.
The left in the array are candidates

From this code, I understand more about the backtrack and recursive method. This code belongs to the depth-first backtrack search.

Donald Knuth’s Dancing Link algorithm is quite interesting, there are lots of articles and blog about using it solving the SuDoKu. But right now for me it is really hard to understand.

For SuDoKu, there are following thins to do in future:
1. Implement the SuDoKu solver using Dancing Link algorithm
2. Find out the algorithm of SuDoKu Generator, like how to generate a puzzle based on difficulty, and how to find out there is only one solution.
3. How to implement an algorithm that simulate human's solving logic?

Here are several link about dancing link and SoDuKu:
Dancing Link C implementation
Dancing link for Java
Donald Knuth Home page
Donald Knuth's Dancing links paper(pdf)

code Kata 3, How many n-digit binary numbers are there that don't have two adjacent 1 bits?

2009-11-13T22:34:00.001-05:00

Today I read a new Kata from Dave's CodeKataCodeKata fifteen: A diversion

The Question:
Think of binary numbers: sequences of 0's and 1's. How many n-digit binary numbers are there that don't have two adjacent 1 bits? For example, for three-digit numbers, five of the possible eight combinations meet the criteria: 000, 001, 010, ~~011~~, 100, 101, ~~110~~, ~~111~~. What is the number for sequences of length 4, 5, 10, n?

Having worked out the pattern, there's a second part to the question: can you prove why that relationship exists?

The answer is in the replied comments, it is a fibonacci series:
f(1) = 2 => 0,1
f(2) = 3 => 00, 01, 10
f(n) = f(n-1) + f(n-2)

Now I will try to explain why it is like this using my own thoughts:
f(3) can be thought of 2 groups, one is begin with 0, and the other is begin with 1
begin with 0 group: 0 + xx, => f(2)
begin with 1 group: since it begins with 1, it must be followed by 0, so it becomes 10 + x , =>f(1)

so f(3) = f(2) + f(1)

same thing with f(n)
begins with 0: 0 + xxx...x ( n-1 digit) => f(n-1)
begins with 1: 10 + xxx..x ( n-2 digit) => f(n-2)

so f(n) = f(n-1) + f(n-2)

Now I will give the solution in Ruby:


def non_adjacent_one_array(n)
if(n == 1)
["0", "1"]
elsif (n== 2)
["00", "01", "10"]
else
array1 = non_adjacent_one_array(n-1).collect { |i| "0"+i}
array2 = non_adjacent_one_array(n-2).collect { |i| "10"+i}
array1+array2   
end
end

s=non_adjacent_one_array(5)
puts "n=5, number is #{s.length}\n"
puts s
puts "\n"

1.upto(10) do |i|
s = non_adjacent_one_array(i)
puts "n=#{i}  number=#{s.length}\n"
end

And the results are:

n=5, number is 13
00000
00001
00010
00100
00101
01000
01001
01010
10000
10001
10010
10100
10101

n=1 number=2
n=2 number=3
n=3 number=5
n=4 number=8
n=5 number=13
n=6 number=21
n=7 number=34
n=8 number=55
n=9 number=89
n=10 number=144

Build an iPhone application without Interface Builder

2009-11-12T23:14:00.006-05:00

I am still a beginner of iPhone developer. I finished a tutorial book. Right now I am trying to read another one. But I found my problem: I only know how to build the application using the Interface Builder, but I don’t know the underlying mechanism of the application, for example; what is the life cycle of the application, how does the application delegate, ViewController and View interact each other?

Using Interface Builder makes the application developing very easy, it separate out the presentation logic with the application logic. I like this idea, but for beginner only know the Interface Builder is not enough:
- It makes your job easier, but also hides the application information, and blocks you to understand the underlying mechanism. For example, When I investigate the sample code LocateMe, it uses UITabBarController and NavigationgController for each tab, when I read the source code, I don’t know the where the variable navigationController come from, I don’t know how it created. It is set in the nib file, and you have to read the reference document to see how it works.

- Sometime you need to customize your app, Interface Builder will not help you, you have to do it programmatically. For example, I tried to add a UIScrollView using the interface builder, but I found it does not scroll as I expected.

I google the solution and found this video is helpful:

Building iPhone Applications without Interface Builder from Troy Mcilvena on Vimeo.

And this blog:Why would you use Interface Builder for iPhone Development?

I just summarize the detailed steps:

1. In Xcode wizard, choose Window-based Application, then delete the MainWindow.xib, remove the property with the key ‘Main nib file base name’ (the raw key name is ‘NSMainNibFile’) from your Info.plist file.

2. In main.m, add your AppDelegate class name in the last argument of the UIApplicationMain method.

int main(int argc, char *argv[]) {

NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
int retVal = UIApplicationMain(argc, argv, nil, @"NoNibAppDelegate");
[pool release];
return retVal;
}

3. In AppDelegate class, initialize and configure the window and Controller object.

For example: NoNibAppDelegate.m

- (void)applicationDidFinishLaunching:(UIApplication *)application {

// Override point for customization after application launch
window = [[UIWindow alloc] initWithFrame:[ [UIScreen mainScreen] bounds]];

RootViewController *root = [[RootViewController alloc] init];
[window addSubview:root.view];
[window makeKeyAndVisible];
}

4. In your ViewController class, implement the loadView() method
From UIViewController Reference document: If you create your views manually, you must override this method and use it to create your views

- (void)loadView
{

UIView *contentView = [[ButtonView alloc] initWithFrame:[[UIScreen mainScreen] applicationFrame]];

self.view = contentView;
[contentView release];

}

5. In View class, implement the initWithFrame: method
“If you create a view object programmatically, this method is the designated initializer for the UIView class.” - From UIView reference document.

- (id)initWithFrame:(CGRect)frame {
if (self = [super initWithFrame:frame]) {

self.backgroundColor = [UIColor lightGrayColor];

UILabel *label = [[UILabel alloc] initWithFrame:CGRectMake(0.0f, 0.0f, 320.0f, 30.0f)];
label.text = @"Hello World";
label.center = self.center;
label.backgroundColor = [UIColor clearColor];
label.textAlignment = UITextAlignmentCenter;

[self addSubview:label];
}
return self;
}

6. You might want to register an UIControl event listener:
- Intitialize the UIControl object, for example a UIButton:

[buttonPress addTarget:self action:@selector(buttonClicked:) forControlEvents:UIControlEventTouchUpInside];

- implement the event call back method:

-(void)buttonClicked:(id)sender
{
UIButton *button = (UIButton *)sender;
NSString *text = button.currentTitle;

NSString *string = [ [NSString alloc] initWithFormat:@"Button %@ pressed.",text];

NSLog(@"button clicked. button title:%@", text);

self.label.text = string;
[string release];

}

You can download my sample code from here.

From this investigation, I understand the iPhone application life cycle and event flow; know the underlying mechanism and get the confidence to customize application in future.

For learning iPhone, I am trying to follow the Shu-Ha-Ri learning model: at the Shu level, which is called following and copying, so I just follow one book: Beginning iPhone 3 Development: Exploring the iPhone SDK. I just follow step by step, copy every line of the code from the book. This is a very good book for the beginner who never touch the apple development environment like me, but it does not cover the deeper information, which I am really want to understand now.

So I think it is time for me to go to Ha level: which is breaking, it means trying to collecting different information and trying to read different article and books, Now I learned that it is time for me and I am ready to read Apple reference document, read advanced iPhone books, investigate the sample code, joining the forum, and practice.

Tutorial gives you a starting point, Interface Builder is a crutch, it is time for you to remove them if you want to get improved.

Here is my reading list for iPhone in fufure:
- Apple iPhone reference documents and API documents
- Apple iPhone sample codes
- The iPhone Developer's Cookbook: Building Applications with the iPhone SDK
This is a really good book, but not for beginner, it tells your more detailed information which is not covered in other books. I am waiting for the 2nd edtion.

References:
- iPhone Application Programming Guide

- The iPhone Developer's Cookbook: Building Applications with the iPhone SDK

Code Kata 2 - Reverse words in a string using Ruby

2009-11-08T22:55:00.004-05:00

Last time I posted the blog Code Kata - Reverse words in a string, in that blog I provided the java solution. Today I will try to solve this problem using Ruby.

Question: How to reverse words in a string?
For example, given a string "The quick brown fox jumped over the lazy dog", reversed string needs to be “dog lazy the over jumped fox brown quick The”.
And this time there is no memory constraint, we just focus on providing an elegant and concise code.

The algorithm has two steps:
1. First reverse the entire string by character
For example: "The quick brown fox jumped over the lazy dog" becomes
“god yzal eht revo depmuj xof nworb kciuq ehT”
2. Then reverse each word by character
For example: “god yzal eht revo depmuj xof nworb kciuq ehT” becomes
“dog lazy the over jumped fox brown quick The”

The ruby code would be:

The method reverse_string() will return an array instead of a string, but it does not matter here. You can see how easy to implement using Ruby compare to Java!

My Presentation: The OO design principles

2009-11-04T23:34:00.007-05:00

The OO Design Principles

View more presentations from Steve Zhang.

This is my presentation about OO software design principles: http://www.slideshare.net/intellizhang/the-oo-design-principles

The whole slide is copied from the following two books:
1. Agile Software Development: Principles, Patterns, and Practices, by Uncle Bob

2. The Pragmatic Programmer: From Journeyman to Master, by Dave Thomas and Andy Hunt

Code Kata – Reverse words in a string

2009-10-26T22:02:00.011-04:00

Dave Thomas creates a website called Code Kata, Who encourage developers improve their skills by keep practicing coding, trying to find better solutions for some questions.

Today I met an interesting question: How to reverse words in a string?
For example, given a string "The quick brown fox jumped over the lazy dog", reversed string needs to be “dog lazy the over jumped fox brown quick The”. The thing is you can not use extra memory to do it, you had better manipulate the reverse logic in the original text string.

To solve this question, I remembered I read a good book several years ago, it is called Programming Pearls ,written by John Bentley.

In chapter 2, section “The power of Primitive”, the author gives the solution of transform an array vector, which is clean, elegant and efficient. I used this solution to solve this problem.
The algorithm has two steps:
1. First reverse the entire string by character
For example: "The quick brown fox jumped over the lazy dog" becomes
“god yzal eht revo depmuj xof nworb kciuq ehT”
2. Then reverse each word by character
For example: “god yzal eht revo depmuj xof nworb kciuq ehT” becomes
“dog lazy the over jumped fox brown quick The”

You see it is pretty easy, right? No need extra memeory, no need substring.This is the power of the algorithm.

Here I post the java version of the solution, you can easily implement this algorithm in C or other language.

public class StringRerverse {

private int wordStartIndex;
private int wordEndIndex;
private StringBuffer sBuffer;

String reverse(String string)
{

this.sBuffer = new StringBuffer(string);

sBuffer.reverse();

int size = sBuffer.length();
int pos = 0;

while(pos < size)
{
if(findNextWord(pos) == true)
{
reverseWord(sBuffer,wordStartIndex,wordEndIndex);
pos = wordEndIndex+1;
}
else
{
break;
}
}

return sBuffer.toString();

}
boolean findNextWord(int startFrom)
{
int start = startFrom;

while (start< sBuffer.length()-1 && sBuffer.charAt(start) == ' ')
{
start++;
}
if(start == sBuffer.length() -1 )
{
return false;
}

wordStartIndex = start;

int end = start+1;
while(end < sBuffer.length()-1 && sBuffer.charAt(end) != ' ' )
{
end++;
}
if(end == sBuffer.length() -1 )
{
wordEndIndex = end;
}
else
{
wordEndIndex= end-1;
}
return true;
}

void reverseWord(StringBuffer sBuffer,int wordStartIndex, int wordEndIndex)
{
int length = wordEndIndex - wordStartIndex + 1;
for(int j = 0; j<length/2; j++)
{

int startIndex = wordStartIndex + j;
int endIndex = wordEndIndex - j;
char c = sBuffer.charAt(startIndex);
sBuffer.setCharAt(startIndex, sBuffer.charAt(endIndex));
sBuffer.setCharAt(endIndex, c);
}
}

public static void main(String[] args)
{
StringRerverse reverse = new StringRerverse();

String string = new String("The quick brown fox jumped over the lazy dog");
String reverseString = reverse.reverse(string);
System.out.println("reversed String: "+reverseString);

}
}

From Good to Great Developer

2009-10-25T15:01:00.006-04:00

Last week I watch a presentation called “From good to great developer“ from www.infoq.com.
Here is the link : http://www.infoq.com/presentations/Good-to-Great-Developer-Chris-Hedgate

Chris Hedgate made a really great presentation. He pointed out that good developer writes code quickly but hard to maintain; but great developer has long term goal and try to write clean code and make it easier to maintain.

He also gave the advices how to become a great developer. I really like the 4-stage learning model he introduced: unconscious incompetence, conscious incompetence, conscious competence and unconscious competence. The learner in different stage will take different actions: inspiration, training, practice and reflection.

Talking about becoming a great developer, which is the same idea coming from Agile. It means craftsmanship, professionalism, discipline, and mastery. I think every serious developer should have a goal to become a great developer or master programmer, just like kong-fu master or samurai. It is a whole life’s practice.

Being a great developer is not an easy job. You need to understand the agile principles, understand design patterns, have disciplines and patience to write clean codes and keep refactoring; you have to constantly reflect yourself; you need to keep learning new technologies. And you need to help other people and try to motivate and inspire them. This is raising the bar, it is hard, but this is the only way to create a remarkable career for you, the only way to give you a passion, and the only way to transcend yourself.

Here I add some other videos which share the same philosophy:

1. Craftsmanship and Ethics, by Robert Martin:
http://www.infoq.com/presentations/craftmanship-ethics

2. Developing Expertise: Herding Racehorses, Racing Sheep, by Dave Thomas
http://www.infoq.com/presentations/Developing-Expertise-Dave-Thomas

Dave Thomas is the author of “Pragmatic Programmer, from journeyman to master”. This video talks about the Dreyfus model, which is 5-stage learning model. The details about it is in the Book “Pragmatic learning and thinking” by Andy Hunt.

3. I Come to Bury Agile, Not to Praise It, by Alistar CockBurn
http://www.infoq.com/presentations/cockburn-bury-not-praise-agile
This video talks about a Shu-Ha-Ri, a 3-stage learning model, I first learned Shu-Ha-Ri from his book “Agile Software Development”

4. Deliberate Practice in Software Development, by Marry Poppendieck,
http://www.infoq.com/presentations/poppendieck-deliberate-practice-in-software-development
Marry Poppendieck is regarded the first person who apply Toyota’s Lean technology into software development, this video she provide the 10-year rule theory, emphasizes the craftsmanship and clean code.

5. Productive Programmer: On the Lam from the Furniture Police, by Neal Ford
http://library.theserverside.com/detail/RES/1242309506_447.html?asrc=vcatssc_sitepost_05_14_09
The author of “Productive programmer”, he talked about the principles in his book, the book of “Peopleware” and “pragmatic learning and thinking”

6. Transcendence and Passing Through the Gate, by Dave West
http://www.infoq.com/presentations/transcendence-gate-dave-west

This video is really interesting, it talks about agile, but he use the concept of Zen. He mentioned the famous 10 Bulls pictures which describe 10 different stages of a human being to describe the agile development.

Learning from the Book Of Five Rings

2009-10-12T16:58:00.004-04:00

Miamoto Musashi was a very famous Japanese saumari, who wrote a book called The book of Five Rings. I first heard of him by Alistir CockBurn’s book – Agile Software Devleopment.

The good thing is Alistair pointed out what we can learn from his book into our software development. Here is what Alistair pointed out:

• Do not develop an attachment to any one weapon or anyone school of fighting.
Use the rang of them without getting stuck in anyone.
• Practice and observe reflectively.
Practice hard, win big (pointed out by Jeff Sutherland, the SCRUM creator)
• Win
Pay more attention to winning than to looking good,
“Do not do anything useless”

Here I also would like to copy what his rules for learning the art ( from page 41):
1. Think of what is right and true.
2. Practice and cultivate the science.
3. Become acquainted with the arts.
4. Know the principles of the crafts.
5. Understand the harm and benefit in everything.
6. Learn to see everything accurately.
7. Become aware of what is not obvious.
8. Be careful even in small matters.
9. Do not do anything useless.

The interesting thing is that in the book, Mushasi use carpentry as a metaphor for mastering his science of martial art. (from page 9, Likening the Science of Martial Arts to Carpentry). It seems every word can be used to software development, used as principles of mastering programmer's skills.

In future I will post more about this book.

My vision of a good software developing team

2009-10-12T15:54:00.004-04:00

We need to do things in the right way. In Chinese, “way” means “TAO”. If we follow the TAO of the software development, then we can deliver the great product in a short time.

Following are my vision for a good software developing team, and what the TAO should be:

- Follow good design
We need follow good OO design principles, and good design patterns. A good design is a foundation of a software product, and a good design is the DNA of a good product.

- Build Quality in
We need to balance between long-term benefits with short term benefits
Our software should be: flexibility, maintainability, and readability.

- Write clean code, constantly keep refactoring, continuously improving design
I like Uncle Bob’s saying” If code is messy, so does the product”.
Bad code slows down the productivity
Only one way to go fast is going well, writing clean code.

- Design for maintenance
The only way to develop maintainable code is to design the maintainability at design phase

- Deliver DRY project
DRY means “Don’t Repeat Yourself”.
Here I would like to mention the definition of what does mean a project is Done:
1. Deliver the product to our customer
2. Developers improve their skills;
3. Team gain knowledge;
4. The design and code is easier for future project
I think all these 4 things together means a project is done, but currently we only focus on No. 1, which cause on the problem in future: since the code is not reusable, we have to rebuild the entire app, or we have to duplicate all the old code, which either by fixing duplicating bugs or by spending long time to fix the issues by adding new codes. Finally we spend more time and money by reinventing the wheels.

- Test Driven Development
We need are not only use JUnit, but also to make sure our design is easier for unit test, our framework is easier to using unit test.

- Build automation
Our build automation should support everywhere, server side code, client side code, and test code.
Everything we need should be checked in the repository, including dependent libraries, config files, property files, database script, build tools, etc.

- Use static analysis tools
Currently there a lots of open source static analysis tools. For example they can find the potential bugs; tell you how complex of your code, how messy your code is, and tell you the coverage of your test code, etc. which will help us improving our code quality greatly

- Foster knowledge sharing and internal training
We need to build an environment which can foster knowledge management in our company, which is called ‘ba’ in “Lean technology”. We need to use Wiki, internal forum, internal training to share the knowledge for all the developers.
Knowledge management is very important

- Following the Agile
We need not only do the agile practices, but need also fully understand the agile principles and its values.
And each team needs to use different agile methods.
Developer needs to follow XP;
Manager team needs to follow Scrum
The whole company needs to follow Lean technology

- Training developer to be professionalism
We need our developers have the concept of:
Professionalism, craftsmanship, discipline, ethics, etc.

Next I want to discuss more about the gap between the manager and developer. I think there is a role we are missing – “coach”. I think a good coach will be a bridge connecting developers and managers, his role is filling the gap.
I would like to make a metaphor: thinking our software team as a sports team- whether basketball team or a hokey team. In a sports team a coach plays a very important role:
- Design the wining strategy for the team
- Instruct each player not only what to do but also how to do;
- start timeout when he found the issue during the game, either by substituting a new player or set up a new strategy;
- Training players and improve their skills;
- Motivate players and find their potentials;

So my suggestion is we need a coach or several coaches in our development team.

Be prepared to leave

2009-10-02T19:25:00.007-04:00

I was totally motivated by the book -The passionate programmer. Now I am trying to copy the author’s idea and write an essay using his style, the name is called “Be prepared to leave”.

Usually people start to update their resume when they want to find a new job, then they realize their problems: they found it is hard to review what they did in the past few years; they realize their skills might be outdated; they don’t know where to go, just randomly submit their resumes. Every thing seems to be by accident, not deliberately.

So how can we plan our career deliberately? My answer is be prepared to leave. If you have this mind set, you have a positive attitude of doing your daily jobs. Suppose you are going to leave or be fired, then it force you to update your resume regularly, to update your resume, it forces you to review yourself constantly:

• For every project you finished, you ask yourself: how did I finish it, what kind of skills I used, how can I add a good achievement in my resume? How can I improve it? How can I use the current new technology into my project, so that I can improve my work and also I can add it into my resume? If you summarize and review your work constantly, you will find you have many good points to put into your resume.

• Find your blind spots. Since you are going to leave, you need to take chance trying to learn what you don’t know. Think it proactively, when you find your blind spot, then try to fix it as soon as possible. When you find you have a challenge to do some job, then it is probably a blind spot for you, and it is a good opportunity for you to get improved. For example you found it is very painful for you to deploy or build a project, because it needs you manually do many different steps and easy to make error. So why not improve the build script and improve the build automation process? If you have this attitude, you will find every job is interesting, even it is a maintain job.

• Research the market regularly, try to find the skills the market need and what skills you need to improve and learn; try to find the company you want to go and find out what kind of technologies they are using, then try to learn it, try to use them into your current project.

In summary when you have a “be prepared to leave” attitude, you will not find your daily job is boring any more, you will find many opportunities to improve yourself. Sooner or later your skills and thoughts will go to next level, and you will be easily got a good new job when chances come.

IPhone development: memory management rules

2009-10-02T14:27:00.005-04:00

Right now I am learning the IPhone development using objective-C, I have to admit that I am a new beginner of objective-C, even if I have lots of experience of C/C++ development, the syntax of object-C confuse me a lot.

Right now I run into a problem: when should we release the object? Here is the example:

YellowViewController *yellowController =
[ [YellowViewController alloc] initWithNibName:@"YellowView"
bundle:nil];
self.yellowViewController = yellowController;
[yellowController release];

It seems an idiom for objective-c memory management, since I’ve seen lots of source code did in the similar way. I was really confused, since I treated the objective-c object as a c++ pointer. In the above example, yellowController and self.yellowViewController are pointers, they are pointing to the same memory address, if we release yellowController, then how about self.yellowViewController? Will it become an invalid object because it is pointing to the memory address which was released already?

Through me research, then I understand the memory management rule of objective-c. It uses reference counting method to manage the memory:
- When create an object using alloc, the reference count will be 1;
- When release method is called, decreases reference counter by 1;
- When copy or retain method is called, increases the reference counter by 1;

Then I have a question: how about the assignment operation? What does it exactly do? Will it increase the reference counter or will it copy the whole object data to the variable self.yellowViewController?

Finally I realize that the key is in the definition of the instance variable self.yellowViewController:
It’s definition is like this:
@property (retain, nonatomic) YellowViewController *yellowViewController;

Which means we do the assignment operation to the class variable self.yellowViewController, the retain method will be called, which means the reference counter will be increased by 1.

We can use the following methods to check the reference counter:
NSLog([NSString stringWithFormat:@"%u", [yellowController retainCount]]);
self. yellowViewController = yellowController;
NSLog([NSString stringWithFormat:@"%u", [yellowController retainCount]]);

In summary, here is the simple rules for object-c memory management:

Retention Count rules: (copy from here)
1. Within a given block, the use of -copy, -alloc and -retain should equal the use of -release and -autorelease.
2. Objects created using convenience constructors (e.g. NSString's stringWithString) are considered autoreleased.
3. Implement a -dealloc method to release the instance variables you own

Some good references:
1. Very simple rules for memory management in Cocoa
2. Learn Objective-C
3. Forum thread: newbie memory management question ("release")