Saturday, February 18, 2012

Null analysis for fields in Eclipse JDT withdrawn for Juno

Null analysis for fields with Eclipse JDT , introduced in Juno M5, has been withdrawn and will not be available in the Juno release. Only Juno M5 will contain this feature. The reason for withdrawal is the fact that becuase of implementation constraints, it could not support analysis for alien field references i.e. non-static fields of objects other than 'this'. We will rework the implementation and come back with a full fledged support for fields, including annotations to support null analysis for fields. See https://bugs.eclipse.org/bugs/show_bug.cgi?id=247564#c176 for more details.



Apologies to those who were looking forward to a complete null annotations support for fields as well. We are considering making the current implementation of the feature available as a plugin patch. I will keep you posted.

Also, even when using annotations for method return and parameters, be careful in dealing with fields because we don't warn there. Eg:

class A{
     Object o = null;

     void goo(@NonNull Object param) {}
     @NonNull Object foo() {
            goo(this.o);
            return this.o;
       }
}

Monday, January 30, 2012

Null Analysis for Fields with Eclipse JDT

It is common knowledge that Eclipse has an intra-procedural null analysis capability, augmented recently with null annotations for inter-procedural analysis in Juno M4. However, a big flaw and pain point till now was that the anlsysis would only work for local variables/parameters, and NOT for fields! Strange, innit? Well, a bunch of problems with handling fields prevented the initial null analysis implementation to be generic enough to handle fields as well. Access from different threads, through different objects, initialization of even final fields in different constructors were some of the roadblocks because of which this work was put on the backburner. However, even though we haven't found a holy grail to tackle these problems, we've come out with atleast a basic implementation of null analysis for fields! Rejoice!

A. The Problem
Prior to Juno M5, the following code snippet would raise null warnings only on the local variable, but not on the field.


B. The Solution
See the Juno M5 new and noteworthy page for the new null analysis for fields.
Note that for non-constant fields, we only raise "potential NPE" warnings even if the field has been assigned null or compared against null because we assume that there's always a chance of another thread modifying it between the assignment/comaparison and the actual reference. So consider the following snippet


C. Not quite there yet?
There are a few limitations with the current null analysis for fields:
  • Yet to be done for final fields that are initialized at the time of declaration. This will be done with the fix for bug 237236 soon.
  • The analysis only works for non-static fields of the current object or static fields of the current type ONLY. That means, in the following code snippet. you get null warnings on direct access of field1 or access via "this", but not if field1 is accessed via another object 'test'.
class Test{
   public Object field1;

   void foo(Test test) {
      if (field1 == null) { 
         System.out.println( field1 .toString());  // "potential" NPE warning
     } else {
        // do something
     }
     System.out.println(field1.toString());  // Potential NPE warning here

     if (this.field1 == null) { 
         System.out.println( this.field1 .toString());  // "potential" NPE warning
     } else {
        // do something
     }
     System.out.println(this.field1.toString());  // Potential NPE warning here
    // access through object 'test' will raise no warnings. See below

     if (test.field1 == null) { 
         System.out.println( test.field1 .toString());  // no warning
     } else {
        // do something
     }
     System.out.println(test.field1.toString());  // no warning


  }
}

This is a limitation of the current code analysis infrastructure and we hope to address this sometime soon. The soon to be released null annotations support for fields will greatly assuage this problem in the short term.

Please do start using this new feature. Even with its limitations, its quite useful and we've found a long list of unsafe code patterns in the Eclipse SDK itself. Deepak's post here presents one such example


Apart from this, Juno M5 has more interesting items. See my previous post on pre-built indexes, Stephan's post on Resource leak warnings and the Juno M5 new and noteworthy page for more.

Stay tuned!



Thursday, January 26, 2012

Faster Java search using pre-built indexes with Eclipse JDT

JDT indexes referenced libraries (or JARs) in your projects for use in the Java search. However, in most cases these libraries seldom change, and the indexing may be costly and take time when you invoke search for the first time after adding the JAR on the buildpath.

From 3.8M5 onwards, you can avoid this redundant indexing by providing a pre-built index for each classpath entry in the classpath container. This feature was recently implemented in bug 356620.

So how do you generate the indexes in the first place?
1. Create the JAR
2. Go to Run Configurations>Eclipse Application. Give a name to the configuration, viz. indexer, and in 'Main' tab choose Run an application, and use the list to select org.eclipse.jdt.core.JavaIndexer

3. Go to the 'Arguments' tab and use the arguments -option <indexFileLocation> <jarFileLocation> in addition to the existing ones.

4. The index file will be generated at the specified location.

Now when the JAR in question is being added to the build path of a project, the classpath containers can add the index location to classpath, which will look as follows:

<classpathentry kind="lib" path="C:/Users/IBM_ADMIN/Desktop/TestIndex.jar">
<attributes>
<attribute name="javadoc_location" value="file://C://abc"/>
</attributes>
</classpathentry>

Voila! JDT will no longer index the JAR and even the first search will be lightning fast!

Note: This feature is mainly intended for plug-ins that implement their own classpath container, not for the end user.

Tuesday, December 6, 2011

Inter-procedural Null analysis using annotations in Eclipse JDT

The JDT team is extremely excited with the release of the new annotation based null analysis. Before writing anything about it I want to thank Stephan Herrmann (JDT committer from Germany) for all the hard work he has put in on this feature! Feel free to drop him a word of appreciation on stephan AT cs.tu-berlin.de

What is JDT's annotation based null analysis?
To give a bit of a background, you must've noticed that JDT java compiler does some static analysis and based on that warns if you deference a variable that has been analysed to be (potentially) null at runtime, or compare a variable against null when its nullness at runtime can be determined at compile time.

[See line numbers to locate error messages]
However, the warnings are only generated for local variables inside a method. There's no way to find out the change of nullness due to a method call, or for parameters of a method. Consider the example on the left. o1 is a parameter and since we don't know how foo(..) will be called and what value will be passed to o1, we can't do any analysis on it unless its state definitely changes inside foo(..). Also note that even though o1 is assigned return value of method  bar(), which returns null, the compiler cannot figure that out during static analysis and assumes the return value of bar() to be an unknown value.

Null annotations offer a solution here. They are a way of enforcing null contracts in your code to make sure you catch such (potential) NPE's before they occur at runtime and become a stop-ship bug for your product.


How do I use null annotations?
JDT now comes bundled with a JAR (org.eclipse.jdt.annotation) containing null annotations. The types of annotations offered are:
NonNull annotation
Default is @org.eclipse.jdt.annotation.NonNull -
  • when this annotation occurs on a return type, it enforces a non null return value for a method.
  • when this annotation occurs on a method parameter, it enforces a non null value to be passed to the method call and that only a non null value can be assigned to this parameter inside the method.

Nullable annotation
Default is @org.eclipse.jdt.annotation.Nullable-
  • when this annotation occurs on a method return type or parameter, it says that the returned value can be null, and so can be the argument.

NonNullByDefault annotation
Default is @org.eclipse.jdt.annotation.NonNullByDefault-

  • This can be applied to a package (in package-info.java), class or method to enforce nullability for all enclosed elements. The annotation alone enforces non nullability of enclosed method return value and parameter values.
  • If applied with a an argument i.e. NonNullBeDefault (false), it will cancel out any global default for the particular element.

To use these annotations in your project, right click Project>Build Path>Add External Archives and browse to org.eclipse.jdt.annotation JAR in the "plugins" directory of your eclipse install location. (We are working on the UI for this.)

One can also use their own annotation types as null annotations. Just make sure they are on the project's buildpath.

To enable null annotation based analysis and specify the annotation names, navigate to Preferences>Java>Compiler>Errors/Warning>Null analysis.

(Note also the "use non-null as default" option, which you can use to set a universal (workspace/project level) default to be applied to all method return types and parameters.)







What new errors/warnings should I expect to see with null annotations enabled?
As seen in the above dialog, there are two kind of diagnostics of primary importance - violation of null specification and potential violation of null specification. In addition to these the existing null analysis gets enhanced, and now for annotated method parameters a null state is available inside a method.

Violation of null spec is issued in cases when:
1. You pass a null value as an argument to a method whose corresponding parameter is annotated with @NonNull
2. You assign a @NonNull annotated parameter a null value.
3. You return a null value in a method that is annotated with @NonNull
4. You inherit a method annotated with @NonNull but now loosen the contract by annotating child method as @Nullable.
5. You inherit a method with a parameter annotated with @Null but now tighten the contract by annotating child method's corresponding parameter as @NonNull.

[See line numbers to locate error messages]



Potential violation of null spec is issued when:
1. You pass a value evaluated to be null on some execution path as an argument to a method whose corresponding parameter is annotated with @NonNull
2. You assign a @NonNull annotated parameter a value ascertained to be null on some execution path.
3. You return a possibly null value in a method that is annotated with @NonNull.

[See line numbers to locate error messages]


Insufficient null info is issued when there's not sufficient information to evaluate a given value to be (possibly) null and that value is passed as an argument, assigned to a parameter or returned from a method.

[See line numbers to locate error messages]


Redundant null annotation is issued when an annotation has the same effect as one applied already in an enclosing element.

[See line numbers to locate error messages]

Note that this is a work in progress and we will continue to polish the same for the Juno release. We're also working on ways to help people use this functionality easily (eg.: Support for nullity profiles for libraries and Quick fixes for null annotations. ) .
For a sneak preview of this feature, drop in at the EclipseCon 2012 tutorial - How To Train the JDT Dragon. If you find any issues while testing this feature, please open a bug. I hope you're as excited as we are about this cool new feature!

Monday, November 14, 2011

Of Compliance and Source settings in Eclipse java compiler


Going through a few stack overflow posts and a few bugs, I realized that there's still a lot of confusion about the  source and compliance level settings available in JDT. Well, frankly, almost everyone I know was confused when they started using JDT, even the current JDT committers themselves!
These settings are available in Preferences>Java>Compiler, or in your project's .settings folder>org.eclipse.jdt.core.prefs file.



So, what do the settings imply and what do they do?

Source compatibility:
This setting should be used to select upto which java version the user intends to use the language features. Eg.: setting it to 1.7 means that the user can use strings in switch, the diamond construct, etc. but setting it to 1.6 will mean that the user can not use such 1.7-only language features.
For the JDT compiler itself, we use a compliance of 1.4. Yes, quite old-fashioned you'd say, but this was a conscious choice to enable people to use the JDT compiler even if they dont have a version of java higher than 1.4 installed on their machines. You can also chose a source setting for your project on such considerations.

Compiler Compliance level:
This setting is a kind of a switch to make the JDT compiler "compliant" to a certain Oracle javac version. This is because not all bugs fixed in the Oracle javac compiler get backported to earlier releases. So, there are many differences between each javac version. There are even cases when only one javac version differs from the others, before and after it. So, we provided the compliance setting for users to replicate that difference in behaviour in the JDT compiler. This is to enable people who compile with ECJ and ship with javac or vice-versa, to see consistent errors (or the lack thereof).

Take, for example, the following case:

package com.example;

import java.util.List;


public class ErasureBug {


    public String output(List<Integer> integers) {


        return "";
    }


    public void output(List<Double> doubles) {


    }
}


This code compiles without error only in javac 6, but fails with a duplicate method erasure in both javac 5 and javac 7. 


So, in Eclipse when the user sets the project "compliance" to 1.6, he/she will be able to compile the code as shown, while in 1.5 and 1.7 modes, the code will fail.




Generated .class files compatibility:
This setting changes the generated bytecode to be compatible to a certain VM version. This is required at runtime when the VM available is lower than the one for which the code is compiled. So, if you set source level at 1.3 , use 1.3 features your code and expect your clients to still be able to use the older 1.2 or lesser VM to execute your code, you should set the .class file compatibility setting to 1.2 or less. This does not work for higher java versions, because suppose you use annotations in your code which only came in 1.5, or instructions such as invokedynamic in 1.7, which introduced a new Bootstrap method attribute, earlier VM's may still not be able to execute the class files. So we cannot guarantee backward compatibility and the UI thus will not allow you to set class file compatibility lower than the source level. However, if you're using features such as generics in 1.5 source level, it's quite safe for a 1.4 VM to execute those. So, you can manually set the org.eclipse.jdt.core.compiler.codegen.targetPlatform property to 1.4 in project>settings folder>org.eclipse.jdt.core.prefs file.


Note that the source level cannot be more than the compliance level.


Hope this gives more clarity on the settings.


EclipseCon update:
I have the following submissions for EclipseCon 2012. Do vote if you find them interesting:



Tuesday, October 18, 2011

Of invalid references to system libraries and execution environments

Picture this:

  • You have a plugin that is supposed to be compatible with systems running even on 1.4 JRE i.e. your plugin execution environment is 1.4.
  • An unsuspecting developer unintentionally checks out this plugin project into his workspace where he uses a 1.6 JRE by default.
  • He starts writing Generics and is even allowed to do so because a 1.6 library is associated with the project in his workspace. The poor guy doesn't yet know what blunder he has done! :)
See an example below where in a plugin with 1.6 compliance, a developer accidentally used Autocloseable (coming from his workspace default 1.7 JRE).

API tooling provides a mechanism to figure out such illegal references to system libraries. In order to set up API tooling for your plugin project, right click on the project > PDE tools > API Tooling setup. Choose your project and click next. When it shows "Add API nature and API analysis builder", click finish. Now make sure your project settings for API tooling (right click > Properties > Plug-in development > API errors/warnings) are set as shown below:

By default, the "Supported Environments" will have no Execution Environments descriptions installed (From 3.8 onwards you will get an error whenever no execution environment description is installed). To install these EE descriptions, click on "install them now" or go to the "Install New Software.." page. Choose the "Eclipse Project update site" if already available or add the URL http://download.eclipse.org/eclipse/updates/<version> , where <version> = 3.6, 3.7, etc.
Uncheck "group by category" option and search for descriptions. Install these descriptions and you're all set to go. (Note: the above steps should be taken if you move to 3.8 Juno and suddenly find the "Missing EE descriptions" error in your project). 

Now with the above "invalid references to system libraries" option set to error/warning, you should get a warning for usage of Autocloseable in the above plugin project, as shown below:


These settings when done once on the project level and released into the repository will make sure that nobody accidentally references any API/class one is not supposed to. Read Olivier's post to know more about project vs. workspace settings

Monday, August 1, 2011

Agile Development of mobile applications using IBM Rational Team Concert and Phonegap for Android, iOS and others


In earlier articles - Developing applications for Android™ using IBM Rational Team Concert in an agile way and Developing applications for Android™ using IBM Rational Team Concert andRational Rhapsody in an agile way we saw how to leverage the power of planning, collaboration, process, source control, build management, etc that IBM Rational Team Concert (RTC) has to offer, to develop and manage Android applications in a team working across geographies and across different platforms. We also saw how a team can model the application, specify UML class diagrams, use case diagrams, and link them with the requirements through Rational Rhapsody. The buck does not stop here. In a world where there are numerous different platforms such as iOS, Windows Mobile, Android, Meego, Blackberry, etc, how does a development team create applications that work seamlessly across different devices supporting different platforms? The brute force approach, ofcourse, is to write fresh code for each platform and then test and manage the application differently for each device or platform configuration. And as it sounds, this approach is not at all scalable and requires immense investment of resources – time, people and money. The other confounding problem that mobile developers can typically come across is how to glue together native and web components into a single mobile application(hybrid application). The way out of this conundrum is offered by new, open source technologies such as PhoneGap.

In this article, we explore what is PhoneGap and how does one use it. We then see, with the help of a demo, how a team can easily develop, test and maintain a cross-platform application working on Rational Team Concert's Eclipse Client and using PhoneGap APIs. This is facilitated by the MDS Applaud PhoneGap Eclipse plugin.


About PhoneGap

According to the website, "PhoneGap is an open source solution for building cross-platform mobile apps with modern, standards-based Web technologies. Based on HTML5, PhoneGap leverages web technologies developers already know best... HTML and JavaScript."

The definition itself does not make things very clear though. Let us try understand a bit more. So today, suppose you have a web application that you want to deploy on multiple platform, say Android, iPhone, etc. In this app, you want to use one or more of the platform services to obtain some kind of data or carry out some kind of processing. Lets take, for example, an application that lets you invite friends to a movie as soon as you book a movie ticket. The app lets you select which friends to invite directly from your phone's contacts book. That means, if you're the developer of such an app, you'll most probably have to deal with consuming the platform's contact book using the phone APIs, for doing which, each platform would obviously have its own different ways. So there are 2 problems here:

1) How can you query the platform's contact book from within your web application which is being written in javascript and has no knowledge of the platform's APIs.
2) How can you do so in a platform agnostic way, so that your web app works not just on Android but also on iPhone, notwithstanding the different ways of using the contacts book in both.

PhoneGap aspires to be the answer to both of the above questions, and also allows creation of hybrid applications, consisting of both native and web components. How? It provides APIs which abstract the platform's contact book for you, so that you only deal with the PhoneGap API and let PhoneGap do the rest of the magic for you.




Download and installation

1) OS: Windows x86 / Linux x86 .
2) IBM Rational Team Concert: Download/buy from RTC downloads page. Note that if you're using an existing Eclipse installation to install RTC, make sure you have Eclipse 3.5 or lower (Eclipse 3.6 is not yet supported for Android development). RTC can also be installed directly on Eclipse 3.5.x (see http://jazz.net/library/techtip/384).

3)
Android SDK: Download the SDK from Android website - http://developer.android.com/sdk/index.html.

4)
Android Development Tools(ADT): This is an eclipse plugin- the equivalent of JDT(Java Development Tools)- for Android Development. ADT can be downloaded and installed using instructions given at http://developer.android.com/sdk/eclipse-adt.html. Please follow the detailed instructions on this page for installing the ADT plugin, and also for setting the location of Android SDK in Eclipse.

5) It is a good idea to read through the following step by step guide to set up RTC and prepare it for Android Development: http://jazzpractices.wordpress.com/2010/08/10/how-to-set-up-rtc-for-android-development/.


Installing PhoneGap

Refer to my earlier blogpost on developing Phonegap applications using Eclipse to read about how to install Phonegap on the RTC Eclipse client.

*Note: You can similarly develop Windows mobile applications using RTC's Visual Studio client.



Creating a PhoneGap Android project

Follow the steps below to create a hello world PhoneGap Android application:

  1. Create a new PhoneGap project by clicking on the PhoneGap command you see on the coolbar.
  2. If you don't have the source code for PhoneGap, check "Use Built-in PhoneGap" on the project wizard
  3. Click Next and create the new Android project in the Android project wizard.
  4. Launch the new application as an Android application and you will get an application as shown below.

(Note: For those who do not have the PhoneGap plugin for Eclipse, you can also simply create an Android project, making sure that the android main activity class extends PhoneGap's DroidGap class instead of Activity class. Make sure you add "phonegap.jar" as an external library to the build path.) 
 
You see that a PhoneGap app is nothing but an android app with the main Activity class extending PhoneGap's DroidGap class and it's onCreate() method loading an HTML page instead of performing any native functionality. You can replace this HTML page by one of your own choice in the assets/www folder. Check out this website for some good examples. Similar applications can also be created on other platforms using the PhoneGap classes specific to that platform. The javascript code can be directly re-used.

The Demo

The demo showcases how a distributed team working on a cross-platform, hybrid mobile application can leverage PhoneGap APIs, and also the functionality provided by RTC through the complete lifecycle of the application, right from planning to implementation to design and testing, and even defect tracking.
In the demo, the application to be developed has a native activity to start with, which displays a list of all phone contacts. Clicking a particular contact displays the contact details in another native activity and provides a button to send an email to that contact. Clicking the button launches a web based email client (this is a PhoneGap activity), which has an option to cc more contacts from the phone's contact book. This makes use of the PhoneGap contacts API to fetch a list of the phone contacts from the web based activity. 
 
Check out the video below:



With the recent release of the Rational's Jazz based solution for Application Lifecycle Management, also called Collaborative Lifecycle Management or CLM, it is possible to do much more by integrating multiple tools with RTC. To see Rational Team Concert and CLM live in action, do visit IBM Rational Innovate India 2011, Bangalore. You can catch me there at the RTC solutions center. Also, if you're interested in building JavaME applications, you can also read up my jazz.net article - Developing Java ME applications using Rational Team Concert inan agile way to see how to do so using RTC.


For more information

Android is a trademark of Google Inc. Use of this trademark is subject to Google Permissions.