Endless Scrolling List Using jQuery

The mobile site for Pinterest got me all excited about endless scrolling list. This makes sense for mobile sites where we need to minimize user input. It turns out that the behavior is quite easy to implement using jQuery.

Our goal is to fetch new data and add to the list just before the user is about to scroll to the very bottom of the list.

The amount of the document that is left to be shown below the window frame is:

$(document).height() - $(window).scrollTop() - $(window).height()

When this amount reaches a minimum value, say 100 pixels, we can begin to fetch additional data and add items to the list.

Let’s create a test application. First, we need an element for the list where we will add items.

<div id="list_area"></div>

Attach a scroll event handler for the window.

$(function() {
 addContent();
 $(window).scroll(function () { 
  if ($(document).height() -
   $(window).scrollTop() - 
   $(window).height() <= 100) {
   console.log("Time to add content");
   addContent();
  }
 });
});

Example implementation of addContent().

var i = 0;
function addContent() {
 var div = $("#list_area");
 for (var j = 0; j < 50; ++j, ++i) {
   div.append($("<p>", {"text": "Paragraph " + i}));
 } 
}

HTML5 File Upload with Progress Bar Using jQuery

XMLHttpRequest Level 2, HTML5 File API and Progress event work together to provide a completely new file upload model. There are numerous benefits to this new model compared to the old multi-part form approach.

  • You can upload files using Ajax.
  • You can show progress indicator.
  • The server side code becomes a lot simpler.

Every new web application should take a look the new approach.

Unfortunately, jQuery is somewhat lagging in its support for the new approach. There are a few specific work arounds you have to take to make things to work. By work around, I mean, things that you must do that could have been easily taken care of by jQuery.

Note: If you are not using jQuery and just want to know how to upload file using XMLHttpRequest Level 2, then read this other article. The article below assumes that you know the fundamentals of XMLHttpRequest Level 2 already and just want to know how to use jQuery to do the work.

Let’s start with the form. There are many ways to get the list of files to upload. You can use drag and drop. Or, as is the case in this article, we will use a simple input of type file. Also, to display progress, you can use the HTML5 progress element. But, for a better look, we will choose to use jQuery UI’s progress widget. Our HTML markup will be very simple.

<input type="file" id="fileToUpload"/><br/>
<div id="progressbar"></div>
<button id="upload_btn">Start Uploading</button>

From the ready function, we setup the UI. This is just regular jQuery stuff and we are not doing anything with file upload API yet.

$(function(){
    $("#progressbar").progressbar();
    $("#progressbar").hide();
    $("#upload_btn").click(upload);
});

All the action happens from the upload() method.

function upload(){
    var fileIn = $("#fileToUpload")[0];
    //Has any file been selected yet?
    if (fileIn.files === undefined || fileIn.files.length == 0) {
        alert("Please select a file");
        return;
    }

    //We will upload only one file in this demo
    var file = fileIn.files[0];
    //Show the progress bar
    $("#progressbar").show();

    $.ajax({
        url: "MediaUploader?fileName=" + file.name + "&mimeType=" + file.type,
        type: "POST",
        data: file,
        processData: false, //Work around #1
        contentType: file.type, //Work around #2
        success: function(){
            $("#progressbar").hide();
        },
        error: function(){alert("Failed");},
        //Work around #3
        xhr: function() {
            myXhr = $.ajaxSettings.xhr();
            if(myXhr.upload){
                myXhr.upload.addEventListener('progress',showProgress, false);
            } else {
                console.log("Upload progress is not supported.");
            }
            return myXhr;
        }
    });
}

Alright, let’s go through the work arounds.

  1. We must set processData to false. Here’s why. We set the File object as the data field. jQuery doesn’t recognize File object and tries to use it to generate regular form post data, which obviously fails.
  2. The content type needs to be set correctly to the MIME type of the file. Otherwise, some browsers will set it to “application/x-www-form-urlencoded”. The server will then try to parse the request body as form input, which will obviously fail.
  3. This is the big one. jQuery provides no way of attaching a “progress” event listener for the XMLHttpRequest object. The work around is to register your own XMLHttpRequest object creation method and attach the event handler from there.

Now, all there is left to do is show the progress.

function showProgress(evt) {
    if (evt.lengthComputable) {
        var percentComplete = (evt.loaded / evt.total) * 100;
        $('#progressbar').progressbar("option", "value", percentComplete );
    }  
}

The server side is, of course, quite straightforward. Your best bet is to write a Servlet. Various JAX-RS implementations provide custom extensions to handle HTML5 file upload. You can also use that option if you are not worried about getting locked down to an implementation. Here, we will use a Servlet.

protected void doPost(HttpServletRequest request, HttpServletResponse response) 
  throws ... {
    String fileName = request.getParameter("fileName"); 
    InputStream is = request.getInputStream();
    //Read the file contents from the input stream
    byte buff[] = new byte[256];
    int len;

    while ((len = is.read(buff)) > 0) {
        //...
    }
}

Git Tutorial for Beginners Part II

In Part I, I tried to introduce Git to non-believers who have their brains formatted by CVS. We learned to create a repository and commit files to it. Part II will be very short. We will learn to look at the history of changes made to a file and learn to revive an old version of the file.

Let’s say, that you have a file called test.c. This file has been committed several times in the past. You can look at the history of changes by running this command.

git log test.c

This will display something like this.

commit 9faa6e6d4df82766de1ecc63ff5e79463e21c7cb
Author: Bibhas Bhattacharya <bibhas@example.com>
Date: Wed Aug 15 10:03:29 2012 -0400
Comments2

commit f8648963ac296d974dedd5530a56a024c495219c
Author: Bibhas Bhattacharya <bibhas@example.com>
Date: Wed Aug 15 10:01:43 2012 -0400

Comments1

The date and author of every commit made for the file is displayed. In CVS, each commit creates a new version number. In Git, you get a unique key for the commit. This is shown in bold face above. Git generates the key by using SHA-1 hash of the Git tree (something that closely resembles the files and directories as they existed at the time of the commit).

So, anyway, the important thing is that there is no CVS like version number. Every version has a unique key.

Now, let’s say that you made a terrible mistake to the file. Perhaps your cat went in the middle of the night and typed in gibberish to it and saved it. Now what? Well, first we need to find out what has changed since a specific commit. For that, run this command:

git diff 9faa6e6d4df82766de1ecc63ff5e79463e21c7cb test.c

The hash value of the commit you are trying to compare with goes into the command.

This will print out a terrible output that only Martians can understand.

index a4b000a..01dfe10 100644
--- a/src/test.c
+++ b/src/test.c
@@ -1,2 +1,2 @@
-New line
-Line 2
+New line was here
+Only a memory 3

Fortunately, git works with many excellent visual diff programs. Most installations probably have vimdiff configured by default. To view the difference in a graphical tool, run this command:

git difftool 9faa6e6d4df82766de1ecc63ff5e79463e21c7cb test.c

To configure a different diff tool, read this article. Right about now, I am starting to miss my Eclipse diff view. Oh, how many times has it come to my aid in times of major distress!

Anyway, getting back to the mischief caused by your cat. We now have to fix the problem. The diff may show the source of the error and you may be able to manually fix the problem. If you must start from scratch, just checkout the older version.

git checkout 9faa6e6d4df82766de1ecc63ff5e79463e21c7cb test.c

This will completely replace the file with the older version.

Setting up a Git Server in Linux

Hello all. I needed to setup a Git server for my team and didn’t want to commit to a paid GitHub account yet. So, I set out to setup a Git server of my own. Instructions are sketchy at best. But, I think I got it to work.

I am using CentOS 5.0 i386.

First, we need to configure Yum to use the RedHat extended Linux repository. For i386 system, run this command:

rpm -Uhv http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/
    rpmforge-release-0.3.6-1.el5.rf.i386.rpm

If you are using a 64bit OS, run this instead:

rpm -Uhv http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS/
    rpmforge-release-0.3.6-1.el5.rf.x86_64.rpm

Then, install Git by running these commands.

yum -y install git
yum -y install git-daemon

Now, we are all set to start the server. First, create a folder that will be the root of all repositories. Let’s say /root/git.

Start Git daemon as follows:

git daemon --reuseaddr --base-path=/root/git --export-all --verbose 
    --enable=receive-pack &

That’s it. If you are running a firewall, drill a hole in port 9418.

Let’s quickly test things out.

We will first create a repository in the server. Recall, we used /root/git as the root of all repositories in the server. So go there and create a folder for a repository.

cd /root/git
mkdir ProjectA
cd ProjectA
git init --bare

OK, now we have a bare repository on the server called “ProjectA”.

Now, go to a client machine. Run these commands to push to the remote repository ProjectA.

mkdir ProjectA
git init
touch README.txt
git add .
git commit -m "Project creation"
git push git://SERVER_IP_OR_HOSTNAME/ProjectA master

This should work and get you in business.

Connecting to Amazon EC2 from Mac OSX

I saw all kinds of posts saying that ssh in Mac doesn’t read Amazon’s key file. That’s just bogus. I did have some struggles but I got things to work.

First, copy the key file, say demokey.pem, on the hard drive somewhere. I normally carry the key file in a USB drive. It will not work from a USB drive! That’s where I wasted a lot of time.

Then change the permission of the key file:

chmod 400 demokey.pem

Then, run ssh:

ssh -i demokey.pem root@YOUR_EC2_INSTANCE_HOSTNAME

Similarly, copy files using scp:

scp -i demokey.pem somefile.txt root@YOUR_EC2_INSTANCE_HOSTNAME:/root

Voila! Enjoy!

Doing Post-Redirect-Get Pattern in JSF 2

JSF 2.0 introduces the ability to send GET requests. This makes it possible to make bookmarkable pages. It also makes it possible to implement the Post-Redirect-Get (PRG) pattern. What is the PRG pattern? It is nothing new and we have been using it from day one of web application. It goes as follows:

  1. User submits a form. A POST request is sent.
  2. The server changes something in the database. For example, create a new customer.
  3. The server sends a redirect reply (302 or 301) to the browser. A POST request should never return HTML markup for too many reasons to get into here. The Location header of the reply contains a URI to the next page. Any necessary data, such as the ID of the newly created customer, needs to be added to the URI as parameters. For example,  display_customer.xhtml?customerId=10.
  4. Browser sends a GET request for the next page as set in the Location header of the previous reply. The server sends back HTML markup for that page. This page is also inherently bookmarkable, since all the URL parameters are there.

This essential pattern was not possible prior to JSF 2.0. Unfortunately, many badly written books and tutorials have made it difficult to understand how GET request works. That’s because they never explained it in the context of PRG pattern.

Let’s learn how to implement PRG pattern using a real example. Here, we will create a new customer.

First, the Customer class.

public class Customer {
    private String name;
    private String email;
    private int id;
//Getter and setter methods omitted for brevity
//...
}

Next, the managed bean class. Pay attention to how the addCustomer() method returns the outcome. We will discuss it more later.

@ManagedBean
public class Controller {
    Customer customer = new Customer();

    public Customer getCustomer() {
        return customer;
    }
    public void setCustomer(Customer customer) {
        this.customer = customer;
    }

    public String addCustomer() {
        //Add customer to database
        System.out.println("Adding customer: " + customer.getName());
        //In real life, we will get a unique ID set
        customer.setId(10);

        return "display_customer?faces-redirect=true&includeViewParams=true";
    }

    public void loadCustomer() {
        //Look up customer from database
        System.out.println("Loading customer by ID: " + customer.getId());
        customer = lookupCustomer(customer.getId());
    }
    public Customer lookupCustomer(int id) {
        //Use DAO to retrieve customer data
        //...
    }
}

Of course, in real life, addCustomer() will use a DAO class to add the customer to the database. Here we fake a unique ID of 10. Note the two extra parameters we are supplying to the “display_customer” outcome:

  1. faces-redirect – This causes a 302 redirect reply to be sent back.
  2. includeViewParams – This causes JSF to add all necessary URL parameters to the redirected URI in the Location header. How does JSF know what URL parameters to add? There is the trick. JSF will inspect the meta data of the target view – display_customer.xhtml – and find out the required view parameters. It will automatically add those to the redirected URI. So, let’s see how these views are defined.

First, let’s have a look at the view that contains the form – add_customer_form.xhtml. There is nothing special here.

<h:form>
Name: <h:inputText value="#{controller.customer.name}" /><br/>
E-mail: <h:inputText value="#{controller.customer.email}" /><br/>
<h:commandButton action="#{controller.addCustomer()}" value="Submit" />
</h:form>

Next the display_customer.xhtml file. This is where the magic happens. We add the metadata to the root.

<f:metadata>
    <f:viewParam name="customerId" value="#{controller.customer.id}" />
    <f:event listener="#{controller.loadCustomer()}" type="preRenderView"/>
</f:metadata>
<body> 
<h1>Customer Details</h1>
<p>
ID: #{controller.customer.id} <br/>
Name: #{controller.customer.name}<br/>
</p>
</body>

The <f:viewParam> metadata clearly says that the view needs the “customerId” URL parameter. This URL parameter is bound to the customer.id property of the managed bean. This has two effects:

  1. When constructing the Location URL of the 302 reply, JSF copies the value of customer.id to the customerId URL parameter.  In our case, the URI will look something like display_customer.xhtml?customerId=10.
  2. When a GET request arrives for display_customer.xhtml, JSF copies the customerId URL parameter to customer.id.

Also, as a part of the GET request processing, JSF calls the pre-render listener loadCustomer(). This method initializes the customer property of the managed bean.

Now, not only have you implemented the PRG pattern, you also got a bookmarkable page that shows customer details.

PRG is nothing new. We have followed it religiously in Servlet/JSP and Struts days. The possibility to implement it is new in JSF. Frankly speaking, I like the way JSF automatically adds the required URL parameter to generate the redirected URL. This is better than way things are done in Struts. So, what are you waiting for. Get started with PRG in your JSF apps!

Basic Lockdown of MySQL

MySQL is stunningly insecure right after installation. It has these two problems:

  1. The main administrative user “root” has no password set.
  2. There is a notion of an anonymous user, once again without any password. Essentially, anyone can log in and pretty much do anything.

The lockdown steps are mentioned in the documentation. Here is a summary.

First, log in as the root user.

mysql -u root

Switch to the mysql database.

use mysql;

Set a password for the root user.

UPDATE user SET Password = PASSWORD('some_pass') where user='root';

Set a password for the anonymous user. It has an empty user name.

UPDATE user SET Password = PASSWORD('some_pass') where user='';

Alternatively, you can even delete the anonymous user. I didn’t try this, but seems like a good idea.

delete from user where user='';

Finally, flush the changes so that changes are applied without a server restart.

flush privileges;

You probably want to create a user that will be used by the applications to connect to the database. Here are the steps to add a user called “monty” with rights to access the “test” database.

use test;
GRANT ALL PRIVILEGES ON *.* TO 'monty'@'%' IDENTIFIED BY 'some_pass' WITH GRANT OPTION;

Running MySQL on Amazon EC2

In a previous article, I talked about setting up JBoss AS 7 in Amazon EC2. Today, I will talk about MySQL installation.

The SUSE Linux Enterprise Server 11 AMI has MySQL pre-installed. But, I had all kinds of problems trying to start the server. So, I decided to uninstall all the pre-installed packages and start from scratch. In real life though, you will probably go with RDS. But, if you want to build a simple system and want to install MySQL on your own, these steps may help.

Make sure that you are logged in as root. Installation of MySQL from RPM requires root privilege.

First of all, remove all pre-installed packages. You can get a list:

rpm -qa | grep -i mysql

Then uninstall all of them. For example:

rpm -e mysql libmysqlclient15-5.0.94-0.2.4.1 <and so on>

Completely remove the /var/lib/mysql folder. This is where the actual database data files are stored. Without this step, a future installation will run into problem.

cd /var/lib
rm -rf mysql

Download the MySQL server RPM:

curl -o mysql.rpm \
  http://cdn.mysql.com/Downloads/MySQL-5.5/MySQL-server-5.5.27-1.sles11.x86_64.rpm

Then, install the RPM.

rpm -Uvh mysql.rpm

This will copy the files and create the mysql user, which we will use later.

Download the MySQL Client package.

curl -o mysql-client.rpm \
  http://cdn.mysql.com/Downloads/MySQL-5.5/MySQL-client-5.5.27-1.sles11.x86_64.rpm

Install the package.

rpm -Uvh mysql-client.rpm

In SUSE, all MySQL binary executables are directly stored under /usr/bin. Since, this folder is already in your PATH, you can run various MySQL commands from anywhere.

First, you must initialize the database. It is essential that you use the mysql user here:

mysql_install_db --user=mysql

This will make mysql user the owner of various files.

Finally, run the server. Once again, make sure that you use the mysql user for this:

mysqld_safe --user=mysql &

Verify that server has started by running a few commands:

mysqladmin version
mysqlshow

Shutdown the server.

mysqladmin -u root shutdown

Here, “root” is the MySQL user and not the UNIX user.

Only thing left to do now is have MySQL start and stop automatically as the machine boots up and shuts down. The RPM installer has already created the /etc/init.d/mysql file. We will make  small change.

Open /etc/init.d/mysql in an editor.

Find these two lines:

basedir=
datadir=

Change them as follows:

basedir=/usr
datadir=/var/lib/mysql/

To be extra sure, you can run the “mysqladmin variables” command and look up the correct values of the basedir and datadir variables.

Reboot the EC2 instance and verify that MySQL has started automatically.

You should now have a fully functional MySQL installation. The biggest gotcha in the process is the file system security. You run all the steps by logging in as root. But, actual work done by MySQL has to be done by the mysql user. This is why commands like mysql_install_db and mysqld_safe are run with the mysql user setting.

Create a DataSource by Script in JBoss 7

JBoss AS 7 has a new administrative architecture and simplified command line. Its really much easier to use scripting to create a new JDBC data source.

As you know, creating a data source is a two step process:

  1. Define a JDBC driver. Copy the necessary JAR files so that the servers can find them.
  2. Create a new data source for that driver. Give it a JNDI name and configure the connection pool.

JBoss 7 greatly simplifies the driver creation. Essentially, you deploy the JAR file like a EAR or WAR. If you are using the domain mode, specify the server group for this deployment. The domain controller will automatically copy the JAR file to all the machines. This saves a ton of trouble.

Let’s go through the script line by line:

First, deploy the JDBC driver. If you are in domain mode, use this command. This will distribute the driver to all server groups and copy the JAR file to all machines.

deploy --all-server-groups mysql-connector-java-5.1.21-bin.jar --name=MySQLDriver

If you are in standalone mode, use this command:

deploy mysql-connector-java-5.1.21-bin.jar --name=MySQLDriver

Note that there is no quote around the deployment’s name. Also, JBoss is very picky about what characters you can use. Avoid spaces and any special characters.

Now, create the data source and its connection pool. I broke up the command in multiple lines here to better understand things. But it should be all in a single line in the script.

/subsystem=datasources/data-source="MySqlDS2":add(
  jndi-name="java:/jdbc/MySqlDS2",
  max-pool-size=10,min-pool-size=5,
  driver-name="MySQLDriver",
  connection-url="jdbc:mysql://localhost/test",
  user-name="monty",password="some_pass")

This creates a data source with JNDI name “java:/jdbc/MySqlDS2”. Note: The JNDI name needs to start with either “java:/” or “java:jboss/”.

Finally, enable the data source so you can use it right away.

/subsystem=datasources/data-source=MySqlDS2:enable

If you want to be extra cautious, test the connection right from the script. This step is optional:

/subsystem=datasources/data-source=MySqlDS2:test-connection-in-pool

That’s it.

Now, add all of these lines in a single file (one command per line). Save the file as say script.txt in <JBOSS>/bin folder. Copy the JDBC driver JAR file in the same folder. Then run the script:

./jboss-cli.sh --connect --file=script.txt

If you want to quickly test it out from an application, do that:

try {
    InitialContext ctx = new InitialContext();
    DataSource ds = (DataSource) ctx.lookup("java:/jdbc/MySqlDS2");
    ds.getConnection().close();
} catch (Exception e) {
    e.printStackTrace();
}

Developing JAX-RS Services With JBoss AS 7

This morning, I tried to write a RESTful service using JBoss AS 7 and JBoss plugins for Eclipse Indigo. The process to setup a dynamic web project for JAX-RS is a little tricky.

First of all, do’t just go ahead and try to add the JAX-RS facet to the dynamic web project. Instead, right click the project and select Add JAX-RS 1.1 Support.

Then, open web.xml, and add these lines:

<servlet>
<servlet-name>javax.ws.rs.core.Application</servlet-name>
</servlet>

<servlet-mapping>
   <servlet-name>javax.ws.rs.core.Application</servlet-name>
   <url-pattern>/rest/*</url-pattern>
</servlet-mapping>

You can use anything else in place of “/rest/*”. Any URI starting with this will be routed to the JAX-RS servlet.

Let’s say that we have a class:

@Path("/mybean")
public class MyBean {
  @Path("/value")
  @Produces("text/plain")
  @GET
  public String getValue() {
     return "Some value";
  }
}

Let’s say that the context root of the web project is “MyWeb”. Now, you can access the getValue() method of the MyBean resource using the URI: /MyWeb/rest/mybean/value.