1. Drupal Modules

A module is a collection of php functions that link into Drupal, providing additional functionality to the Drupal installation.
Because the module code executes within the Drupal context, it can access all the variables and structures and use all the functions of Drupal core.

1.1. Choose a short name

The first step in creating a module is to choose a short name for it. This short name will be used in all file and function names in your module. In this example, we will be using “nodes_to_text_file” as the short name.

Make sure your short name starts with a letter and only contains lower-case letters and underscores. Also make sure your module does not have the same short name as any theme you will be using.

1.2. Create the directory

In the modules directory of your Drupal 7 install, create a folder with your short name as its name.

1.3 empty module file

Create a nodes_to_text_file.module file in your nodes_to_text_file directory. It is to this file that we will add the functions that will hook into Drupal. In the next section, we will add the first function.
Module files begin with the opening php tag. Add it to the module file:

<?php

However, omit the closing php tag ?>. Omitting it is a convention, and including it might cause strange runtime behavior on certain server setups.

1.4 Creating the info file

Every module has its info file. It contains metadata about the module and hence, tells Drupal about the module. Without this file, the module will not appear in the module listing of your Drupal installation.

Create a nodes_to_text_file.info file in your nodes_to_text_file directory:

name = Nodes To Text File
description = A module that creates a text file containing all content of certain types of nodes whenever a node of one of these types is added, edited or deleted
core = 7.x

The above 3 properties form the minimal setup: name sets the name of the module, description describes what the module does and core determines with which version of Drupal this module is meant to be used.

After adding the info file, the module should be visible in your module listing. Go to Modules and scroll down to the “Other” section. The Nodes To Text File module should be listed there. Enable it.

2. Module Hooks

Hooks are the most important concept to grasp when implementing Drupal modules. Hooks provide the points at which you can insert your actions. They can be thought of as event listeners. An event such as deleting a node would trigger the hook “hook_node_delete”. In this case, Drupal will scan all modules for functions with the name mymodule_node_delete, where mymodule is the module’s name. If your module implements hook_node_delete, that function will always execute after a node gets deleted.

For a list of all available hooks in Drupal core, check the Drupal 7 hooks documentation.

2.1 Implementing the hook_help hook

The first hook we are going to implement is hook_help. This hook should always be implemented. It provides help and information about the module. To implement an hook, you need to replace “hook” in the hook name(f.e. hook_help) with your module’s short name and create a function in the .module file with that name.
In this case, nodes_to_text_file_help:

/**
 * Implements hook_help.
 *
 * Displays help and module information.
 *
 * @param path
 *   Which path of the site we're using to display help
 * @param arg
 *   Array that holds the current path as returned from arg() function
 */
function nodes_to_text_file_help($path, $arg) {
  switch ($path) {
    case "admin/help#nodes_to_text_file":
      return '<p>'.  t("A module that creates a text file containing all content of certain types of nodes whenever a node of one of these types is added, edited or deleted") .'</p>';
      break;
  }
}

The path parameter contains the information about where the user is currently at. If the user is currently at the help section of our module, we return a paragraph that gives information about our module. Note that we use the Drupal t($string) function, which translates the content if possible and which should be used on any string that is shown to the user.

Now, check Modules page of your Drupal site again. You should see a link ‘Help’ on the right of your module. If you click it, you will see the paragraph returned by the nodes_to_text_file_help function of our module.

2.2. Implementing the hooks hook_node_insert, hook_node_update and hook_node_delete

We will now implement the hooks hook_node_insert, hook_node_update and hook_node_delete.
All these hooks will call the same function nodes_to_text_file_write_file($node):

/**
 * Implements hook_node_delete.
 */
function nodes_to_text_file_node_delete($node){
	nodes_to_text_file_write_file($node);
}

/**
 * Implements hook_node_update.
 */
function nodes_to_text_file_node_update($node){
	nodes_to_text_file_write_file($node);
}

/**
 * Implements hook_node_insert.
 */
function nodes_to_text_file_node_insert($node){
	nodes_to_text_file_write_file($node);
}

For now, add the following implementation of the function:

function nodes_to_text_file_write_file($node){
	watchdog('We should write the file again!','Info Message');
}

watchdog is a useful function to debug hooks. They will be added to your Drupal server logging at Reports>Recent log messages:

<watchdog screenshot>

3. Database API

For this app, we will need two queries. One that fetches all the published nodes of the given content types of which the user is the owner:

/**
 * Custom content function.
 *
 * Retrieve data from database
 *
 * @return
 *   A result set of the targeted nodes.
 */
function nodes_to_text_file_contents($contentTypes){
  //Use Database db_select API to retrieve nodes.
  global $user;
  $uid = $user->uid;

  $queryResult = db_select('node', 'n')
    ->fields('n', array('nid', 'title', 'created'))
    ->condition('uid', $uid) //owner is current user.
    ->condition('status', 1) //Published.
    ->condition('type', $contentTypes, 'in') //Node type is in $contentTypes
    ->orderBy('created', 'DESC') //Most recent first.
    ->execute();
  return $queryResult;
}

This query uses the new(starting from Drupal 7) db_select function of the Database API. Other than the old db_query function, db_select makes it possible to construct the query in an object oriented matter.

db_select maps to a sql select statement(as do db_insert, db_update and db_delete map to their sql equivalents). It takes a table name(f.e. ‘node’) and an alias (f.e. ‘n’) as its arguments. This comes down to:

select * from node as n

fields selects the fields listed in the array in its second argument on the alias n:

select nid, title, created from node as n

condition adds a condition to the query. The first is the field, the second the value, the third the operator:

select nid, title, created from node as n where published = 1 and type in ('article','page') and uid = 1

if the values in the $contentTypes array are ‘article’ and ‘page’ and user Steffen has uid 1. Note that the “global $user” statement gets the global $user variable, which contains all information about the user that is currently logged in.

orderBy then sort the results according to the field in its first argument:

select nid, title, created from node as n where published = 1 and type in ('article','page') and uid = 1 order by created asc

The execute method then compiles and runs the query and returns the result.

The other query we need – one that fetches all content types of the core Node Module – uses the same API and is quite simular:

/**
 * Function that retrieves the content types from the database.
 *
 * Retrieve content types from database
 *
 * @return
 *   A result set of content types.
 */
function nodes_to_text_file_retrieve_content_types(){
  $queryResult = db_select('node_type', 'nt')
    ->fields('nt', array('type', 'name'))
    ->condition('module', 'node') //only node content types
    ->orderBy('type', 'ASC')
    ->execute();
  return $queryResult;
}

4. Drupal File API

We are now implementing our nodes_to_text_file_write_file function that is called by the node_insert, node_update and node_delete hooks. We briefly discuss the part in which we determine whether the file should be written and in which we fetch the contents first.

/**
 * Custom write file function.
 *
 * Creates the text file and stores it on the server.
 */
function nodes_to_text_file_write_file($node){
	$contentTypes = variable_get('nodes_to_text_file_content_types', array());
	$nodeType = $node->type;

	$elementFound = false;
	foreach ($contentTypes as &$value) {
    	if ($nodeType === $value){
    		$elementFound = true;
    		break;
    	}
	}

	if ($elementFound){
    	$queryResult = nodes_to_text_file_contents($contentTypes);

    	global $user;
    	$username = $user->name;

   	...
}

First, we fetch the node content types which should be included in the file using variable_get($name,$default) As we will see in the next section, variable_set($name,$value) – typically invoked in a module configuration form – sets Drupal system wide variables, which can then be fetched by variable_get.
We then match the node type of the node that was inserted, updated or deleted with the array of content types. If a match is found($elementFound), execution continues. We fetch the nodes using the query of the previous section and get a reference to the username of the currently logged in user.

Then – where the … dots are in the above function – we add the following and write out the fetched content to a file:

	if (! file_prepare_directory(file_build_uri($username))){
    		drupal_mkdir(file_build_uri($username));
    	}

    	$filename = file_build_uri($username . '/content.txt');
    	$fh = fopen($filename, 'w') or die("can't open file");

   	 	while($record = $queryResult->fetchAssoc()) {
    		$stringData = $record['nid'] . ' - ' . $record['title'] . ' - ' . $record['created'];
    		fwrite($fh, $stringData."\n");
    	} 

    	fclose($fh);

We write the fetched contents to a context.txt file in the default Drupal public files location, in a folder with the username as its name. If this folder does not exist yet, we create it first:

if (! file_prepare_directory(file_build_uri($username))){
   drupal_mkdir(file_build_uri($username));
}

Then we construct the filename:

$filename = file_build_uri($username . '/content.txt');

If the username is “Steffen”, then the filename will be:
<drupal installation location>/sites/default/files/steffen/content.txt

Then we open the file for writing and iterate over the queryResult. We print out the nid, the title and the created timestamp of every record to a new line in the file. We then close the file again:

   $fh = fopen($filename, 'w') or die("can't open file");

   while($record = $queryResult->fetchAssoc()) {
      $stringData = $record['nid'] . ' - ' . $record['title'] . ' - ' . $record['created'];
      fwrite($fh, $stringData."\n");
   } 

   fclose($fh);

We are using some functions of the Drupal File API here:
file_build_uri: This function constructs, given a relative path, a URI into Drupal’s default files location(this is usually something like ‘sites/default/files/’).
file_prepare_directory: This function checks that the directory exists and is writable.
drupal_mkdir: This function creates a directory using Drupal’s default mode.

To respectively open the file for writing, write to it and close it again, we use the php functions fopen, fwrite and fclose.

5. Drupal Form API

Our module is implemented now. However, we want to be able to configure the node content types for which the file should be written. We already saw that we were fetching this array of content types like this:

$contentTypes = variable_get('nodes_to_text_file_content_types', array());

What remains is to explain how we can set this array of content types first through a module configuration form.

We will implement hook_menu so we get a configuration form at our module at the relative path admin/config/content/nodes_to_text_file:

/**
 * Implements hook_menu().
 */
function nodes_to_text_file_menu() {
  $items = array();  

  $items['admin/config/content/nodes_to_text_file'] = array(
    'title' => 'Nodes To Text File',
    'description' => 'Configuration for Nodes To Text File module',
    'page callback' => 'drupal_get_form',
    'page arguments' => array('nodes_to_text_file_form'),
    'access arguments' => array('access administration pages'),
    'type' => MENU_NORMAL_ITEM
  );

  return $items;
}

Using the hook_menu hook, modules register paths to define how URL requests are handled. In this case, we are setting a path that makes a configuration form available from the Drupal Configuration page:

We passed the form already to the array of the ‘page arguments’ property of the link, but we didnt define it yet. We are doing so now:

/**
 * Form function, called by drupal_get_form()
 * in nodes_to_text_file_menu().
 */
function nodes_to_text_file_form($form, &$form_state) {
	$queryResult = nodes_to_text_file_retrieve_content_types();

	$options = array();
	while($record = $queryResult->fetchAssoc()) {
    	$options[$record['type']] = t($record['name']);
    } 

  	$form['nodes_to_text_file_content_types'] = array(
    	'#type' => 'checkboxes',
    	'#title' => t('Content types to include'),
    	'#default_value' => variable_get('nodes_to_text_file_content_types', array()),
    	'#description' => t('The content types to include.'),
    	'#options' => $options
  	);

  return system_settings_form($form);
}

First we are creating the array of possible content types so we can use it later to generate the necessary checkboxes on the form:

$queryResult = nodes_to_text_file_retrieve_content_types();

	$options = array();
	while($record = $queryResult->fetchAssoc()) {
    	$options[$record['type']] = t($record['name']);
    }

In Drupal, forms should be constructed using the Form API. We are using it to construct our form:

$form['nodes_to_text_file_content_types'] = array(
    	'#type' => 'checkboxes',
    	'#title' => t('Content types to include'),
    	'#default_value' => variable_get('nodes_to_text_file_content_types', array()),
    	'#description' => t('The content types to include.'),
    	'#options' => $options
  	);

after which we return the form. Note that the Form API is pretty straightforward. It is generally enough to consult the API page to know what types of fields are available and which properties are available and which ones are required.

6. The entire module file for reference

<?php
/**
 * Implements hook_help.
 *
 * Displays help and module information.
 *
 * @param path
 *   Which path of the site we're using to display help
 * @param arg
 *   Array that holds the current path as returned from arg() function
 */
function nodes_to_text_file_help($path, $arg) {
  switch ($path) {
    case "admin/help#nodes_to_text_file":
      return '<p>'.  t("A module that creates a text file containing all content of certain types of nodes whenever a node of one of these types is added, edited or deleted") .'</p>';
      break;
  }
}

/**
 * Implements hook_node_delete.
 */
function nodes_to_text_file_node_delete($node){
	nodes_to_text_file_write_file($node);
}

/**
 * Implements hook_node_update.
 */
function nodes_to_text_file_node_update($node){
	nodes_to_text_file_write_file($node);
}

/**
 * Implements hook_node_insert.
 */
function nodes_to_text_file_node_insert($node){
	nodes_to_text_file_write_file($node);
}

/**
 * Custom write file function.
 *
 * Creates the text file and stores it on the server.
 */
function nodes_to_text_file_write_file($node){
	$contentTypes = variable_get('nodes_to_text_file_content_types', array());
	$nodeType = $node->type;

	$elementFound = false;
	foreach ($contentTypes as &$value) {
    	if ($nodeType === $value){
    		$elementFound = true;
    		break;
    	}
	}

	if ($elementFound){
    	$queryResult = nodes_to_text_file_contents($contentTypes);

    	global $user;
    	$username = $user->name;

   	 	if (! file_prepare_directory(file_build_uri($username))){
    		drupal_mkdir(file_build_uri($username));
    	}

    	$filename = file_build_uri($username . '/content.txt');
    	$fh = fopen($filename, 'w') or die("can't open file");

   	 	while($record = $queryResult->fetchAssoc()) {
    		$stringData = $record['nid'] . ' - ' . $record['title'] . ' - ' . $record['created'];
    		fwrite($fh, $stringData."\n");
    	} 

    	fclose($fh);
	}
}

/**
 * Custom content function.
 *
 * Retrieve data from database
 *
 * @return
 *   A result set of the targeted nodes.
 */
function nodes_to_text_file_contents($contentTypes){
  //Use Database db_select API to retrieve nodes.
  global $user;
  $uid = $user->uid;

  $queryResult = db_select('node', 'n')
    ->fields('n', array('nid', 'title', 'created'))
    ->condition('uid', $uid) //owner is current user.
    ->condition('status', 1) //Published.
    ->condition('type', $contentTypes, 'in') //Node type is in $contentTypes
    ->orderBy('created', 'DESC') //Most recent first.
    ->execute();
  return $queryResult;
}

/**
 * Function that retrieves the content types from the database.
 *
 * Retrieve content types from database
 *
 * @return
 *   A result set of content types.
 */
function nodes_to_text_file_retrieve_content_types(){
  $queryResult = db_select('node_type', 'nt')
    ->fields('nt', array('type', 'name'))
    ->condition('module', 'node') //only node content types
    ->orderBy('type', 'ASC')
    ->execute();
  return $queryResult;
}

/**
 * Form function, called by drupal_get_form()
 * in nodes_to_text_file_menu().
 */
function nodes_to_text_file_form($form, &$form_state) {
	$queryResult = nodes_to_text_file_retrieve_content_types();

	$options = array();
	while($record = $queryResult->fetchAssoc()) {
    	$options[$record['type']] = t($record['name']);
    } 

  	$form['nodes_to_text_file_content_types'] = array(
    	'#type' => 'checkboxes',
    	'#title' => t('Content types to include'),
    	'#default_value' => variable_get('nodes_to_text_file_content_types', array()),
    	'#description' => t('The content types to include.'),
    	'#options' => $options
  	);

  return system_settings_form($form);
}

/**
 * Implements hook_menu().
 */
function nodes_to_text_file_menu() {
  $items = array();  

  $items['admin/config/content/nodes_to_text_file'] = array(
    'title' => 'Nodes To Text File',
    'description' => 'Configuration for Nodes To Text File module',
    'page callback' => 'drupal_get_form',
    'page arguments' => array('nodes_to_text_file_form'),
    'access arguments' => array('access administration pages'),
    'type' => MENU_NORMAL_ITEM
  );

  return $items;
}

Making a backup of a Mysql database and restoring it

Making a backup of a MySql database is straightforward. I am giving the statements below for a Mac, but on Windows the same mysqldump and mysql commands can be used from the bin folder of the MySql installation.

First navigate to your MySql installation folder. This will probably be something like:

cd /usr/local/mysql

And then run mysqldump:

sudo ./bin/mysqldump -u <username> -p <password> testdatabase > /development/testdatabase.sql

where <username> is your mysql username and <password> is your mysql password.

This will generate a sql script in the development folder that is able to recreate all tables and indexes and reinsert all data that are available in the testdatabase, by running the following command on the target server:

sudo ./bin/mysql testdatabase < /development/testdatabase.sql

assuming the testdatabase.sql was first sent to the development folder on the target server by using the scp command or some graphical client.

However, when running the above mysqldump command on Mac or Windows, all tablenames are stored in lowercase in the target sql script file by default. When it is then imported on a Linux server, all tablenames are in lowercase as well.

This is a problem, since the case of the tablenames matters on Linux machines. It is possible that applications – written in PHP, Java or something else – that will make use of the database expect some or all of these tablenames to be in upper case or have its first letter capitalized. These applications will then not run correctly.

Case sensitivity: the problem when migrating a MySql database from Windows/Mac to Linux

Hence, it is necessary the cases are preserved in the generated sql script. The answer to this Stackoverflow question suggests to restart the mysql database on the Mac/windows server with the lower_case_table_names parameter set to 0. For more information about this, please check the Mysql documentation about the lower_case_table_names parameter.

However, you might not have the permissions to take the database offline, restart it with this parameter and then run mysqldump again. You are then stuck with the script with only lower case table names. I think the only option is then to edit the script and replace all lower case occurences with the version with the proper case.

The method that I describe below is just a quick way to do this using Java.

Getting the list of tablenames with their case right

Most applications – such as Java web applications (using JPA/Hibernate for example), and Drupal and WordPress installations – are able to automatically create the database tables they need if they are missing. Hence, you can run the application once and make sure the tables are generated. You then open a MySql terminal.

On the Linux machine, you would run the following from the mysql installation bin folder:

mysql --user=<user> --password=<password>

After which you will get the mysql terminal.
Assuming the tables were already generated by the web application in the testdatabase database, you run:

mysql> show tables from testdatabase;

Which in my case renders:

+--------------------------------+
| Tables_in_testdatabase         |
+--------------------------------+
| Account_                       |
| Address                        |
| AnnouncementsDelivery          |
| AnnouncementsEntry             |
| AnnouncementsFlag              |
| AssetCategory                  |
| AssetCategoryProperty          |
| AssetEntries_AssetCategories   |
| AssetEntries_AssetTags         |
| AssetEntry                     |
| AssetLink                      |
| AssetTag                       |
| AssetTagProperty               |
| AssetTagStats                  |
| AssetVocabulary                |
| BlogsEntry                     |
| BlogsStatsUser                 |
| BookmarksEntry                 |
| BookmarksFolder                |
| BrowserTracker                 |
| CalEvent                       |
| Chat_Entry                     |
| Chat_Status                    |
| ClassName_                     |
| ClusterGroup                   |
| Company                        |
| Contact_                       |
| Counter                        |
| Country                        |
| CyrusUser                      |
| CyrusVirtual                   |
| DLFileEntry                    |
| DLFileRank                     |
| DLFileShortcut                 |
| DLFileVersion                  |
| DLFolder                       |
| EmailAddress                   |
| ExpandoColumn                  |
| ExpandoRow                     |
| ExpandoTable                   |
| ExpandoValue                   |
| Group_                         |
| Groups_Orgs                    |
| Groups_Permissions             |
| Groups_Roles                   |
| Groups_UserGroups              |
| IGFolder                       |
| IGImage                        |
| Image                          |
| JournalArticle                 |
| JournalArticleImage            |
| JournalArticleResource         |
| JournalContentSearch           |
| JournalFeed                    |
| JournalStructure               |
| JournalTemplate                |
| Layout                         |
| LayoutPrototype                |
| LayoutSet                      |
| LayoutSetPrototype             |
| ListType                       |
| Lock_                          |
| MBBan                          |
| MBCategory                     |
| MBDiscussion                   |
| MBMailingList                  |
| MBMessage                      |
| MBMessageFlag                  |
| MBStatsUser                    |
| MBThread                       |
| Mail_Account                   |
| Mail_Attachment                |
| Mail_Folder                    |
| Mail_Message                   |
| MembershipRequest              |
| OpenSocial_Gadget              |
| OrgGroupPermission             |
| OrgGroupRole                   |
| OrgLabor                       |
| Organization_                  |
| PasswordPolicy                 |
| PasswordPolicyRel              |
| PasswordTracker                |
| Permission_                    |
| Phone                          |
| PluginSetting                  |
| PollsChoice                    |
| PollsQuestion                  |
| PollsVote                      |
| Portlet                        |
| PortletItem                    |
| PortletPreferences             |
| QUARTZ_BLOB_TRIGGERS           |
| QUARTZ_CALENDARS               |
| QUARTZ_CRON_TRIGGERS           |
| QUARTZ_FIRED_TRIGGERS          |
| QUARTZ_JOB_DETAILS             |
| QUARTZ_JOB_LISTENERS           |
| QUARTZ_LOCKS                   |
| QUARTZ_PAUSED_TRIGGER_GRPS     |
| QUARTZ_SCHEDULER_STATE         |
| QUARTZ_SIMPLE_TRIGGERS         |
| QUARTZ_TRIGGERS                |
| QUARTZ_TRIGGER_LISTENERS       |
| RatingsEntry                   |
| RatingsStats                   |
| Region                         |
| Release_                       |
| ResourceAction                 |
| ResourceCode                   |
| ResourcePermission             |
| Resource_                      |
| Role_                          |
| Roles_Permissions              |
| SCFrameworkVersi_SCProductVers |
| SCFrameworkVersion             |
| SCLicense                      |
| SCLicenses_SCProductEntries    |
| SCProductEntry                 |
| SCProductScreenshot            |
| SCProductVersion               |
| SN_MeetupsEntry                |
| SN_MeetupsRegistration         |
| SN_WallEntry                   |
| ServiceComponent               |
| Shard                          |
| ShoppingCart                   |
| ShoppingCategory               |
| ShoppingCoupon                 |
| ShoppingItem                   |
| ShoppingItemField              |
| ShoppingItemPrice              |
| ShoppingOrder                  |
| ShoppingOrderItem              |
| SocialActivity                 |
| SocialEquityAssetEntry         |
| SocialEquityGroupSetting       |
| SocialEquityHistory            |
| SocialEquityLog                |
| SocialEquitySetting            |
| SocialEquityUser               |
| SocialRelation                 |
| SocialRequest                  |
| Subscription                   |
| TasksProposal                  |
| TasksReview                    |
| Team                           |
| Ticket                         |
| UserGroup                      |
| UserGroupGroupRole             |
| UserGroupRole                  |
| UserIdMapper                   |
| UserTracker                    |
| UserTrackerPath                |
| User_                          |
| Users_Groups                   |
| Users_Orgs                     |
| Users_Permissions              |
| Users_Roles                    |
| Users_Teams                    |
| Users_UserGroups               |
| Vocabulary                     |
| WSRP_WSRPConsumer              |
| WSRP_WSRPConsumerPortlet       |
| WSRP_WSRPProducer              |
| WebDAVProps                    |
| Website                        |
| WikiNode                       |
| WikiPage                       |
| WikiPageResource               |
| WorkflowDefinitionLink         |
| WorkflowInstanceLink           |
| blocked_user                   |
| deleted_message                |
| private_message                |
| read_message                   |
+--------------------------------+
176 rows in set (0.55 sec)

The above is a list of tablenames for the Liferay 6.0 portal and migrating these tables is a case in which I actually used the method described here. I stripped the list of all pipes(|) and saved the list of tablenames into a txt file, one tablename per line.

If you cannot generate the list of tablenames with the right case automatically like above, you will need to manually create this file, severely slowing up the method used in this tutorial.

Setting the case right with some Java code

The below Java class takes the testdatabase.sql file and the tablenames.txt file as input and writes the sql script with the cases set right to a testdatabase-converted.sql file:

public class SetCaseRight {

	private static List<String> readFile(String fileName){
		try{
			List<String> lines = new ArrayList<String>();
			FileInputStream fstream = new FileInputStream(fileName);
			DataInputStream in = new DataInputStream(fstream);
			BufferedReader br = new BufferedReader(new InputStreamReader(in));
			String strLine;
			while ((strLine = br.readLine()) != null)   {
				lines.add(strLine);
			}
			in.close();
			return lines;
		}catch (Exception e){
			e.printStackTrace();
			return null;
		}
	}

	private static String readFileAsString(String fileName){
        try {
			StringBuffer fileData = new StringBuffer(1000);
			BufferedReader reader = new BufferedReader(new FileReader(fileName));
			char[] buf = new char[1024];
			int numRead=0;
			while((numRead=reader.read(buf)) != -1){
			    String readData = String.valueOf(buf, 0, numRead);
			    fileData.append(readData);
			    buf = new char[1024];
			}
			reader.close();
			return fileData.toString();
		} catch (Exception e) {
			e.printStackTrace();
			return null;
		}
    }

	private static void writeStringToFile(String s, String fileName){
		try {
			BufferedWriter out = new BufferedWriter(new FileWriter(fileName));
			out.write(s);
			out.close();
		}
		catch (IOException e) {
			e.printStackTrace();
		}
	}

	public static void main(String[] args){
		String sqlSourceFile;
		String tableNamesTxtFile;
		String sqlTargetFile;
		if (args.length >= 1){ sqlSourceFile = args[0];} else { sqlSourceFile = "testdatabase.sql";}
		if (args.length >= 2){ tableNamesTxtFile = args[1];} else { tableNamesTxtFile = "tablenames.txt";}
		if (args.length >= 3){ sqlTargetFile = args[2];} else { sqlTargetFile = "testdatabase-converted.sql";}
		String sql = getSql(sqlSourceFile);
		for (String tableName: readTableNames(tableNamesTxtFile)){
			sql = sql.replaceAll("`(?i)" + tableName.trim()+"`", "`"+tableName.trim()+"`");
		}
		writeStringToFile(sql, sqlTargetFile);
	}

	public static String getSql(String sqlSourceFile){
		return readFileAsString(sqlSourceFile);
	}

	public static List<String> readTableNames(String tableNamesTxtFile){
		return readFile(tableNamesTxtFile);
	}
}

It is also possible to run the above class command line – without having to compile it yourself – by using this SetCaseRight class file:

java SetCaseRight testdatabase.sql tablenames.txt testdatabase-converted.sql

in a folder where java is on the PATH and where the first argument is the sqlTargetFile, the second argument is the tableNamesTxtFile and the third argument is the sqlTargetFile.

The resulting sql file is then ready to be imported on the Linux server.

Setting up the theme project

We are assuming you will be using Eclipse, but will always tell what is going on under the hood, so other IDE(or commandline!) guys should also be able to follow.

You should start out by downloading the Liferay Plugins SDK from the bottom of the Liferay additional files page.
Eclipse users should also install the Liferay IDE Eclipse Plugin and add a Liferay Server(and runtime) after. More info on how to do this can be found in the Liferay documentation on installing the Liferay IDE or at this very own blog, in the last part of the getting started with portals and Spring Portlet MVC post.

If you were already working with Liferay, you probably had these installed already anyway and the only thing you need to do then, is start your Liferay server.

Then, within Eclipse, choose New>Project>Liferay>Liferay Project and then check Theme in the box.

What the theme plugin does is run a script from the <liferay-sdk-home>/themes folder that generates a Liferay theme project for you, in your Eclipse workspace.
You can run the script manually by entering

./create.sh integrating-stuff "Integrating Stuff"

or

create.bat integrating-stuff "Integrating Stuff"

in a terminal from the said <sdkhome>/themes folder.

The Liferay IDE also takes a look at your Liferay runtime and copies a bunch of files to the docroot folder of your project.

These files represent the _styled theme, a very basic Liferay theme, which can be found in the portal source in the portal-web/docroot/html/themes folder or in the docroot/html/themes folder of your Liferay install.
As the Liferay documentation indicates, almost all themes will use the “_styled” theme as a base. If you want to start from scratch, the parent of your theme would be the “_unstyled” theme. Inheriting from “_unstyled” would give you quite a bit more work but would give you full control of the theme.

Any of the files that were copied to your docroot folder can be overridden by your theme by placing them in your _diffs folder. In other words, the _diffs folder will contain all files that differ from your parent theme.

Note that, as already indicated above, the “_styled” theme is not production ready. Since our _diffs folder is still empty, our theme – which is currently not overriding anything and is an exact representation of the “_styled” theme – would not look that great if we would deploy it to Liferay right now.
Therefore, it is a better idea to copy the content of the _diffs folder of the production ready theme you want to start from into your own theme’s _diffs folder. To start from the default “Classic” Liferay theme, download the portal source(link) and copy the _diffs folder from the Classic theme at portal-web/docroot/html/themes to your own theme’s _diffs folder.
Now your theme is production ready and ready for deployment.

2. Deploying the theme to Liferay

You probably already noticed the ant build script in the root of your project. If you run the default ant goal “deploy” of this script, the theme is published to your Liferay server.
(If you are not doing this from within Eclipse, you need to properly set the values in the build.properties file in the ssdk’s theme folder.)

Let’s deploy and enable our theme on our Liferay server, which for now, looks exactly the same as the default “Classic” theme. If the theme deployed successfully, the Liferay logging should contain something like:

3. Overriding parts of the parent theme

The content of the_diffs folder we copied from the Classic theme consisted of 3 folders: css, images and js(javascript). In the _styled theme there is one additional folder, templates, of which the classic theme does not override anything. This is recommended practice: themes should try to avoid overriding template files since they are likely to change in between Liferay versions so changing them might make porting the theme to a new Liferay version more difficult.

a. overriding css

Now, to add a nice gradient background image and change the default text size and font, we have to modify our custom.css file.

We will change the

body {
	background: #EEF0F2;
	font-size: 11px;
}

into:

body {
        background-color: #4F555B;
	background-image: url("../images/layout/bg-grad.gif");
	background-repeat: repeat-x;
	color: #FFFFFF;
	font-family: Verdana,Helvetica,sans-serif;
	font-size: 11px;
}

and also create a layout_folder in our images folder and add the file bg-grad.gif to it.

If we redeploy the theme and load our Liferay page, we see that the background has changed and text appears a bit different. Pretty easy, huh.

b. Overriding and adding images

In the previous section, we already added a background image to the theme. If you want to override an image, you just need to add it to the images folder as well, but the image needs to have the same folder structure and file name as the one you want to override.
For example, if you want to override some emoticons, you would create an emoticons folder(you need to look at the _styled theme files to know what you can override) and start putting names there of files that also exist in the parent theme(angry.gif, blank.gif,..).

c. Overriding the custom javascript

This can be done the same way images are overridden. However, the _styled theme only has one javascript file: main.js, which, with Liferay 6, only contains some logic that keeps the Liferay breadcrumbs always on screen.

d. Overriding the template files

As mentioned already, the Liferay documentation advises to avoid overriding the template files. That being said, almost all projects I know off involved creating themes that were modifying these templates.
If one wants to add or remove parts of the page and the modifications go beyond styling dom elements, these templates are usually changed. The view technology these files use is Velocity.

The two most important ones are portal-normal.vm – which renders the whole page – and portlet.vm – which renders one portlet. The other ones respectively define additional velocity variables(init_custom.vm), render the navigation bar(navigation.vm) and render the popup pages(portal_pop_up.vm).

Let’s take a look at the portal-normal.vm Velocity template:

<!DOCTYPE html>

#parse ($init)

<html class="#language("lang.dir")" dir="#language("lang.dir")" lang="$w3c_language_id">

<head>
	<title>$the_title - $company_name</title>

	$theme.include($top_head_include)
</head>

<body class="$css_class">

#if($is_signed_in)
	#dockbar()
#end

<div id="wrapper">
	<a href="#main-content" id="skip-to-content">#language("skip-to-content")</a>

	<header id="banner" role="banner">
		<hgroup id="heading">
			<h1 class="company-title">
				<a class="logo" href="$company_url" title="#language("go-to") $company_name">
					<span>$company_name</span>
				</a>
			</h1>

			<h2 class="community-title">
				<a href="$community_default_url" title="#language("go-to") $community_name">
					<span>$community_name</span>
				</a>
			</h2>

			<h3 class="page-title">
				<span>$the_title</span>
			</h3>
		</hgroup>

		#if(!$is_signed_in)
			<a href="$sign_in_url" id="sign-in" rel="nofollow">$sign_in_text</a>
		#end

		#if ($update_available_url)
			<div class="popup-alert-notice">
				<a class="update-available" href="$update_available_url">#language("updates-are-available-for-liferay")</a>
			</div>
		#end

		#if ($has_navigation)
			#parse ("$full_templates_path/navigation.vm")
		#end
	</header>

	<div id="content">
		<nav class="site-breadcrumbs" id="breadcrumbs">
			<h1>
				<span>#language("breadcrumbs")</span>
			</h1>

			#breadcrumbs()
		</nav>

		#if ($selectable)
			$theme.include($content_include)
		#else
			$portletDisplay.recycle()

			$portletDisplay.setTitle($the_title)

			$theme.wrapPortlet("portlet.vm", $content_include)
		#end
	</div>

	<footer id="footer" role="contentinfo">
		<p class="powered-by">
			#language("powered-by") <a href="http://www.liferay.com" rel="external">Liferay</a>
		</p>
	</footer>
</div>

</body>

$theme.include($bottom_include)

</html>

If one would want (and is allowed to) remove the “Powered by Liferay” line, he can easily do this by just removing the line containing the text.

If the static htmldesign/prototypes are already made by web designers, it is easy for developers to port this created design to Liferay by copying the html to the portal-normal.vm file and inserting the Velocity directives where required.
All these Velocity templates are clearly readable. When taking a look at the above file, it is obvious how to replace the whole Liferay header, for example.

4. More advanced stuff

There is more advanced stuff one can use to fine tune a theme and is not covered by this tutorial – such as adding settings or color schemes to a theme. Although this tutorial gets you started from a practical point of view, it is probably a good idea to scan through the Liferay documentation on themes as well.

WebServer class

Everything that we are going to use is already present in the Android API. We are not going to need any additional libs.

A pragmatic implementation looks like this:

package com.integratingstuff.android.webserver;

import java.io.IOException;
import java.net.ServerSocket;
import java.net.Socket;
import java.net.SocketException;

import org.apache.http.HttpException;
import org.apache.http.impl.DefaultConnectionReuseStrategy;
import org.apache.http.impl.DefaultHttpResponseFactory;
import org.apache.http.impl.DefaultHttpServerConnection;
import org.apache.http.params.BasicHttpParams;
import org.apache.http.protocol.BasicHttpContext;
import org.apache.http.protocol.BasicHttpProcessor;
import org.apache.http.protocol.HttpRequestHandlerRegistry;
import org.apache.http.protocol.HttpService;
import org.apache.http.protocol.ResponseConnControl;
import org.apache.http.protocol.ResponseContent;
import org.apache.http.protocol.ResponseDate;
import org.apache.http.protocol.ResponseServer;

import android.content.Context;

public class WebServer {

	public static boolean RUNNING = false;
	public static int serverPort = 8080;

	private static final String ALL_PATTERN = "*";
	private static final String EXCEL_PATTERN = "/*.xls";
	private static final String HOME_PATTERN = "/home.html";

	private Context context = null;

	private BasicHttpProcessor httpproc = null;
	private BasicHttpContext httpContext = null;
	private HttpService httpService = null;
	private HttpRequestHandlerRegistry registry = null;

	public WebServer(Context context) {
		this.setContext(context);

		httpproc = new BasicHttpProcessor();
		httpContext = new BasicHttpContext();

		httpproc.addInterceptor(new ResponseDate());
		httpproc.addInterceptor(new ResponseServer());
		httpproc.addInterceptor(new ResponseContent());
		httpproc.addInterceptor(new ResponseConnControl());

		httpService = new HttpService(httpproc,
		    new DefaultConnectionReuseStrategy(), new DefaultHttpResponseFactory());

		registry = new HttpRequestHandlerRegistry();

		//registry.register(HOME_PATTERN, new HomeCommandHandler(context));

		httpService.setHandlerResolver(registry);
	}

	private ServerSocket serverSocket;

	public void runServer() {
		try {
			serverSocket = new ServerSocket(serverPort);

			serverSocket.setReuseAddress(true);

			while (RUNNING) {
				try {
					final Socket socket = serverSocket.accept();

					DefaultHttpServerConnection serverConnection = new DefaultHttpServerConnection();

					serverConnection.bind(socket, new BasicHttpParams());

					httpService.handleRequest(serverConnection, httpContext);

					serverConnection.shutdown();
				} catch (IOException e) {
					e.printStackTrace();
				} catch (HttpException e) {
					e.printStackTrace();
				}
			}

			serverSocket.close();
		} catch (SocketException e) {
			e.printStackTrace();
		} catch (IOException e) {
			e.printStackTrace();
		}

		RUNNING = false;
	}

	public synchronized void startServer() {
		RUNNING = true;
		runServer();
	}

	public synchronized void stopServer() {
		RUNNING = false;
		if (serverSocket != null) {
			try {
				serverSocket.close();
			} catch (IOException e) {
				e.printStackTrace();
			}
		}
	}

	public void setContext(Context context) {
		this.context = context;
	}

	public Context getContext() {
		return context;
	}
}

In the WebServer contructor we are initializing an HttpService. We are using all the default implementations here. The interesting part is the construction of the HttpRequestHandlerRegistry. As we will see, we will add a custom requesthandler to this registry which shall build the response for a particular request.

In the runServer method we actually start the server by initiating a low level ServerSocket and make it accept connections. We then continiously bind the socket to our webserver implementation and tell it to handle the request.

The startServer and stopServer methods are the methods actually used to start and stop the server.

In Android apps, a server like this should be run as an Android Service. So, we are implementing the following WebServerService:

package com.integratingstuff.android.webserver;

import android.app.Service;
import android.content.Intent;
import android.os.IBinder;
import android.util.Log;

public class WebServerService extends Service {

	private WebServer server = null;

	@Override
	public void onCreate() {
		Log.i("HTTPSERVICE", "Creating and starting httpService");
		super.onCreate();
		server = new WebServer(this);
		server.startServer();
	}

	@Override
	public void onDestroy() {
		Log.i("HTTPSERVICE", "Destroying httpService");
		server.stopServer();
		super.onDestroy();
	}

	@Override
	public IBinder onBind(Intent intent) {
		return null;
	}
}

Which we can start from the Android application like this:

Intent webServerService = new Intent(context, WebServerService.class);
context.startService(webServerService);

HttpRequestHandler

Our WebServer is now running, but we did not register any requesthandlers yet. In our webserver, we commented the following line:

registry.register(HOME_PATTERN, new HomeCommandHandler(context));

Let’s uncomment it and implement the HomeCommandHandler:

package com.integratingstuff.android.webserver;

import java.io.IOException;
import java.io.OutputStream;
import java.io.OutputStreamWriter;

import org.apache.http.HttpEntity;
import org.apache.http.HttpException;
import org.apache.http.HttpRequest;
import org.apache.http.HttpResponse;
import org.apache.http.entity.ContentProducer;
import org.apache.http.entity.EntityTemplate;
import org.apache.http.protocol.HttpContext;
import org.apache.http.protocol.HttpRequestHandler;

import android.content.Context;

public class HomeCommandHandler implements HttpRequestHandler {
	private Context context = null;

	public HomeCommandHandler(Context context) {
		this.context = context;
	}

	@Override
	public void handle(HttpRequest request, HttpResponse response,
	    HttpContext httpContext) throws HttpException, IOException {
		HttpEntity entity = new EntityTemplate(new ContentProducer() {
			public void writeTo(final OutputStream outstream) throws IOException {
				OutputStreamWriter writer = new OutputStreamWriter(outstream, "UTF-8");
				String resp = "<html><head></head><body><h1>Home<h1><p>This is the homepage.</p></body></html>";

				writer.write(resp);
				writer.flush();
			}
		});
		response.setHeader("Content-Type", "text/html");
		response.setEntity(entity);
	}

	public Context getContext() {
		return context;
	}
}

We are mapping the pattern home.html to this requesthandler. This handler will send some basic html back as its response.

If you now surf to http://<ip-of-device>:8080/home.html, you should see the homepage with the “This is the homepage” message.

So now, our webserver is up and running and responding to home.html requests. You can easily add more requesthandlers to the registry, that match specific patterns or wildcard * patterns.

MultipartParser

The above can be easily found elsewhere, so to give this article some added value, I will quickly indicate below how to parse form date/get the contents of files contained in the request.

Code

We will need to parse a multipart request manually. I am just going to give the code snippets and I am not going into the parsing intrinsics, but basically, the http protocol defines boundary delimiters for multipart messages. What we are doing with the following MultipartParser is read the entire sent form into a Multivalued map(a map of which the value is a list of a certain type of object, so every key is mapped to a list of 1 or more values). Note that the MultipartParser actually uses a CaseInsensitiveMultivaluedMap that wraps a regular MultivaluedMap, because keys like “name” and “NAME” map to the same formproperty.

MultipartParser

package com.integratingstuff.android.webserver;

import java.io.IOException;
import java.io.InputStream;

public class MultipartParser {

	public final static String SEP = "\n";

	private InputStream is;
	private byte[] boundaryBA;
	static private byte[] boundaryDelimiterBA = "--".getBytes();

	private MultivaluedMap partHeaders;
	private PartInputStream partIS;

	private static int BOUNDARY_TYPE_START = 0;
	private static int BOUNDARY_TYPE_END = 1;
	// private static int BOUNDARY_TYPE_INVALID = 2;

	private byte[] buff;
	// the next byte to return
	private int buffIdx = 0;
	// The number of bytes that were set on the buffer (red from is);
	private int buffSize = 0;
	// the position of the next boundary ("--boundary"), -1 if does not exist in
	// this buffer
	private int boundryIdx = -1;
	// The index of the byte that is suspected of been the boundary
	private int saveIdx = 0;

	// This is a temp array that is used to read stuff into it.
	private byte[] temp = new byte[1024];

	public MultipartParser(InputStream is, String boundary) {
		this.is = is;
		boundaryBA = ("--" + boundary).getBytes();
		// make sure to allocate a buffer that is at least double then the
		// boundary length
		int buffLength = Math.max(8192, boundaryBA.length * 2);
		buff = new byte[buffLength];

	}

	private void shiftBuff() {
		System.arraycopy(buff, buffIdx, buff, 0, buffSize - buffIdx);

		buffSize -= buffIdx;
		saveIdx -= buffIdx;
		if (saveIdx < 0)
			saveIdx = 0;
		/*
		 * throw new RuntimeException("This should never happend, we found a bug.");
		 */
		boundryIdx = Math.max(-1, boundryIdx - buffIdx);
		buffIdx = 0;

		// for debug purposes
		// Arrays.fill(buff, buffIdx, buff.length-1, (byte)0);

	}

	public boolean nextPart() throws IOException {
		// if this is the first next just get rid of the PREAMBLE
		if (partIS == null) {
			partIS = new PartInputStream();
		}
		// clear the part/preamble bytes that were not read
		digestPartStream();
		if (digestBoundary() == BOUNDARY_TYPE_END)
			return false;
		partIS.setState(PartInputStream.STATE_NOT_ACTIVE);
		partIS = new PartInputStream();
		partHeaders = parseHeaders();
		return partHeaders != null;
	}

	public InputStream getPartBodyStream() {
		return partIS;
	}

	public MultivaluedMap getPartHeaders() {
		return partHeaders;
	}

	// read till end of stream (next boundary)
	private void digestPartStream() throws IOException {
		while (partIS.read(temp) != -1) {
		}
	}

	private boolean compareByte(byte[] a, int aOffset, byte[] b, int bOffset,
	    int length) {
		for (int i = 0; i < length; i++) {
			if (a[aOffset + i] != b[bOffset + i])
				return false;
		}
		return true;
	}

	private int digestBoundary() throws IOException {
		// it might be that there is a new line before the boundary
		digestNewLine();

		// promote pointers to the end of the boundary
		buffIdx += boundaryBA.length;
		saveIdx += boundaryBA.length; // DO NOT DELETE

		// check if this is an end boundary
		int unredBytes = verifyByteReadyForRead(2);
		if (unredBytes >= 2) {
			if (compareByte(buff, buffIdx, boundaryDelimiterBA, 0,
			    boundaryDelimiterBA.length))
				return BOUNDARY_TYPE_END;
		}
		// OK
		digestNewLine();
		boundryIdx = -1;
		findBounderyIfNeeded();
		return BOUNDARY_TYPE_START;
	}

	private void findBounderyIfNeeded() {
		if (boundryIdx == -1) {
			boundryIdx = indexOf(buff, saveIdx, buffSize, boundaryBA);
			if (boundryIdx != -1) {
				int nlSize = 0;
				if (boundryIdx > 1) {
					if (buff[boundryIdx - 2] == '\r' && buff[boundryIdx - 1] == '\n')
						nlSize = 2;
					else
						nlSize = 1;
				}
				if (boundryIdx == 1) {
					nlSize = 1;
				}

				saveIdx = boundryIdx - nlSize;
			} else {
				// the boundary was not found, but we can promote the save till
				// boundary size + NL size
				saveIdx = Math.max(saveIdx, buffSize - (boundaryBA.length + 2));
			}
		}
	}

	private int verifyByteReadyForRead(int required) throws IOException {
		int unreadBytes = buffSize - buffIdx - 1;
		if (unreadBytes < required) {
			fetch(required - unreadBytes);
			unreadBytes = buffSize - buffIdx;
		}
		return unreadBytes;
	}

	private int fetch(int minmum) throws IOException {
		int res = 0;
		int max2featch = buff.length - buffSize;

		if (max2featch < minmum) {
			shiftBuff();
			max2featch = buff.length - buffSize;
		}

		while (res < minmum && max2featch > 0) {
			max2featch = buff.length - buffSize;

			int read = is.read(buff, buffSize, max2featch);
			if (read == -1) {
				if (res == 0)
					return -1;
				else
					break;
			}
			res += read;
			buffSize += read;
		}
		findBounderyIfNeeded();
		return res;

	}

	private void digestNewLine() throws IOException {
		// make sure we have enough byte to read
		int unreadBytes = verifyByteReadyForRead(2);
		int size = 0;

		if (unreadBytes >= 2 && buff[buffIdx] == '\r' && buff[buffIdx + 1] == '\n')
			size = 2;
		else if (buff[buffIdx] == '\r')
			size = 1;
		else if (buff[buffIdx] == '\n')
			size = 1;
		buffIdx += size;
		if (saveIdx < buffIdx)
			saveIdx = buffIdx;
	}

	private int indexOf(byte[] ba, int start, int end, byte[] what) {
		for (int i = start; i < end - what.length + 1; i++) {
			// only if the first byte equals do the compare (to improve
			// performance)
			if (ba[i] == what[0])
				if (compareByte(ba, i, what, 0, what.length))
					return i;
		}
		return -1;
	}

	private MultivaluedMap parseHeaders() throws IOException {

		MultivaluedMap headers = new CaseInsensitiveMultivaluedMap();

		String line;
		do {
			line = readLine();
			if (line == null || line.equals(""))
				break;
			int semIdx = line.indexOf(":");
			headers.add(line.substring(0, semIdx).trim(), line.substring(semIdx + 1)
			    .trim());

		} while (true);
		if (saveIdx < buffIdx)
			saveIdx = buffIdx;
		return headers;
	}

	private String readLine() throws IOException {

		int lineIdx = 0;
		int breakeSize = 0;
		while (lineIdx <= verifyByteReadyForRead(lineIdx)) {
			if (buff[buffIdx + lineIdx] == '\n') {
				breakeSize = 1;
				break;
			}
			if (buff[buffIdx + lineIdx] == '\r') {
				if ((verifyByteReadyForRead(lineIdx + 1) >= lineIdx + 1)
				    && (buff[buffIdx + lineIdx + 1] == '\n')) {
					breakeSize = 2;
					break;
				} else {
					breakeSize = 1;
					break;
				}
			}
			lineIdx++;
		}

		// got to the end of input without NL
		if (lineIdx == 0) {
			buffIdx += breakeSize;
			return null;
		}

		String hdr = new String(buff, buffIdx, lineIdx);
		buffIdx += lineIdx + breakeSize;
		return hdr;
	}

	public class PartInputStream extends InputStream {
		// The state of the part Stream
		// 0 active
		// 1 not active (the Parser already moved to the next part.)
		private int state = 0;
		public final static int STATE_ACTIVE = 0;
		public final static int STATE_NOT_ACTIVE = 1;

		public void setState(int status) {
			this.state = status;

		}

		@Override
		public int read(byte[] b, int off, int len) throws IOException {
			if (state == STATE_NOT_ACTIVE) {
				throw new IOException(
				    "Stream allready closed: the PartInputStream is not accesable after moving to the next part");
			}
			int available = verifyNumOfByteToReadB4Boundary(len);
			if (available < 1) {
				return available
                        }
			int size2copy = Math.min(len, available);
			System.arraycopy(buff, buffIdx, b, off, size2copy);
			buffIdx += size2copy;
			return size2copy;
		}

		@Override
		public int read() throws IOException {
			if (state == STATE_NOT_ACTIVE) {
				throw new IOException(
				    "Stream allready closed: the PartInputStream is not accesable after moving to the next part");
			}
			int i = verifyNumOfByteToReadB4Boundary(1);
			if (i < 1)
				return -1;
			// make sure that the return value is 0 - 255
			int res = buff[buffIdx] & 0xff;
			if (res < 0) {
				int t = 0;
				t++;
			}
			buffIdx++;
			return res;

		}

		private int verifyNumOfByteToReadB4Boundary(int minmum) throws IOException {
			int availabe = saveIdx - buffIdx;
			if (availabe >= minmum)
				return availabe;
			//
			if (saveIdx <= boundryIdx) {
				if (availabe == 0)
					return -1;
				return availabe;
			}
			int fetched = fetch(minmum - availabe);
			availabe = saveIdx - buffIdx;
			if (availabe == 0 && fetched == -1)
				return -1;

			return availabe;

		}

		@Override
		public int available() {
			return saveIdx - buffIdx;
		}
	}
}

The Selenium 2 test

We implement a Selenium 2 test similar to the one we introduced in Converting Selenium 1 to Selenium 2 tests. We are using System.getProperty calls now. A System.getProperty call fetches a certain system property. These system properties will be set by Maven first, as we will see later.

import java.io.File;

import org.junit.Test;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxBinary;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.support.PageFactory;
import org.openqa.selenium.support.ui.ExpectedCondition;
import org.openqa.selenium.support.ui.WebDriverWait;

public class PortalPageTest {

  private FirefoxDriver createFirefoxDriver() {
    File firefoxBin = new File(System.getProperty("selenium.pathToFirefoxBinary"));
    FirefoxBinary firefoxBinary = new FirefoxBinary(firefoxBin);
    FirefoxDriver driver = new FirefoxDriver(firefoxBinary, null);
    return driver;
  }

  @Test
  public void testPortalPage() {
    final FirefoxDriver firefoxDriver = createFirefoxDriver();

    firefoxDriver.get(System.getProperty("selenium.baseURL"));

    final PortalPage portalPage = new PortalPage();
    WebDriverWait wait = new WebDriverWait(firefoxDriver, 20 * 1000, 200);
    wait.until(new ExpectedCondition() {
      @Override
      public Boolean apply(WebDriver driver) {
        PageFactory.initElements(firefoxDriver, portalPage);
        return portalPage.testPortlet.getText().equals("This is our test portlet.");
      }
    });
  }
}

And the related Page object:

import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.FindBy;
public class PortalPage {

  @FindBy(id = "testPortlet")
  public WebElement testPortlet;

}

The Maven profile

In summary, our Maven profile will:

1. Clean the Liferay server directories during the clean phase.
2. Start the Liferay server, deploy the application and wait 5 minutes for the application to finish deploying during the pre-integration-test phase.
3. Execute the Selenium 2 tests. It will not fail if any of the tests fails but instead, it will write the results away.
4. Stop the Liferay server during the post-integration-test phase.

Profile breakup

We build up the profile part by part.

1. Activation

<activation>
    <property>
      <name>liferay.home.dir</name>
    </property>
</activation>

The profile is getting activated when we supply a liferay.home.dir. However, this is not the only property we are going to need for a successful run. We need to give all the necessary info about the browser we are going to run the Selenium tests with and the server we are going to run the tests on. So, next to the liferay.home.dir, we will also need to supply the selenium.firefox.binary and the liferay.port properties.
Hence, to do an install with the profile enabled, we will need to run Maven with a goal like this:

clean install –Dliferay.home.dir=/development/Liferay
–Dselenium.firefox.binary=/opt/firefox/firefox.sh –Dliferay.port=8080

2. Cargo plugin

<plugin>
  <groupId>org.codehaus.cargo</groupId>
  <artifactId>cargo-maven2-plugin</artifactId>
  <version>1.0.6</version>
  <executions>
    <execution>
      <id>start-liferay</id>
      <phase>pre-integration-test</phase>
      <goals>
        <goal>start</goal>
      </goals>
    </execution>
    <execution>
      <id>stop-liferay</id>
      <phase>post-integration-test</phase>
      <goals>
        <goal>stop</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <skip>false</skip>
    <wait>false</wait>
    <container>
      <containerId>tomcat6x</containerId>
      <home>${liferay.home.dir}/tomcat-6.0.23</home>
      <timeout>30000</timeout>
      <type>installed</type>
      <systemProperties>
        <file.encoding>UTF8</file.encoding>
        <external-properties>${liferay.home.dir}/portal-ext.properties
        </external-properties>
      </systemProperties>
    </container>
    <configuration>
      <home>${liferay.home.dir}/tomcat-6.0.23</home>
      <type>existing</type>
      <properties>
        <cargo.jvmargs>-Xmx1024m -XX:MaxPermSize=512m</cargo.jvmargs>
        <cargo.servlet.port>${liferay.port}</cargo.servlet.port>
        <cargo.logging>high</cargo.logging>
      </properties>
    </configuration>
  </configuration>
</plugin>

The Cargo plugin is going to start and stop Liferay in a talled application server container. The server is started before integration testing begins and stopped after integration testing. In the configuration part of the plugin, we configure our server: where it is located, the vmargs we want to start it with, etc..
The <skip> part of the configuration means we want Cargo to run and start the server. If set to true, the execution of this plugin would be skipped and no container would be started.
The <wait> part of the configuration determines whether we want to wait after the Cargo container is started or not. If set to true, we would need to manually intervene(as in pressing some buttons) to continue the Maven process. For automated integration testing, this obviously needs to be set to false.
For more information about configuring the cargo plugin, visit the Cargo maven plugin reference.
One thing to keep in mind is that the cargo.servlet.port should be the port Liferay runs on(8080 by default). If this port is not configured correctly, Cargo will timeout because it cannot determine whether the server started or not. It is the port Cargo listens on, not the port the server starts on. The server starts on the default port listed in its own configuration files.

3. Liferay plugin

<plugin>
  <groupId>com.liferay.maven.plugins</groupId>
  <artifactId>liferay-maven-plugin</artifactId>
  <version>6.0.5</version>
  <executions>
    <execution>
      <id>liferay-deploy</id>
      <phase>pre-integration-test</phase>
      <goals>
        <goal>deploy</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <autoDeployDir>${liferay.home.dir}/deploy</autoDeployDir>
  </configuration>
</plugin>

The above plugin deploys our war(the product of the pom) to the Liferay server. It also executes before integration testing begins. Note that in the default case, Maven plugins execute in the order in which they are defined. So in this case, the Liferay server is started first, after which the application(the portlets) is deployed to the started server.
Note that this plugin deploys the application, but it does NOT wait for the app to be deployed. Maven execution continues right away, so we will need to build something in to wait for the app to be deployed, before we actually start running the Selenium 2 tests.

4. Antrun plugin

<plugin>
  <artifactId>maven-antrun-plugin</artifactId>
  <executions>
    <execution>
      <id>wait-for-liferay</id>
      <phase>pre-integration-test</phase>
      <configuration>
        <tasks>
          <echo>Wait 300 seconds for Liferay..</echo>
          <waitfor maxwait="300" maxwaitunit="second">
            <available file="errors.log" />
          </waitfor>
        </tasks>
      </configuration>
      <goals>
        <goal>run</goal>
      </goals>
    </execution>
    <execution>
      <id>clean-liferay</id>
      <phase>clean</phase>
      <configuration>
        <tasks>
          <echo>Cleaning Liferay..</echo>
          <delete
            dir="${liferay.home.dir}/tomcat-6.0.23/webapps/${project.build.finalName}"
            quiet="true" />
          <delete dir="${liferay.home.dir}/tomcat-6.0.23/work"
            quiet="true" />
          <delete dir="${liferay.home.dir}/tomcat-6.0.23/temp"
            quiet="true" />
        </tasks>
      </configuration>
      <goals>
        <goal>run</goal>
      </goals>
    </execution>
  </executions>
</plugin>

The antrun plugin enables all Ant functionality within a Maven context.
This the dirty part of our profile. If there is something to improve, it is this part.
We are essentially doing two things here.
Before integration testing begins, we are waiting 300 seconds. This gives the application enough time to deploy, so it is ready once the selenium tests start running. Note that you could use something more intelligent here than plain waiting, such as polling the server to check whether a certain page is loading or even building a Liferay hook that when deployed, can be used to ask Liferay through a webservice whether it has finished deploying or not. However, that would require building something custom and is outside of the scope of this article.
The second ant execution configured for the plugin, cleans the Liferay server in the clean phase, so we start with a clean Liferay server every run. We had to add this because we were having issues with the new deploy conflicting with certain parts of an old deploy from time to time. Among other things, this makes sure the old version is not deployed first when the server starts. Same as with the other execution, one could come up with more sophisticated ways to do this, but it should be enough to illustrate the idea.

5. Surefire plugin

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.7.2</version>
  <configuration>
    <skip>true</skip>
  </configuration>
</plugin>

The surefire plugin is the default plugin to run unit tests with. Because we want to use the failsafe plugin in the next section, the execution of this plugin should be skipped. Therefore, we are setting <skip> to true.

6. Failsafe plugin

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>2.9</version>
  <executions>
    <execution>
      <id>selenium2-firefox-test</id>
      <phase>integration-test</phase>
      <goals>
        <goal>integration-test</goal>
      </goals>
      <configuration>
        <includes>
          <include>**/SeleniumIntegrationTestSuite.java
                </include>
        </includes>
        <systemPropertyVariables>
          <selenium.baseURL>http://localhost:${liferay.port}/web/guest/testing
          </selenium.baseURL>
          <selenium.pathToFirefoxBinary>${selenium.firefox.binary}
          </selenium.pathToFirefoxBinary>
        </systemPropertyVariables>
        <reportsDirectory>${project.build.directory}/failsafe-reports/firefox
        </reportsDirectory>
      </configuration>
    </execution>
  </executions>
</plugin>

With the failsafe plugin, we are going to run our Selenium 2 test. The main difference with the surefire plugin is that this plugin will make sure the Maven execution keeps executing is some integration tests fail.
There is also some additional configuration required.
Basically, we make a Suite class to which we add our test:

import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;

@RunWith(Suite.class)
@SuiteClasses({ PortalPageTest.class })
public class SeleniumIntegrationTestSuite{
}

All tests defined in this suite will be run using the supplied firefox executable. Note that the path to the firefox bin is not part of the failsafe plugin, but is a system property that we are setting. This system property is then fetched within the running Selenium 2 test. Same for the Selenium base url. Note that in order to run the test (suite) successfully, you will need to add a page “testing” to your Liferay portal and make sure the portlet is deployed on that page first.
Finally, the outcome of all tests will be written to htmlfiles in the supplied reportsDirectory.

The complete profile

To end, we are showing the entire Maven profile here for reference:

<profile>
  <id>liferay</id>
  <activation>
    <property>
      <name>liferay.home.dir</name>
    </property>
  </activation>
  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.cargo</groupId>
        <artifactId>cargo-maven2-plugin</artifactId>
        <version>1.0.6</version>
        <executions>
          <execution>
            <id>start-liferay</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>start</goal>
            </goals>
          </execution>
          <execution>
            <id>stop-liferay</id>
            <phase>post-integration-test</phase>
            <goals>
              <goal>stop</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <skip>false</skip>
          <wait>false</wait>
          <container>
            <containerId>tomcat6x</containerId>
            <home>${liferay.home.dir}/tomcat-6.0.23
            </home>
            <timeout>30000</timeout>
            <type>installed</type>
            <systemProperties>
              <file.encoding>UTF8</file.encoding>
              <external-properties>${liferay.home.dir}/portal-ext.properties
              </external-properties>
            </systemProperties>
          </container>
          <configuration>
            <home>${liferay.home.dir}/tomcat-6.0.23</home>
            <type>existing</type>
            <properties>
              <cargo.jvmargs>-Xmx1024m -XX:MaxPermSize=512m
              </cargo.jvmargs>
              <cargo.servlet.port>${liferay.port}</cargo.servlet.port>
              <cargo.logging>high</cargo.logging>
            </properties>
          </configuration>
        </configuration>
      </plugin>

      <plugin>
        <groupId>com.liferay.maven.plugins</groupId>
        <artifactId>liferay-maven-plugin</artifactId>
        <version>6.0.5</version>
        <executions>
          <execution>
            <id>liferay-deploy</id>
            <phase>pre-integration-test</phase>
            <goals>
              <goal>deploy</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <autoDeployDir>${liferay.home.dir}/deploy</autoDeployDir>
        </configuration>
      </plugin>

      <plugin>
        <artifactId>maven-antrun-plugin</artifactId>
        <executions>
          <execution>
            <id>wait-for-liferay</id>
            <phase>pre-integration-test</phase>
            <configuration>
              <tasks>
                <echo>Wait 300 seconds for Liferay..</echo>
                <waitfor maxwait="300" maxwaitunit="second">
                  <available file="errors.log" />
                </waitfor>
              </tasks>
            </configuration>
            <goals>
              <goal>run</goal>
            </goals>
          </execution>
          <execution>
            <id>clean-liferay</id>
            <phase>clean</phase>
            <configuration>
              <tasks>
                <echo>Cleaning Liferay..</echo>
                <delete
                  dir="${liferay.home.dir}/tomcat-6.0.23/webapps/${project.build.finalName}"
                  quiet="true" />
                <delete dir="${liferay.home.dir}/tomcat-6.0.23/work"
                  quiet="true" />
                <delete dir="${liferay.home.dir}/tomcat-6.0.23/temp"
                  quiet="true" />
              </tasks>
            </configuration>
            <goals>
              <goal>run</goal>
            </goals>
          </execution>
        </executions>
      </plugin>

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-failsafe-plugin</artifactId>
        <version>2.9</version>
        <executions>
          <execution>
            <id>selenium2-firefox-test</id>
            <phase>integration-test</phase>
            <goals>
              <goal>integration-test</goal>
            </goals>
            <configuration>
              <includes>
                <include>**/SeleniumIntegrationTestSuite.java
                </include>
              </includes>
              <systemPropertyVariables>
                <selenium.baseURL>http://localhost:${liferay.port}/web/guest/testing
                </selenium.baseURL>
                <selenium.pathToFirefoxBinary>${selenium.firefox.binary}
                </selenium.pathToFirefoxBinary>
              </systemPropertyVariables>
              <reportsDirectory>${project.build.directory}/failsafe-reports/firefox
              </reportsDirectory>
            </configuration>
          </execution>
        </executions>
      </plugin>

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.7.2</version>
        <configuration>
          <skip>true</skip>
        </configuration>
      </plugin>

    </plugins>
  </build>
</profile>

What is Selenium?

As the Selenium homepage states, Selenium automates browsers. With Selenium, you can script your browser. Usually, it is used for automated testing of web applications, but sometimes it is used to automate certain repetitive web-based tasks as well.

Selenium tests

When using Selenium tests, there is one big choice one has to make: whether to use Selenium IDE or Selenium WebDriver.

Selenium IDE tests

Selenium IDE is a Firefox add-on that will do simple record-and-playback of interactions with the browser.
You can download it from the Selenium website and install it as a plugin. Once it is installed, open Firefox, go to Tools>Selenium IDE. We are going to record a simple test that goes to dzone.com and clicks on the “New Links” link.
To record, just press the red button in the IDE. Selenium IDE starts recording now. Everytime you click or type something in Firefox, the action will be recorded in the script, and in the end, Selenium IDE will be able to reproduce everything you did when interacting with the browser.

In our case, the resulting Selenium 1 tests looks like this(click on Source in Selenium IDE):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head profile="http://selenium-ide.openqa.org/profiles/test-case">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<link rel="selenium.base" href="http://www.dzone.com/" />
<title>New Test</title>
</head>
<body>
<table cellpadding="1" cellspacing="1" border="1">
<thead>
<tr><td rowspan="1" colspan="3">New Test</td></tr>
</thead><tbody>
<tr>
	<td>open</td>
	<td>/links/index.html</td>
	<td></td>
</tr>
<tr>
	<td>clickAndWait</td>
	<td>link=New links</td>
	<td></td>
</tr>
</tbody></table>
</body>
</html>

To run the test, you can just click the green arrow. If you do, Selenium IDE goes to the dzone.com website and clicks the “New Links” link.

Selenium WebDriver tests

Selenium WebDriver is a collection of language specific bindings(Java, Python,..) to drive a browser. Unlike Selenium IDE, Selenium WebDriver uses native calls to the browser.
It was introduced with Selenium 2.0, which is a new and extended version of the Selenium 1.0 API. Usually, when talking about Selenium 2.0, people mean Selenium making use of the WebDriver API.
Instead of recording tests with a plugin from your browser, Selenium WebDriver expects you to write code . If you want to code along in Java, you will have to fire up your Java IDE – I am going to use Eclipse myself – and make sure you import the Selenium library jar and the driver jars into the project. If you are using Maven, the dependencies would look like:

<dependency>
        <groupId>org.seleniumhq.selenium</groupId>
	<artifactId>selenium-htmlunit-driver</artifactId>
	<version>${selenium.version}</version>
</dependency>
<dependency>
        <groupId>org.seleniumhq.selenium</groupId>
	<artifactId>selenium-chrome-driver</artifactId>
	<version>${selenium.version}</version>
</dependency>
<dependency>
	<groupId>org.seleniumhq.selenium</groupId>
	<artifactId>selenium-firefox-driver</artifactId>
	<version>${selenium.version}</version>
</dependency>
<dependency>
        <groupId>org.seleniumhq.selenium</groupId>
	<artifactId>selenium-ie-driver</artifactId>
	<version>${selenium.version}</version>
lt;/dependency>
<dependency>
	<groupId>org.seleniumhq.selenium</groupId>
	<artifactId>selenium-support</artifactId>
	<version>${selenium.version}</version>
</dependency>

Note: Selenium libraries for other programming languages such as Python are available as well. The approach for those is just the same.

Our Selenium 2 test looks like this:

public class SeleniumTestCase {
	@Test
	public void testFramework() {
		File firefoxBin =
                     new File("C:\\Users\\Public\\Mozilla Firefox 3.6\\firefox.exe");
		FirefoxBinary firefoxBinary = new FirefoxBinary(firefoxBin);
		final FirefoxDriver firefoxDriver =
                     new FirefoxDriver(firefoxBinary, null);

		firefoxDriver.get("http://www.dzone.com");

		final DzonePage dzonePage = new DzonePage();

		WebDriverWait wait = new WebDriverWait(firefoxDriver, 30*1000, 200);
		wait.until(new ExpectedCondition() {
		      @Override
		      public Boolean apply(WebDriver driver) {
		    	  PageFactory.initElements(firefoxDriver, dzonePage);
		    	  return dzonePage.newLinksLink.isDisplayed();
		      }
		});

		dzonePage.newLinksLink.click();
	}
}

First, the driver is instantiated. For Firefox, it is necessary to supply the driver with the location of the Firefox binary. For some drivers, such as the InternetExplorerDriver, this is not necessary.
Then we instruct the driver to send a http get request to www.dzone.com. After that, we wait for the page to load. We do this by instantiating WebDriverWait, which wraps our driver into a construct which allows it to wait for a condition. In this case, we want to wait for the “new links” link to be displayed, so we can click it.
Note that in Selenium 2 there is a Page class concept. This page is an abstract representation of the opened webpage and looks like this:

public class DzonePage{
	@FindBy(linkText = "New links")
	public WebElement newLinksLink;

	@FindBy(tagName = "body")
	public WebElement body;
}

From within Eclipse, this test can be run like “Run as>jUnit Test”.

Converting from Selenium 1 to Selenium 2

If you have a bunch of selenium 1 tests you have to convert, you can consider automating the conversion. The selenium1 tests can easily be parsed as xml files and you can come up with code snippets that map one to one on the selenium1 commands.
However, writing a framework like that is beyond the scope of this article. I am just going to discuss some issues I encountered when doing such conversion.

Coming up with Selenium WebDriver equivalents of the Selenium IDE commands.

Although many mappings are rather straightforward, and there is also a guide for converting Selenium 1 to Selenium 2 tests, some are a bit harder to come up with.
For example, it is often necessary to wait for a certain element to load after doing a click. We already saw that in our example. Generic code that waits on the element that needs to be loaded, could be implemented like this:

protected final void waitForExistsAndVisible(final WebElement element) {
     seleniumContext.getWebDriverWait().until(new ExpectedCondition() {
	@Override
	public Boolean apply(WebDriver driver) {
		return element.isDisplayed();
	}
     });
}

From time to time, there are some catches too. For example, to wait for an element not to be present, one could expect driver.findElement to return null if the element is not present. However, this method never returns null. If the element is not present, a NoSuchElementException is thrown instead. Hence, the method needs to look like:

protected final void waitForElementNotPresent(final String htmlId) {
     seleniumContext.getWebDriverWait().until(new ExpectedCondition() {
	@Override
	public Boolean apply(WebDriver driver) {
	     try {
		driver.findElement(By.id(htmlId));
		return false;
	     } catch (NoSuchElementException e) {
		return true;
	     }
	}
     });
}

There are also some more exotic ones, that dig a bit deeper into the API, such as code to assert and dismiss an alert:

protected final boolean assertAndDismissAlert(String alertText) {
     Alert alert = seleniumContext.getWebDriver().switchTo().alert();
     if (alert != null) {
	boolean ok = alertText.equals(alert.getText());
	alert.dismiss();
	return ok;
     } else {
	return false;
     }
}

Or to check whether an element has focus:

protected final boolean hasFocus(WebElement element) {
     WebElement focusedElement = seleniumContext.getWebDriver()
          .switchTo().activeElement();
     return element.equals(focusedElement);
}

Selenium WebDriver issues

Since Selenium WebDriver is a rather new library, and its a rather ambitious project, I guess it is to be expected there are some issues with it. For example, there appear to be some irregularities between what a human user sees/can interact with in the browserwindow and what the driver sees/can interact with.

a. Element is visible for user, but not for Selenium 2

Selenium 2 takes some assumptions that do not always hold. For example, if a dom element has a height or width of 0, Selenium 2 thinks it is not visible/not clickable, while in reality, it often is. If an element has a css float attribute set, the height becomes 0 but the element is still visible.
A workaround that usually works is to click an element inside the element that has height 0.

b. Element not found

There are differences in behavior between drivers. This problem is an intrinsic effect of the WebDriver using the browser native APIs.
Because of this, different browserdrivers can react a bit different. For example, in Firefox, clicking on a span element within a link works, in IE6 it does not.
Also, different browsers have slightly different xpath implementations, which is something to keep in mind too.

c. Click problems

Selenium sometimes doesn’t correctly click an element on certain browsers. This appears to be a viewport problem. If the element is not in the current browser view, Selenium2 does not scroll to the link, the click does not trigger anything, although no Exception is thrown. This can happen with other commands on WebElements too, such as when calling clear() on a input WebElement.

d. Selenium 2 sometimes shows very specific OS/browser dependent behaviour

The previous problem is probably a more specific case of this one, and we are giving another example of it here.
With Firefox, unlike with IE, the driver always scrolls to the link first before it clicks it. However, once we had a bunch of tests failing for Firefox on Windows while all these tests were green for Firefox on Ubuntu.
Apparently, there was some Javascript on the page that caused a breadcrumbs bar to scroll with the page. On Windows, to click on a link, the Selenium2 driver scrolled to the link(to make it visible) and then wanted to click on it. However, right after the scroll, the breadcrumbs appeared over the link to click, and by the time the driver simulated the click, the link was not clickable anymore so nothing happened and all the mentioned tests timed out.

Introduction

Usually, we do not look into JVM intrinsics. We take the JVM as is.
Some of the world’s finest engineers are working on the JVM(s) and I am sure we can’t improve their work, definitely not without getting seriously involved.
However, as a Java developer, you are often confronted with performance issues, usually working memory related.
The problem I run into most is the dreaded OutOfMemoryError: PermGen Space error. I thought it would be nice to know more about it, so I looked into what causes it exactly.

Java memory structure

To understand the error, we have to look into how the jvm memory is structured.
There are two memory regions in the JVM: the heap and the stack. Local variables and methods reside on the stack, everything else on the heap.
This Java heap memory is structured again into regions, called generations. The longer an object lives, the higher the chance it will be promoted to an older generation. Young generations(such as Eden on Sun JVM) are more garbage collected than older generations(survivor and tenured on Sun JVM). However, there is also some separate heap space called permanent generation. Since it is a separate region, it is not considered part of the Java Heap space. Objects in this space are relatively permanent. Class definitions are stored here, as are static instances.

Without getting into details, Classloaders deploy and undeploy classes all the time. For example, this happens when an application is deployed or undeployed on a webserver. On web servers, all applications have their own Classloader. When an application is deployed or undeployed, its class definitions and Classloader are respectively put into and removed from the permanent generation heap.

OutOfMemoryError: PermGen Space

The OutOfMemoryError: PermGen Space error occurs when the permanent generation heap is full. Although this error can occur in normal circumstances, usually, this error is caused by a memory leak.

In short, such a memory leak means that a classloader and its classes cannot be garbage collected after they have been undeployed/discarded.

To give an example on how this can happen, let’s say we have a Payment class, which is part of a jar in a web application that is deployed on some webserver. In the lib folder of the web server, there is some logging framework present, which has a Log class with the method register(Class clazz) with which classes can be registered for logging. Let’s say that the Payment class gets registered by this method and the Log class starts keeping a reference to the clazz object. When the Payment class gets undeployed, it is still registered with the Log class. The Log class will still have a reference to it and hence, it will never be garbage collected. Moreover, since the Payment Class has a reference to its ClassLoader in turn, the ClassLoader itself will never be garbage collected either, and so will none of the classes it loaded.

An even more typical example is with the use of proxy objects. Spring and Hibernate often make proxies of certain classes. Such proxy classes are loaded by a classloader as well, and often, the generated class definitions – which are loaded like classes and stored in permanent generation heap space – are never discarded, which causes the permanent generation heap space to fill up.

Avoiding the error

1. Increasing the maximum size of the permgen heap

The first thing one can do is to make the size of the permanent generation heap space bigger.
This cannot be done with the usual –Xms(set initial heap size) and –Xmx(set maximum heap size) JVM arguments, since as mentioned, the permanent generation heap space is entirely separate from the regular Java Heap space, and these arguments set the space for this regular Java heap space. However, there are similar arguments which can be used(at least with the Sun/OpenJDK jvms) to make the size of the permanent generation heap bigger:

-XX:MaxPermSize=256m

would set its maximum size to 256m, which is 4 times bigger than the default size.

2. Use common sense when using static fields on classes.

Make sure you do not write classes that have static variables keeping references to class definitions and the like.

Using JDK dynamic proxies instead of cglib proxies

Some third party frameworks, such as cglib(although it might be better for newer versions of the library?), appear to be permgen monsters.

So using jdk dynamic proxies instead of cglib might be a good idea when getting the error.

Also, newer versions of Hibernate appear to not use cglib as a bytecode provider anymore, so upgrading your version of Hibernate, might drastically lower your chances on getting the error.

Summary

In general, when getting the error, one needs to determine why certain class definitions are not garbage collected. Once that is known, it should be possible to battle the error.

Liferay hooks

After playing around a bit with a portal, and maybe writing a few portlets for it, you probably start wondering how you can override some of the portal features. You might want to use a branded theme for the whole portal or integrate the portal login with your own database of users/own authentication system.
Liferay has quite a few types of so called “plugins”, such as themes and layout templates, which enable developers to achieve the aforementioned effects. The most powerful type of plugin is called a “hook”. Liferay hooks let you change the portal functionality itself.

Building a hook

We are going to build a simple hook that logs info about every request to Liferay. Before and after every request we will log something. This could be a useful hook if it is necessary to log the ips of the clients or if one wants to calculate how long every Liferay request takes.

Project Structure

The project structure of a hook looks very similar to that of a a regular webapp. Hooks, just like portlets, are packaged as wars. You can even deploy portlets and hooks in the same war file, although this is conceptually not a good idea, because hooks override the behavior of the whole portal, and portlets are supposed to be small pluggable components that don’t impact the portal itself.

Since we are using Maven, we use a typical Maven project structure.

Note: It is possible to build hooks using the Eclipse Liferay IDE plugin, but for this article, we choose to explain building a hook from scratch.

The files in our hook war will be the following:

• /src/main/java/com/integratingstuff/liferay/hooks/CustomPostEventAction
• /src/main/java/com/integratingstuff/liferay/hooks/CustomPreEventAction
• /src/main/resources/portal-hook.properties
• /src/main/webapp/WEB-INF/liferay-hook.xml
• /src/main/webapp/WEB-INF/liferay-plugin-package.properties
• /pom.xml

We see that all the typical Maven directories: src/main/java for the Java files, src/main/resources for the resource file and src/main/webapp for the web content. The file that defines our hook, liferay-hook-xml, resides in this last directory.

Note: It is not necessary to have a web.xml file in your source code. Upon hook deploy however, Liferay will automatically create it for the deployed war.

Defining hooks

Hooks are defined in a liferay-hook.xml file.

The liferay-hook.xml for our portlet looks like this:

<hook>
	<portal-properties>portal-hook.properties</portal-properties>
	<!--
	<custom-jsp-dir>/WEB-INF/jsps</custom-jsp-dir>	-->
	<struts-action>
		<struts-action-path>/portal/layout</struts-action-path>
		<struts-action-impl>com.integratingstuff.liferay.hooks.CustomLayoutAction</struts-action-impl>
	</struts-action>
	<service>
		<service-type>com.liferay.portal.service.UserLocalService</service-type>
		<service-impl> com.integratingstuff.liferay.hooks.CustomUserLocalServiceImpl</service-impl>
	</service>
	 -->
</hook>

It is only overriding some portal properties, but for completeness, we also comment on the most common and important other uses of hooks in this section.

Overriding portal properties

We are only supplying a portal.properties file to override some properties of our portal(this can be done on portal level also, but note that pointing to hook classes in the actual portal.properties file is not a good idea since theses classes are not on the classpath of the portal itself).
Not all portal properties can be overridden with a hook. You can read which properties can be overridden by taking a look at the liferay hook dtd wiki page.

Overriding portal jsps

There are some other tags that can be used in a liferay-hook.xml file.
It is possible to override portal specific jsps by supplying a custom-jsp-dir and then putting jsps in that directory. For example, if we define a custom-jsp-dir “/WEB-INF/jsps” and then we create /WEB-INF/jsps/html/portlet/blogs/view.jsp in our project structure, this portal specific jsp will be overridden. The one from the hook will be used instead of the portal one.

Overriding portal services

Liferay contains a lot of services that are defined as spring beans in … of the portal source code. All these services can be overridden by using the service tag of liferay-hook.xml. For example, we could replace the portal implementation of the com.liferay.portal.service.UserLocalService – the service that enables one to save/update/lookup information about portal users – interface with our own this way.

Soon: overriding actions

As stated in this blog article, in the next releases of Liferay, it will be possible to override Liferay Struts actions with hooks as well.

Differently from Liferay services, these Struts actions are not defined as Spring beans, and in the past, they required the EXT model to be overridden. These Struts Liferay actions are concrete classes, not interfaces, are part of the internal Liferay code and dependent heavily on its implementation. It often is not a good idea to override them, since changing your version of Liferay would likely break them, but still, it is not uncommon to override some of the important ones like LayoutAction(to get custom rendering behavior of any page fragment) and LoginAction(to get custom portal login functionality that can not be covered by the Liferay event model) if need be.

Soon: adding servlet filters

Apparently, soon it will also become possible to add servlet-filter and servlet-mapping declarations to your hook, making it possible to add custom servlet filters to Liferay itself.

Our hook

Our portal-hook.properties file looks like this:

servlet.service.events.pre=com.integratingstuff.liferay.hooks.CustomPreEventAction
servlet.service.events.post=com.integratingstuff.liferay.hooks.CustomPostEventAction

Basically, we are adding events before Liferay starts to process a request(but after any Liferay servlet filter) and after the processing of every request.

As stated in the comments of the portal.properties documentation(http://www.liferay.com/community/wiki/-/wiki/Main/Portal+Properties+6.0.5), when overriding Portal Events properties, we have to point to classes that extend com.liferay.portal.kernel.events.Action.

So when implementing these classes, we have to extend this Action class:

public class CustomPreEventAction extends Action{
	public void run(HttpServletRequest request, HttpServletResponse response)
			throws ActionException {
		log.info(“Request from ip: ” + request.getRemoteAddr();
		request.setAttribute(“startOfRequest”,new Date());
	}
}

public class CustomPostEventAction extends Action{
	public void run(HttpServletRequest request, HttpServletResponse response)
			throws ActionException {
		Date now = new Date();
		Date startOfRequest = request.getAttribute(“startOfRequest”);
		if (startOfRequest != null){
			log.info(“Request ook: ” + (now.getTime() - startOfRequest.getTime())+ “ms”);
		}
	}
}

Note though that the request and response that are passed in these actions are actually wrapper objects that are managed by Liferay itself.
Some things cannot be done with these wrapper objects. Invalidating or modifying the session of the request for example.

Other files

Optional: liferay-plugin-package.properties

There is also the optional liferay-plugin-package.properties file. It is good practice to add it. It also offers some extra features.

name=IntegratingStuffDemo-hook
module-group-id=liferay-ee
module-incremental-version=1
tags=
short-description=
change-log=
page-url=http://www.liferay.com
author=Liferay, Inc.
licenses=EE

#portal.dependency.jars=portal-impl.jar, struts.jar, commons-logging.jar, log4j.jar, slf4j-api.jar, slf4j-log4j12.jar, commons-codec.jar

Notice the commented line. With portal.dependency.jars you can supply jars that have to be copied from the portal root to the lib folder of the hook. This way, you can easily write a hook that depends on the portal implementation for example(for example, if you want to override one of the Struts actions that are part of portal-impl, with struts-action).

Even if you do not specify any portal.dependency.jars, Liferay will still copy its log4j.jar, commons-logging.jar and util-java.jar into the lib folder of the hook. Adding these this way(or having it in the lib folder of the war) is not necessary.

pom.xml

If you are using Maven, you will need the following dependencies in your pom.xml file, of which some are Liferay specific:

<dependency>
   <groupId>com.liferay.portal</groupId>
   <artifactId>util-java</artifactId>
   <version>6.0.5</version>
   <scope>provided</scope>
</dependency>

<dependency>
   <groupId>com.liferay.portal</groupId>
   <artifactId>portal-service</artifactId>
   <version>6.0.5</version>
   <scope>provided</scope>
</dependency>
<dependency>
   <groupId>javax.servlet</groupId>
   <artifactId>servlet-api</artifactId>
   <version>2.5</version>
   <scope>provided</scope>
</dependency>
<dependency>
   <groupId>javax.servlet.jsp</groupId>
   <artifactId>jsp-api</artifactId>
   <version>2.1</version>
   <scope>provided</scope>
</dependency>

<dependency>
   <groupId>javax.portlet</groupId>
   <artifactId>portlet-api</artifactId>
   <version>2.0</version>
   <scope>provided</scope>
</dependency>

Deploying the hook

Deploying the hook is very straightforward. You can just drop the resulting war in the deploy directory of your Liferay install or add it to your server in your Eclipse/other IDE.

Portals

A portal is a collection of windowed mini web applications, called portlets, which support features like personalization, content aggregation, integration into a foreign portal, authentication and customization. The portal itself usually offers things like an eventing system, single sign on, a portal search, tons of community stuff and facilitates easy communication between the different windows/portlets.

In the screenshot below we see an example of a portal page. Notice all the different windows, each one representing a small web application.

Popularity

Some years ago, portals were heavily hyped, as the future of the web even, but these days, not anymore. I even met some people who think they are plain useless and should be buried. Then I met some enthusiasts again who see more uses for them than I do.

Uses

Community websites

A portal like Liferay offers tons of features out of the box. If you need an instant messaging system, wikis, forums, message boards, document management, auditing, polls, a chat system, friends lists, blogs and calendars, combined with custom development, Liferay is probably one of the best choices around. You definitely do not want to start building all these things – which have been implemented a 1000 times before – yourself. You could use seperate packages, such as JForum or something, but these would still need a lot of integration and custom work. You could go with a PHP solution such as Drupal or WordPress, but if you are a Java developer/member of a team of Java developers, you are probably not a fan of doing all the custom development in php. Hence, Liferay.
To put it bold, if I would have to make a Facebook, LinkedIn or Youtube, I would make it with Liferay.
Let’s hope people are not going to fire load issue questions at me now. If they do, I am just going to point them to the Liferay Performance whitepapers.

Since I am on a portal/Liferay marketing roll anyway, people can view a list of sites that use Liferay here. Unlike what people might be thinking by now, I am not getting paid by Liferay for this article.

Enterprise websites backed by a SOA

But what if you do not need all the community stuff?
In which case is using the portlet specification over the servlet specification useful by nature, without having to fallback on other portal features to defend its use?
This can only be when all the seperate portlets add value over building the solution as a few classic web applications. If one portlet allows to select a customer, some other portlets could be fed this selection as input for example. This beats making a view interface to get a customer for every seperate web application.
Unfortunately, this will only work when there is already a clean modularization on the backend. If the backend consists of one database and one huge model, things will get too intertwined, and such modularization on the view level will become a headache instead of an advantage.
This is why I think an enterprise portal needs to be backed by a service oriented architecture, in which there are clearly seperated services which should be integrated with each other. The power of portlets comes from the ease with which they integrate disparate sources. So, they should be used when there is not one source of content but when there are multiple sources of content.

When not to use

Most web applications do not need a community and most web applications are (arguably) not an integration of many disparate content sources. Hence, in my opinion, portals are rather a niche product than a good overall solution.

Some argue that portals offer another level of abstraction over web development and that more things are taken care of for a developer. Sure, but at the same time, the developer is confronted with a lot more complexity, is constrained more and in my experience, many portal projects end up modifying the portal software itself which ties the developed software to some particular portal, destroying the portability argument. For Liferay for example, people usually rely on the Liferay specific window state “exclusive” for ajax requests, use the Liferay specific action-url-redirect to apply the post-redirect-get pattern and when custom authentication logic is necessary, Liferay hooks are built which rely on specific Liferay api. Sometimes you wonder why there is only a portlet spec and not a portal spec.

Portals are not a one-size-fits-all solution. If you do not have the feeling a portal is a perfect fit for your needs, you probably should not use one.

Portlets

We know now what a portlet is. A mini web application. A window on a portal page. But we didn’t dive into the technical details until now.

The portlet container

In short, the portal utilizes a portlet container to manage the lifecycle of the portlets just like a servlet container is used to manage the lifecycle of servlets. A portlet container is responsible for the initialization, request processing and destruction of portlets. The Java Portlet Specification defines the contract between a compliant portlet container and portlets. This standardization allows for portability of portlets between portal implementations.

Portlet versus servlet development

Portlet development is very similar to servlet development. The portlet API is modeled after the servlet API. The Portlet, PortletContext, PortletRequest and PortletResponse are very similar to their servlet counterparts. The major difference is that portlets only render a fragment of an html page, instead of a whole page.

The portlet specification

However, there still are some differences between portlet and servlet development.
We discuss the 3 most important portlet specific features now.

a. Different phases

With portlets, a request has at least two distinct phases: the action phase and the render phase. The action phase is executed only once. This is the moment where any backend actions occur. In the render phase the view is rendered to the user. Unlike the action phase, the render phase can be executed multiple times for a single request. With servlets, there is no such distinction on the API level, and these phases is something a portlet developer has to get used to.

b. Portlet modes

A portlet can have different display modes. The portlet mode determines what content the portlet should generate. The Portlet API defines 3 portlet modes: view, edit and help. In view mode, a user typically views data. In edit mode, a user typically modifies data. In help mode, a user can consult help about the portlet. A portlet developer can add any number of custom portlet modes to a portlet.

c. Window states

A window state indicates the amount of portal page space that should be assigned to a portlet. The portlet API defines 3 window states: normal, minimized and maximized. Any portal is allowed to define additional window states. Liferay, for example, has one additional window state, “exclusive”, which just renders the page fragment coming from the portlet, without decorating it with the entire portal page. Very useful when there is a need for Ajax integration.

In the screenshot below, we see the portal page with the portlets again. This time however, some of them have the portlet window state “minimized”, the others still have window state “normal.” Note that every portlet has some icons. These will either change the window state(to maximize it for example), or they will change the portlet mode, from “view” mode to “edit” mode for example.

Writing a portlet

Spring Portlet Mvc

The Spring Portlet Mvc framework is a mirror image of the Spring Web Mvc framework, and uses the same underlying view abstractions and integration technology. Therefore, different Web Mvc classes will be reused.
It is also important to realize that when using Spring Portlet Mvc, you will no longer write your own Portlet but use the Spring Mvc one(just like you dont write your own servlets when using Spring Web Mvc).

Project structure

We are going to create one of the simplest Spring Portlet Mvc portlets possible.

The files we will need are the following:

• /src/main/java/com/integratingstuff/portlets/test/SampleController
• /src/main/webapp/WEB-INF/portlet.xml
• /src/main/webapp/WEB-INF/springContextConfig.xml
• /src/main/webapp/WEB-INF/web.xml
• /src/main/webapp/WEB-INF/jsp/demo.jsp
• /pom.xml

In comparison with a regular webapp, there is only one portlet specific file: portlet.xml.

portlet.xml

The portlet.xml file is our most important file. It defines our portlet.

<?xml version="1.0" encoding="ISO-8859-1"?>
<portlet-app xmlns="http://java.sun.com/xml/ns/portlet/portlet-app_2_0.xsd"
	version="2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://java.sun.com/xml/ns/portlet/portlet-app_2_0.xsd">

	<portlet>
		<portlet-name>sample</portlet-name>
		<portlet-class>org.springframework.web.portlet.DispatcherPortlet</portlet-class>
		<init-param>
			<name>contextConfigLocation</name>
			<value>
				/WEB-INF/springContextConfig.xml
			</value>
		</init-param>
		<supports>
			<mime-type>text/html</mime-type>
			<portlet-mode>view</portlet-mode>
		</supports>
		<portlet-info>
			<title>Portlet Mvc Demo</title>
		</portlet-info>
	</portlet>

</portlet-app>

Portlet Mvc is designed around a portlet that dispatches requests to spring portlet mvc controllers. However, the DispatcherPortlet does more than only that. It also makes sure that the portlet is completely integrated with the Spring ApplicationContext and the developer is able to use every other Spring feature.

You will probably notice that we, in this tutorial, are developing a portlet that only supports the “view” portlet mode.

Note also how we point the portlet to our spring contextConfigLocation. If we would not add this init-param, the DispatcherPortlet will look for the default [portlet-name]-portlet.xml file in the WEB-INF directory. If this file would not be present, an exception would be thrown at deploy time.

springContextConfig.xml

On initialization of the DispatcherPortlet, the framework will create the bean definitions defined in this file.

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:util="http://www.springframework.org/schema/util"
	xsi:schemaLocation="
		http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
		http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-3.0.xsd"
	>

	<bean id="sampleController" class="com.integratingstuff.portlets.test.SampleController">
	</bean>

	<bean id="portletModeHandlerMapping" class="org.springframework.web.portlet.handler.PortletModeHandlerMapping">
	    <property name="portletModeMap">
	        <map>
	        	<entry key="view" value-ref="sampleController"/>
	        </map>
	    </property>
	</bean>

	<bean id="viewResolver" class="org.springframework.web.servlet.view.UrlBasedViewResolver">
	    <property name="prefix" value="/WEB-INF/jsp/"/>
	    <property name="suffix" value=".jsp"/>
	    <property name="viewClass"><value>org.springframework.web.servlet.view.JstlView</value></property>
	</bean>

</beans>

We have one custom controller, which we will discuss later.
Note how this controller is mapped on the view portlet mode in the PortletModeHandlerMapping bean, which is the bean the DispatcherPortlet will use to decide which controller to execute.

The last bean to discuss is our viewResolver. Note that this bean is a member of the regular spring mvc framework and not a member of a portlet specific package. This is because spring portlet mvc reuses all spring mvc view technologies, as we already said when we introduced Spring Portlet mvc. How this is possible is discussed in the next section.

web.xml

We also need a valid web.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<web-app id="WebApp_ID" version="2.4" xmlns="http://java.sun.com/xml/ns/j2ee"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">

	<display-name>portletMvcDemo</display-name>

	<servlet>
		<servlet-name>ViewRendererServlet</servlet-name>
		<servlet-class>org.springframework.web.servlet.ViewRendererServlet</servlet-class>
	</servlet>
	<servlet-mapping>
		<servlet-name>ViewRendererServlet</servlet-name>
		<url-pattern>/WEB-INF/servlet/view</url-pattern>
	</servlet-mapping>

</web-app>

When using Spring Portlet mvc, this file will always declare the ViewRendererServlet. To be able to reuse all the view technologies from Spring Web Mvc, the PortletRequest and Response need to be converted to a HttpServletRequest and Response in order to execute the render method of the (regular Spring Web Mvc) View. In order to do this, DispatcherPortlet uses the special ViewRendererServlet which only exists for this purpose.

SampleController

package com.integratingstuff.portlets.test;

import javax.portlet.RenderRequest;
import javax.portlet.RenderResponse;

import org.springframework.web.portlet.ModelAndView;
import org.springframework.web.portlet.mvc.AbstractController;

public class SampleController extends AbstractController {

	public ModelAndView handleRenderRequestInternal(RenderRequest request, RenderResponse response) throws Exception {
		ModelAndView mav = new ModelAndView("demo");
		mav.addObject("message", "Check it out!");
		return mav;
	}

}

Note that there are actually two methods to implement for our subclass of AbstractController: handleActionRequestInternal and handleRenderRequestInternal. We are not doing any backend action, we are just building and rendering the view, hence we only override the handleRenderRequestInternal method.

demo.jsp

We are returning a “demo” view from our controller, which our viewResolver resolves to the following demo.jsp:

<%@ page contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
This is our test portlet.<br/>
Message: ${message}

This is just a regular jsp.

pom.xml

This project was developed as a Maven project. This Maven pom.xml file is not essential for portlets at all, but for the people copy pasting along, the pom is printed here for easy reference:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>com.integratingstuff.portlets.test</groupId>
	<artifactId>portletMvcDemo</artifactId>
	<version>1.0</version>
	<packaging>war</packaging>
	<dependencies>

		<dependency>
			<groupId>javax.servlet</groupId>
			<artifactId>servlet-api</artifactId>
			<version>2.5</version>
			<scope>provided</scope>
		</dependency>
		<dependency>
			<groupId>javax.servlet.jsp</groupId>
			<artifactId>jsp-api</artifactId>
			<version>2.1</version>
			<scope>provided</scope>
		</dependency>

		<dependency>
			<groupId>junit</groupId>
			<artifactId>junit</artifactId>
			<version>4.8.2</version>
			<scope>test</scope>
		</dependency>

		<dependency>
			<groupId>org.slf4j</groupId>
			<artifactId>slf4j-api</artifactId>
			<version>1.6.1</version>
		</dependency>
		<dependency>
			<groupId>org.slf4j</groupId>
			<artifactId>slf4j-log4j12</artifactId>
			<version>1.6.1</version>
		</dependency>
		<dependency>
			<groupId>commons-collections</groupId>
			<artifactId>commons-collections</artifactId>
			<version>3.2.1</version>
		</dependency>
		<dependency>
			<groupId>commons-digester</groupId>
			<artifactId>commons-digester</artifactId>
			<version>2.1</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-web</artifactId>
			<version>3.0.5.RELEASE</version>
		</dependency>
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-webmvc-portlet</artifactId>
			<version>3.0.5.RELEASE</version>
		</dependency>
		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-test</artifactId>
			<version>3.0.5.RELEASE</version>
		</dependency>
		<dependency>
			<groupId>org.springframework.webflow</groupId>
			<artifactId>spring-webflow</artifactId>
			<version>2.2.1.RELEASE</version>
			<exclusions>
				<exclusion>
					<groupId>org.springframework.webflow</groupId>
					<artifactId>spring-js-resources</artifactId>
				</exclusion>
			</exclusions>
		</dependency>

		<dependency>
			<groupId>javax.servlet</groupId>
			<artifactId>jstl</artifactId>
			<version>1.1.2</version>
		</dependency>
		<dependency>
			<groupId>taglibs</groupId>
			<artifactId>standard</artifactId>
			<version>1.1.2</version>
		</dependency>

		<dependency>
			<groupId>javax.portlet</groupId>
			<artifactId>portlet-api</artifactId>
			<version>2.0</version>
			<scope>provided</scope>
		</dependency>

	</dependencies>

</project>

We can now build our portlet. If we run “maven package” on our project, a war is generated which is deployable on a portal.

Optional: liferay-portlet.xml

Usually, a liferay portlet also has a liferay-portlet.xml. This is only really necessary if you want to use some Liferay specific features, but when using Liferay, it always is good practice to include. If you want, add the following liferay-portlet.xml to your portlet application:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE liferay-portlet-app PUBLIC "-//Liferay//DTD Portlet Application 6.0.0//EN" "http://www.liferay.com/dtd/liferay-portlet-app_6_0_0.dtd" >
<liferay-portlet-app>
	<portlet>
		<portlet-name>sample</portlet-name>
		<action-url-redirect>false</action-url-redirect>
	</portlet>
</liferay-portlet-app>

Deploying the portlet to Liferay

1. Download Liferay

First, download Liferay Community Edition 6.0 from http://www.liferay.com/downloads. We use the version bundled with Tomcat. Extract the downloaded zip to your harddrive.

2. Install the Eclipse Liferay IDE

Although it is not necessary to use the Liferay IDE for Eclipse, we recommend it in this article. You can read about it at Liferay IDE Overview and figure out how to install it at the Liferay IDE Installation Guide.

In short, you just have to add http://releases.liferay.com/tools/ide/eclipse/helios/stable/ as an update site within Eclipse(Help>Install New Software in Eclipse) and install the plugins present on that location.

Note: There is also a Liferay Developer Studio, which is a bit more extended than the regular Liferay IDE, but for the purpose of this article, the Liferay IDE suffices.

Add the Liferay Server to the Eclipse Servers panel

Open the Eclipse Servers panel(Window>Show View>Servers) and rightclick in this window. Choose New>Server, open the map “Liferay, Inc.” and select Liferay v6.0 CE Server(Tomcat 6) as Server Type. Click next and enter the Liferay tomcat directory: point to the tomcat within your extracted Liferay installation. Add the server.

Start the server

Rightclick on the server and press “Start”. The server will now be started and you will be able to access the portal in your browser through http://localhost:8080

Note: If you have trouble starting the server, add the following vm arguments -Xms1024M -Xmx1024M -XX:MaxPermSize=256M and increase the server timeout(double click on the server in the Servers window).

Add the portlet to the server

Rightclick on the server again and choose “Add and Remove”. You can now choose to add any projects eligible to be deployed on the server(if your project is not, you probably need to add the dynamic web module Eclipse facet to your project). Our portlet project should be in the list under “Available”. Move it to the server(“Configured”). It should automatically be published(if not, rightclick on the server again and press “Publish”).

Add the portlet to a portal page

Now Liferay is started, log on to Liferay(test@liferay.com with password test is the default test user on Liferay 6.0) and choose Add on the menu in the top, then >More>Undefined>our portlet. The portlet we made will then be added to the page.

The portlet we developed is now deployed and visible on a portal. See how we already edited its background by using the portal “Look and Feel” feature.

In most cases, a migration from Spring Security 2 to Spring Security 3 is rather straightforward, but when custom spring security filters are present, additional work needs to be done.

In this article we discuss all changes required to do the migration. We also recommend the order in which the changes are discussed as the order in which to do them.

1. Importing the dependencies

One of the biggest overall changes between Spring Security 2 and Spring Security 3 is that Spring Security 3 takes a more modular approach. Spring Security 3 was divided into modules. The modules you will encounter in pretty much any webapp that is secured with spring-security are:

1. spring-security-core(core classes such as Authentication, Voter, and the like)
2. spring-security-web(all core classes that are dependent on servlet api, such as all spring security filters)
3. spring-security-config(needed to use the spring security xml namespace in spring applicationContext files)

There are lot of other specific modules, such as spring-security-ldap and spring-security-asm.

A maven dependency such as

<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-core</artifactId>
<version>2.0.6.RELEASE</version>
</dependency>

would change to the following dependencies:

<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-core</artifactId>
<version>3.0.5.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-web</artifactId>
<version>3.0.5.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-config</artifactId>
<version>3.0.5.RELEASE</version>
</dependency>

2. Changing the classnames

One of the effects of the modularisation is that a lot of spring security framework classes were moved to a more specific package.

SecurityContextHolder’s package changed from org.springframework.security.context to org.springframework.security.core.context for example.

In order to do a successful migration, it is necessary to change the import declaration of all spring security classes in classes that make use of them.

3. Handling small changes to the API

Usually, just changing the imported spring security class’s package name is enough. However, the api of some spring security framework classes was changed as well. Usually, these changes are rather small and easy to deal with.

Some of these small changes that come to mind:

• SpringSecurityException does not exist any longer. The quickest migration is to let it extend from RuntimeException or NestedRuntimeException instead of SpringSecurityException. Note that the specific Spring Security exceptions, such as AccessDeniedException and AuthenticationException still exist, but just extend RuntimeException directly now.
• The decide method of AccessDecisionVoter now takes a collection of ConfigAttribute instead of a ConfigAttributeDefinition. ConfigAttributeDefinition was basically just a decorator for a collection of ConfigAttribute elements.
• SavedRequest is now an interface instead of a concrete class. In order to instantiate a default implementation, instantiate DefaultSavedRequest.
• The default User of the UserDetails interface does not have a setAuthorities method anymore. The authorities are passed to the constructor, in the implementation the variable is final.

4. Rewriting the filters

Usually, a migration from Spring Security 2 to Spring Security 3 is straightforward. However, if you implemented custom spring security filters, chances are you will have a bit more work. The basic filter class from the Spring Security specific SpringSecurityFilter changed to the generic Spring web GenericFilterBean and the API of this class changed a lot more than the API of other classes.

Especially AbstractProcessingFilter(now AbstractAuthenticationProcessingFilter) changed. In Spring Security 3.0, the determineFailureUrl and determineTargetUrl methods were removed in favour of adding separate Handler instances(AuthenticationSuccessHandler, AuthenticationFailureHandler,..) to these classes, which makes the filters more configurable/reusable. A similar story for LogoutFilter(determineRedirectUrl removed in favour of adding a LogoutSuccessHandler).

5. Changing the security related applicationContext files

Last but not least, the spring applicationContext in which all the spring security beans are defined needs to change to.

At the very least, the spring security namespace schema will need to be changed from http://www.springframework.org/schema/security/spring-security-2.0.xsd to http://www.springframework.org/schema/security/spring-security-3.0.xsd. Forgetting to do this whill result in a BeanDefinitionParsingException upon application deploy.

The spring security xml namespace underwent quite some changes too.
One of the most notable changes is the removal of custom-authentication-provider and custom-filter inside authenticationProvider and custom spring security filter beans. Authentication providers are now injected explicitly or configured as <security:authentication-provider ref=””> within the authenticationManager bean. Custom filters can be declared explicitly in the spring security filter chain bean declaration, or they can be included with <custom-filter> within the <http> element in case this one is used.
Another notable change within the spring security xml namespace is that authentication-provider is now declared as a child of authentication-manager.

It is also necessary to change the classnames of the spring security framework classes that changed package in the applicationContext file(s).

And finally, some beans expect other properties than they used to. These injections need to be corrected too. For example, FilterSecurityInterceptor does not take a String anymore but needs to be injected with or a proper securityMetadataSource or needs to be declared with a fully configured security:filter-security-metadata-source tag.