Tuesday, January 26, 2010

The Opensource Marginal Cost

Recently I had a nice discussion on Twtitter. The topic was opensource. I find it interesting a lot of people still believe that opensource is an ideology, or a development model. It is interesting too there are a lot of people who are trying to create new ways to define opensource, such as "Open source is a business tactic, not a business model"[1], I do not understand the problem in considering opensource a business model. Although I am an engineer, I have studied business administration and I am very comfortable with opensource as a "Business Model". Moreover, I believe that a "tactic" is based on some business model. So, I do not see the difference. As a result I will consider “business tactic” and “business model” as synonyms.





Many of us are used to selling opensource to the "technical departments". But, of times when we are facing a top level management people, for various reasons, we have experience difficult in explaining opensource as a suitable alternative! One of the reasons for this difficulty is we are used to and prepared to explain opensource from technical perspective. But, our technical arguments are not well understood by many management types. Most management types understand business models, business strategies and business tactic, whichever you prefer to call them. That is not a “knock” against management types. They understand their domains just as engineers understand their technical domains. So, you can imagine the difficulty in management understanding the technical reasons for using open source. Likewise, you can imagine the difficulty engineers have in understanding some business models, strategies, tactics, etc.


I think the question really comes down to, “How are CIOs and managers to trust in something, or rely on something, they do not fully understand from a business perspective or a technical perspective?” It is for this reason, I believe if we better positioned the business case and business reasons for using opensource as an alternative rather than using technical reasons, we would have a much better chance in convincing CIOs and managers opensource solutions can indeed reduce their cost and also provide benefits to their customers.


In the study of economics there is a concept called "Marginal Costs"[2], In general terms, marginal cost at each level of production includes any additional costs required to produce the next unit. If producing additional vehicles requires, for example, building a new factory, the marginal cost of those “extra” vehicles includes the cost of the new factory. In practice, the analysis is segregated into short and long-run cases, and over the long run, all costs are marginal. At each level of production and time period being considered, marginal costs include all costs which vary with the level of production, and other costs are considered fixed costs. This is a concept that can by also applied into opensource industry.



For JBoss Enterprise subscriptions, or any other product family, Red Hat has a cost to keep a number of employees contributing, testing, certifying, collaborating, writing documentation, training a support team, consulting team etc. For this reason, the"Opensource Marginal Cost" is really low. That's why Red Hat never charges expensive prices for their products. That is also one of the biggest advantages for any company who follows this kind of business model in order to create a really profitable company offering opensource solutions. Obviously, Red Hat receives many contributions, but any contribution must be certified, and supported by Red Hat. Tor this reason, even for collaborations, there is a cost to test and to provide the security that this contribution will not cause any damage to "Subscription Buyers". However, this cost is still low, compared to "Proprietary Software".

"Opensource Marginal Cost" offers companies to save as much as 80% in the software acquisition.


Industry will accept "opensource" when the advantages are tangible factors, and not only merely words of text. I cannot see any company keeping their doors open creating innovative solutions without a way sell them.

In fact, I saw a huge company trying jump in the opensource market without a good plan, and the result turned out to be catastrophic. This particular company was acquired by another proprietary vendor. Some think this end result is a "proprietary win over the opensource". However, in fact, in my humble opinion this is a "strategic company win over a company with a poor opensource strategy ".


I live into a country where a lot of companies are looking for ways to reduce their costs in software, not only because opensource is "a new trend", but because some of them must trim their “IT” budgets to make them more competitive. I have seen many companies who have reduced their cost of software licencing, and with the saved software licensing costs they offered their employees not only better salaries, but also better environments for employees to work in as well as additional training and incentives for education.


Opensource is not only a way to develop software, it is a way to make employees happier by saving money which can then be invested in what is really relevant for the company, the employees!





There are many others factos that shows how opensource is an alternative for the future, replacing the retrograde proprietary industry software, such as bio alternatives as true substitutes for gasoline and its derivatives. In the end, everything is economics. We must open our minds, and not only think just as engineers, but also begin to think in terms of long term business.


Monday, January 18, 2010

Introducing project: breakingwoods - Components Repository and resources for JBoss ESB

Introduction


First of all, the main objective of this project is to be a source of new ideas, resources, components and actions for JBoss ESB.  Nobody is interested in creating a fork (or anything like that), but we believe that it could be a nice way to contribute to the evolution of this incredible project, as well as to be an accurate information source and filter for candidate components to be integrated into the core JBoss ESB project.





At breakingwoods, anybody will be able to contribute.  All contributions are welcome, whether that be a new ESB component, a review of an existing compoenent, ideas, fixes, testing etc... all will be really appreciated  The only thing you need is a Gmail/Google account.


Freedom to Contribute


Everybody at the breakingwoods's team believes that JBoss ESB is a very robust solution, but would be even better if we would add more Adapters, Listeners, Gateways (in JBoss ESB it might be the same), as well as new Actions. The same happened with another popular opensource ESB solution: Mule, made by MuleSource.  The Mule community has Mule Forge, which is an extension's repository for MuleESB.  We hope that breakingwoods can be exactly the same for JBoss ESB, offering lots of components, and showing valuable information about which new ideas are rocking or the most downloaded, rated, commented etc.





Why this project is not hosted under JBoss.ORG?


Firstly we would like to make the project strong in its own right.  Once it is hosted at Google, we hope people will feel comfortable enough to contribute with whatever they want, once it is not so close to the crowd and the spotlight. Nevertheless, maybe in the future, we could move from Google Code to JBoss.ORG.  This will depend of the success of this initiative.  Alternatively, it could be great to have it isolated.


How can I contribute?


There are several ways in which you can help to make this project a success:
  • To Propose / To Create / To Review/ To test: New Adapters/Gateways/Listeners
  • To Propose / To Create / To Review/ To test: New ESB Actions
  • To Propose / To Create / To Review/ To test: New Ideas, Designs, Quickstarts


Already at breakingwoods:
  • esbgen:  A basic CLI tool similar to seam-gen.  Helps you create the first esb project.
  • TwitterAction:  Action for publishing an ESB Message in Twitter.
  • EMailListener:  Periodically polls in the configured e-mail account, publishing incoming emails to JBoss ESB as an ESB Message.
  • Google Spreadsheet Listener:  Connects to Spreadsheets hosted by GoogleDocs, publishing new records to JBoss ESB.
  •  Apache DBUtils:  Action that executes plain SQLs.


There are new Actions and Listeners in the pipeline.  The following are some of the next Listeners we are planning.  Maybe you will be able to help:


  • Apache Camel Listener/ Action
  • SalesForce Listener / Action
  • Infinispan Listener/ Action
  • Terracota Listener / Action
  • AMQP Listener/Action
  • Where is yours ? :)


We are working in some of these, so stay tuned, as lots of good things will appear here at this project.


Another good point, is that most of these components will be compatible to JBoss SOA Platform.  Obviously Red Hat will not support problems regarding SalesForce, Terracota or anything that is not part of the JBoss SOA-P supported distribution.  However, you will still be able to add these components into your solutions, possibly reducing your development time and increasing your productivity.


If you have chance, check it out here: http://code.google.com/p/breakingwoods/


Enjoy one of the best things about open source development: "To learn with others essentially doing something useful".







breakingwoods Team.

Saturday, January 2, 2010

JBoss ESB on the Google's Clouds: Spreeadsheet Listener

Recently my brother started a new kind of service in his very small consultancy company, he is offering a kind of Consultancy over the Google Apps focused in small business customers, although I believe that Google Apps is not suitable just for small ones, but for bigger companies/clients as well.




Google Apps offers lots of good functionalities and services, such as: a) E-Mail, b) Site, c) Agenda, d) Documents (An online Office suite). I believe that this is a good strategy, so you might face in a near feature people replacing the older Excel files by new Google Spreadsheets, and than I decide work on it a little bit, creating a new Gateway to interact with this Google Service.

Getting Started with Google APIs

Google offers a good documentation over their APIs[1], in fact, everything that Google does, you are able to interact with just using Java, Python or even JavaScript[2].

You might use Google Spreadsheets for many purposes, since a basic data collecting, or as my friend JP Viragine told me: "Maybe we could use as a "Decision Table on the Clouds", or just a excel replacement. The big advantage, beyond the fact you don't need to install anything in your computer, is the "Collaboration", because you might share the online file with another people that will be able for editing any time and anywhere.

Another big difference when we are handling such kind of technology is the simple fact that we are totally stateless, once the Http allows us connect, and after the processing the http will give an answer, and that's it, is not that easy we use "Observers/Observables" when we are talking about HTTP (by default Stateless protocol), so my strategy was use "Timers" for polling the http service. I had not found a way to receive a notification from the service using Java, maybe it would make my job easier.

New JBoss ESB Listeners


If you create your first JBoss ESB gateway, you will be able to create as much as you need! The only thing that I have to recall when I have to create a new Gateway/Listeners is the wiki section that Tom Fennelly wrote [3]. So everything you will see here in this entry is very well documented there, you must say thanks to Tom, he did a really good job documenting about this subject.


JBoss ESB in Action with GoogleDocs

The first step is declare our listener into Jboss-esb.xml, there we will describe everything we need, such as properties and its values as well as the Listener class itself:



The most important configuration from this listener is the property: gatewayClass, which we are using: org.jboss.soa.jbossesb.gateways.google.GoogleDocsGateway , which is not an ou-of-the-box class from JBoss ESB, it is totally new, and you will see it working pretty soon. Moreover, we defined other properties that will be used by the Listener classes further, this properties are: documentName, feedURL, username, password and so on.

I created a very ugly class GoogleDocsGateway, that for this entry is handling just the spreadsheet, this is a class that extends AbstractThreadedManagedLifecycle, after this the rest is quite easy. When you are extending this class, you must keep in mind that there are certain tasks to think about:

1 -Define what you will do in the doInitialise() method, this is the method called when JBoss ESB invokes this service by the first time.

2- The doRun() is the most important one, this is a method that or will be waiting for an event notification, or you must try get a way to poll the protocol that you are interacting with, in order to don't consume unnecessary resources.

package org.jboss.soa.jbossesb.gateways.google;

import java.util.Timer;
import java.util.TimerTask;

import org.jboss.soa.esb.ConfigurationException;
import org.jboss.soa.esb.Service;
import org.jboss.soa.esb.client.ServiceInvoker;
import org.jboss.soa.esb.helpers.ConfigTree;
import org.jboss.soa.esb.listeners.ListenerTagNames;
import org.jboss.soa.esb.listeners.lifecycle.AbstractThreadedManagedLifecycle;
import org.jboss.soa.esb.listeners.lifecycle.ManagedLifecycleException;
import org.jboss.soa.esb.listeners.message.MessageDeliverException;
import org.jboss.soa.esb.message.Message;
import org.jboss.soa.esb.message.format.MessageFactory;

/**
 * Google Gateway Class
 * 
 * @author esilva
 */
public class GoogleDocsGateway extends AbstractThreadedManagedLifecycle {

 private ConfigTree listenerConfig;
 private Service service;
 private ServiceInvoker serviceInvoker;

 private GoogleSpreadSheet sheet = new GoogleSpreadSheet();

 protected boolean firstExecution = true;

 public GoogleDocsGateway(ConfigTree config) throws ConfigurationException {
  super(config);

  this.listenerConfig = config;

  String serviceCategory = listenerConfig
    .getRequiredAttribute(ListenerTagNames.TARGET_SERVICE_CATEGORY_TAG);

  String serviceName = listenerConfig
    .getRequiredAttribute(ListenerTagNames.TARGET_SERVICE_NAME_TAG);

  service = new Service(serviceCategory, serviceName);

  sheet.setService(config.getAttribute("servicename"));

  sheet.setUsername(config.getAttribute("username"));

  sheet.setPassword(config.getAttribute("password"));

  sheet.setFeedurl(config.getAttribute("feedurl"));

  sheet.setDocumentName(config.getAttribute("documentname"));

 }

 @Override
 protected void doRun() {

  publishMessageToESB();

  int delay = 10000; 
  int period = 5000; 
  Timer timer = new Timer();

  timer.scheduleAtFixedRate(new TimerTask() {
   public void run() {

    try {
     if (sheet.hasChanges()) {

      System.out
        .println("Changes found, publishing the message again");

      publishMessageToESB();

     }

     else {

      System.out.println("No updates found");

     }

    } catch (Exception e) {

     e.printStackTrace();
    }

   }

  }, delay, period);

 }

 private void publishMessageToESB() {
  Message esbMessage = MessageFactory.getInstance().getMessage();

  try {

   esbMessage.getBody().add(sheet.load());

  } catch (Exception e) {

   e.printStackTrace();
  }
  try {
   serviceInvoker.deliverAsync(esbMessage);

  } catch (MessageDeliverException e) {

   e.printStackTrace();
  }
 }

 @Override
 protected void doInitialise() throws ManagedLifecycleException {

  try {
   serviceInvoker = new ServiceInvoker(service);

   sheet.initialize();

  } catch (MessageDeliverException e) {

   throw new ManagedLifecycleException(
     "Failed to create ServiceInvoker for Service listening Google Service: "
       + service + "'.");
  }

  catch (Exception e) {

   throw new ManagedLifecycleException(e);
  }

 }

}



I created a new class called: GoogleSpreadSheet, that is an abstraction over the tasks we must to do to interact with the Google Spreadsheets, and through the doRun() method I am using a kind of scheduler to poll the url of my spreadsheet according the property configuration made by the "user" of this gateway.

package org.jboss.soa.jbossesb.gateways.google;

import java.net.URL;
import java.util.List;

import org.quartz.Job;
import org.quartz.JobExecutionContext;
import org.quartz.JobExecutionException;

import com.google.gdata.client.spreadsheet.SpreadsheetService;
import com.google.gdata.data.spreadsheet.ListEntry;
import com.google.gdata.data.spreadsheet.ListFeed;
import com.google.gdata.data.spreadsheet.SpreadsheetEntry;
import com.google.gdata.data.spreadsheet.SpreadsheetFeed;
import com.google.gdata.data.spreadsheet.WorksheetEntry;

public class GoogleSpreadSheet implements Job {

 protected static SpreadsheetService myService;

 protected URL metafeedUrl;

 protected SpreadsheetFeed feed;

 protected List spreadsheets;

 protected SpreadsheetEntry entry = null;

 private String service;
 private String username;
 private String password;
 private String feedurl;
 private String documentName;
 private String lastUpdate;

 protected boolean on = false;

 public String getService() {
  return service;
 }

 public void setService(String service) {
  this.service = service;
 }

 public String getUsername() {
  return username;
 }

 public void setUsername(String username) {
  this.username = username;
 }

 public String getPassword() {
  return password;
 }

 public void setPassword(String password) {
  this.password = password;
 }

 public String getFeedurl() {
  return feedurl;
 }

 public void setFeedurl(String feedurl) {
  this.feedurl = feedurl;
 }

 public String getDocumentName() {
  return documentName;
 }

 public void setDocumentName(String documentName) {
  this.documentName = documentName;
 }

 public String getLastUpdate() {
  return lastUpdate;
 }

 public void setLastUpdate(String lastUpdate) {
  this.lastUpdate = lastUpdate;
 }

 public GoogleSpreadSheet() {

 }

 public void initialize() throws Exception {

  myService = new SpreadsheetService(service);

  System.out
    .println("Initializing Connection between JBoss ESB and Google Docs...");

  myService.setUserCredentials(username, password);

  metafeedUrl = new URL(feedurl);

  feed = myService.getFeed(metafeedUrl, SpreadsheetFeed.class);

  spreadsheets = feed.getEntries();

  entry = null;

  for (int i = 0; i < spreadsheets.size(); i++) {

   entry = spreadsheets.get(i);

   if (entry.getTitle().getPlainText().equalsIgnoreCase(documentName)) {

    setLastUpdate(entry.getUpdated().toString());

   }
  }

 }

 public String load() throws Exception {

  StringBuilder builderOut = new StringBuilder();

  entry = null;

  for (int i = 0; i < spreadsheets.size(); i++) {

   entry = spreadsheets.get(i);

   if (entry.getTitle().getPlainText().equalsIgnoreCase(documentName)) {

    List worksheets = entry.getWorksheets();

    for (int j = 0; j < worksheets.size(); j++) {

     WorksheetEntry worksheet = worksheets.get(j);

     URL listFeedUrl = worksheet.getListFeedUrl();

     ListFeed listFeed = myService.getFeed(listFeedUrl,
       ListFeed.class);

     if (listFeed.getEntries().size() > 0) {

      StringBuilder header = new StringBuilder();

      for (String tag : listFeed.getEntries().get(0)
        .getCustomElements().getTags()) {

       header.append(tag + ",");

      }

      builderOut.append(header.toString().substring(0,
        header.lastIndexOf(",")));

     }

     for (ListEntry entrada : listFeed.getEntries()) {

      StringBuilder row = new StringBuilder();

      for (String tag : entrada.getCustomElements().getTags()) {

       row.append(entrada.getCustomElements()
         .getValue(tag)
         + ",");

      }

      builderOut.append(row.toString().substring(0,
        row.lastIndexOf(",")));

     }
    }

   }

  }

  return builderOut.toString();

 }

 public boolean hasChanges() throws Exception {

  SpreadsheetFeed feed = myService.getFeed(metafeedUrl,
    SpreadsheetFeed.class);

  List spreadsheets = feed.getEntries();

  entry = null;

  for (int i = 0; i < spreadsheets.size(); i++) {

   entry = spreadsheets.get(i);

   if (entry.getTitle().getPlainText().equalsIgnoreCase(documentName)) {

    if (getLastUpdate().equalsIgnoreCase(
      entry.getUpdated().toString())) {

     System.out.println(getLastUpdate() + "==========="
       + entry.getUpdated().toString());

     return false;

    } else {

     this.setLastUpdate(entry.getUpdated().toString());
     System.out.println("Changes arriving: " + getLastUpdate()
       + "===========" + entry.getUpdated().toString());
     return true;

    }

   }

  }
  return false;

 }

 public GoogleSpreadSheet(String service, String username, String password,
   String feedurl, String documentName, String lastUpdate) {

 }

 public String load(String service, String username, String password,
   String feedurl, String documentName, String lastUpdate)
   throws Exception {

  SpreadsheetService myService = new SpreadsheetService(service);

  myService.setUserCredentials(username, password);

  URL metafeedUrl = new URL(feedurl);

  SpreadsheetFeed feed = myService.getFeed(metafeedUrl,
    SpreadsheetFeed.class);

  List spreadsheets = feed.getEntries();

  SpreadsheetEntry entry = null;

  StringBuilder builderOut = new StringBuilder();

  for (int i = 0; i < spreadsheets.size(); i++) {

   entry = spreadsheets.get(i);

   if (entry.getTitle().getPlainText().equalsIgnoreCase(documentName)) {

    System.out.println("Last Update: " + entry.getUpdated());

    List worksheets = entry.getWorksheets();

    for (int j = 0; j < worksheets.size(); j++) {

     WorksheetEntry worksheet = worksheets.get(j);

     URL listFeedUrl = worksheet.getListFeedUrl();

     ListFeed listFeed = myService.getFeed(listFeedUrl,
       ListFeed.class);

     if (listFeed.getEntries().size() > 0) {

      StringBuilder header = new StringBuilder();

      for (String tag : listFeed.getEntries().get(0)
        .getCustomElements().getTags()) {

       header.append(tag + ",");

      }

      System.out.println(header.toString().substring(0,
        header.lastIndexOf(",")));
      builderOut.append(header.toString().substring(0,
        header.lastIndexOf(",")));

     }

     for (ListEntry entrada : listFeed.getEntries()) {

      StringBuilder row = new StringBuilder();

      for (String tag : entrada.getCustomElements().getTags()) {

       row.append(entrada.getCustomElements()
         .getValue(tag)
         + ",");

      }

      System.out.println(row.toString().substring(0,
        row.lastIndexOf(",")));
      builderOut.append(row.toString().substring(0,
        row.lastIndexOf(",")));

     }
    }

   }

  }
  return builderOut.toString();

 }

 @Override
 public void execute(JobExecutionContext ctx) throws JobExecutionException {

  try {
   System.out.println("Has Changes: " + this.hasChanges());
  } catch (Exception e) {

   e.printStackTrace();
  }

 }

}



Instead to use lots of confusing images, I will use a screencast:



I hope you liked that idea, and you can feel yourself more comfortable for creating new Listeners/Gateways for JBoss ESB.


Rerefences

[1] - http://code.google.com/intl/pt-BR/apis/spreadsheets/
[2] - http://code.google.com/intl/pt-BR/apis/spreadsheets/data/3.0/developers_guide_java.html
[3] - http://community.jboss.org/docs/DOC-13193