Gad J. Meir wrote a couple of lines in his blog about my lecture :)

I gave a lecture in the Mamram performance day, which was briefly mentioned by Gad J. Meir on his Hebrew blog :), skip to the end of the post to read about it. Thanks Gad.

Why continuous delivery is a must practice for software development teams?

My good friend, @elad_roz said something interesting about YAML. He said that YAML is better format then JSON, not because it's more simple, but because it forces you to work with right text indentation due to it's nature of figuring out the data structure according to the text indentation. We all know that source code / data file which is correctly and uniformly indented, is much simpler to understand and maintain.

Same principal goes for software development. We know that in the new challenging world of increasing demands from the software quality, complexity and the speed software is delivered, our only saviour is automation. If we don't automate things like build, deployment, integration tests, regression tests and functional tests, we don't have a chance to deliver valuable software to our customers.

The point in continuous delivery practice, is that it enforces the development team, the QA team and the Managers, to invest heavily in automation. If each code change, is a potential production version, to keep up the speed, you have to automate everything. There is no time for manual regression testing, there is no time to write deployment manuals and there is no time to execute any of the steps needed to promote a software change to production, manually.

The pace and the complexity of the business requirements from software development teams is constantly increasing. The only way teams can continue to successfully deliver the expectation, is by increasing the pace of the development by embracing the latest technologies and agile practices. Continuous delivery is a practice which will help you to understand the benefits of automation because it's not always clear to development teams and managers what is the benefit, in the first look. The reason for the unclarity is usually due to the fact that automation is hard to start with. If you don't have a single automatic functional test, you will have to invest a lot of resources, just to get started. And if you already started, each new feature or a bug fix you deliver, has to come with automation. This, from one point of view, extends the time to production floor of the feature, but from a different point of view, pays off in a later stage, where you discover problems in a very early stage of the development process, and don't waste valuable time on manual testing and version promotions. Another reason why this pays off greatly is the human nature. People are not very good at doing the same steps over and over again. They eventually get bored, and then the quality of execution will suffer.Automation saves your team from this boredom, and saves a LOT of time.

As the continuous delivery starts to be very trendy, some of the teams might start working with this paradigm, without fully understanding the practice. This might happen due to different reasons starting from leadership enforcing the teams to work in this manner, but not providing the teams time and resources to learn and adapt, or even a business pressure to deliver, where the management decides to cut in the automation, because it's the easiest place to cut the development times. Teams with this problem, will fail to deliver quality and speed. It's might not be seen right away, but only after some iterations of the development / production cycle.

Conclusion: if you decided to go the wonderful path of continuous delivery, first you must understand the philosophy, then you must be ready to invest heavily in the infrastructures of such a practice, and only then you will be able to enjoy the major benefits.

Web application multi language support - rule of thumb

Often, web application, support different languages. Some of the applications, even automatically selects a language, after checking the location of your request. For example, the site is checking your source IP address, and if you are coming from a country, they support, the site will automatically select the proper language.
These sites, must always support, easy "switch language" feature. It should be visible, on all pages, preferable on the site header, so it does not take more then 1-2 seconds for a user to to switch the language, if he prefers a different one. This is a must from my perspective to any web site, that support custom languages.

So why I decided to write this? Because it seems that google does not respect this practice. Unfortunately, Google Adsense, selects automatically Hebrew, for me, as i'm located in Israel and I cannot find easy way to switch it back to English (actually i cannot find this option at all, on the new UI of the Adsence application). I cannot stand the terms in Hebrew, it takes me so much longer, to understand what they are talking about.

Shame on you google.

So, when you design a web application, please make sure that this fundamental feature, is found easily by your users.

:)

When doGet is not really doGet and why it is dangerous?

HTTP specification defines 7 HTTP methods: GET, PUT, HEAD, TRACE, POST, DELETE and OPTIONS. When you write a HttpServlet you may decide to implement each and every one of these methods. For example if i implement the doPost method, this method will be invoked when a HTTP POST request is sent to my server. Simple enough. I don't expect any other method to be executed when i implement the POST method.

Now if we look at the HTTP specification at the definition of the HEAD method we see :

The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response.

Now what happens if you implement the doGet method, and don't implement the doHead? I would expect the servlet container return 405 method not allowed status, as it would do for any other HTTP method that i did not implement. But what really happens is the servlet container is calling the doGet instead, and drops the body (if you wrote anything to it). I can understand why this was the servlet container implementation, according the the specification the functionality still holds if we drop the body, we still return the HTTP headers and everybody happy...

So what is the problem?

The problem is performance. If the body generation is "CPU / IO / MEMORY" intensive operation you would like to do this operation as little as possible, so what you really should do in this case is implement the doHead method and calculate the HTTP headers for this request and set them, then implement the doGet method in the same way in addition to the body generation and setting. Because the default servlet container is working in this way, it is very easy not to pay attention to this, and implement only the doGet (which is very common thing to do) and don't think of implement anything else.

Now for browser based applications, this is fine, because when writing a web UI you usually will implement only doGet and doPost methods, but when working with servlets you often do not write regular user UI application. It is very low level API for this, so it is used for web services, or other processing methods. At this time i would expect the container not to act as it acts.

By the way, if you implement doPost for example and execute HEAD on the same servlet, you get 405 method not allowed status, as expected, so the only confusion happens when you implement doGet.

I even looked at the servlet 2.5 spec itself to see if the spec authors are mentioning this behavior anywhere, but could not find anything. Hence this is just container's optimization :)

I guess that in some cases this could even lead to some memory problems if the servlet container implementation is poor...

Beware...

Gilad Bracha's lecture on Newspeak programming language

Yesterday i atteneded a very interesting lecture by Gilad Bracha about Newspeak programming language. Here is a quote from his web site about the language design:

"Newspeak is a new programming language in the tradition of Self and Smalltalk. Newspeak is highly dynamic and reflective - but designed to support modularity and security. It supports both object-oriented and functional programming."

You can read more here.

Gilad was a co-author of the second and third editions of the Java Language Specification, a major contributor to the second edition of the Java Virtual Machine Specification.

In the lecture he gave yesterday he talked about the major design problems in the Java language and how Newspeak is different. The topics were:

1. The problem with Java's global namespace.
2. Software modularity (built in the Newspeak language, a paper on this topic here)
3. What it means to be a dynamic languge.
4. Built in security features.
5. High reflectivity of the language.
6. Interoperability.
7. Threading models (what is wrong with Java's and where the world of languages is going to - Actors model).

I enjoyed the lecture very much, Gilad is a great computer science theoretican in addition to his high practice in real word (When worked at Sun on Java and in Cadence Design Systems on Newspeak). I think that this is where his power is, the conjunction of theory and practice.

You can read more about him and about Newspeak in the folloing links:

newspeaklanguage.org/

bracha.org/Site/Home.html

jaxenter.com/gilad-bracha-java-is-becoming-cobol-2-0-10456.html

Solr 1.4 - Nutch 1.0 integration

Solr is open source enterprise search platform build on top of the famous Apache Lucene project. Nutch is open source web-search software used mostly as a web crawler. I needed to check out the Solr search engine, and integrate it with Nutch web crawler. So i followed the tutorial here which differs a bit with the versions (the tutorial was written with Solr 1.3.0). The versions i used:

Ubuntu 9.04
Solr 1.4.0
Nutch 1.0

Here is the list of problems i encountered and their solution:

0. Install java :) - I had a plain installation of Ubuntu so, this is basic.
then add JAVA_HOME to your .bashrc file in your home directory (if JAVA_HOME is not set), just add the line
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/
and replace the path with your java installation.

1. In solrconfig.xml, the boolean value must be a full xml notation, and not a partial one, other wise the Solr server won't start with error about parsing boolean value.

2. Change in regex-urlfilter.txt the domain name from

# allow urls in foofactory.fi domain

+^http://([a-z0-9\-A-Z]*\.)*lucidimagination.com/

to your domain name in the end - like:

# allow urls in foofactory.fi domain

+^http://([a-z0-9\-A-Z]*\.)*blogspot.com/ for example.

Other wise you will see the following output:

Generator: Selecting best-scoring urls due for fetch.

Generator: starting

Generator: segment: crawl/segments/20100111014122

Generator: filtering: true

Generator: jobtracker is 'local', generating exactly one partition.

Generator: 0 records selected for fetching, exiting ...

which is basically says that there is nothing to fetch ..

All the rest worked like a charm :)

Note that nutch distribution comes only with linux / unix shell scripts so running this on windows is a bit tricky, but doable via Cygwin, i preferred using my Virtual Boxed ubuntu installation..

Have fun

Simple MRU Cache for Java

A generic, and synchronized implementation of "Most Recently Used" cache can be found here. This implementation is based on simple HashMap and LinkedList, and used by extending the abstract class. Very useful :)

A brilliant java decompiler

Each of us, every now and then needs a java de-compiler. Up until now i was using Jad. Jad is a command line tool, which make it very easy to decompile java class files into readable java source code. Then i discovered the DJ decompiler, which is just a GUI on top of the jad command line utility. I must say that is was very handy to work with DJ, it saved me tons of time, and made the work easier. So now i needed to use a decompiler once again, so i googled "java decompiler" and the first google answer was the JD java decompiler. It was new to me, so i checked it out. It has 2 versions. First is a standalone version, which is a exe file, that is written in c and c++ and hence works very fast and does not need any jdk or JRE installations. The second version is actually an eclipse plugin, which i did not check, mainly because i was very very happy with the standalone version... It's brilliant. Fast, intuitive, minimalistic, download and run (no installation needed, no dependency), small footprint, and very good search and class type browsing capabilities (very much like in eclipse, even same icons). JD is a free software, the only restriction is not to embed it in other commercial tools. You can download it from here. My complements go to the sole developer of this handy utility : Emmanuel Dupuy.

The full Oracle commitment on the MySql issue.

You can read the whole oracle commitment here.

It is very interesting, i will post my interpretation on this issue soon.