Information Bits

I’m currently working for a company which is a startup in the information retrieval domain founded a year ago a open community platform called Information Retrieval Facility. An international symposium was organized last week this were my impressions:
My impression from overhearing some discussions is that basically, from my point of view, there are three overlapping problem fields: first there are the pure engineering based problems which are sometimes caused by not being aware of current technology and possibilities, in fact most of the solutions are 10 or more years behind the current State of the Art. With professional software engineering, EVERY company is able to solve them, the only thing matters if the technical solution itself supports the actual revenue generating product.
The second problem field is “The big ball of mud” of the patent information itself. The raw information itself is very inconsistent, has a lot of errors, comes in nearly any possible flavor and format, is not standardized in any way (and will never be in near future) and is, nowadays, more a legal document as a technical description. There is no other way as to handle each small bucket of mud carefully and with as much domain knowledge as possible. Because patents are human oriented, only actual domain experts will be capable to work through this. So demand here is to find a way to provide a supporting work flow as well as “Conversational Interfaces” (language oriented, not solemnly voice oriented) to help the mining of information out of the pile of mud. And simple things as flexible filter combinations can help a lot as well as to build up a working feedback loop (OK, not so simple). To drainage the mud, only Google will be able to do this…
The third field is the area of scientific problems in information- and knowledge retrieval. Either on the structural, semantically, language or domain specific area, there are a lot of not so simple problems which had to be solved and there is mostly no currently existing shortcut. The research here goes in two directions: make the information and the actual (meta)knowledge more computational, as well as provide humans tools to help them to cope with the knowledge itself, through navigation, presentation, abstraction or decision making.
Some other common themes were that it is clear that China will be the biggest challenge in the patent world, patent search is a challenging and very important factor at least for international companies and will become more important for smaller ones also, searching in special features as chemical structures or images, feature- and knowledge extraction from pure text are not well integrated into a flexible tool to support search strategies.

So I have to say the event was very well organized and the idea to build a community platform for different professions seems to work out nicely. Presentations and pictures can be found here.

UPDATE: The videos of all presentations now can also be found here.

Yesterday I watched the complete DARPA Urban Challenge through the whole 7 hours and it was worth the time (at least more interesting as some F1 races). Eleven Teams were qualified for the final race, six of them completed all three missions and all in good time. The goal for every bot was to finish the missions as fast as possible, drive without violating the Californian traffic rules and do not collide with any other bot or with any of the other 37 vehicles on the road. The course was very large, with trees and off-road sections, so the GPS has some problems, as well as with some unmapped streets. Every car gets its random generated missions 5 minutes before starting, so no team has known in advanced what to expect.
The first three cars had one thing in common: they drove really confident, especially the Standford car “Junior” which was not only the fastest one, it also drove absolutely perfect. The second one was, as in 2005, “The Boss” from Carnegie Mellon. The last three were also very interesting to watch: “Skynet” and “Little Ben” were very careful and “polite” drivers, they wait on every crossing until any other car has passed and Skynet was by far the best looking car. MIT’s “Talos” was very interesting to watch. They had more sensors and processing power attached to the car as everyone else, trying to process as much environmental data as possible which results in a very “spastic” driving because every 5 minutes the car was definitely not sure what to do. Talos was also the bully in this race, it collides with Team CarOLO and with Skynet, it also ignores most of the traffic rules (infect it drives like a teenager). But because this was MIT’s first participating in the race, it was a very good show.
The winner will be announced today but I thing “Junior” has made it.

So after the 2004 Grand Challenge then only one car droves 8 miles, now six cars succeed in urban environment. In three years it was possible for the technology to evolve very fast and if you think that at least Lexus now starts to build in robotic behavior into the cars (self parking functionality), it is reasonable that in ten years it is technically feasible to build very reliable robotic vehicles. Maybe this was the last challenge, but I think DARPA has shown what is technologically possible in very short time.

 UPDATE: OK, I was wrong. Dr. William “Red” Whittaker has made it this year with “The Boss”, his car wins the DARPA Urban challenge 2007, second is tis time Stanford. He was actually faster with an also perfect driving performance.

The last day was a tutorial from Eric Evans who explained a lot of his ideas behind Domain Driven Design and showed some very interesting tutorial because allthough he is a very slow speaker, what he says is very precise and pointed. DDD can bridge the gap between requirements or stories to well though tasks and design models. Successful modeling needs extensive interaction between development and domain expert. His talk, the tutorial and the book, which I have read a year ago, make now more sense to me because the patterns discussed were not the whole thing, at least as important is to understand the domain in a common language, an ubiquities language for all participants.
We tried to catch the right bus, and after a long drive we were at the airport. The first thing was that we were not able to check our baggage through to Vienna because of security reasons. This worked on our arrival, so why not now!? Second interesting thing was, our flight had one our delay, so we missed the second one to Vienna. So now I’m sitting in the airport and write this blog … waiting for the first flight at 06:55.

UPDATE: Finally we are back in Vienna!

Today I participated in Robert C. Martin’s Class for Test Driven Development. Required was at least a notebook this was easy to satisfy, my Sony Vaio TZ has it’s first long running task to do. Robert C. Martin is one of the most active people who advocate TDD and he is a great presenter. The content of the course is relatively simple: he provides some FITnesse test cases and the first JUnit test cases including some unimplemented base classes for a simple text adventure game (the thing the older one of us remember playing on the first IBM PCs). The first challenge of most people was to get it up and running on Eclipse because he only provided project files for IntelliJ. After this, he explains some of the basic practices of TDD and he and the audience begane to program. Everyone was very disciplined, so writing a JUnit test, implement the functionality and run also the FITnesse tests was no problem at all. So everyone implemented the stories bit for bit, re-factored the code after the tests successfully run. After 4 hours of the tutorial duration everyone has a basic game running, navigating correctly in a arbitrary maze and the code definitely looks better than normally. FITnesse is sometimes a bit cumbersome, but it works very well and is based on a really simple interface language. The main disadvantage is that someone has to write the business case use cases and this should definitely not be the developer. So for using FITnesse, you have to convince some tester, manager or QA guy to write down what he wants as acceptance test.
Some other tips:

• If you have private functions you want to test, move them to a class there they can be public. Mostly they are in the wrong place
• Clean up and re-factor your code every time a new test runs successfully
• Use comments in code sparingly because it smells of bad programming if the code is not clear to follow
• Keep functions and tests as small as possible
• Try to find useful names for functions and tests
• If you work with modern IDEs, learn their shortcuts for using the advanced refactoring capabilities.
• Design your project first, but be aware that tests change the code and the structure of the intended design because they actively use the functionality!
• Sometimes break the rules if you must, but be aware of them and make it no habit

The first talk was held by Wayne Fenton from eBay who told how the tackle the chellange of skaling eBay. What was interesting is, that the have to deploy their features constantly, at least every 2 weeks. The deployments have not only to be deployable but also to be undeployable without hassle! Their critical data is hosted by Oracle, not so critical things are based on MySQL. The split their data according to the functionality of the application, use Sun Solaris in a virtualization environment and Tibco as Event Bus for monitoring. LDAP is used for configuration and deployment. Very interesting talk.
Robert S. Hanmer talks about fault tolerance and emphasis how important the distinction between error, failure and mitigation of the failure is. He currently writes on a book about this. Because he comes from a Telco and Joe Amstrong (Erlang) was also in the audience there was a lifely discussion what the write granularity for mitigation is (the process itself according to Joe). The have a product up and running with the reliability of 9 nines! 99,9999999 availability, and they can proof it (it is a Ericcson Telco switch).
Kevlin Henney talks about the different nations of performance and basically shows that it is a feature which is cross cutting through the whole system and should influence design, architecture and implementation but depends on the context of the application. Great presentation as ever.
Rebecca Wirfs-Brock talks about her experience as long time architecture reviewer and it was interesting to hear that the best thing you can do is to behave like Columbo: ask questions which seems to be silly and try to get contact to all people that really matter there.
The panel afterwards includes besides Kevlin, Wayne, Rebecca, Klaus Marquadt also part time Joseph Yoder and Markus Völter (he wants a architecture language of some kind to express the intentions) and some others of the audience and goes very lively about the theme if architecture really matters. The important point is, if architects can not implement their models, at least a prototype, architecture has no value for itself. Architecture matters but it is a intrinsic feature and the metaphor that it is something which is inhibited by the developers and kept clean (given by Rebecca) is a good explanation why you should care. Interesting was the question of documentation because one out of the audience brought up the question that the best thing maybe is to make a movie from the architecture representation, not longer as 15 minutes and store the movie. The quoted Jeff Sutherland for this idea and practices it in his company. Fascinating idea I have to admit.

The final conference panel was held by Martin Fowler and includes Eric Meijer, Eric Evans, Eric Dörnenburg and Diana Larsen. The basic question was what was the most interesting thing for all of them, but the discussion was very dynamic including the audience about the question why JavaScript sucks and why DSLs, if context specific, are very useful. Diana was sadly a bit misplaced.

Day two starts with a keynote from Eric Meijer where he talks a bit what are his expectations of LINQ: relieve the developer from complicated tasks such as data access and concurrent algorithms. Was a nice talk but nothing new (but he showed the demo of 3D animation in DHTML).
The next talk from Jean Bezivin about model transformation. Was a bit boring in the first half because of the explanation how the OMG MOF meta-metamodel works. The second half was about the Eclipse Model Discovery project which was definitely more interesting.
Eric Evans talked about Strategic Design after lunch. His conclusion was that you have to find out the core model of the domain and focus your whole work primarily on this part. You should get rid of the supporting or generic domain models by buying COTS solutions our outsource the development. To find the core model is context depended and it is mostly not known by the management or the business. You as architect are responsible to work it out.
Klaus Marquardt was on the stage a second time and talks who you can identify complexity inside projects, what you can do no to get more as necessary and how bad are most metrics for measurement (especially senseless KPIs).
Joseph Yoder presents how he models most of his complex domain models with the Adaptive Object Model Pattern. This was especially interesting because I used the pattern in the no so far past and currently for some highly flexible models.
At last we visited the talk of Jimmy Nilson where he shows how he intends currently to use LINQ for Domain Driven Design. Nothing new here but the guy from Microsoft who developed the SQL implementation has told us something about the design decisions and trade offs of his team.

The JAOO 2007 conference days started and the first key note was held by Robert C. Martin about the craft and profession of developing software. He emphasis that software developers have commit themselves to the ethics of avoiding bad code. Complete agreement with that but I think the reality is that most developers produce absolutely bad code and there is no awareness of ethics as is for most people their whole life.
The first interesting talk was given by Charles Simonyi who presents the first real world application of his Intentional Software Workbench. This was the first time I’ve seen a workable peace of his pet project and I’ve to say I by it. You can build a DSL including the domain specific notation, in this case a Excel-like interface, and let generate the supporting code. Looks promising.
The next talk was given by Klaus Marquardt who is project manager by the medical device company Dräger where they build with agile practices a complete, clinical certified complex system. His final point was getting the right people into the project an emphasis a global view of the project for all developers and stackholders. Interestingly, they made the first retrospective 2 years after the project has finished…
After this, Lex Spoon gives a talk about Scala and he demonstrated in a very short example how Scala manage it to combine functional and object oriented features. Scala is really the first language which is capable to implement new language and type class features purly with libraries. Because it is based on Java, there is no problem to use it in real projects.
Gilad Bracha presented a way how to build a executable grammer with the help of Smalltalk and allthough this was a bit esoteric, the reason why this can be interesting is, that you can extend the grammar with the power of Smalltalk (via inheritance and mixins) without the need to rebuild the parser/lexer combination. There is a parser combinator in Scala, Sparsec, so I will take an additional look at it.
The last talk was from Andreas Zeller talked about Beautiful Debugging, the same topic as he has written in the book Beautiful Code. Interestingly is his current reasearch which shows that neither experience nor language features or unit tests can effectively prevent bugs.
The end key note was Charles Simoniy’s presentation of his flight to the IIS and this was really fascinating. I think every one in the room envies him for this!
Converence Party was not so much a party this year, somehow it was not much energy left, so I decided to leave early and write to the blog (at least we met Roy Osherove, ISerialize, who was disappointed because of the evening program …)

Arrival in Aarhus for the JAOO 2007! With a friend I’m the whole week up in Denmark and we will listen to the presentation in the following 3 days and afterwards attend some of the tutorials, in my case the tutorial of Robert C. Martin about Test Driven Development and Eric Evans on Domain Driven Design. The flight was without problems although the Quick check in at Vienna Airport was not so quick at all because the terminal was unable to print the tickets (no problem, you can simply open the box and see what was left inside the printer …), finally we had to check in in the traditional way. At the arrival in Aarhus we run into Rajeev Dayal from Google who gives a talk about GWT 1.4 on JAOO. We are staying in the same hotel, as seems most of the attendees, which is, to say it nice, a bit old fashioned. But it is certainly enough for the week. We were out for lunch and had a interesting discussion how they develop GWT, about language design, C# and Java and about Rich Internet Clients. Tomorrow the conference will start and it will be a challenge to decide which speaker should be heard.

After long years of sending every interesting bit of information to any one who I though was interested in, now finally a blog will do the job. So no more spam from me to patient friends
So for now links, book recommendations, interesting thoughts or other things worth telling a bigger audience will be posted here.