No, it's not as if all developers exhibit the same behaviours or nervous tics - but a lot of us tend to suffer from two less-than-stellar character traits: the "Not Invented Here" syndrome and its nefarious cousin "I've always done it this way". Not exactly what you're looking for in an industry where there's a continuing debate about code reuse.
Of course it is a lot more fun to claim you're at the forefront of technology and innovation, but half of the time we're just rehashing the same technology - message queues are about as old as UN*X and let's not even talk about the Network Computer brouha. And sometimes this running in circles makes us blind and turn us into snooty stubborn time-wasters. If you're wondering what triggered this post, well twice this week have I been confronted with this near-pathological behaviour.
The first occasion was reading one colleague's analysis for a new logging system to better track changes at the record level. Obviously an important feature that should be done right, and in this case the idea was to extend the logging features currently present in the application - thumbs up in the reuse department. What struck me as odd about it, was that he didn't want to adopt the auditing system that was added to the underlying database system in a not too distant past. What really irked me about it, was that he didn't even mention it and didn't bother to express a good reason as to why we shouldn't use the built-in mechanism.
The second occasion was when a colleague started to explain how they solved a certain asynchronous data processing problem at his previous job, in what we can euphemistically call a 1980's solution: have an external process poll the database every minute. And he insisted that this was the only robust way to do it, and it worked fast because they had hundreds of these batch processes running on an IBM RS-6000. As the problem unfolded, he had to add more and more checks and balances to counteract concurrent modifications by different processes. When I suggested that for this particular situation a message queue might be more appropriate, he dismissed it as costly and overcomplicated. I don't know about you, but message queues are generally built by very smart people who have thought through the different corner cases, and I'll happily follow their path.
Naturally, as we have creative minds (and no, it's not because we express ourselves through code rather than music or painting that we shouldn't be considered "creative") we seek challenges and aim for the sky when we decide not to fix the code but rewrite it - paraphrasing Captain Jack Sparrow: "a bigger boat, a better boat, that boat." And it is extremely important to deliver something that we feel proud of, which is what drives us to work insane hours and then some more, just to get it done by this impossible deadline with more features than were in the requirements.
But aren't we sometimes overdoing it by completely ripping something apart and replacing it with an alternative that lives up to our current standards? None of us were born with programming experience, we all had to learn through "Hello, world!" in a dozen programming languages, and if all is right, we're still learning new things every day. Yet with an ever-growing to do-list and much longer to research-list, I prefer not to duplicate efforts. Which is why open source libraries, with the right licensing policy, are a great blessing.
Steering back to the topic, I feel this interview with Joshua Bloch is a must-read. Joshua Bloch is the author of Effective Java, considered by many to be among the best books on how to use the Java programming language. Here's the single-best quote from that interview: "In order to stay sane, most developers have a can-do attitude, and some take it too far. They say to themselves, 'Yes, there's a library, but I can do better.' Maybe you can, but that doesn't mean you should."
Mind you, when I recently rebuilt a monitoring tool, I did scrap nearly 90% of the existing code and replaced it with new code that made as much use as possible of new features added in recent versions of Java. So please do not read the above as a mindless "holier-than-thou" rant about short-sighted colleagues. What I really hope, is that you remember to keep an open mind, learn about what's waiting for you to use in your applications - either in the core of your development libraries or in what's available from the outside world. And if you do roll your own, at least make a list of the arguments pro and contra.
Thursday, August 7, 2008
Thursday, July 31, 2008
Service-Orientation vs. Object-Orientation
What better way to fill an hour-long lunch-break than by reading up on various websites? Some people like to play cards and others take a stroll around the village. Lots of people leave breadcrumbs in their keybpoard as they read their private emails, but my daily ritual includes trips to MacWorld, MacUser, OreillyNet, Java.Net, JavaWorld and JavaLobby.
Today I bumped into this fresh post by Masoud Kalali on JavaLobby: Service-Orientation vs. Object-Orientation: Understanding the Impedance Mismatch. Even if your language of choice is Revolution and therefore not object-oriented (even if it's object-based, but we'll leave that for some other time) you'll still want to head over to the site and read the article. After all, a smart programmer keeps his horizons broad so he knows which is the right tool for a particular job, right?
Object-oriented programming has long established itself as a solid software development paradigm, improving on procedural programming by offering encaspulation, inheritance and polymorphism. Service-oriented architecture may still be surrounded by a lot of buzzwords,(often referred to as the SOA ALphabet Soup) but has clearly shown the way to seamless interoperability, regardless of development platform or operating system.
Both OOP and SOA promote reuse. And as more and more business models, developed using an object-oriented language like Java, need to expose their logic as services, it naturally becomes important to do it correctly: rather than just exposing all individually available procedures and functions as web services, you should take a step back and think very hard about exactly what services you seek to offer the outside world.
The innards of your system may be as tightly- or loosely-coupled as you prefer, the truth is that there's a lot of methods in your classes that you're only going to use internally. Services are better defined in terms of business processes, anyway: your 'order entry' service would be wise to offer both a high-level 'do it all in one go' service where an entire order can be created using a single XML structure that includes master and detail information, and a fine-grained 'do just one thing at a time' API to create/read/update/delete one part at a time.
So setting up your services becomes a matter of defining the sensible parts of a workflow, starting with a small area of your application and expanding as it makes sense, based on the feedback of the users of your services - which can be colleagues in the IT-department or external partners. In such an approach, services unsurprisingly have a lot in common with 'task oriented' user interfaces, where the user is guided through the different steps of the process - not in the sometimes belittling way of wizards, but as part of a way to empower the user by concentrating on his or her job.
Information Technology in general and software development in particular are in so many ways still in their infancy. And as we're growing up, we're relearning things from the past, re-inventing how combining existing and sometimes considered-obsolete technologies delivers new value and makes our applications more powerful while easier to use. It feels great to take part in all this...
Today I bumped into this fresh post by Masoud Kalali on JavaLobby: Service-Orientation vs. Object-Orientation: Understanding the Impedance Mismatch. Even if your language of choice is Revolution and therefore not object-oriented (even if it's object-based, but we'll leave that for some other time) you'll still want to head over to the site and read the article. After all, a smart programmer keeps his horizons broad so he knows which is the right tool for a particular job, right?
Object-oriented programming has long established itself as a solid software development paradigm, improving on procedural programming by offering encaspulation, inheritance and polymorphism. Service-oriented architecture may still be surrounded by a lot of buzzwords,(often referred to as the SOA ALphabet Soup) but has clearly shown the way to seamless interoperability, regardless of development platform or operating system.
Both OOP and SOA promote reuse. And as more and more business models, developed using an object-oriented language like Java, need to expose their logic as services, it naturally becomes important to do it correctly: rather than just exposing all individually available procedures and functions as web services, you should take a step back and think very hard about exactly what services you seek to offer the outside world.
The innards of your system may be as tightly- or loosely-coupled as you prefer, the truth is that there's a lot of methods in your classes that you're only going to use internally. Services are better defined in terms of business processes, anyway: your 'order entry' service would be wise to offer both a high-level 'do it all in one go' service where an entire order can be created using a single XML structure that includes master and detail information, and a fine-grained 'do just one thing at a time' API to create/read/update/delete one part at a time.
So setting up your services becomes a matter of defining the sensible parts of a workflow, starting with a small area of your application and expanding as it makes sense, based on the feedback of the users of your services - which can be colleagues in the IT-department or external partners. In such an approach, services unsurprisingly have a lot in common with 'task oriented' user interfaces, where the user is guided through the different steps of the process - not in the sometimes belittling way of wizards, but as part of a way to empower the user by concentrating on his or her job.
Information Technology in general and software development in particular are in so many ways still in their infancy. And as we're growing up, we're relearning things from the past, re-inventing how combining existing and sometimes considered-obsolete technologies delivers new value and makes our applications more powerful while easier to use. It feels great to take part in all this...
Tuesday, July 22, 2008
Message in a Bottle
Most people who have spent some time with me, are convinced that I am musically stuck in the eighties: The Cure, Siouxsie and the Banshees, The Sisters of Mercy and all those happy-sounding bands. But don't let the title of this post lead you to believe I'm talking about The Police. No, I'm talking about enterprise integration in the form of message queues.
In my last post, I talked about how interesting it would be if we could combine the easy-to-master power of cross-platform desktop software development tool Revolution with the heavy-weight champion in cross-platform development: Java. Well, the first way we can make these two technologies work together and play into one another's strengths, is by means of message queues.
Back in the old days, most application integration consisted of exchanging files: application A exported data into a file, and then application B would import the data from that file, usually at the click of two buttons. Convenient, even if you had to copy them to and from floppy disks. In the multi-user time-sharing Unix world, pipes have traditionally played an important role in data exchange, as one application connected its output pipe to the other application's input pipe and vice versa, and just wrote data to one another.
And of course, in the golden age of internetworked computers, you're bound to have enjoyed the miracles of TCP/IP and other networking protocols, as you surfed the web, exchanged instant messages and legally downloaded the occasional file. All thanks to the wonderful world of sockets that connected an application on your computer with an application running on another computer somewhere out there.
Sockets are also often used to let two applications talk to one another while running on the same computer - does the word 'localhost' ring a bell? The common theme in this type of information interchange is that both sides agree on the data format and talk to one another directly in one form or another. Now while that's pretty easy to put together as long as the two parties can come to an agreement, it gets complicated as you have to talk to more and more applications.
Think about it: if two apps talk to one another, you have one connection to develop; when a third app joins the party, two extra connections need to be made; a fourth app means adding three more to the mix. The mathematical formula is n x (n - 1) connections to interconnect n applications. And that's not such a strange scenario: nowadays, our applications can't sit there in their ivory towers - our users expect them to talk to one another seamlessly.
So, would you like to write an ever increasing number of connectors? Or would you prefer an approach that takes care of the plumbing? Enter message queues: think of them like a mass-mailing system for applications - you post a message to the queue and it will make sure that your message reaches all the applications that have declared their interest in receiving all or just certain types of messages. In this method, all you have to agree on is the message format and where it will be posted.
I'm sure you'll agree nothing is easier than speaking the same language and delegating the actual delivery of messages? Add to this the promise of delivering your messages to the applications that are hooked into the system - in order and exactly once! That's certainly a much better deal than people losing the USB-stick, deleting the email or forgetting to import those data exchange files, right? And how about this: when you change the data in one client application, you broadcast it via the message queue and all other clients pick up the message and refresh accordingly? Sounds good to me...
But enough theory, how can we do this in practice? The good news is that most Message Queue vendors have signed on to the Java Messaging System standard: JMS. Even better news is that in these days of open-source software development, we can pick up a solid implementation for free: Apache ActiveMQ - yes, from the makers of the server that's running most of the worldwide web. And the best news (not just for Revolution developers), is the STOMP project: a combination of a bridge and a standard protocol, which allows any socket-wielding application to talk to just about all JMS-enabled Message Queues.
You guessed it: the first way of combining Java and Revolution is using sockets to talk to a JMS-enabled Message Queue. Armed with the STOMP-specs and earlier experiments with socket communication, I put together a STOMP library in a few hours yesterday morning. Now don't rush over to the quartam.com website just yet, as it needs more testing and tweaking before it's ready for general use. But I can tell you that sending and receiving messages works like a treat. Yet another reason to add Revolution to your developer toolbox if you haven't done so already.
So where does the bottle come into this story? Well, I happily drank the rest of the bottle of dry white wine that I opened yesterday evening, while typing this post. Or maybe I was just trying to lure you in with the title of one of the most famous songs by The Police. Then again, wouldn't it be nice if all the messages we carefully wrote as a child at the beach, shoved in a coke bottle and threw into the sea, actually made it to their destinations?
In my last post, I talked about how interesting it would be if we could combine the easy-to-master power of cross-platform desktop software development tool Revolution with the heavy-weight champion in cross-platform development: Java. Well, the first way we can make these two technologies work together and play into one another's strengths, is by means of message queues.
Back in the old days, most application integration consisted of exchanging files: application A exported data into a file, and then application B would import the data from that file, usually at the click of two buttons. Convenient, even if you had to copy them to and from floppy disks. In the multi-user time-sharing Unix world, pipes have traditionally played an important role in data exchange, as one application connected its output pipe to the other application's input pipe and vice versa, and just wrote data to one another.
And of course, in the golden age of internetworked computers, you're bound to have enjoyed the miracles of TCP/IP and other networking protocols, as you surfed the web, exchanged instant messages and legally downloaded the occasional file. All thanks to the wonderful world of sockets that connected an application on your computer with an application running on another computer somewhere out there.
Sockets are also often used to let two applications talk to one another while running on the same computer - does the word 'localhost' ring a bell? The common theme in this type of information interchange is that both sides agree on the data format and talk to one another directly in one form or another. Now while that's pretty easy to put together as long as the two parties can come to an agreement, it gets complicated as you have to talk to more and more applications.
Think about it: if two apps talk to one another, you have one connection to develop; when a third app joins the party, two extra connections need to be made; a fourth app means adding three more to the mix. The mathematical formula is n x (n - 1) connections to interconnect n applications. And that's not such a strange scenario: nowadays, our applications can't sit there in their ivory towers - our users expect them to talk to one another seamlessly.
So, would you like to write an ever increasing number of connectors? Or would you prefer an approach that takes care of the plumbing? Enter message queues: think of them like a mass-mailing system for applications - you post a message to the queue and it will make sure that your message reaches all the applications that have declared their interest in receiving all or just certain types of messages. In this method, all you have to agree on is the message format and where it will be posted.
I'm sure you'll agree nothing is easier than speaking the same language and delegating the actual delivery of messages? Add to this the promise of delivering your messages to the applications that are hooked into the system - in order and exactly once! That's certainly a much better deal than people losing the USB-stick, deleting the email or forgetting to import those data exchange files, right? And how about this: when you change the data in one client application, you broadcast it via the message queue and all other clients pick up the message and refresh accordingly? Sounds good to me...
But enough theory, how can we do this in practice? The good news is that most Message Queue vendors have signed on to the Java Messaging System standard: JMS. Even better news is that in these days of open-source software development, we can pick up a solid implementation for free: Apache ActiveMQ - yes, from the makers of the server that's running most of the worldwide web. And the best news (not just for Revolution developers), is the STOMP project: a combination of a bridge and a standard protocol, which allows any socket-wielding application to talk to just about all JMS-enabled Message Queues.
You guessed it: the first way of combining Java and Revolution is using sockets to talk to a JMS-enabled Message Queue. Armed with the STOMP-specs and earlier experiments with socket communication, I put together a STOMP library in a few hours yesterday morning. Now don't rush over to the quartam.com website just yet, as it needs more testing and tweaking before it's ready for general use. But I can tell you that sending and receiving messages works like a treat. Yet another reason to add Revolution to your developer toolbox if you haven't done so already.
So where does the bottle come into this story? Well, I happily drank the rest of the bottle of dry white wine that I opened yesterday evening, while typing this post. Or maybe I was just trying to lure you in with the title of one of the most famous songs by The Police. Then again, wouldn't it be nice if all the messages we carefully wrote as a child at the beach, shoved in a coke bottle and threw into the sea, actually made it to their destinations?
Monday, June 30, 2008
More Items on my Bookshelf
As a follow-up on my previous post, I have a confession to make. When I looked through my recent additions in Delicious Library, I realized that I had accidentally skipped one of my recent purchases. So in an effort to rectify that, here are some more items that recently found themselves added to my collection.
The item that I forgot to mention, was TCP/IP Sockets in Java. One of the tasks at my day-job is the maintenance of a monitoring service for diagnostics machines - also known as sample or specimen analyzers. The communication between our Laboratory Information System and these machines is of the utmost importance to our customers, and the laboratory staff need to be alerted when the analyzers fail to send their data because of a stalled translator service.
The monitoring system is basically divided in two parts: the service daemon which monitors communication, and the dashboard user interface that displays the status. Both of these are written in Java, enabling monitoring on multiple operating systems - the ubiquitous Windows workstations, as well as Mac, Linux, Solaris and other Unix-derivatives. The data exchange between these parts relies on multi-threaded socket communication, and that's why I bought this book, as I'm sure I'll need it when I go about rewriting portions of the engine to make it more scalable and alleviate the limitations that the current version is bumping into.
Which brings me to the next item on my bookshelf: Java Concurrency in Practice. As computers gain multiple cores and both operating systems and CPUs get ever smarter about dividing up the workload among the available resources, not to mention the fact that users don't like waiting around twiddling their thumbs while data is getting processed, it becomes more and more important to learn how to write multi-threaded applications.
Now Java was built from the start with multi-threading in mind. Over the years, the APIs have been improved, and Java 5 and Java 6 added classes that make concurrent programming a lot easier, shielding developers form the low-level plumbing needed to make it happen. And there's a lot more planned in Java 7 to help sorting and other tasks make better use of the processing powers offered by multi-core environments. This book gives you a lot better insight about the tricky little details of concurrency, explaining how processors, in an effort to maximize throughput, may shuffle instructions around in memory to optimize performance. If you're using Java and want to make optimal use of concurrency, this is a must-read.
The last item that I want to mention this time around, is Effective Java (2nd edition). Considered by many as the bible of properly using Java, this book was revised and updated for Java 6, exploring new design patterns and language idioms, showing the reader how to make the most of features ranging from generics to enums to annotations to autoboxing to (you guessed it) concurrency utilities.
As I'm spending more and more time with Java, I'm looking for a deeper understanding of the technology and the language, hoping to make my code clearer, more correct, more robust, and (fingers crossed) more reusable. Don't worry, I'm still using Revolution as often as I can - it's the language that I jump back to whenever I get a chance. It makes things so much easier and with the upcoming browser plug-in, I can avoid the un-fun of Java applets, AJAX, Flash and Silverlight.
But it is equally important to look around, learn from the available technologies and then pick the best tool for the job. Revolution is a wonderful solution for desktop applications that both need to look good and connect to databases and the internet. But when it comes to building highly-scalable multi-threaded applications, there's no replacement for Java. Ah, if only we could marry these two cross-platform technologies. Or maybe we can? That will have to wait for another post, however.
The item that I forgot to mention, was TCP/IP Sockets in Java. One of the tasks at my day-job is the maintenance of a monitoring service for diagnostics machines - also known as sample or specimen analyzers. The communication between our Laboratory Information System and these machines is of the utmost importance to our customers, and the laboratory staff need to be alerted when the analyzers fail to send their data because of a stalled translator service.
The monitoring system is basically divided in two parts: the service daemon which monitors communication, and the dashboard user interface that displays the status. Both of these are written in Java, enabling monitoring on multiple operating systems - the ubiquitous Windows workstations, as well as Mac, Linux, Solaris and other Unix-derivatives. The data exchange between these parts relies on multi-threaded socket communication, and that's why I bought this book, as I'm sure I'll need it when I go about rewriting portions of the engine to make it more scalable and alleviate the limitations that the current version is bumping into.
Which brings me to the next item on my bookshelf: Java Concurrency in Practice. As computers gain multiple cores and both operating systems and CPUs get ever smarter about dividing up the workload among the available resources, not to mention the fact that users don't like waiting around twiddling their thumbs while data is getting processed, it becomes more and more important to learn how to write multi-threaded applications.
Now Java was built from the start with multi-threading in mind. Over the years, the APIs have been improved, and Java 5 and Java 6 added classes that make concurrent programming a lot easier, shielding developers form the low-level plumbing needed to make it happen. And there's a lot more planned in Java 7 to help sorting and other tasks make better use of the processing powers offered by multi-core environments. This book gives you a lot better insight about the tricky little details of concurrency, explaining how processors, in an effort to maximize throughput, may shuffle instructions around in memory to optimize performance. If you're using Java and want to make optimal use of concurrency, this is a must-read.
The last item that I want to mention this time around, is Effective Java (2nd edition). Considered by many as the bible of properly using Java, this book was revised and updated for Java 6, exploring new design patterns and language idioms, showing the reader how to make the most of features ranging from generics to enums to annotations to autoboxing to (you guessed it) concurrency utilities.
As I'm spending more and more time with Java, I'm looking for a deeper understanding of the technology and the language, hoping to make my code clearer, more correct, more robust, and (fingers crossed) more reusable. Don't worry, I'm still using Revolution as often as I can - it's the language that I jump back to whenever I get a chance. It makes things so much easier and with the upcoming browser plug-in, I can avoid the un-fun of Java applets, AJAX, Flash and Silverlight.
But it is equally important to look around, learn from the available technologies and then pick the best tool for the job. Revolution is a wonderful solution for desktop applications that both need to look good and connect to databases and the internet. But when it comes to building highly-scalable multi-threaded applications, there's no replacement for Java. Ah, if only we could marry these two cross-platform technologies. Or maybe we can? That will have to wait for another post, however.
Subscribe to:
Posts (Atom)