XML validation using a Schema

In earlier posts, I examined how we can use the built-in revXML library in LiveCode to validate XML data against a DTD, later refining it with a version check to match evolving requirements. Unfortunately, a Document Type Definition is quite a limited way of XML validation. So this time, we'll improve our defenses again, by incorporating XML Schemas.

Whereas a DTD is limited to defining the basic structure of the XML in terms of elements and attributes, XML Schemas allow you to define validation on the actual content of the elements and attributes. So you can be sure that an element defined as "xs:date" is actually a valid date, or that an attribute defines as "xs:positiveInteger" is actually a positive integer, etc. A full explanation of XML Schemas is beyond the scope of this post, you'll find plenty of information around the web - a good first stop is this W3Schools XML Schema tutorial.

This all sounds very good, but here's the rub: the revXML library offers no built-in support for XML Schemas. So yet again we turn to Java, with its built-in XML Validation API. We can easily execute Java code using LiveCode's shell function - so let's start by writing the XmlValidateSchema class:

import java.io.File;
import java.io.IOException;

import javax.xml.XMLConstants;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.transform.Source;
import javax.xml.transform.dom.DOMSource;
import javax.xml.validation.Schema;
import javax.xml.validation.SchemaFactory;
import javax.xml.validation.Validator;

import org.w3c.dom.Document;
import org.xml.sax.SAXException;

public class XmlValidateSchema {
 public static void main(String[] args) throws SAXException, ParserConfigurationException, IOException {
  final File xmlFile = new File(args[0]);
  final File xsdFile = new File(args[1]);
  // Load the XML file
  final DocumentBuilderFactory docBuilderFactory = DocumentBuilderFactory.newInstance();
  final DocumentBuilder docBuilder = docBuilderFactory.newDocumentBuilder();
  final Document document = docBuilder.parse(xmlFile);
  // Load the XSD file
  final SchemaFactory schemaFactory = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
  final Schema schema = schemaFactory.newSchema(xsdFile);
  final Validator validator = schema.newValidator();
  // Validate the XML document against the XSL schema
  final Source source = new DOMSource(document);
  validator.validate(source);
 }
}

In keeping with earlier Java examples, the code is a bit lazy when it comes to exception handling: if any exception is thrown, it will end up in the output of our shell function call. The only important thing to remember is that the first parameter is the XML file, and the second is the XML Schema Definition (XSD) file.

Let's go to LiveCode and create a new stack for the user interface.



As you can see, there's a field for the Schema text, a field for the XML text, and a button to Validate the XML against the Schema. Since it's perhaps a tad small, here's the content of the Schema field:

<?xml version="1.0"?>
<xs:schema 
    xmlns:xs="http://www.w3.org/2001/XMLSchema"
    targetNamespace="http://www.quartam.com" 
    xmlns="http://www.quartam.com"
    elementFormDefault="qualified">

<xs:element name="RootNode" type="RootNode"/>

<xs:complexType name="RootNode">
  <xs:sequence>
    <xs:element name="BranchNode" type="BranchNode" maxOccurs="unbounded"/>
  </xs:sequence>
  <xs:attribute name="SpecVersion" type="xs:string"/>
</xs:complexType>

<xs:complexType name="BranchNode">
  <xs:sequence>
    <xs:element name="LeafNode" type="xs:string" maxOccurs="unbounded"/>
  </xs:sequence>
</xs:complexType>

</xs:schema>

And here's the content of the XML field:

<?xml version="1.0"?>
<RootNode 
    xmlns="http://www.quartam.com"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.quartam.com schema.xsd"
    SpecVersion="1.0">
  <BranchNode>
    <LeafNode>The first leaf node</LeafNode>
    <LeafNode>The second leaf node</LeafNode>
    <LeafNode>The third leaf node</LeafNode>
  </BranchNode>
  <BranchNode>
    <LeafNode>The fourth leaf node</LeafNode>
    <LeafNode>The fifth leaf node</LeafNode>
  </BranchNode>
</RootNode>

After saving the stack, we copy the compiled XmlValidateSchema.class file into the same directory as the stack. Now we can write the script for the 'Validate' button:

on mouseUp
   --> write Schema and XML to temporary files
   local tSchemaFile, tXmlFile
   put the tempName into tSchemaFile
   put field "Schema" into URL ("file:" & tSchemaFile)
   put the tempName into tXmlFile
   put field "XML" into URL ("file:" & tXmlFile)
   --> assemble the shell command
   local tShellCommand
   put "java XmlValidateSchema" && \
          ShellPath(tXmlFile) && \
          ShellPath(tSchemaFile) \
          into tShellCommand
   --> execute the shell command
   local tHideConsoleWindows, tDefaultFolder, tShellResult
   put the hideConsoleWindows into tHideConsoleWindows
   set the hideConsoleWindows to true
   put the defaultFolder into tDefaultFolder
   set the defaultFolder to AbsolutePathFromStack()
   put shell(tShellCommand) into tShellResult
   set the defaultFolder to tDefaultFolder
   set the hideConsoleWindows to tHideConsoleWindows
   --> cleanup the temporary files
   delete file tSchemaFile
   delete file tXmlFile
   if tShellResult is not empty then
      answer error tShellResult
   end if
end mouseUp

function AbsolutePathFromStack pFileName
   local tAbsolutePath
   put the effective filename of this stack into tAbsolutePath
   set the itemDelimiter to slash
   if pFileName is not empty then
      put pFileName into item -1 of tAbsolutePath
   else
      delete item -1 of tAbsolutePath
   end if
   return tAbsolutePath
end AbsolutePathFromStack

function ShellPath pPath
   if the platform is "Win32" then
      put quote & pPath & quote into pPath
   else
      replace space with backslash & space in pPath
   end if
   return pPath
end ShellPath

This time around, we didn't have to fiddle with the Java classpath, as the XML Validation API is built-in. However, we had to write the XML and Schema to temporary files, to avoid length limitations in the shell command. If we now make a deliberate mistake, say change one of the 'LeafNode' elements into a 'BeafNode' element, we see this error:



Again, the image is a bit small, so here's the content of the error:

ERROR:  'cvc-complex-type.2.4.a: Invalid content was found starting with element 'BeafNode'. One of '{"http://www.quartam.com":LeafNode}' is expected.'
Exception in thread "main" org.xml.sax.SAXParseException: cvc-complex-type.2.4.a: Invalid content was found starting with element 'BeafNode'. One of '{"http://www.quartam.com":LeafNode}' is expected.
	at com.sun.org.apache.xerces.internal.jaxp.validation.Util.toSAXParseException(Util.java:109)
	at com.sun.org.apache.xerces.internal.jaxp.validation.ErrorHandlerAdaptor.error(ErrorHandlerAdaptor.java:104)
	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:382)
	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:316)
	at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator$XSIErrorReporter.reportError(XMLSchemaValidator.java:429)
	at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.reportSchemaError(XMLSchemaValidator.java:3185)
	at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.handleStartElement(XMLSchemaValidator.java:1831)
	at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.startElement(XMLSchemaValidator.java:705)
	at com.sun.org.apache.xerces.internal.jaxp.validation.ValidatorHandlerImpl.startElement(ValidatorHandlerImpl.java:335)
	at com.sun.org.apache.xml.internal.serializer.ToXMLSAXHandler.closeStartTag(ToXMLSAXHandler.java:205)
	at com.sun.org.apache.xml.internal.serializer.ToXMLSAXHandler.characters(ToXMLSAXHandler.java:524)
	at com.sun.org.apache.xml.internal.serializer.ToXMLSAXHandler.characters(ToXMLSAXHandler.java:467)
	at com.sun.org.apache.xalan.internal.xsltc.trax.DOM2TO.parse(DOM2TO.java:229)
	at com.sun.org.apache.xalan.internal.xsltc.trax.DOM2TO.parse(DOM2TO.java:215)
	at com.sun.org.apache.xalan.internal.xsltc.trax.DOM2TO.parse(DOM2TO.java:215)
	at com.sun.org.apache.xalan.internal.xsltc.trax.DOM2TO.parse(DOM2TO.java:215)
	at com.sun.org.apache.xalan.internal.xsltc.trax.DOM2TO.parse(DOM2TO.java:121)
	at com.sun.org.apache.xalan.internal.xsltc.trax.DOM2TO.parse(DOM2TO.java:85)
	at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transformIdentity(TransformerImpl.java:615)
	at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:661)
	at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:300)
	at com.sun.org.apache.xerces.internal.jaxp.validation.ValidatorImpl.process(ValidatorImpl.java:220)
	at com.sun.org.apache.xerces.internal.jaxp.validation.ValidatorImpl.validate(ValidatorImpl.java:141)
	at javax.xml.validation.Validator.validate(Validator.java:82)
	at XmlValidateSchema.main(XmlValidateSchema.java:31)

Thanks to the combination of LiveCode and Java, we can develop cross-platform solution quickly, without having to give up the power of existing libraries. Unfortunately, loading Java every time for a shell call is not the optimal solution, so in another post, we'll investigate how we can run Java code using LiveCode's 'process' communication. Stay tuned...

Stamping PDF files

In a previous post, I examined how we can use LiveCode and the Java-based iText library to concatenate a series of existing PDF files into a single PDF file. Now we will examine how we can 'stamp' a PDF file with an image using the same technique.

The first thing to code is the Java class that we will call using the shell function. Here's what I came up with:

import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;

import com.lowagie.text.DocumentException;
import com.lowagie.text.Image;
import com.lowagie.text.Rectangle;
import com.lowagie.text.pdf.PdfContentByte;
import com.lowagie.text.pdf.PdfReader;
import com.lowagie.text.pdf.PdfStamper;

public class StampPdfFile {
  public static void main(String[] args) throws IOException, DocumentException {
    final String inputFile = args[0];
    final String outputFile = args[1];
    final String imageFile = args[2];
    final String[] coords = args[3].split(",");
    final PdfReader inputReader = new PdfReader(inputFile);
    final OutputStream outputStream = new FileOutputStream(outputFile);
    final PdfStamper outputStamper = new PdfStamper(inputReader, outputStream);
    final int pageCount = inputReader.getNumberOfPages();
    final Image image = Image.getInstance(imageFile);
    final int left = Integer.parseInt(coords[0]);
    final int top = Integer.parseInt(coords[1]);
    final int right = Integer.parseInt(coords[2]);
    final int bottom = Integer.parseInt(coords[3]);
    final int height = bottom - top;
    final int width = right - left;
    image.scaleToFit(width, height);
    for (int pageIndex = 1; pageIndex <= pageCount; pageIndex++) {
      final PdfContentByte overContent = outputStamper.getOverContent(pageIndex);
      final Rectangle pageSize = inputReader.getPageSize(pageIndex);
      image.setAbsolutePosition(left, pageSize.getHeight() - bottom);
      overContent.addImage(image);
    }
    outputStamper.close();
  }
}

In a nutshell, the first parameter is the input file, the second the output file, the third the image file, and the fourth parameter is a comma-separated list of coordinates making up the target rectangle. As usual, the code is a tad lazy when it comes to faulty input parameters and exception handling - if there's a mistake you'll simply get the stacktrace as the output of the shell command.

The most important bit is in the loop over the pages, where we use the outputStamper.getOverContent() method to draw our image on top of the existing content. If you'd rather have the image in the back, as a watermark, you would use the outputStamper.getUnderContent() methopd instead. Also note that setting the image position coordinate system works from the bottomLeft of the page, so we have to use the original page height and subtract the bottom coordinate from it.

Now we can proceed with writing a LiveCode button script:

on mouseUp
   --> determine the input, output and image files
   local tInputFile, tOutputFile, tImageFile
   put ShellPath(AbsolutePathFromStack("demo1.pdf")) \
          into tInputFile
   put ShellPath(AbsolutePathFromStack("stamp.pdf")) \
          into tOutputFile
   put ShellPath(AbsolutePathFromStack("Template.png")) \
          into tImageFile
   --> determine the image target rectangle
   local tImageRect
   put quote & "10,10,103,87" & quote \
          into tImageRect
   --> determine the class path
   local tClassPath
   if the platform is "Win32" then
      put ".;iText-2.1.7.jar" into tClassPath
   else
      put ".:iText-2.1.7.jar" into tClassPath
   end if
   --> assemble the shell command
   local tShellCommand
   put "java -classpath" && tClassPath && \
          "StampPdfFile" && \
          tInputFile && \
          tOutputFile && \
          tImageFile && \
          tImageRect \
          into tShellCommand
   --> execute the shell command
   local tHideConsoleWindows, tDefaultFolder, tShellResult
   put the hideConsoleWindows into tHideConsoleWindows
   set the hideConsoleWindows to true
   put the defaultFolder into tDefaultFolder
   set the defaultFolder to AbsolutePathFromStack()
   put shell(tShellCommand) into tShellResult
   set the defaultFolder to tDefaultFolder
   set the hideConsoleWindows to tHideConsoleWindows
   if tShellResult is not empty then
      answer error tShellResult
   end if
end mouseUp

function AbsolutePathFromStack pFileName
   local tAbsolutePath
   put the effective filename of this stack into tAbsolutePath
   set the itemDelimiter to slash
   if pFileName is not empty then
      put pFileName into item -1 of tAbsolutePath
   else
      delete item -1 of tAbsolutePath
   end if
   return tAbsolutePath
end AbsolutePathFromStack

function ShellPath pPath
   if the platform is "Win32" then
      put quote & pPath & quote into pPath
   else
      replace space with backslash & space in pPath
   end if
   return pPath
end ShellPath

Click the button, and it happily takes the existing PDF files (demo1.pdf), paints the image (Template.png) on top of all pages, and writes a new PDF file (stamp.pdf) in the same folder as our stack. There we have it, another example of using iText from within LiveCode.

Concatenating PDF files

Since the advent of LiveCode 4.5, developers have the ability to 'print' stack content directly to PDF files. And if you need pin-point control over what goes where, you can use Quartam PDF Library to generate PDF files from scripts. That's great if you are in full control of the content, but what if you need to work with existing PDF files? In the next few posts, we will examine how you can tap into the power of the Java-based iText library from LiveCode.
So let's start by downloading a copy of iText version 2.1.7 - do not use version 5.x as the API changed and the following example code won't work.

The first question is: how can we execute Java code from LiveCode? The simplest solution is the shell function: it allows you to execute DOS or Unix commands, as if you typed them in from the command line. Note that on Windows, using this function will show a DOS window, but you can control that by setting the hideConsoleWindows property before calling the shell function.
You can test it out by simply executing the following line from the message box:

  answer shell("java -version")

The second question is: what sort of Java code do we need to write? Well, I fired up a copy of Eclipse, started a new project, and created a new class 'ConcatPdfFiles' in the default package. Then I grabbed my paper copy of iText in action (first edition) and flipped to page 64 as this contains the examples for concatenating PDF files. A little bit of thinking, and I derived the following code:

import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;

import com.lowagie.text.Document;
import com.lowagie.text.DocumentException;
import com.lowagie.text.pdf.PdfCopy;
import com.lowagie.text.pdf.PdfReader;

public class ConcatPdfFiles {
 public static void main(String[] args) throws DocumentException, IOException {
  final String outputFilePath = args[0];
  final OutputStream outputStream = new FileOutputStream(outputFilePath);
  final Document outputDocument = new Document();
  final PdfCopy outputCopy = new PdfCopy(outputDocument, outputStream);
  outputDocument.open();
  for (int i = 1; i < args.length; i++) {
   final PdfReader inputPdfReader = new PdfReader(args[i]);
   final int pageCount = inputPdfReader.getNumberOfPages();
   for (int pageIndex = 0; pageIndex < pageCount; pageIndex++) {
    outputCopy.addPage(outputCopy.getImportedPage(inputPdfReader, pageIndex + 1));
   }
  }
  outputDocument.close();
 }
}

As you can see, the code is a bit lazy when it comes to exception handling: I just let the exceptions get thrown, and this will be the output of our shell call if something goes wrong. Note also that the first argument is the output file, followed by the input files that you want to concatenate into the output file.

More importantly, at this point in time, the code doesn't compile. The problem is, we haven't yet told Eclipse where that iText-2.1.7.jar library file is, so compilation fails. This is sometimes referred to as 'classpath hell' - you have to give Java a list of paths where it can find the necessary additional libraries, not just at compile time but also at runtime as we'll see later.
Because I like to keep everything together in my Java projects, I added a new 'lib' folder to my project, and copied the iText2.1.7.jar file into it. At that point, you can use the contextual menu on the iText.2.1.7.jar file, and add it to the Build Path. Now the code I showed earlier compiles just fine, and we can proceed to the next stage.

The third question is: how do we put everything together in LiveCode? We'll begin by putting all the necessary parts into a single folder: the iText-2.1.7.jar library file, the ConcatPdfFiles.class compiled file and two example PDF files (demo1.pdf and demo2.pdf). Then we fire up LiveCode, create a new stack 'ConcatPdfFiles' and save it in the same folder as the other files, naming it "ConcatPdfFiles.liveCode'. Now we can drop a button onto the stack and start scripting.

Now we need to determine the correct command to be executed by the shell function. It should look something like:

java -classpath <class-path> ConcatPdfFiles <output-file> <input-file-1> <input-file-2> ...

The java executable needs the correct classpath, and we need to pass in compatible file paths.

Let's start with the classpath. This is a list of places that java needs to look for its .class files - as separate files in folders, or stored together in a .jar file. And for extra fun, the separator character is a colon on Unix-based platforms, and a semicolon on Windows. You can have relative paths in this classpath, and '.' (period) is short for the current directory. So rather than building a long class path, we can circumvent the issue by setting the defaultFolder property to change the working directory before calling the shell function. Then our classpath can be as short as:

.:iText-2.1.7.jar

on MacOS X/Linux and

.;iText-2.1.7.jar

on Windows.

The next bit is compatible file paths. The good news: LiveCode uses a '/' (slash) as separator, regardless of the underlying platform, and Java is more than happy to accept '/' in a path, even when it's running on Windows. However, if there are spaces in the path, we need to save them by putting quotes around the path on Windows, and escaping the spaces with a backslash on Unix-based platforms.
And to determine the paths relative to the stack's location on your hard disk, we'll need a helper function that uses the effective filename property of our stack.

So finally, we have a button script as follows:

on mouseUp
   --> determine the input and output files
   local tInputFiles, tOutputFile
   put ShellPath(AbsolutePathFromStack("demo1.pdf")) && \
          ShellPath(AbsolutePathFromStack("demo2.pdf")) \
          into tInputFiles
   put ShellPath(AbsolutePathFromStack("output.pdf")) \
          into tOutputFile
   --> determine the class path
   local tClassPath
   if the platform is "Win32" then
      put ".;iText-2.1.7.jar" into tClassPath
   else
      put ".:iText-2.1.7.jar" into tClassPath
   end if
   --> assemble the shell command
   local tShellCommand
   put "java -classpath" && tClassPath && \
          "ConcatPdfFiles" && \
          tOutputFile && tInputFiles \
          into tShellCommand
   --> execute the shell command
   local tHideConsoleWindows, tDefaultFolder, tShellResult
   put the hideConsoleWindows into tHideConsoleWindows
   set the hideConsoleWindows to true
   put the defaultFolder into tDefaultFolder
   set the defaultFolder to AbsolutePathFromStack()
   put shell(tShellCommand) into tShellResult
   set the defaultFolder to tDefaultFolder
   set the hideConsoleWindows to tHideConsoleWindows
   if tShellResult is not empty then
      answer error tShellResult
   end if
end mouseUp

function AbsolutePathFromStack pFileName
   local tAbsolutePath
   put the effective filename of this stack into tAbsolutePath
   set the itemDelimiter to slash
   if pFileName is not empty then
      put pFileName into item -1 of tAbsolutePath
   else
      delete item -1 of tAbsolutePath
   end if
   return tAbsolutePath
end AbsolutePathFromStack

function ShellPath pPath
   if the platform is "Win32" then
      put quote & pPath & quote into pPath
   else
      replace space with backslash & space in pPath
   end if
   return pPath
end ShellPath

Click the button, and it happily concatenates the two PDF files (demo1.pdf and demo2.pdf) into a single PDF file (output.pdf) in the same folder as our stack. There we have it, our first use of iText from within LiveCode.

I went to the Devoxx 2010 conference and...

and I got the t-shirt to prove it


100% Javaholic was the motto of this year's Java developer conference, organised by the Belkgian JUG and held in Antwerp.
Of course, I wasn't there to collect the t-shirt - unlike some others who shall remain unnamed and went from booth to booth to fetch a t-shirt, even if they weren't interested in the product. You know who you are :-p

and I met Bruno Lowagie

For those who don't know Bruno Lowagie, he's the primary developer behind the iText project, an open-source library for producing PDF files using Java. There must be something in the water around here, as he lives about 30 miles from my house - two Belgians, both writing PDF libraries - what are the odds?
As he was busy signing copies of iText in Action (second edition) the conversation was rather short, but he signed my copy of his book with "To a fellow PDF developer" - what a compliment :-)

and I managed to squeeze LiveCode onto a whiteboard


Given that someone actually added 'HyperTalk :-)' to the board right behind my entry, I think I wasn't the only guy there who knows LiveCode. Next time, poke me and I'll buy you a drink.

and had a very good time overall

Plenty of interesting sessions, friendly environment and excellent organization. I'll definitely go back next time I get the chance.

Revolution is now LiveCode

Earlier today, RunRev announced the availability of LiveCode 4.5 - the new name and version for the development platform Revolution.

If you've come here and don't know what LiveCode/Revolution is: it is a cross-platform development tool for Windows, MacOSX and Linux, sporting a very-high-level-language that allows you to create working solutions in far less time than other tools and languages. The LiveCode Web browser plug-in works very similar to Flash and Silverlight. There are also pre-releases available of LiveCode Server (comparable to a PHP engine) and LiveCode for iOS (yep, IPhone, iPad and iPod Touch).

Whenever I get a chance, I'll use LiveCode to put together a quick tool for my own use, or a prototype/mockup for something that will eventually be developed in another language like Progress OpenEdge ABL or Java. I also offer Quartam Reports for LiveCode, the essential report generator, and Quartam PDF Library, for those occasions when you need to go beyond 'print to pdf file'.

Where Java is built around a relatively small platform-specific Virtual Machine implementation, with a huge bytecode library on top, LiveCode has a far more optimized yet surprisingly lean core implementation for each supported operating system, and a rich language that means you'll type fewer lines of code to accomplish the same result.

I like the name change and the clearer product line-up. And I like how this new version is faster than ever before. Cheers to the RunRev team for wrapping it up after the setback concerning the iPhone/iPad implementation. Combined with Apple's recent liberation of developer tools, the future looks decidedly LiveCode!

XML validation using a DTD and Versions

In a previous post, I invetigated how we could validate an XML file in revTalk by using a DTD. As we all know, requirements evolve and our software needs to adapt likewise, especially at the integration end-points. So if we need to accept more data, we should (a) know that it's coming, and (b) make sure it's there before we process the incoming data.

How do we do that? Well, as we've seen beofre, we can check the structure of the XML file. And to make sure we check it correctly, we should introduce a specification version for our XML structure, and attach it to the root node of the document as an attribute. Then all we need to do is check the root node, extract its SpecificationVersion attribute, and we can apply the correct DTD validation.

Doesn't sound too complicated, does it? Let's expand our current stack design a bit so that it looks like this:


As you can see, the field "DTD" was renamed to "DTD 1.0" and another field "DTD 1.1" was added to hold the DTD for specification version 1.1; finally, I moved down the "Validate" button and modified its script:

constant kMaxVersion = 1.1

on mouseUp
   -- Load the XML into a local variable
   local tXmlText
   put field "XML" into tXmlText
   -- Parse the XML text into a Tree
   local tXmlTree
   put revCreateXmlTree(tXmlText, \
          false, \
          true,  \
          false) \
          into tXmlTree
   if tXmlTree is not an integer then
      answer error \
             "There is an error in the XML structure" & return & \
             tXmlTree  -- contains the full error message
   else
      -- Validate the root node
      local tRootNode
      put revXmlRootNode(tXmlTree) into tRootNode
      if tRootNode is not "RootNode" then
         answer error \
                "The XML root node should be 'RootNode'"
      else
         -- Validate the SpecVersion
         local tSpecVersion
         put revXmlAttribute(tXmlTree, tRootNode, \
                "SpecificationVersion") into tSpecVersion
         if tSpecVersion begins with "xmlerr" then
            answer error \
                   "The SpecificationVersion is missing"
         else if tSpecVersion > kMaxVersion then
            answer error \
                   "The SpecificationVersion " && tSpecVersion && \
                   "is newer than" && kMaxVersion
         else
            -- Load the corresponding DTD
            local tDtdText
            if tSpecVersion is 1.0 then
               put field "DTD 1.0" into tDtdText
            else
               put field "DTD 1.1" into tDtdText
            end if
            -- Validate the XML against the DTD
            local tValidationResult
            put revXmlValidateDTD(tXmlTree, tDtdText) \
                   into tValidationResult
            if tValidationResult is not empty then
               answer error \
                      "The XML structure does not conform" & \
                      return & tValidationResult
            else
               answer information "The XML conforms to the DTD"
            end if
         end if
      end if
      -- Cleanup
      revDeleteXmlTree tXmlTree
   end if
end mouseUp

So how does this new version work?
- first, it parses the XML document
- next, it verifies the root node
- next, it checks the specification version
- next, it loads the appropriate DTD
- finally, it validates the XML against the DTD

If we test it, it correctly informs us that the XML document conforms to our specification version 1.1. What happens if we change the SpecificationVersion to 1.2?


Then we get this error message:


And finally, what happens if we change the SpecificationVersion to the original 1.0?


Then we get this error message:


This is a much safer way to check the incoming data in XML format. Unfortunately, a Document Type Definition is quite a limited way of XML validation. So next time, we'll improve our defenses again, by incorporating XML Schemas.

XML validation using a DTD

One of the positive aspects of the Extensible Markup Language XML is that it is a flexible way to structure data in a human-readable format, in a cross-platform and technology-independent way. No wonder it is widely used as a way to exchange data between applications, and forms the foundation for XML-RPC, SOAP and other Web Service methods.

But it would be naive to think that every XML document that we get is not only well-formed, but also in the format that we expect it to be, with the right elements and attributes. In this post, we'll examine a strategy to validate incoming XML data in our revTalk application, using a Document Type Definition - a.k.a. DTD.

Part of the XML specification since the very start, a DTD describes the structure of the XML elements and attributes. For more information, I advise you to study the excellent introductory tutorials on W3Schools.com. We're here to use it from revTalk, so let's start by creating a new stack for the user interface.


Simply drop two scrolling fields onto it, name them "XML" and "DTD" respectively, and then group each of them separately so we can put a nice group label on top (I have the memory of a goldfish so I might forget which-is-which ;-) ) Finally drop a button at the bottom of the stack and set its name to "Validate" - and now we're ready to start scripting the button.

First things first, we need to parse the XML text into an XML tree to use all the rev XML commands and functions.

on mouseUp
   -- Load DTD and XML into local variables
   local tDtdText, tXmlText
   put field "DTD" into tDtdText
   put field "XML" into tXmlText
   -- Parse the XML text into a Tree
   local tXmlTree
   put revCreateXmlTree(tXmlText, \
          false, \ -- must be well-formed
          true,  \ -- create a tree in memory
          false) \ -- no SAX parser messages
          into tXmlTree
   if tXmlTree is not an integer then
      answer error \
             "There is an error in the XML structure" & return & \
             tXmlTree  -- contains the full error message
   else
      -- Clean up resources
      revDeleteXmlTree tXmlTree
   end if
end mouseUp

We use the revCreateXmlTree function to parse the XML text into a tree structure. If the XML test is not well-formed then we report the error, otherwise we know we have a valid XML tree structure at our disposal - which we need to cleanup after we're done, using the revDeleteXmlTree command. Now that we have the basics covered, we can add the DTD validation to our script.

on mouseUp
   -- Load DTD and XML into local variables
   local tDtdText, tXmlText
   put field "DTD" into tDtdText
   put field "XML" into tXmlText
   -- Parse the XML text into a Tree
   local tXmlTree
   put revCreateXmlTree(tXmlText, \
          false, \  -- must be well-formed
          true, \ -- create a tree in memory
          false) \ -- no SAX parser messages
          into tXmlTree
   if tXmlTree is not an integer then
      answer error \
             "There is an error in the XML structure" & return & \
             tXmlTree  -- contains the full error message
   else
      -- Validate the XML against the DTD
      local tValidationResult
      put revXmlValidateDTD(tXmlTree, tDtdText) \
             into tValidationResult
      if tValidationResult is not empty then
         answer error \
                "XML structure does not conform to the DTD" & return & \
                tValidationResult  -- contains the full error message
      else
         answer information "The XML conforms to the DTD"
      end if
      -- Clean up resources
      revDeleteXmlTree tXmlTree
   end if
end mouseUp

If the XML conforms to the DTD, the revXmlValidateDtd function will return empty, otherwise its output contains the validation error. Pretty straightforward, so let's test this with a simple XML and DTD:


When we click the 'Validate' button, we get the message that the XML conforms to the DTD. Exactly what we were hoping for. Now let's change the XML somewhat to see if it fails when our XML text clearly does not conform to the DTD.


And here's the error message that we get on our screen:


With very little scripting, we have added a first layer of defense against incoming XML data that is not up to our specifications. Next time, we'll elaborate on this example and bolster our defenses.

Fun with detailed files - and stat

In a previous post, I tried to find a way to convert Unix user and group ids into their corresponding names. After I reported my findings to the use-revolution mailing list, Andre Garzia stepped in, suggesting to take a look at the stat command.

Calling 'stat' through the shell function for every file would be costly and slow; using it to fetch the information for an entire folder in one go wouldn't be so bad, but if you already use the detailed files function built into revTalk, it would seem like a waste of cycles. So I decided to rewrite my work handlers, employing a caching mechanism for the id-to-name mapping, and using 'stat' to pick up the mapping information not available in the cache.

--> script-local 'static' variables

local sGroupIdMap, sUserIdMap

--> public functions and commands

command LoadIdMaps
  p_LoadGroupIdMap
  p_LoadUserIdMap
end LoadIdMaps

function GroupIdToName pGroupId, pFile
  local tName, tShellCommand
  put sGroupIdMap[pGroupId] into tName
  if tName is empty and the platform is "MacOS" then
    -- stat is only available on MacOS X
    put p_GetFileStat(pFile, "%Sg") into tName
    put tName into sGroupIdMap[pGroupId]
  end if
  return tName
end GroupIdToName

function UserIdToName pUserId, pFile
  local tName, tShellCommand
  put sUserIdMap[pUserId] into tName
  if tName is empty and the platform is "MacOS" then
    -- stat is only available on MacOS X
    put p_GetFileStat(pFile, "%Su") into tName
    put tName into sUserIdMap[pUserId]
  end if
  return tName
end UserIdToName

--> private functions and commands

private function p_GetFileStat pFile, pFormat
  local tName, tShellCommand
  -- make sure to escape spaces in the file name
  replace space with backslash & space in pFile
  put "stat -f" && pFormat && pFile into tShellCommand
  put word 1 of shell(tShellCommand) into tName
  return tName
end p_GetFileStat

private command p_LoadGroupIdMap
  if the platform is "Win32" then exit p_LoadGroupIdMap
  p_LoadFileIdMap "/etc/group", sGroupIdMap
end p_LoadGroupIdMap

private command p_LoadUserIdMap
  if the platform is "Win32" then exit p_LoadUserIdMap
  p_LoadFileIdMap "/etc/passwd", sUserIdMap
end p_LoadUserIdMap

private command p_LoadFileIdMap pFile, @pIdMap
  local tData, tLine
  put URL ("file:" & pFile) into tData
  set the itemDelimiter to colon
  repeat for each line tLine in tData
    -- skip empty and comment lines
    if tLine is empty \
        or char 1 of tLine is "#" then next repeat
    -- fill the id map
    put item 1 of tLine into pIdMap[item 3 of tLine]
  end repeat
end p_LoadFileIdMap

Armed with the above set of commands and functions, we can still use the detailed files function and obtain user and group names from a cache, calling into 'stat' as necessary. Let's update our sample button script:

on mouseUp
  local tOldFolder, tNewFolder, tDetailedFiles
  answer folder "Select a folder"
  if the result is "Cancel" then exit mouseUp
  put it into tNewFolder
  --> extract the detailed files information
  put the defaultFolder into tOldFolder
  set the defaultFolder to tNewFolder
  put the detailed files into tDetailedFiles
  --> load the group and user id maps
  LoadIdMaps
  --> build the extended file information
  local tDetailedFile, tExtendedFiles
  local tFile, tUserId, tGroupId
  repeat for each line tDetailedFile in tDetailedFiles
    -- skip hidden files
    if char 1 of tDetailedFile is "." then next repeat
    -- fill the extended file entry
    put urlDecode(item 1 of tDetailedFile) into tFile
    put item 8 of tDetailedFile into tUserId
    put item 9 of tDetailedFile into tGroupId
    put tFileName & tab & \
        UserIdToName(tUserId, tFile) & tab & \
        GroupIdToName(tGroupId, tFile) & return \
        after tExtendedFiles
  end repeat
  --> wrap things up
  set the defaultFolder to tOldFolder
  put char 1 to -2 of tExtendedFiles into field "Files"
end mouseUp

The more observant amongst you may think "If all you needed was a file's user and group name, why not just 'stat' and get it over with?" - well, the detailed files function returns a lot of useful information in one go, including the size of the resource fork of a file on MacOS systems.
Just one more thing: the 'stat' function was added in MacOS X 10.4 Tiger - so if your application still needs to support MacOS X 10.3 Panther, you're going to have to merge in the 'nireport' call from the previous post. Consider it an exercise for the reader.

On to the next challenge!

Fun with detailed files

Earlier today, the following question was posted on the use-revolution mailing list: file owner & group names. Rev veteran Monte Goulding was wondering how to convert file owner and group ids into actual user and group names. I happily logged into the Solaris server at work and dug around a bit, finally replying that he could parse this information from the following two files:

/etc/group
/etc/passwd

And what would such a parsing script look like in revTalk? I'm glad you asked:

command LoadGroupIdMap @pGroupIdMap
  p_LoadFileIdMap "/etc/group", pGroupIdMap
end LoadGroupIdMap

command LoadUserIdMap @pUserIdMap
  p_LoadFileIdMap "/etc/passwd", pUserIdMap
end LoadUserIdMap

private command p_LoadFileIdMap pFile, @pIdMap
  local tData, tLine
  put URL ("file:" & pFile) into tData
  set the itemDelimiter to colon
  repeat for each line tLine in tData
    -- skip empty and comment lines
    if tLine is empty \
        or char 1 of tLine is "#" then next repeat
    -- fill the id map
    put item 1 of tLine into pIdMap[item 3 of tLine]
  end repeat
end p_LoadFileIdMap

Given that both files use a colon as item delimiter, and have the name as first item and the id as third item, I extracted the parsing into a separate private command. Put the above handlers in your stack script, and then fetch each id-to-name-mapping before interpreting the detailed files.
Here's a simple button script that uses the above commands:

on mouseUp
  local tOldFolder, tNewFolder, tDetailedFiles
  answer folder "Select a folder"
  if the result is "Cancel" then exit mouseUp
  put it into tNewFolder
  --> extract the detailed files information
  put the defaultFolder into tOldFolder
  set the defaultFolder to tNewFolder
  put the detailed files into tDetailedFiles
  set the defaultFolder to tOldFolder
  --> load the group and user id maps
  local tGroupIdMap, tUserIdMap
  LoadGroupIdMap tGroupIdMap
  LoadUserIdMap tUserIdMap
  --> build the extended file information
  local tDetailedFile, tExtendedFiles
  repeat for each line tDetailedFile in tDetailedFiles
    -- skip hidden files
    if char 1 of tDetailedFile is "." then next repeat
    -- fill the extended file entry
    put urlDecode(item 1 of tDetailedFile) & tab & \
        tUserIdMap[item 8 of tDetailedFile] & tab & \
        tGroupIdMap[item 9 of tDetailedFile] & return \
        after tExtendedFiles
  end repeat
  --> wrap things up
  put char 1 to -2 of tExtendedFiles into field "Files"
end mouseUp

Once I got home, I wanted to verify this on my MacOS X machine, only to discover that it wasn't properly mapping the uid and gid to user and group names. Intrigued, I dug a bit deeper and found out that MacOS X actually relies on a DirectoryService when it's not running in single-user mode. Lovely, but how can we get what we need using revTalk?
Well, it turns out you can use the nireport command via the shell function to fetch a list of the group and user names with id. So let's add another helper command to do things the MacOS X way, and update our existing commands to use the correct approach, depending on the platform.

command LoadGroupIdMap @pGroupIdMap
  if the platform is "MacOS" then
    p_LoadNireportIdMap "/groups gid name", pGroupIdMap
  else if the platform is "Linux" then
    p_LoadFileIdMap "/etc/group", pGroupIdMap
  end if
end LoadGroupIdMap

command LoadUserIdMap @pUserIdMap
  if the platform is "MacOS" then
    p_LoadNireportIdMap "/users uid name", pUserIdMap
  else if the platform is "Linux" then
    p_LoadFileIdMap "/etc/passwd", pUserIdMap
  end if
end LoadUserIdMap

private command p_LoadFileIdMap pFile, @pIdMap
  local tData, tLine
  put URL ("file:" & pFile) into tData
  set the itemDelimiter to colon
  repeat for each line tLine in tData
    -- skip empty and comment lines
    if tLine is empty \
        or char 1 of tLine is "#" then next repeat
    -- fill the id map
    put item 1 of tLine into pIdMap[item 3 of tLine]
  end repeat
end p_LoadFileIdMap

private command p_LoadNireportIdMap pParams, @pIdMap
  local tData, tLine
  put shell ("nireport ." && pParams) into tData
  set the itemDelimiter to tab
  repeat for each line tLine in tData
    put item 2 of tLine into pIdMap[item 1 of tLine]
  end repeat
end p_LoadNireportIdMap

This seemed to do the trick on my iMac PowerPC G5 running MacOS X 10.4 Tiger - but when I tried it on my MacBook Pro Intel running MacOS X 10.6 Snow Leopard, it failed as there is no "nireport" command available. As it turns out, Apple dropped NetInfo when MacOS X 10.5 Leopard came out. So my solution is not quite complete, unfortunately.

To be continued...

BorderLayout example - with resizing

In a previous post, I examined how you can mimic Java's BorderLayout behavior in revTalk. At the end of that post, I promised to explain how you can add resizers to the West and East panels. Just as a reminder, here's what we had accomplished in the previous installment:



Before we get to the meat of the resizers, let's do some preparation work. To clearly show the user that he can drag around the resizers, we'll make a local copy of the vertical divide resize cursor that the RunRev team ships with the IDE. Open the message box and type:

copy image "vdividecursor.gif" of card id 1030 of stack "revMacCursors" to this card

We'll be using that as our cursor later, but it doesn't have to be visible for that to work - so you can hide it now. Next, we add a rectangular button, carefully placing it over the touching edges of the West and Center panels. Set the button's name to "WestResizer" and its width to 5, and make sure its height is the same as the West and Center panel. Here's what it should look like:



Before we add the script, we'll also add two custom properties to the West graphic:
-set the uMinWidth custom property to 75
-set the uMaxWidth custom property to 150

Now that we have the visuals worked out, let's script the resizer button:

local sIsResizing
local sMinX, sMaxX

on mouseEnter
   lock cursor
   set the cursor to the short id of image "vdividecursor.gif"
end mouseEnter

on mouseLeave
   unlock cursor
end mouseLeave

on mouseDown
   local tMouseOffset
   put item 1 of the mouseLoc - item 1 of the location of me \
       into tMouseOffset
   put the uMinWidth of graphic "West" + tMouseOffset into sMinX
   put the uMaxWidth of graphic "West" + tMouseOffset into sMaxX
   put true into sIsResizing
   pass mouseDown
end mouseDown

on mouseUp
   put false into sIsResizing
   pass mouseUp
end mouseUp

on mouseRelease
   put false into sIsResizing
   pass mouseRelease
end mouseRelease

on mouseMove x,y
   if sIsResizing is true then
      local tNewX
      put min(max(x, sMinX), sMaxX) into tNewX
      lock screen
      set the left of me to tNewX - 3
      set the widthFromLeft of graphic "West" to tNewX
      dispatch "resizeCenterPanel" to this card
      unlock screen
   end if
   pass mouseMove
end mouseMove

While it's not too long a script, some explanation is in order:
- the mouseEnter and mouseLeave messages are handled to update the cursor
- the mouseDown message is handled to setup a few script local variables:
-> the sIsResizing variable is used as a flag to track the status
-> the sMinX and sMaxX variables are calculated once, based on the offset between the mouse and the resizer button, and taking into account the minimum and maximum width we set earlier
-> calculating those once helps cut down on the math later on
- the mouseUp and mouseRelease messages are handled to turn off the sIsResizing flag
- the mouseMove message is handled to resize the West and Center panels
-> but only if the sIsResizing flag is true
-> it uses the sMinX and sMaxX variables to make sure we don't go too far either way
-> it updates the resizer button itself, the West panel, and the Center panel.

To make this work, we obviously also have to add a handler for our custom 'resizeCenterPanel' event. We'll add that to the card script in a bit, after we've finished the resizer for the East panel. So duplicate the WestResizer button, name the copy EastResizer and move it over to the right, placing it over the touching edges of the East and Center panels.

Before we add the script, we'll also add two custom properties to the East graphic:
-set the uMinWidth custom property to 100
-set the uMaxWidth custom property to 200

Next, update its script to:

local sIsResizing
local sMinX, sMaxX

on mouseEnter
   set the lockCursor to true
   set the cursor to the short id of image "vdividecursor.gif"
end mouseEnter

on mouseLeave
   set the lockCursor to false
end mouseLeave

on mouseDown
   local tMouseOffset
   put item 1 of the mouseLoc - item 1 of the location of me \
       into tMouseOffset
   put the width of this stack - \
       the uMinWidth of graphic "East" + \
       tMouseOffset into sMaxX
   put the width of this stack - \
       the uMaxWidth of graphic "East" + \
       tMouseOffset into sMinX
   put true into sIsResizing
   pass mouseDown
end mouseDown

on mouseUp
   put false into sIsResizing
   pass mouseUp
end mouseUp

on mouseRelease
   put false into sIsResizing
   pass mouseRelease
end mouseRelease

on mouseMove x,y
   if sIsResizing is true then
      local tNewX
      put min(max(x, sMinX), sMaxX) into tNewX
      lock screen
      set the left of me to tNewX - 3
      set the left of graphic "East" to tNewX
      set the widthFromLeft of graphic "East" to \
          the width of this stack - tNewX
      dispatch "resizeCenterPanel" to this card
      unlock screen
   end if
   pass mouseMove
end mouseMove

The difference with the WestResizer is that the EastResizer also has to take into account the width of the stack, but that's not too complicated. Finally we'll update the card script, adding a handler for the 'resizeCenterPanel' custom event, and we'll also take care of updating the resizer's location and height when the stack is resized. The new card script looks like this:

on resizeStack pNewWidth, pNewHeight
   local tCenterHeight, tCenterWidth
   --
   lock screen
   -- update the north and south panels
   set the widthFromLeft of graphic "North" to pNewWidth
   set the widthFromLeft of graphic "South" to pNewWidth
   set the bottom of graphic "South" to pNewHeight
   put the top of graphic "South" - \
       the bottom of graphic "North" \
       into tCenterHeight
   -- update the west and east panels
   set the heightFromTop of graphic "West" to tCenterHeight
   set the heightFromTop of graphic "East" to tCenterHeight
   set the right of graphic "East" to pNewWidth
   put the left of graphic "East" - \
       the right of graphic "West" \
       into tCenterWidth
   -- update the west and east resizers
   set the heightFromTop of button "WestResizer" to tCenterHeight
   set the heightFromTop of button "EastResizer" to tCenterHeight
   set the left of button "EastResizer" to \
       the left of graphic "East" - 3
   -- update the center panel
   set the widthFromLeft of graphic "Center" to tCenterWidth
   set the heightFromTop of graphic "Center" to tCenterHeight
   --
   unlock screen
   pass resizeStack
end resizeStack

--> custom event handlers

on resizeCenterPanel
   local tCenterWidth, tCenterHeight
   put the top of graphic "South" - \
       the bottom of graphic "North" \
       into tCenterHeight
   put the left of graphic "East" - \
       the right of graphic "West" \
       into tCenterWidth
   lock screen
   set the left of graphic "Center" to the right of graphic "West"
   set the top of graphic "Center" to the bottom of graphic "North"
   set the widthFromLeft of graphic "Center" to tCenterWidth
   set the heightFromTop of graphic "Center" to tCenterHeight
   unlock screen
   pass resizeCenterPanel
end resizeCenterPanel

--> helper property setters

setProp widthFromLeft pNewWidth
   local tRectangle
   --
   lock screen
   put the rectangle of the target into tRectangle
   put item 1 of tRectangle + pNewWidth into item 3 of tRectangle
   set the rectangle of the target to tRectangle
   unlock screen
end widthFromLeft

setProp heightFromTop pNewHeight
   local tRectangle
   --
   lock screen
   put the rectangle of the target into tRectangle
   put item 2 of tRectangle + pNewHeight into item 4 of tRectangle
   set the rectangle of the target to tRectangle
   unlock screen
end heightFromTop

And there we have it: a BorderLayout with resizers for the West and East panels.



Once you're happy with the way the resizers work, you can change the style of the WestResizer and EastResizer button to transparent. They'll continue to work just fine, but won't be visible. With a little more work, you can also add resizers for the North and South panels. But I'll leave that as an exercise to the reader...

BorderLayout example script

As you may know, I spend quite a bit of my time in Java, not merely building back-end systems with message queues and raw socket communication, but also creating user interfaces in Swing. One of the most compelling features of Swing is its system of LayoutManagers - and certainly the easiest-to-use is the BorderLayout. Today, I'm going to explain how you can mimic this type of layout handling in revTalk.

The principle of the BorderLayout is quite simple: it divides the window into 5 panels: North, East, South, West and Center. As a picture says more than a thousand words, here's what this layout generally looks like:



The North panel is a good place to store a toolbar, the South panel usually plays host for a status area, while the West panel may show an outline, and the East panel a property grid, leaving the Center for the actual content.

For this example, I chose the easy route, creating a single rectangle graphic for each panel. I also picked distinct colors so you can more easily see the effect of resizing the window once the pieces of the puzzle fall into place. And it turns out it isn't that much work to implement, as I set the card script to:

on resizeStack pNewWidth, pNewHeight
   local tCenterHeight, tCenterWidth
   --
   lock screen
   -- update the north and south panels
   set the widthFromLeft of graphic "North" to pNewWidth
   set the widthFromLeft of graphic "South" to pNewWidth
   set the bottom of graphic "South" to pNewHeight
   put the top of graphic "South" - \
       the bottom of graphic "North" \
       into tCenterHeight
   -- update the west and east panels
   set the heightFromTop of graphic "West" to \
       tCenterHeight
   set the heightFromTop of graphic "East" to \
       tCenterHeight
   set the right of graphic "East" to pNewWidth
   put the left of graphic "East" - \
       the right of graphic "West" \
       into tCenterWidth
   -- update the center panel
   set the widthFromLeft of graphic "Center" to \
       tCenterWidth
   set the heightFromTop of graphic "Center" to \
       tCenterHeight
   --
   unlock screen
   pass resizeStack
end resizeStack

--> helper property setters

setProp widthFromLeft pNewWidth
   local tRectangle
   --
   lock screen
   put the rectangle of the target into tRectangle
   put item 1 of tRectangle + pNewWidth \
       into item 3 of tRectangle
   set the rectangle of the target to tRectangle
   unlock screen
end widthFromLeft

setProp heightFromTop pNewHeight
   local tRectangle
   --
   lock screen
   put the rectangle of the target into tRectangle
   put item 2 of tRectangle + pNewHeight \
       into item 4 of tRectangle
   set the rectangle of the target to tRectangle
   unlock screen
end heightFromTop

Let's see what is happening here. The script handles the resizeStack event, which has 4 parameters (new width, new height, old width, old height) of which we only use the first two.

As the North and South panels use up the entire width, I can simply set their width to the new width of the stack. However, setting the width of a control in revTalk works from the location of the control (which is the center, not the topleft).
So I use a fake helper property setter 'widthFromLeft' to update the rectangle of the control instead. Finally, we move the South panel to the bottom edge of the window, and calculate the distance between the top of the south panel and the bottom of the north panel, as this is the new height for our West, East and Center panels.

Armed with this information, I follow a similar path for the West and East panels, updating their height using a fake helper property setter 'heightFromTop', and moving the East panel to the right edge of the window, calculating the distance between the east and west panels to determine the new width of the Center panel. And now it is dead-easy to update the Center panel with the calculated width and height.

That's all there is to it! Admittedly, this example merely resizes 5 graphics, but you can extend this logic to groups, handling the resizeControl message in your group scripts to automatically update the layout of the group's contents.

Here's what it looks like when stretched horizontally.



And here's what it looks like when stretched vertically.



Straightforward to implement with very little code. Next time, I'll add some resizers for the West and East panels.

DataGrid Filter example - with sorting

The other day I posted an example of how you can add a filter to a DataGrid in Revolution. At the end of that blog post, I remarked how I had deliberately turned off column sorting. Fear not, it is straightforward to add support for this, again thanks to the DataGrid lessons available on the runrev.com website.

Just as a reminder, here's a screenshot of the example stack:



So what do we need to do to allow the user to click on the column header and apply the sort and filter at the same time? It's quite a simple bity of scripting, but before we get to that, you need to make some minor modifications to the datagrid columns.
- Switch to the 'pointer' tool, and select the datagrid, then open the Object Inspector, and switch to the 'Columns' panel
-> select the column 'index' and set its Sort by column type to 'Numeric'
-> select the column 'name' and make sure its Sort by column type is 'Text'

Now edit the script of the datagrid, and set it to:

local sOriginalData
local sIndexMap
local sFilter

setProp dgData pData
  put pData into sOriginalData
  RebuildMap
end dgData

getProp dgData
  return sOriginalData
end dgData

setProp dgFilter pFilter
  put pFilter into sFilter
  RebuildMap
end dgFilter

getProp dgFilter
  return sFilter
end dgFilter

--> datagrid callbacks

on GetDataForLine pLine, @pData
  local tKey
  -- retrieve the original line of data
  put sIndexMap[pLine] into tKey
  put sOriginalData[tKey] into pData
end GetDataForLine

on SortDataGridColumn pColumn
  SortMap pColumn
  HiliteAndStoreSortByColumn pColumn
  dispatch "RefreshList" to me
end SortDataGridColumn

--> private commands and functions

private command RebuildMap
  local tKeys, tKey, tIndex
  put empty into sIndexMap
  put 0 into tIndex
  put the keys of sOriginalData into tKeys
  sort tKeys numeric
  repeat for each line tKey in tKeys
    if sFilter is empty \
    or sOriginalData[tKey]["name"] contains sFilter \
    then
      add 1 to tIndex
      put tKey into sIndexMap[tIndex]
    end if
  end repeat
  set the dgNumberOfRecords of me to tIndex
  SortMap
  dispatch "ResetList" to me
end RebuildMap

private command SortMap pColumn
  local tSortDirection, tSortType, tSortCaseSensitive
  local tSortCommand
  local tKeys, tKey, tNewIndex, tNewIndexMap
  -- works whether a column is specified or not
  if pColumn is empty then
    -- find the current sort column
    put the dgProps["sort by column"] of me into pColumn
    if pColumn is empty then
      exit SortMap
    end if
  end if
  -- assemble and do the sort command
  put the dgColumnSortDirection[pColumn] of me \
    into tSortDirection
  put the dgColumnSortType[pColumn] of me \
    into tSortType
  put the dgColumnSortCaseSensitive[pColumn] of me \
    into tSortCaseSensitive
  put "sort lines of tKeys" && tSortDirection \
    into tSortCommand
  switch tSortType
    case "numeric"
      put " numeric" after tSortCommand
      break
    case "international"
      put " international" after tSortCommand
      break
    case "datetime"
    case "system datetime"'
      put " dateTime" after tSortCommand
      break
  end switch
  put " by ColumnValueForLine(pColumn, each)" \
    after tSortCommand
  set the caseSensitive to \
    (tSortCaseSensitive is true)
  set the useSystemDate to \
    (tSortType is "system datetime")
  put the keys of sIndexMap into tKeys
  do tSortCommand
  -- rebuild the index map
  put 0 into tNewIndex
  repeat for each line tKey in tKeys
    add 1 to tNewIndex
    put sIndexMap[tKey] into \
      tNewIndexMap[tNewIndex]
  end repeat
  put tNewIndexMap into sIndexMap
end SortMap

private function ColumnValueForLine pColumn, pLine
  local tKey
  -- retrieve the original line of data
  put sIndexMap[pLine] into tKey
  return sOriginalData[tKey][pColumn]
end ColumnValueForLine

What's changed in comparison to the last implementation? Well, the bulk of the necessary work is done in the new private command SortMap, which reads the column sort properties and uses a custom sort function to get the filtered lines in the right order on the basis of the column value in the original lines of data. Then it builds a new IndexMap to replace the old one, and the regular refresh routines of the datagrid will "just work."

So what happens when we sort the data descending on the name, without applying a filter?



Looking good - does the filter still work? Let's type 'url' into the filter field.



You may have noticed that I also added getProp-handlers for dgData and dgFilter, making the above script easily reusable. With just a bit of lateral thinking and coding, you too can build just about anything, thanks to Revolution.

DataGrid Filter example

One of the powerful additions to Revolution 3.5 is the DataGrid - a custom control group that allows you to display any type of data with astonishing flexibility. Recently, I had to add filtering capabilities to a data grid for one of my projects. In this post, I will explain how I accomplished it, thanks to the DataGrid lessons available on the runrev.com website.

Here's a screenshot of the example stack:



Execute the following steps, if you want to follow along:
- Create a new stack
- Drag a button to the topleft of the card and set its name to "Fill"
- Drag a text field to the topright of the card
- Drag a label field next to the text field and set its content to "Filter:"
- Drag a data grid onto the card and configure it using the Object Inspector
-> In the 'Basic Properties' panel, turn off the checkbox "Allow Text Editing"
-> Switch to the 'Columns' panel, and add 2 columns "index" and "name"
-> Edit the script of the DataGrid, and set it to the following:

local sOriginalData
local sIndexMap
local sFilter

setProp dgData pData
  put pData into sOriginalData
  RebuildMap
end dgData

setProp dgFilter pFilter
  put pFilter into sFilter
  RebuildMap
end dgFilter

private command RebuildMap
  local tKeys, tKey, tIndex
  put 0 into tIndex
  put the keys of sOriginalData into tKeys
  sort tKeys numeric
  repeat for each line tKey in tKeys
    if sFilter is empty \
    or sFilter is in sOriginalData[tKey]["name"] \
    then
      add 1 to tIndex
      put tKey into sIndexMap[tIndex]
    end if
  end repeat
  set the dgNumberOfRecords of me to tIndex
  dispatch "ResetList" to me
end RebuildMap

on GetDataForLine pLine, @pData
  local tKey
  put sIndexMap[pLine] into tKey
  put sOriginalData[tKey] into pData
end GetDataForLine

on SortDataGridColumn pColumn
  -- prevent column sorting
end SortDataGridColumn

The above data grid script will take care of filtering the data. Now it's just a matter of wiring up the text field.
-> Edit the script of the text field and set it to the following:

local sUpdateMsg

on keyUp pKey
  ScheduleUpdate
  pass keyUp
end keyUp

on deleteKey
  ScheduleUpdate
  pass deleteKey
end deleteKey

on backspaceKey
  ScheduleUpdate
  pass backspaceKey
end backspaceKey

on pasteKey
  ScheduleUpdate
  pass pasteKey
end pasteKey

on cutKey
  ScheduleUpdate
  pass cutKey
end cutKey

command ScheduleUpdate
  if sUpdateMsg is empty then
    send "UpdateFilter" to me in 100 milliseconds
    put the result into sUpdateMsg
  end if
end ScheduleUpdate

on UpdateFilter
  set the dgFilter of group "DataGrid 1" \
    to the text of me
  put empty into sUpdateMsg
end UpdateFilter

By trapping the different key events, we can update the data grid filter as we go along. Note that I used the ScheduleUpdate helper command to try and minimize the number of refreshes for quick typers. Play around with the interval until you find the sweet spot of responsive user interface.

Finally, it is time to fill the data gird with some data for testing, so edit the script of the button and set it to the following:

on mouseUp
  local tData, tIndex, tName
  put 0 into tIndex
  repeat for each line tName in the functionNames
    add 1 to tIndex
    put tIndex into tData[tIndex]["index"]
    put tName into tData[tIndex]["name"]
  end repeat
  set the dgData of group "DataGrid 1" to tData
end mouseUp

This simple script will fill our data grid with a list of the built-in functions that revTalk has to offer. Click the button, and type away in the filter text field to see the effect. Here's what happens when I type "cos":


Looks pretty good, doesn't it? It certainly was good enough for my use, but there are a few things to bear in mind about this implementation:
- I didn't have to edit the data in the grid.
- I didn't have to support sorting for my particular project.
Both of these limitations can be overcome, but I'll leave that for another time...

Forums moved over to quartam.on-rev.com

Hi All,

After much deliberation, I have decided to move the forums over from ning.com to my new on-rev based website: http://quartam.on-rev.com/forums/index.php.
And here's an even shorter link: http://forums.quartam.com.

All posts were imported as plain text, and the topics were locked.
Unfortunately, you will have to re-register as a user on the new forum.

I apologize for any inconvenience, caused by this switch.

Best regards,

Jan Schenkel
--
Quartam Reports & PDF Library for Revolution
www.quartam.com

But the waiting makes me curious

Be patient, is very good advice, but the waiting makes me curious.
It's a line from the song 'Very Good advice' written for the original Disney 'Alice in Wonderland' movie and recently remade by The Cure's Robert Smith for the 'Almost Alice' soundtrack. And I just couldn't think of a better introduction to this blog entry on the revTalk 'wait' command.

When your script is in a tight loop, screen updates my not happen until after the whole script is finished.

on mouseUp
  repeat with i = 1 to 5000
    put i into field "Output"
  end repeat
end mouseUp

Rather than continuously updating the content of the field, the field may only get updated after the repeat loop. While that's good enough in this silly example, it's not suitable for keeping the user updated on the progress of a real-world data crunching routine.

You can give the engine some breathing room by inserting a call to the 'wait' command.

on mouseUp
  repeat with i = 1 to 5000
    put i into field "Output"
    -- allow screen to redraw
    wait 0 milliseconds
  end repeat
end mouseUp

Now you'll see the content of the field refreshed regularly; but it also makes your loop much slower.

on mouseUp
  constant kRefresh = 25
  repeat with i = 1 to 5000
    put i into field "Output"
    if i mod kRefresh is 0 then
      -- allow screen to redraw
      wait 0 milliseconds
    end if
  end repeat
end mouseUp

By only redrawing the screen every 25th iteration, we can still refresh the screen with less impact on performance. Play with the 'kRefresh' constant value to find the right redraw/performance ratio.

But that's not all the 'wait' command can do: if used properly, you can give your users a way to 'cancel' out of a tight loop.

on mouseUp
  constant kRefresh = 25
  enable button "Cancel"
  set the uCancel of me to false
  repeat with i = 1 to 5000
    put i into field "Output"
    if i mod kRefresh is 0 then
      -- allow screen to redraw and user to cancel
      wait 0 milliseconds with messages 
      if the uCancel of me is true then
        answer "Are you sure you want to cancel?" \
            with "Yes" or "No"
        if it is "Yes"
        then exit repeat
        else set the uCancel of me to false
      end if
    end if
  end repeat
  disable button "Cancel"
end mouseUp

Now we add a button "Cancel" and give it the following script:

on mouseUp
  set the uCancel of button "TightLoop" to true
end mouseUp

And presto! Not only does the screen redraw, but we now allow the user to stop the time-consuming process in our tight loop. Of course you'll have to add your logic to 'revert' any changes you've made in the process, but the above provides you with a skeleton script for handling this sort of situation.

As with all good things in life, 'wait with messages' comes at a price: as the user can now click your button twice, or trigger other scripts, it is very important to disable those parts of your user interface that shouldn't be executed 'concurrently' - my favourite method is to switch to a 'limited' environment, disabling all controls, displaying a blended white overlay over them as well as the 'Cancel' button making it the only enabled control. After the loop, the stack returns to the 'unlimited' environment, hiding the overlay and enabling the controls.

Some of you may wonder what happens when a script issues 'wait 1 second with messages' and the user clicks on another button: the other button's script is executed first, and then the engine checks if the wait timeout has passed, and resumes the original script. This means that if your other button script takes 5 seconds to complete, your first button script will remain suspended - the revEngine is not multi-threaded. But with a little planning, you can provide a much better user experience.

In memoriam: Bill Marriott

As usual after the Holidays, I had some catching up to do with emails on the use-revolution mailing list. It was quite a shock when I read the news that William Marriott, marketing director at RunRev, had passed away.

To the rev developer community, he was the guy who worked his way up from the trenches, growing from a vocal advocate of quality, to the man who held surveys to figure out who we were and what we wanted, and helped in so many ways to reshape the image of the RunRev company. For the revSelect third-party add-on developers, he was the guy who not only gracefully hosted the webinars, but was in general someone you could easily work with to get results (a character trait shared at RunRev HQ, I might add).

At the RunRevLive '08 conference in Las Vegas, I had the pleasure of taking up an afternoon of his time to talk about how I could improve my existing products, and exchange thoughts on an assortment of ideas. His combined marketing and programming skills, plus his sense of humour, make him a hard act to follow.

My condolences and thoughts are with his family. May he rest in peace, or find a way to smuggle his laptop into heaven.