 Bob Balaban | Bob Balaban May 16 2008 12:20:20 PMGreetings, Geeks! Just wanted to let you know that I will be presenting a session (and attending many others) at the Irish Lotus User Group (ILUG) get-together in Dublin next month (June 4-6). My session title and abstract are: Achieving "Peaceful" Coexistence Between Microsoft Exchange and Lotus Notes While many organizations strive to standardize their messaging infrastructures on a single platform, many others do not. Whether through merger, acquisition or simple user preference, many organizations face a need to achieve peaceful coexistence in the messaging wars, perhaps for only a short time (while they transition from one to the other), but perhaps (if they can pull it off) for a very long time. The technical challenges involved in maintaining a well-performing dual messaging infrastructure are not trivial, but they can be met successfully with proper planning and the right technology. The big issues are all around directory synchronization, message transport, rich text, scheduling logic differences and data formats. Come to this session to learn how to overcome these technical challenges how to keep both Notes and Exchange up and running, and how to avoid mutually assured destruction! Hope to see you there/then! Bob Balaban May 13 2008 07:13:39 AMGreetings, Geeks! I'm hoping that many of you out there are fans (or recovering fans) of The Simpsons, in which case you'll have no trouble recognizing the names in this week's post as the characters in the always-fighting cat and mouse TV show beloved by all the children of fictional Springfield. If you're closer to my age (Boomer) than to Gen-X, and don't have any kids that you watch TV with, the equivalent characters of our time were probably Ben and Jerry (the cat and mouse cartoons, NOT the ice cream makers!) So, the thing I want to get at thoday is, why can't we all just get along? Nah, that's not it. Let me try again: Why is it so darn hard to release quality software? Nope, that's not really it either (but it's a good question). Ok, I think I have it now: If you were working in a relatively small software-making company (let's say, between 20 and 200 employees, and by "software maker" I mean, creates and sells software as a primary business), and you had to figure out how to organize the people doing the actual software creation and (we hope) testing, how would you set that up? Of course we need to constrain the question a little bit more, so as to be answerable in our lifetimes. Here are the (arbitrarily ranked) goals I would want to maximize in such a situation:
Minimize the nuber of defects (bugs, glitches, doc errors, user errors whatever) reported by customers (defined as, the people you pay you for your software). Notice that I lump "user errors" in with the more common kinds of problems. I do that because, IMHO, a "user error" (the user did something "wrong", resulting in a bad outcome, but the software is "working as designed") is rather more likely to result from a problem with the User Interface (UI) or perhaps with the documentation, leading the user to expect a result other the one s/he got when they did whatever it was. We could argue whether these are "real" bugs, or "equal" in some way to "real" software problems, but I don't care. By my definition, this kind of error is a quality problem in the product.
Maximize the amount of automated testing a product can undergo before release. I put this item in here because experience (and arithmetic) has shown me that release cycles are dramatically reduced when you apply automated testing to a product. I won't belabor the point, and yes, it's true that you can't generally achieve 100% automation (at reasonable cost), but test automation is a good thing.
Keep as many employees happy in their jobs as possible. Yeah, yeah, I know. There are a LOT of factors that affect employee satisfaction, but here's one thing I have noticed myself, and at more than one company where I have worked: implementing caste systems leads to unhappiness among employees. Creating sub-divisions within the broader "development organization" where one group of people (let's call them the "Lords") is responsible for the fun, creative, and more highly paid work of writing (inventing, architecting, designing, coding...) software, and another group (the "Serfs") are responsible for taking the work product of the Lords and testing it (looking for defects, broadly defined, which might also include things like unacceptable performance, poor UI, etc.). Most companies at which I have been an employee implement (whether blindly, or on purpose) this kind of 2-tiered system. The Lords (developers) get more money, and more status than the Serfs (QA/QE *, pick your terminology). Some people feel that this system is "natural", after all, the developer's job is harder and more creative. The "QE" role is to receive and test, a rather more "mundane", yet unfortunately necessary step in the release cycle. Anyhow, the point here is, your job (as the hypothetical keeper/maker of the org chart in this example) is to try to not have a caste system.
Minimize the cost of producing quality software. Since we're talking about the software business, there has to be a business constraint on all of this. Headcount is expensive. Back in the days when Lotus was mostly a spreadsheet company, the normal ratio of QA to Developer headcount on any major project was 3:1 or so. For every developer creating code, there were 3 people testing it. Today that would be inconceivable. So? Now what? We can probably all agree that there's a difficult problem here. What's the answer? Speaking for myself, I don't have an "answer". There probably is no one single "answer". What I have, though, is a hypothesis. Which is: [hypothesis] If you view the problem of "software product quality" from a broad perspective (as I described it above), then your approach to building the software cannot treat "QA" (or "QE", or whatever you call it) as a distinct operation from "development". ** [/hypothesis] One of the things this implies for software companies is that "Development" should probably not be a separate organization, or department from "Testing" (QA, etc.). I would almost advocate that within a "Software Development" organization (let's say, a team dedicated to creating and evolving a single product, or suite of products), there is indeed reason to declare and nurture a "specialization" of skill that is distinct (i.e., not all architects/designers/coders need to be expert in testing, and not all testers need to be expert in software design/implementation), but really, isn't there an awful lot of overlap? Don't testers benefit from understanding the product's construction? Don't developers benefit from an understanding of a testing process, so that they can (as one possible example) instrument their code appropriately? For sure, once you get to thinking about test automation, someonehas to build the automation harnesses, script the test runs, etc. Right? Is that not "development"? So, is there a benefit to organizing around a "QD" (Quality Development?) job function, and organize a primarily "development" team where perhaps there is some internal specialization (but not necessarily a clear distinction) between creating software and making sure it works properly? What do YOU think? How would YOU organize for quality, if you could? ------------------------------------------------------------------------------------------------- (Footnotes)
* In fact, when I first started working at Lotus Development in the late '80s, the "testing" department was called "QA" (Quality Assurance). Around the time that we shipped Notes V4 (mid-'90s, before IBM bought Lotus), the Lotus QA organization working on Notes was merged with the Iris organization (developers of Notes), which had been composed almost entirely of developers. Somewhere in there, the term "QA" was explicitly switched to "QE", or "Quality Engineering". When I asked why, I was told that people felt it was a better name, because it more accurately reflected the engineering basis of the task, and that it might improve the level of respect given to the job title and to the people. To this day, the terminology (at the Lotus division of IBM, anyway) remains "QE", though I, personally, haven't detected any significant reduction in the caste differential (though of course these things are always in the eye of the beholder, YMMV).
**Again, the underlying assumption here is that you're trying to optimize for quality. Naturally, many, many companies (businesses) do not inherently optimize for quality, they try to optimize for profits, or maybe for market share, or maybe for personal career growth, or maybe for something else. I'm not primarily a business person (maybe if I were, I would still be running my own company!), so I can't say whether quality is the thing for which a business should globally optimize, and certainly, from an employee perspective, working on a "quality" product doesn't make me happy if I can't get paid for it. But it does seem to me that quality should be an important objective of product development, a "differentiator", if you like. :-) But that's another discussion. Bob Balaban May 5 2008 12:11:47 PMGreetings, Geeks! How many of you have ever written LotusScript code that needed to call C entry points in some DLL (such as, maybe, the Notes C API, in nnotes.dll)? I've done it, many of you probably have too. Well, I spent a few days recently trying to figure out how to do the same thing, but from a C# ("c-sharp") program running as "managed code" in .NET. I'm using Visual Studio 2005 as my IDE. In the end I cracked it, and it wasn't too difficult. The real secret was to find the magic incantation equivalent to "DECLARE" in LotusScript. Background: I inherited some C# code that uses the Notes COM interfaces to do some stuff with user mail files. The COM interfaces are pretty much the same as the LotusScript back-end classes, they work great in a .NET environment for accessing NSF data. As an enhancement to the existing code (which loops over all documents in a user mail NSF and does things to them), I wanted to find out which documents were unread for the mail file's owner. This involved two extra steps that I needed to research: - Figure out the name of the mailbox owner. NotesPeek came to my rescue (as it has so many times before), and I found what I needed. It's a Profile document called "calendarprofile" (just use NotesDatabase.GetProfileDOcument("calendarprofile", "") ), and the name is on an item named "Owner". So far, so good. It would be a good idea to make sure that the name you get is in "distinguished name" format (i.e., cn=xxx/o=yyy...)
- Figure out if the curernt document is unread for the name we found in step 1. Unfortunately, there is no back-end class support for this functionality, you have to drop down to the C API. I knew how to do this in LotusScript, but not from .NET. Fortunately, some Google research found me a couple of articles that were very helpful (links at the end of this post). Knowing what C API calls to use, I was able to write a C# class to tell me whether a NOTEID is in the unread list or not.
It turns out that the magic incantation in C# is easy to do, once you know it. You have to tell CLR ("common language runtime") where the DLL is that you want to access, and what the signature (call name and parameter types) of the entry point you need is. Just like Declare statements in LotusScript (with different syntax, of course). Here's an extract from my code showing the Notes entry points I needed: /* * STATUS LNPUBLIC NSFDbOpen( const char far *PathName, DBHANDLE far *rethDB); */ [DllImport("nnotes.dll")] public static extern STATUS NSFDbOpen(String path, ref HANDLE phDB); /* * STATUS LNPUBLIC NSFDbClose(DBHANDLE hDB); */ [DllImport("nnotes.dll")] public static extern STATUS NSFDbClose(HANDLE hDb); /* * STATUS LNPUBLIC NSFDbGetUnreadNoteTable2( DBHANDLE hDB, char far *UserName, WORD UserNameLength, BOOL fCreateIfNotAvailable, BOOL fUpdateUnread, HANDLE far *rethUnreadList); */ [DllImport("nnotes.dll")] public static extern STATUS NSFDbGetUnreadNoteTable2( HANDLE hDb, String user, ushort namelen, bool create, bool update, ref HANDLE hList); /* * STATUS LNPUBLIC NSFDbUpdateUnread( DBHANDLE hDataDB, HANDLE hUnreadList); */ [DllImport("nnotes.dll")] public static extern STATUS NSFDbUpdateUnread(HANDLE hDb, HANDLE hUnreadList); /* * BOOL LNPUBLIC IDIsPresent(HANDLE hTable, DWORD id); */ [DllImport("nnotes.dll")] public static extern bool IDIsPresent(HANDLE hTable, DWORD id); /* * STATUS LNPUBLIC OSMemFree(HANDLE Handle); */ [DllImport("nnotes.dll")] public static extern STATUS OSMemFree(HANDLE h); Note that every call is preceded by " [DllImport("nnotes.dll")]", that's a requirement (and the DLL must be on the system PATH). There's no "#include" directive in C# to bring in header files, so you have to map all the special Notes datatypes (like DBHANDLE, NOTEID, etc.) to native C# types. I did this with "using" statements, like this: using HANDLE = System.UInt32; using DWORD = System.UInt32; using STATUS = System.UInt16; After doing this, I just called the routines as if they were native C# code. CLR took care of mapping string buffers to "unmanaged" memory and so on. It worked great! Since this post is already too long, I won't post the class I wrote to use these CAPI calls, but it's very simple. There's an Init() routine that takes a server, db and username, and opens the database. With the username, it gets the IDTable representing the user's unread list (and updates it). Then, every time I get a new Document, I convert it's NOTEID string to a number, and see if that ID is in the unread list. On shutdown I free the IDList and close the Database (actually, I could almost certainly close the database after retrieving the IDList....) Not bad! The articles I read said there was a way to invoke all kinds of entry points: using structs, passing callback functions, everything (someday I'm sure I'll need to do those things too, just didn't have to this time). Here are the 2 links I promised: http://msdn.microsoft.com/en-us/magazine/cc301501.aspx http://msdn.microsoft.com/en-us/library/ms123402.aspx (I swear this one was there on Saturday, but today when I went to verify it, I got "content not found". Oh well, keep trying...) Enjoy! Bob Balaban April 22 2008 06:55:00 AMGreetings, Geeks! Today I wish to talk about a favorite topic of mine, Notes-style Replication. We all know about it and love it, naturally. It's one of the great foundational pillars of Notes and Domino, it's value to the product and to the applications built on the platform cannot be under-estimated. BUT (I claim) it is time to start thinking about the need for an enhanced replication facility for Notes and especially for Domino. The need (I claim) for a couple of new kinds of Notes/NSF replication has been around for a while, as I'll explain. However, I think the need will accelerate in the next 1 or 2 releases of Notes/Domino, if people (Developers, mainly) start really embracing some of the new features that have appeared in N/D v8.0, and which are likely to appear in v8.5. What "new kinds" of replication am I talking about? Two, mainly: 1) Multi-NSF "packages" 2) Non-NSF files in the N/D data tree Multi-NSF applications have been around for years. The very first consulting project I worked on after leaving Iris Associates in 1997 involved distributing data across as many as 7 Notes databases per dataset. The reason, of course, was that (at the time, in V4.6) the maximum allowed size of a single NSF was something like 2GB (it might have been 4GB, I can't remember specifically). And databases larger than 1GB led to pitifully poor performance. Aside from the difficulty of coding and app set up like that, the fact that a given installation required several instances of an NSF to replicate all at once meant severe headaches, both at application install time (setting up multiple replication instances across a few servers, and ensuring that all the app NSFs replicated at the same time), but it was a headache for the admins, too. They had to know either not to touch ANY of the NSFs in a given group, or to apply certain kinds of changes to ALL of them, at once. The other type of "replication" that I mention above is also not terribly new. Ever since Domino became an HTTP platform (1996 or so, with V4.5), developers have been storing pieces of their app as HTML files in the "data tree" (domino\data\domino\html, and associated folders) on disk, instead of "inside" NSF files. My extensive research shows (Ok, I called a couple of people...) that there were (are) two reasons for this: 1) Simplicity of referencing with HTML links. Domino-generated URLs were then, and are now, rather complex to deal with. But if you put an HTML file in a certain place on the server disk, you always know how to reference it. 2) Performance. I am told by a reliable source (they guy at Lotus who actually maintains the HTTP server code) that it's 2 or 3 times faster to serve a file off of the server's disk than it is to serve the same file embedded somewhere in an NSF (on a Page, perhaps, or as a file resource). This is especially true because in more recent versions of Domino, the amount of caching that the HTTP server tries to do on disk has been scaled way back. Ok, so you have a Domino Web application that relies on specific files on disk. Replicating the NSF (the major piece of the app) to another server is no big deal, but how do you install and/or keep non-NSF files synchronized? There are ways (run a scheduled agent that can access a mapped drive to another server and copy files, buy a third party utility that does it, write an agent to detect modified disk files, attach them to an NSF, replicate, then have another agent detach newer files to disk, etc. None of these solutions is particularly satisfying, especially because none of them (that I am aware) handles merges and conflicts very well. So I think we'd all pretty much agree that this issue is not a new one. So why bring it up now? I'm mentioning it now because I see the need for a solution becoming more important, not less. There are two sets of enhancements (one released in v8.0, one coming in v8.5) that will potentially cause this problem to become even worse, in the sense that more people will require it. The two new features are: 1) Composite Applications (released for Domino in v8.0) 2) Dojo JavaScript libraries shipped with the Domino Server (scheduled for v8.5, this year). The pressures on multi-file replication are clear: CompApps involving Notes-based components pretty much require you to use multiple NSFs. The "Application" database itself has almost nothing in it, except some attached XML files that define the CompApp. There will be references in there to 1 (or possibly several) additional NSFs containing data and/or logic components that make up parts of the overall application. At application launch time, a Notes user double-clicks on the icon representing "the composite application" (the mostly empty one), and the right thing happens in the Notes Client - the multi-pane window opens up and the right components appear in the right places. BUT, it only works if ALL the referenced NSFs are where they should be. I've had many experiences of trying to open a CompApp that was replicated to a server that was not the one used to originally install the app. If one database comprising part of the overall app is not replicated, or if it is replicated to the new server, but to a location different from its originally referenced one, stuff is broken. Furthermore, go tell a Domino server administrator (even an experienced one) that you have a Composite Application on ServerA that needs to be moved (or copied) to ServerB, with replication enabled between the two instances. Go ahead, I dare ya! This exposes another, related problem with multi-NSF applications -- you can't easily figure out from looking at the "main" NSF in a Composite Application what OTHER NSFs are required to make the app work, unless you're willing to go read some XML files or poke around in Domino Designer, looking for database references. Yet, the whole group of them MUST be replicated as a unit, and must follow the original folder layout, neither of which is generally a requirement for "traditional" Notes replication. The second new featureset, scheduled to appear in v8.5, is a terrific enhancement for developers working on Domino-based Web application development. Shipping a set of Dojo JavaScript libraries with the product (and exposing easy ways of hooking those libraries into your app via Domino Designer) is a huge step forward for Web apps on Domino. Dojo is a popular, browser based set of JavaScript code, distributed as open source. It provides some very nice features to the Web application developer, among them: browser isolation; sophisticated AJAX support; accessibility features; a full complement of UI widgets, from which a JavaScript developer can inherit and customize new widgets, and so on. Now, for performance and other reasons (as mentioned above), the plan (which could, of course, change at any time) is to ship the Dojo libraries as a set of .JS files on disk. This will work great if your Dojo-enabled Web apps all run on newly-installed servers, all of the same release of Domino. But if not, then you may have a problem. Let's say (hypothetically) that Dojo version 1.2 ships with all Domino 8.5 servers, on all OS platforms. Then let's say that Domino 8.5.1 is released with Dojo v1.3, which has some nice bug fixes and/or new features in it. Now let's assume that I've deployed an 8.5 Web App using Dojo 1.2 on one of my new 8.5 servers. If I have a second server that gets 8.5.1 and I want to now deploy my app there, I could have a problem. I now need to pick not only which version of the server I should QE my app for, I have two different versions of Dojo to deal with, too. Do I pick one or the other? If so, how do I get the 8.5.1 version of Dojo (hypothetically v1.3) over to my 8.5 server (which normally only has Dojo v1.2 on it)? PLUS, if I've deployed pieces of my JavaScript based client as .JS files to disk, how do I synchronize those between the two servers (or between any 2 servers, even if they are running identical versions of Domino and Dojo)? My point is that, although I think Dojo is a terrific enhancement for Domino, using it will increase the level of deployment complexity and synchronization complexity for both developers and admins. An enhancement to the way Domino does replication to include non-NSF files as well as to be able to treat a collection of related NSFs as a unit would be very, very helpful (Note: Of course I am not naive enough to expect that non-NSF file replication could have all the features of NSF-to-NSF replication, but that's a topic for another post...) Bob Balaban April 8 2008 06:50:02 PMGreetings, Geeks! Naturally, I missed my own anniversary -- the first posting I made on this blog was March 30, 2007, just over a year ago. As my dad might say: "Hoo hah!" (if you're reading this and you're my father, Hi Dad!). Pretty cool, if I say so myself, I really wasn't sure when I started whether I'd be able to keep it going this long. As it turned out, it wasn't as hard to come up with topics as I thought it would be. Here are some stats: 41 -- Approximate number of main postings in the blog (yes, I was too lazy to count...) 1100 -- Approximate number of comments posted 2 -- Number of guest blog postings (1 by Mark Vincenzes, one by Steve Nikopoulos) between 3000 and 9500 -- the range of the number of page hits on the blog per month 166 - largest number of comments on any one post ( this one, I wonder how many of these suggestions will make it into 8.5?) a LOT - the amount of fun I've had doing this little project Thanks to you, fellow Geeks, for making it worthwhile to keep this thing going. Keep posting! Remember: "In theory" means "not really"!  Bob Balaban April 4 2008 06:23:05 AM  Greetings, Geeks! Today our topic is....recursion. No, fellow geeks, this does NOT mean that we are going to practice cursing, and cursing again.... The thing is, recursion (loosely defined as the technique whereby a subroutine processes a piece of a data structure, and then invokes itself ("recursively") to process another piece. The classic example that we all learn in Computer Science 101 class is... "walking a tree". Note that this has very little, if anything to do with the technique we learn in real life, called "walking the dog" (and YES, I KNOW that that is also a yo-yo maneuver....sheesh). There's even a pretty good example of a recursive tree-walker routine, written in LotusScript, in the Designer Help file (search or index to "Domparser class", go to that article, and follow the "Example" link). In this case, the code is reading in an XML file, parsing it into a tree using the standard XML DOM parser, and then travelling the tree to visit every node. Because the algorithm is recursive, it does a "depth-first" tree walk, meaning that it visits the first child of the top-level node, then the first child of that node, then the first child of that node, etc., until it reaches a node that has no children. Along the way, the "welktree" subroutine calls itself to "process" the next child node. In this way, it stores the "context" at each layer of the tree on the LS invocation stack. When it hits a node with no children, it "processes" that node, then backs up a layer, and continues to the next child of tht layer's parent node. Yes, it sounds confusing, but if you step through the code in the debugger, you'll get it pretty quickly. One of our developoers actually copied this code and used it in developing an enhancement to one of our products. It was at that point that he discovered a serious limitation of recursion as an algorithm, and thereby learned an important lesson. To wit: Just because they teach you a "standard" algorithm for data structure traversal in CS101, that doesn't automatically mean that you should actually USE it in code that you sell to people for money, and therefore have to support for real. Why do I say this? Because, dear Geeks, it turned out that the whole thing blew up when the XML DOM tree got big; especially when it got "deep" (meaning, a lot of nested layers). Why? Because, unlike in Windows (and many other operating systems), the size of the calling stack in LotusScript is fixed. Yes, that is correct. The calling stack for a LotusScript program is pre-allocated at program start time, and it never changes. In the Windows C (and .NET) runtime, by contrast, you can specify a calling stack's initial size (there is one stack per thred in most OS processes), but if you run out, the OS (or more properly, the language runtime servics) will grow it automtically. But in LS, when you (for example) recurse too deeply at runtime, you get an "out of memory" exception. I'm not sure how big the LS pre-allocated stack is, but if I had to guess, I would guess 64KB. So, copying that particular example didn't work out so well for us. My suggestion for fixing this problem was to process the nodes iteratively instead of recursively -- do them in a loop. If you traverse the tree "breadth first" instead of "depth first", you can just build a queue of nodes as you "walk" the tree, and at the same time, you can pull nodes off the front of the queue for "processing". We did indeed verify that for the same data inputs, this new technique did not run out of memory. Here's the original example, copied from Designer Help:
Original version: This agent parses the origXML file, walks the DOM node tree and creates a report in outputFile about the nodes encountered. (Declarations)
Dim domParser As NotesDOMParser
Dim LF As String Sub Initialize
Dim session As NotesSession
Dim db As NotesDatabase
Dim inputStream As NotesStream, outputStream As NotesStream
Dim docNode As NotesDOMDocumentNode
Dim origXML As String, outputFile As String
origXML = "c:\dxl\xmldom.xml"
outputFile = "c:\dxl\DOMtree.txt"
Dim header As String
header = "Walk Tree agent"
LF = Chr(13)+Chr(10)
On Error Goto errh
Set session = New NotesSession
Set db = session.CurrentDatabase
'create the output file
Set outputStream =session.CreateStream
outputStream.Open (outputFile)
outputStream.Truncate
'write report title
outputStream.WriteText ("DOM Parser Report - " )
outputStream.WriteText (header+LF)
'open the XML file
Set inputStream = session.CreateStream
inputStream.Open (origXML)
If inputStream.Bytes = 0 Then
outputStream.WriteText (origXML+" is empty"+LF)
Goto results
End If
'create DOM parser and process
Set domParser=session.CreateDOMParser(inputStream, outputStream)
domParser.Process
'get the document node
Set docNode = domParser.Document
Call walkTree(docNode)
results:
Call outputStream.Close
Exit Sub
errh:
outputStream.WriteText ("errh: "+Cstr(Err)+": "+Error+LF)
Resume results
End Sub Sub walkTree ( node As notesdomnode)
Dim child As notesdomnode
Dim elt As notesdomnode
Dim attrs As notesdomnamednodemap
Dim a As notesdomattributenode
Dim piNode As Notesdomprocessinginstructionnode
LF = Chr(13)+Chr(10)
If Not node.IsNull Then
Select Case node.NodeType
Case DOMNODETYPE_DOCUMENT_NODE: ' If it is a Document node
domParser.Output( "Document node: "+node.Nodename )
Set child = node.FirstChild ' Get the first node
Dim numChildNodes As Integer
numChildNodes = node.NumberOfChildNodes
domParser.Output(" has "+Cstr(numChildNodes)+" Child Nodes"+LF)
While numChildNodes > 0
Set child = child.NextSibling ' Get next node
numChildNodes = numChildNodes - 1
Call walkTree(child)
Wend
Case DOMNODETYPE_DOCUMENTTYPE_NODE: ' It is a tag
domParser.Output("Document Type node: "+ node.NodeName+LF)
Case DOMNODETYPE_TEXT_NODE: ' Plain text node
domParser.Output("Text node: "+node.NodeValue+LF)
Case DOMNODETYPE_ELEMENT_NODE: ' Most nodes are Elements
domParser.Output("Element node: "+node.NodeName )
Set elt = node
Dim numAttributes As Integer, numChildren As Integer
numAttributes = elt.attributes.numberofentries
domParser.Output(" has "+Cstr(numAttributes)+" Attributes"+LF)
Set attrs = elt.Attributes ' Get attributes
Dim i As Integer
For i = 1 To numAttributes ' Loop through them
Set a = attrs.GetItem(i)
' Print attr. name & value
domParser.Output("Attribute "+a.NodeName+": "+a.NodeValue+LF)
Next
numChildren = elt.NumberOfChildNodes
Set child = elt.FirstChild ' Get child
While numChildren > 0
Call walkTree(child)
Set child = child.NextSibling ' Get next child
numChildren = numChildren - 1
Wend
domParser.Output( elt.nodeName+LF)
Case DOMNODETYPE_COMMENT_NODE: ' Comments
domParser.Output("Comment node: "+node.NodeValue+LF)
Case DOMNODETYPE_PROCESSINGINSTRUCTION_NODE: ' Handle PI nodes
Set piNode = node
domParser.Output("Processing Instruction node: " )
domParser.Output(" with Target "+piNode.Target+_
" and Data "+piNode.Data+LF)
Case DOMNODETYPE_CDATASECTION_NODE: ' CDATA sections
domParser.Output("CDATA Section node: "+node.NodeName)
domParser.Output(" has value of "+node.NodeValue+LF)
Case DOMNODETYPE_ENTITYREFERENCE_NODE: ' Handle entities
domParser.Output("Entity Reference node: "+node.NodeName+LF)
Case Else:
domParser.Output("Ignoring node: "+Cstr(node.NodeType)+LF)
End Select 'node.NodeType
End If 'Not node.IsNull
End Sub ------------------End of Example-------------------------- It's a perfectly fine example, it's just that there are a lot of real-world cases where it will crash (at least, in LotusScript it will). Here's a new version of the example that I re-wrote to use a node queue. The startup code parses the input XML file, and gets the top-level ("document") node. It puts that node on a queue (I wrote a LS "Queue" class to manage the queue, it's in the Declarations section). Then, as long as there's something in the queue, we grab the first Node, and see if it has any children. If so, we do a loop and put each child at the end of the queue. You could modify the code to "process" (in this example "process" just means to write out some info about the current node) the node before you put it on the queue, or to do it when you take it off the front of the queue, it doesn't matter that much. So, in sum, we're stuffing nodes onto the tail of the queue as we encounter them, and pulling them off the front for processing. We're done when the queue is empty. Not too hard, eh? Enjoy!
New version:
'TreeWalker:
Option Public
Option Declare
Dim domParser As NotesDOMParser
Dim LF As String
Const INCREMENT = 5
Public Class Queue
Private q() As NotesDOMNode
Private first As Integer
Private last As Integer
Sub new()
first = 0
last = -1
Redim q(INCREMENT - 1) ' INCREMENT represents the count, not the ubound
End Sub
Public Sub Add(node As NotesDOMNode)
Dim origFirst As Integer
Dim i As Integer
origFirst = Me.first
' does the queue have space to just add?
If Me.last < Ubound(Me.q) Then
Me.last = Me.last + 1
Set Me.q(last) = node
Goto XEnd
End If
' if there's space at the head of the queue, shift everything over
' (this is cheaper than redim-ing the array)
If Me.first > 0 Then
i = 0
While Me.first <= Me.last
Set Me.q(i) = Me.q(Me.first)
Set Me.q(Me.first) = Nothing
Me.first = Me.first + 1
i = i + 1
Wend
Me.last = Me.last - origfirst
Me.first = 0
' now add the new node
Me.last = Me.last + 1
Set Me.q(last) = node
Goto XEnd
End If
' if we reach here, the array is full but we still need to add a node,
' so no choice but to expand the array
Redim Preserve Me.q(Ubound(Me.q) + INCREMENT )
Me.last = Me.last + 1
Set Me.q(last) = node
Goto XEnd
XEnd:
End Sub
Public Function FirstNode() As NotesDOMNode
If Me.first <= Me.last Then
Set FirstNode = Me.q(Me.first)
Set Me.q(Me.first) = Nothing
Else
Set FirstNode = Nothing
Exit Function
End If
&
nbsp; Me.first = Me.first + 1
End Function
End Class
Sub Initialize
Dim session As NotesSession
Dim db As NotesDatabase
Dim inputStream As NotesStream, outputStream As NotesStream
Dim docNode As NotesDOMDocumentNode
Dim origXML As String, outputFile As String
origXML = "c:\temp\xmldom.xml"
outputFile = "c:\temp\DOMtree.txt"
Dim header As String
header = "Walk Tree agent"
LF = Chr(13)+Chr(10)
On Error Goto errh
Set session = New NotesSession
Set db = session.CurrentDatabase
'create the output file
Set outputStream =session.CreateStream
outputStream.Open (outputFile)
outputStream.Truncate
'write report title
outputStream.WriteText ("DOM Parser Report - " )
outputStream.WriteText (header+LF)
'open the XML file
Set inputStream = session.CreateStream
inputStream.Open (origXML)
If inputStream.Bytes = 0 Then
outputStream.WriteText (origXML+" is empty"+LF)
Goto results
End If
'create DOM parser and process
Set domParser=session.CreateDOMParser(inputStream, outputStream)
domParser.Process
'get the document node
Set docNode = domParser.Document
' Call walkTree(docNode)
Dim q As New Queue()
q.Add docNode ' get the first node in there
Process q
results:
Call outputStream.Close
Exit Sub
errh:
outputStream.WriteText ("errh: "+Cstr(Err)+": "+Error+LF)
Resume results
End Sub
Sub walkTree ( node As notesdomnode)
Dim child As notesdomnode
Dim elt As notesdomnode
Dim attrs As notesdomnamednodemap
Dim a As notesdomattributenode
Dim piNode As Notesdomprocessinginstructionnode
LF = Chr(13)+Chr(10)
If Not node.IsNull Then
Select Case node.NodeType
Case DOMNODETYPE_DOCUMENT_NODE: ' If it is a Document node
domParser.Output( "Document node: "+node.Nodename )
Set child = node.FirstChild ' Get the first node
Dim numChildNodes As Integer
numChildNodes = node.NumberOfChildNodes
domParser.Output(" has "+Cstr(numChildNodes)+" Child Nodes"+LF)
While numChildNodes > 0
Set child = child.NextSibling ' Get next node
numChildNodes = numChildNodes - 1
Call walkTree(child)
Wend
Case DOMNODETYPE_DOCUMENTTYPE_NODE: ' It is a tag
domParser.Output("Document Type node: "+ node.NodeName+LF)
Case DOMNODETYPE_TEXT_NODE: ' Plain text node
domParser.Output("Text node: "+node.NodeValue+LF)
Case DOMNODETYPE_ELEMENT_NODE: ' Most nodes are Elements
domParser.Output("Element node: "+node.NodeName )
Set elt = node
Dim numAttributes As Integer, numChildren As Integer
numAttributes = elt.attributes.numberofentries
domParser.Output(" has "+Cstr(numAttributes)+" Attributes"+LF)
Set attrs = elt.Attributes ' Get attributes
Dim i As Integer
For i = 1 To numAttributes ' Loop through them
Set a = attrs.GetItem(i)
domParser.Output("Attribute "+a.NodeName+": "+a.NodeValue+LF)
Next
numChildren = elt.NumberOfChildNodes
Set child = elt.FirstChild ' Get child
While numChildren > 0
Call walkTree(child)
Set child = child.NextSibling ' Get next child
numChildren = numChildren - 1
Wend
domParser.Output( elt.nodeName+LF)
Case DOMNODETYPE_COMMENT_NODE: ' Comments
domParser.Output("Comment node: "+node.NodeValue+LF)
Case DOMNODETYPE_PROCESSINGINSTRUCTION_NODE: ' Handle PI nodes
Set piNode = node
domParser.Output("Processing Instruction node: " )
domParser.Output(" with Target "+piNode.Target+_
" and Data "+piNode.Data+LF)
Case DOMNODETYPE_CDATASECTION_NODE: ' CDATA sections
domParser.Output("CDATA Section node: "+node.NodeName)
domParser.Output(" has value of "+node.NodeValue+LF)
Case DOMNODETYPE_ENTITYREFERENCE_NODE: ' Handle entities
domParser.Output("Entity Reference node: "+node.NodeName+LF)
Case Else:
domParser.Output("Ignoring node: "+Cstr(node.NodeType)+LF)
End Select 'node.NodeType
End If 'Not node.IsNull
End Sub
Sub Process(q As Queue)
Dim node As NotesDOMNode
Dim count As Integer
Dim child As NotesDOMNode
Dim i As Integer
Set node = q.FirstNode
While Not (node Is Nothing)
count = node.NumberOfChildNodes
If count > 0 Then
For i = 0 To count-1
If i = 0 Then
Set child = node.FirstChild
Else
Set child = child.NextSibling
End If
Bob Balaban March 31 2008 05:10:01 AM Greetings, Geeks! Hope to see you at this annual Boston event (April 30 - May 2). Here are the 2 sessions I will be presenting: Achieving "Peaceful" Coexistence Between Microsoft Exchange and Lotus Notes While many organizations strive to standardize their messaging infrastructures on a single platform, many others do not. Whether through merger, acquisition or simple user preference, many organizations face a need to achieve peaceful coexistence in the messaging wars, perhaps for only a short time (while they transition from one to the other), but perhaps (if they can pull it off) for a very long time. The technical challenges involved in maintaining a well-performing dual messaging infrastructure are not trivial, but they can be met successfully with proper planning and the right technology. The big issues are all around directory synchronization, message transport, rich text, scheduling logic differences and data formats. Come to this session to learn how to overcome these technical challenges how to keep both Notes and Exchange up and running, and how to avoid mutually assured destruction!
Can Domino Applications Be "Web 2.0"-Compliant? Lotus Domino-based Web applications have been around for well over 10 years. The upcoming release of version 8.5 promises lots of new tools and functionality to make it vastly easier to create great-looking and well-performing Web apps. But will these new style apps and the tools you use to build them be architecturally compliant with "Web 2.0"? Come to this session to hear what "Web 2.0" really means for Domino Web application development, to hear how you can keep your Domino apps "standards compliant", and perhaps to learn some new Best Practices for designing and creating Web applications. Bob Balaban March 24 2008 01:43:04 PM Greetings, Geeks! I'm very late to the SNTT club, but I'm hoping to be able to throw in some decent techie tips at least semi-regularly, in addition to the more high-falutin' thought-pieces and questionnaires. So! Have you ever needed to figure out what folders a document is in? Here's a brief synopsis of the documented way to do it, and then I'll show you a neat way I discovered that works better and faster. You can go to the Designer Help database, select the Index tab, and find the "FolderReferences property" entry. You'll see in the doc that this is a property on NotesDocument (works in Java, too), but there are some constraints: the db has to have the "FolerReferencesEnabled" property turned on. But, there's no UI to turn it on (no InfoBox checkbox or anything), you have to do it from a program. And the db has to include 2 hidden views, which you can copy from a recent mail template: $FolderInfo and $FolderRefInfo. Eh, not great, but (so far) not so bad. Then you should go look at Technote number 1092899, which gives a bit more information on how folder references work. It turns out there are a couple of limitations. For one thing, references to folders containing a document are only accumulated once you turn the database-level feature on, it doesn't go back and figure out anything about documents that were put into folders before the feature was enabled. And, having documents in privatefolders causes problems (like, doesn't work....). And they warn you that turning on this feature can slow down "database operations". So, for the project I'm working on, as I did this research it sure started to sound like "not the best way to peel this onion" (meaning, I started to get the idea that if I went down this road, it was going to mean a lot of tears for me). Luckily I came up with another way to do it that seems to work pretty well, and wasn't too hard to code up. I'll describe the algorithm, then show you some code snippets. Algorithm: 1. Get a list of all the folders in the database (scan the array of View objects returned by db.Views, save the actual folders (view.IsFolder) in an array 2. Figure out what documents I want to test for folder-ization (this will vary depending on the application, but that's not important) 3. For each document in whatever list (or DocumentCollection, or whatever): 4. Iterate over each folder object (which is really an instance of the View class), and for each one: 5. Create a ViewNavigator object, using the View.CreateViewNavFrom( )
6. Call ViewNav.Count. If the result is 0, the document is NOT in the folder, otherwise, it is
Cool, huh? I must say, I was very pleased to have figured this out. Why does it work? The trick is in using the ViewNavigator class. When you call any of the CreateViewNavXXX methods on the View class, you will ALWAYS get a non-null result, it always creates the object. BUT, if you use CreateViewNavFrom(document), the navigator class has nothing to do if the document is not in the view/folder. So Count will be 0. It's really nice that the View and ViewNavigator classes were coded to be tolerant of what is essentially an error condition! You're asking for a navigator object where the first position in the navigator's "list" is the document you specify. There is a call in the CAPI that this logic uses (NIFLocateNote(), you can look it up) which, given a document id, finds that document in a view or folder collection.
Code:
This code is written in C#, because the project I happen to be working on requires a .NET implementation. However, this would work just as well if it were in LotusScript or java.
// "allFolders" is a previously-populated array of NotesView objects representing all the folders in a db
string f = null; // list of folder names, delimited by ";"
for (int i = 0; i < allFolders.Count; i++)
{
NotesView v = (NotesView)allFolders[i];
try
{
NotesViewNavigator nav = v.CreateViewNavFrom(doc, 0);
if (nav != null && nav.Count > 0)
f += (f == null) ? v.Name : "; " + v.Name;
}
catch (Exception e) {} // guess it didn't work, keep going
}
Simple, huh? Note that this is a doubly-nested loop: outer loop over documents, inner loop over array of folders. If you can keep the numbers reasonable, this isn't too slow.
Enjoy!
lol, g2g, ttyl
Bob Balaban March 23 2008 07:16:19 AMGreetings, Geeks! Given the time of year, I was feeling a little bit lazy, so I decided to resurrect a couple of postings I wrote a few years ago on a late, lamented "group blog" called "3C-Interop". It used to be at http://www.3cinterop.com, and the contributors were industry experts such as Karen, Rocky, Peter, and Amy. The idea was to focus on topics related to technology inter-operability, and some good content resulted. The blog kind of dried up after Amy got a job at Microsoft, Rocky and I took jobs at IBM, and Karen went to work at the Burton Group. Of course, you can still find cached stuff on Google.... Anyhow, since Binary Tree (my new employer) is big in this area, I decided to go back and review my earlier thinking on the topic of migration vs. integration. What appears below is the original text of what was a 2-part posting in 3C-Interop, unmodified. Of course, if I were to write this now, 3 1/2 years later, it would be a little different. But perhaps not much different. In fact, I think I'll let y'all absorb this for a few days, then post my own critique. The world (this part of it, anyway) has, after all, evolved in the intervening time. Here goes: Integration vs. Migration (December, 2004)
"Interoperability" I view "interoperability" as the capacity of pieces of an application or infrastructure's architecture to talk to other pieces, regardless of whether those pieces are located on the same machine or not, implemented in the same language or not, running on the same operating system or not. This implies quite a wide range of possibility in terms of implementation and communication technologies, but that's what you want when you architect an application -- the ability to move big boxes of functionality around until the arrangement makes sense. "I'll put the UI over here, the database over there, and the business logic will be a combination of this kind of component and a few of those kinds over there." Knowing that the big boxes can talk to each other when and how you need them to makes it possible to architect the solution with full (or at least solid) awareness of the performance, implementation, security and maintenance implications. As a slight digression, this is exactly why Web Services shows a lot of promise in the appdev world -- it's a technology that uses basic transport and data encoding models to let application "boxes" talk to each other. Of course, it's not completely baked yet.
Another characteristic of interoperability is that (if you do it right) you're planning it from the beginning. It's one of those things that seems obvious when you start to architect or design an application, but which, if you don't plan for it up front, can be very difficult to add on later. Taking such buzzwords as "looseley coupled" and "services oriented" into account early on tend to increase the ease of interoperability. You buy yourself flexibility later if, for example, you want to replace the UI, or the RDBMS, or the middle-tier business logic platform. Which, of course, sounds a lot like "migration".
"Migration" I see migration as a different kind of beast. Migration happens when you take a working piece of your architecture and re-host it elsewhere (a different OS, a different "platform" or "framework"). This is something you do AFTER the app is up and running, typically. It generally involves some kind of "porting" or re-coding, either because the OS is different, or because the services on which the functionality rests are different, or differently implemented (e.g., porting a J2EE servlet coded in Java to a C# ASP in .NET), even if the functionality is supposed to be the same. So, is migration fundamentally different from building for interoperability? In a sense no, because architecting for interoperability up front makes any migrations you do later a lot cheaper, at a component level. From a big-picture, full-application point of view, however, I think it's true that architecting for interoperability ultimately means that you will have LESS need to migrate, or at least, IF you have to migrate, it won't be the entire app. And that's a good thing. Part 3 of this little series [editor's note: see below] will address some additional points on this topic: Why do people migrate apps? When should you migrate? When should you not migrate? If you're going to migrate, how should you do it? And last, but not least, is "migrate" etymologically related to "migraine"? After some briefings at IBM a few weeks ago and at Microsoft this past week, [editor's note: remember, this was written in 2004] it seems that it's no longer good enough to sell developers and their management on your platform for doing new development. Now you have to have a strategy and a marketing plan to "convert" organizations from the competition's stuff to your own, including any already-existing apps. I don't know, maybe it was always that way, and maybe it has to be that way in a relatively mature market. So both behemoths are focusing a lot of energy on "migration", and both have their sights set on "harvesting" the large Domino installed base. Godzilla vs. Mothra? Don't get me started. IBM wants Domino apps to migrate to Workplace [editor's note: heh heh] , and Microsoft wants them to migrate to .NET and whatever Windows server platform is current (win2003 today). First off, let me say that these efforts by both companies, while understandable, are misguided. Neither Lotus Workplace nor .NET is nearly at par functionally with Domino (and Notes) in the collaborative space -- yet. Both are working hard to get there, of course, but until they can offer true functional replacements, why should Domino guys and gals even consider throwing out perfectly good, operational Notes/Domino apps and starting over? They shouldn't (IMHO). So, back to the point in my 2 prior postings on this topic in this blog: Integration is what you do to solve particular problems you may be having with existing apps; migration is what you do when you want to port your app to another framework or platform. Often it is true that what begins as integration (moving a piece of an existing, working app somewhere else) ends as migration (all the pieces end up getting moved). But not always. And, just like JIT, migration happens anyway. Why do people migrate things? I can think of reasons (some valid, some not) which apply at different levels in an organization:
Strategic reasons: "My company is moving to .NET"
Political reasons: "The new CIO hates Domino"
Tactical reasons: "We have more Java developers than C#"
Personal reasons: "I want C# on my resume" The issue question for solution architects and developers is: given that you're going to do some of this, what's the best way to get it done so as to a) minimize cost and b) minimize downtime of the app while making changes.
How to turn "migration" into "integration" If you look at the problem as "Which piece to Migrate first?", you're back to where you should be -- thinking about how to "integrate" the app. The choices tend to be: UI Business logic Other content management/workflow logic Data store Web Service(s) Each has its own problems, solutions and trade-offs, and there is no one-size-fits-all solution, technique or product. Different platforms/frameworks have different strengths and weaknesses. So you pick the low-hanging fruit first, go for maximum impact with minimum time/effort, and demonstrate an early success. Then you decide whether you need to keep going with the next hunk of app, and the next, and so on. Bob Balaban March 10 2008 06:30:48 PMGreetings, Geeks! To quote one of my all-time favorite TV characters, Roseanne Roseannadanna (ok, so I'm a baby-boomer, so sue me), "What's all this I hear about discombobulated obligations?" Or, in plain English, "disconnected applications". Today I took the Amtrak Acela (the "fast" train, or, as one of my European friends calls it, the "faster" train) from Newark, New Jersey (home of the corporate hq of my new company, Binary Tree) back home to Boston. It's a 4 hour ride, give or take, and I got bored reading "Office 2007 For Dummies", so I decided ot go online to check my email. Well! It turns out that the fancy, shmancy "Fast" train from Newark (New Jersiey) to Boston (Massachusetts), does NOT have wireless Internet connectivity!! I thought I was going to DIE!! I said to the conductor guy, "HEY! Mister CONDUCTOR guy on the fancy shmancy "fast" train from Newark New Jersity to Boston Massachusetts!! How come there's no wireless Internet connectivity here?? Huh? Is this the fancy shmancy FAST Amtrak train, or what??" He said, "HEY! Mr. Passenger guy. You sound just like Roseanne Roseannadanna from that old Baby Boomer TV show! NO, we do NOT have wireless Internet connectivity on this train! Unless, of course, you purchased yourself a fancy shmancy telephone company broadband wireless connection card! Did you do that, Mr. I-want-to-be-connected-everywhere, Mr. "Baby Boomer" TV-watching guy? Did you? Huh?" Ok, enough, before I get sued by somebody. Here's the question -- does anyone still care in a major way about disconnected apps? It's been a huge differentiating/selling point of Notes for decades, and I use it all the time. I love having a local replica of my email file, so that I can still read and write email when I can't get an Internet connection for one reason or another. I also make use of the local-replica feature when I can get a connection, but it's slow. I let my dbs replicate in the background, and get decent performance using a local copy in the meantime.Admittedly, I use it a lot less than I used to. Just 2 or 3 years ago, wi-fi was way less ubiquitous, and in airports and hotels I frequently had to rely on slow dial-up connections. But even now, I find that on airplanes (and yes, even on fancy-shmancy trains), I'm forced to be offline. In these situations Notes' replication/disconnected support is a real win. But now I'm wondering a) how big a percentage of most people's time is this case? and b) is it stabilized, or shrinking? I've noticed over the last few years that the Microsoft suite of Web and collaborative applications pretty much assumes all-connected-all-the-time. And they have some valid rationales for making that trade-off -- it's certainly easier to create the software baking in that assumption, and it's certainly true that more people are actually connected a greater percentage of the time. So, weigh in for me, fellow Geeks! Is "offline" support still important? Is it of growing, steady, or shinking importance, In Your Humble Opinions? Will anyone still care in 3 years? 5? Goodnight Geeks! (and goodnight, little Roseanne Roseannadanna!) | |