Bob Balaban's Blog

     
    alt

    Bob Balaban

     

    Token bucket!

    Bob Balaban  June 8 2014 05:41:25 AM
    Greetings, geeks!

    I came across this problem on a project I'm doing: given a RESTful API on a web application server, how do you "throttle" calls so that a given authenticated user can only make a certain number of calls (N) per time interval (W), where both N and W are configurable. The solution I came up with is basically a “Token Bucket” with a sliding time window. What surprised me when I got something working was how easy it really was to implement.

    The context in which this bit of code operates is:
      - You know who the caller is (they're already authenticated)
      - The application can maintain a "user context" that you can retrieve on each HTTP "call" or invocation

    The algorithm is as follows:
    Given a per-user TreeSet (a built-in Java collection class that implements a sorted set) containing long values representing java.util.Date time values (milliseconds). The actual set will contain Long values, not long, but that’s transparent to the algorithm. The set contains a list of times the user has invoked the API, within the last interval W

       - When an invocation arrives at the server, set NOW = long value of “now”, from java.util.Date.

      - For the current user’s list of previous call times, do:
          - Inspect the oldest entry, if older than (NOW – W), remove it
          - Until all old entries are removed, or the list is empty.

      - If the number of entries remaining in the set + 1 is > N, return error
      - Else, add NOW to the set and continue

    The application server context is per-user, and you cache a TreeSet instance for each authenticated client in the context. Here's a link to the file:
    TimeBasedAccessList.java

    As it's only 44 lines, here's all the code...

    import java.util.Date;
    import java.util.TreeSet;

    public class TimeBasedAccessList {

           private volatile TreeSet timeSet = new TreeSet();
           private int nCalls;                        // count of allowed calls
           private long timeWindow;        // in millisec
           
           public TimeBasedAccessList(int N, long M) {
                   nCalls = N;
                   timeWindow = M;
           }

           public synchronized boolean isCallOK() {
                   long now = new Date().getTime();
                   long earliest now - this.timeWindow;
                   
                   // remove entries older than "earliest"
                   // can't use "for" loop with removals inside
                   while (!this.timeSet.isEmpty()) {
                           Long L = this.timeSet.first();         // "lowest" valued element is earliest timestamp
                           if (L.longValue() <= earliest) {
                                   this.timeSet.remove(L);
                           }
                           else {
                                   // no more entries out of window
                                   break;
                           }
                   } // end for
                   
                   // see how many entries there are
                   if (this.timeSet.size() >= this.nCalls) {
                           return false;
                   }
                   
                   // add new entry
                   this.timeSet.add(new Long(now));
                   return true;
           } // end isCallOK
           
    } // end class
    Not much, is it? There's meant to be an instance of the class for each authenticated user. The constructor captures the size of the time window and the number of allowed calls in that time. All the interesting stuff happens in the isCallOK() method. It's possible on a web server that two calls from the same user could come in at the same time, and be dispatched on separate threads. Therefore the isCallOK() method needs to be synchronized to be thread-safe.

    In my particular project, I'm using a Tomcat app server, and the consumer of this class is a Tomcat filter. Filters are cool, you designate one to the server in your application (.WAR file), and Tomcat invokes your filter code before any servlets or JSPs that you have configured. The filter can examine the request data, modify things, do server-wide logging, whatever. In my case, I'm using this particular filter as a gateway: if the user is making too many calls, the filter just returns an HTTP error code (I picked 503 - Unavailable, which usually means "I'm busy, try again later"). If not, then the filter passes control on to either another filter, or to the designated endpoint for the caller's request..

    A few comments on the details of the code...

    The TreeSet object (built-in class in the Java Collections, um, collection) can only store Objects, not scalars, like "long", so we have to convert between "long" and "Long" (the Object version). The only slightly tricky bit of this was looping over the TreeSet and removing entries outside the specified time window. Originally I coded it as a "for" loop: for (Long L : this.timeSet), etc. The problem was that it threw an exception. The reason is documented, though a little obscure. When you do a "for" loop over a collection object, the JVM actually builds the looping functionality using an Iterator instance that it gets from the collection object. That part's fine, but when you use an Iterator (either explicitly by coding it, or invisibly, as with a for loop), you're supposed to make any changes to the contents of the collection (such as removing items) via the Iterator, not directly to the collection object itself. That's because if you don't do it through the Iterator, the Iterator's knowledge of what's in the collection gets out of sync with the collection's actual contents.

    I could have fixed the loop by explicitly getting the Iterator instance and using it to, um, iterate, as well as to remove old entries. But instead I chose to code it by iteratively calling TreeSet.first(), which returns the smallest valued (and in this case, therefore the earliest) entry, or null if the set is empty. The loop exits when the first entry is found that is inside the specified time window, or there are no entries left.

    So, the TreeSet class is the hero here, because it automatically sorts the entries for me. I looked around for a Queue class where I could add new entries to one end, and remove old entries from the other, because that would eliminate the need for sorting overhead. I didn't find one, though, and I was too lazy to write one, so I went with TreeSet. The efficiency/speed of this code is still very good, testing whether the next server invocation for a given user is ok to proceed (calling the isCallOK() method) only consumes a few milliseconds.

    So, there you have it. Geek ya later!

    Follow me on Twitter @LooseleafLLC
    This article ©Copyright 2013 by Looseleaf Software LLC, all rights reserved. You may link to this page, but may not copy without prior approval.

      "Employee of the Quarter" (not)

      Bob Balaban  December 27 2013 03:08:49 PM
      Greetings, geeks!

      This one is not a technical topic, so feel free to move on. If you are my current (or potential future) employer, you should also probably move on at this point.

      In my work email today, I received a shotgun message announcing the current "Employee of the Quarter". The lucky designee receives preferential parking (near the building entrance) for 3 months. There was a long-ish description of how great the EotQ is, and why they were selected for this honor.

      Not to take away from someone who is obviously (from the email description) a very hard worker, get-it-done type of person, but I got to thinking in general about honors like this.

      So here, for anyone who's interested, is my list of reasons "Why I Will Never Be Employee of the Quarter", a fact to which I am entirely reconciled. The language of my list is taken from the award email:

      I do not give 110% (arithmetically impossible).
      I do not work tirelessly (I sometimes get tired).
      I do not go above and beyond every day (once in a while, maybe, but see previous comment about getting tired).
      I am not often viewed as exemplary by managers (I probably argue with them too much)

      So there you have it. There are probably more reasons than I've given, but I stopped thinking about it after these 4. Maybe my Geek pals can supply some more (about me, or about themselves...)

      Geek ya later!

      Follow me on Twitter @LooseleafLLC
      This article ©Copyright 2013 by Looseleaf Software LLC, all rights reserved. You may link to this page, but may not copy without prior approval.

      Geek-o-Terica 17: Accuracy vs. Precision, The Time Drift Chronicles

      Bob Balaban  September 8 2013 07:09:08 AM
      Greetings, geeks!

      Ever need a software timer to see how long it took to do something? This is a common task, right? And the basic technique is simple: capture the current time, do your thing, preferably for a while so that you average out a large number of operations to eliminate random perturbations, capture the current time again, and report on the difference between the two times.

      Sounds easy, right? But there are complications. Such as: What is the relative granularity of the time span you are measuring, vs. the granularity of the timer? Here’s a brief anecdote to illustrate what I mean.

      In a previous job I was asked to do benchmarking of the data flowing through a piece of software (it happened to be a server backup system, but that’s not the important point). The test I devised involved copying about 3 terabytes (TB) – that’s 3000 gigabytes – from one place on the network to another, and seeing how long it took. The software I was using to do the data movement had built-in timers and logging, so I didn’t have to do anything special: just set up 3TB of random stuff somewhere, point the software at it and provide a destination location. The log would then contain the elapsed time.

      I knew it would take a while (more than a day, in fact, for reasons I won’t bother to explain here), so I kicked off the test, wrote down the date and time, and went home early (I love those kinds of tests…)

      When I got in the next morning, the test was still running. I hung around for another couple of hours until it was done, wrote down the end date/time, and went looking for the execution log. I noticed two odd things right away: 1) My computed elapsed time, based on writing down the start and end times, was something like 16.5 hours, but the elapsed time in the log was closer to 18 hours; 2) The start and end time recorded in the execution log were in microseconds.

      So the logged results were very precise, but not very accurate. So here’s the first point I want to make about accuracy and precision:

      Point 1 About Accuracy and Precision: For 99% of the real-world “timing” applications you will ever encounter, it makes no sense for the granularity of the event itself to be out of proportion to the granularity of the timer. If the thing you’re timing (copy 3TB of data over a network) takes hours (or days), then what’s the point of timing it in microseconds? Is it going to matter if the resulting elapsed time is 1 day and 2 microseconds, vs. 1 day and 3 microseconds? Very doubtful, as you’re looking at a difference in the 11th decimal place, or thereabouts, which is extremely unlikely to be significant, especially given Point 2.

      Point 2 About Accuracy and Precision: Software timers drift. That is, the “current time” measured in milliseconds, microseconds or nanoseconds is an estimate, not an absolute. The finer-grained (more “precise”) the measurement, the more likely it is to drift from “real” (clock) time as you measure longer and longer intervals. This explains why the log for my experiment was off by a few hours: had the software used millisecond-, or even better, second-level granularity, the results would have been much closer to wall clock time.

      I got to thinking about how I might demonstrate this issue in a simple Java program. Java has two different built-in “timer” capabilities. The java.util.Date class will give you its date/time value in milliseconds, measured since the “reference date” (usually January 1, 1970, 12AM is time 0). So you can create a Date instance as your start time, create one as your end time and subtract the values returned by Date.getTime(). That’s your elapsed time in milliseconds.

      There’s also a nanosecond-level timer in Java (1000 nanoseconds == 1 microsecond), System.nanoTime(). The documentation for this call isn’t precise about when time 0 is for this timer, but I suspect it’s related either to computer boot time, or JVM start time (it would take too many bits to measure the current time as number of nanoseconds since January 1, 1970).

      I wanted to see if the two timer techniques tracked each other over some number of minutes, so I wrote this Java program: NanoTime.java

      The basic idea is to sample the timers (millisec. and nanosec.) every 10 seconds (make the thread sleep in between) or so, and see if they stay about the same relative to each other, or if the difference between them gets bigger as time passes. So, I get the “start” time in both milliseconds and nanoseconds, then loop for N minutes, sleeping for 10 seconds after every iteration of the loop. Inside the loop I get the elapsed time from “start” in both milliseconds and nanoseconds. The “drift” value is computed by simply dividing the nanosecond value by 1000000 (convert to millisec.)  and subtract from the millisecond elapsed time.

      I know the 2 values will never be the same. For one thing, getting the current timer values requires two Java calls. By the time we’ve made the second call, some time has already passed since the first call. But that’s not the important thing. The important thing is to monitor the change in value of “drift” as the program continues to run. If both timers are equally accurate, then the value of “drift” should not vary very much over, say, 10 or 15 minutes.

      In fact, however, it does (on my laptop). Here’s a selection from the program output:

      At time  17:53:22  elapsed millisec = 0 elapsec nanosec = 3124  Drift in millisec = 0
      At time  17:53:32  elapsed millisec = 10120 elapsec nanosec 10119357918  Drift in millisec = 1
      At time  17:53:42  elapsed millisec = 20121 elapsec nanosec 20119601550  Drift in millisec = 2
      At time  17:53:52  elapsed millisec = 30121 elapsec nanosec 30119829118  Drift in millisec = 2
      At time  17:54:02  elapsed millisec = 40122 elapsec nanosec 40120027683  Drift in millisec = 2
      At time  17:54:12  elapsed millisec = 50123 elapsec nanosec 50120249897  Drift in millisec = 3
      At time  17:54:22  elapsed millisec = 60123 elapsec nanosec 60120394024  Drift in millisec = 3

      SNIP…

      At time  18:07:02  elapsed millisec = 820167 elapsec nanosec = 820136832058  Drift in millisec = 31
      At time  18:07:12  elapsed millisec = 830167 elapsec nanosec = 830137065428  Drift in millisec = 30
      At time  18:07:22  elapsed millisec = 840168 elapsec nanosec = 840137282287  Drift in millisec = 31

      SNIP…

      At time  18:23:42  elapsed millisec = 1820224 elapsec nanosec = 1820158420728  Drift in millisec = 66
      At time  18:23:52  elapsed millisec = 1830224 elapsec nanosec = 1830158597874  Drift in millisec = 66
      At time  18:24:02  elapsed millisec = 1840225 elapsec nanosec = 1840158851769  Drift in millisec = 67
      At time  18:24:12  elapsed millisec = 1850226 elapsec nanosec = 1850159074430  Drift in millisec = 67
      Done!

      The “Drift” number is the milliseconds number minus the nanoseconds number (scaled to milliseconds, of course). So the drift values behavior is very interesting. It shows that the elapsed time derived in milliseconds advances faster than the elapsed time derived in nanoseconds. Had the drift remained more or less constant over time, we’d chalk up the difference between the two values to program execution time: it takes a finite amount of time to execute a line of code, and the two time samples can’t occur at exactly the same moment. Plus, of course, other things are going on in the Java VM (garbage collection, perhaps other background thread execution), and on the computer itself (many processes time-sharing the CPUs…). But the fact that the drift value keeps increasing over time indicates a real difference in the sampling techniques.

      I ran the program again on another laptop, and got some very interesting results:

      At  12:37:47  elapsed millisec = 0 elapsec nanosec = 2165  Drift in millisec = 0
      At  12:37:57  elapsed millisec = 10080 elapsec nanosec = 10080167661  Drift in millisec = 0
      At  12:38:07  elapsed millisec = 20080 elapsec nanosec = 20080718639  Drift in millisec = 0
      At  12:38:17  elapsed millisec = 30080 elapsec nanosec = 30081314176  Drift in millisec = -1
      At  12:38:27  elapsed millisec = 40080 elapsec nanosec = 40081868017  Drift in millisec = -1
      At  12:38:37  elapsed millisec = 50080 elapsec nanosec = 50082447490  Drift in millisec = -2

      SNIP…

      At  12:43:47  elapsed millisec = 360080 elapsec nanosec = 360100194438  Drift in millisec = -20
      At  12:43:57  elapsed millisec = 370080 elapsec nanosec = 370100772026  Drift in millisec = -20
      At  12:44:07  elapsed millisec = 380080 elapsec nanosec = 380101345283  Drift in millisec = -21
      At  12:44:17  elapsed millisec = 390080 elapsec nanosec = 390101918540  Drift in millisec = -21
      At  12:44:27  elapsed millisec = 400080 elapsec nanosec = 400102496756  Drift in millisec = -22
      At  12:44:37  elapsed millisec = 410080 elapsec nanosec = 410103064147  Drift in millisec = -23

      SNIP…

      At  12:51:27  elapsed millisec = 820079 elapsec nanosec = 820126346016  Drift in millisec = -47
      At  12:51:37  elapsed millisec = 830079 elapsec nanosec = 830126909636  Drift in millisec = -47
      At  12:51:47  elapsed millisec = 840079 elapsec nanosec = 840127478912  Drift in millisec = -48
      At  12:51:57  elapsed millisec = 850079 elapsec nanosec = 850128065229  Drift in millisec = -49
      At  12:52:07  elapsed millisec = 860079 elapsec nanosec = 860128631991  Drift in millisec = -49
      At  12:52:17  elapsed millisec = 870079 elapsec nanosec = 870129212163  Drift in millisec = -50
      At  12:52:27  elapsed millisec = 880079 elapsec nanosec = 880129787934  Drift in millisec = -50
      At  12:52:37  elapsed millisec = 890079 elapsec nanosec = 890130352950  Drift in millisec = -51
      At  12:52:47  elapsed millisec = 900079 elapsec nanosec = 900130939617  Drift in millisec = -51
      At  12:52:57  elapsed millisec = 910079 elapsec nanosec = 910131482563  Drift in millisec = -52
      At  12:53:07  elapsed millisec = 920079 elapsec nanosec = 920132072303  Drift in millisec = -53
      At  12:53:17  elapsed millisec = 930079 elapsec nanosec = 930132644303  Drift in millisec = -53
      At  12:53:27  elapsed millisec = 940079 elapsec nanosec = 940133205897  Drift in millisec = -54
      At  12:53:37  elapsed millisec = 950079 elapsec nanosec = 950133785300  Drift in millisec = -54
      Done!

      Cool, huh? Similar behavior in that the two values diverge at a more or less constant rate over time. BUT, in the opposite direction! In this case, the nanosecond value advances faster than the millisecond value. Go figure.

      One additional interesting question might be: which value is “right”, or “more accurate”? I don’t have an answer to that question, and within the world of this test program, there’s absolutely no way to tell. What you’d need to do is measure the results of the two sampling techniques against an object outside measurement, a 3rd clock that doesn’t depend on the computer running the Java program. After all, a “gap” of 50-60 milliseconds over a time interval of 30 minutes is a pretty small variation, approximately .003%.

      Geek ya later!

      Follow me on Twitter @LooseleafLLC
      This article ©Copyright 2013 by Looseleaf Software LLC, all rights reserved. You may link to this page, but may not copy without prior approval.

      Some Interesting Tech I’ve Been Involved With Lately

      Bob Balaban  April 20 2013 04:58:50 PM
      Greetings, geeks!

      Some of you have been wondering what I've been up to lately. No, not a stealth-mode startup, not a new API for Notes/Domino, nothing like that.
      In fact, I'm at about the 6-month mark in a whole new (to me) tech area: storage and storage management. It's one of those things that's a whole lot more complicated than you'd ever think, until you start to dig into it. Me, I thought a "disk" started and ended with that box on the end of a USB cable that you plug into a laptop. No. That's just the simplest kind of disk.

      Anyway, I now work at a company called NetApp, that makes the most advanced storage systems (and no, a "storage system" is not a closet-organizer!) on the market. I tried (as always) to explain to my mom and dad what I do all day, and it was hard, they're not "computer people". But I found some awesomely cool writeups that at least start to illustrate what goes on in the world of storage, and I thought I'd share.

      First, CERN. We all know that anything having to do with CERN is automatically very cool. Here's some stuff on what NetApp does for CERN:
      http://www.netapp.com/us/media/ds-3365-0812.pdf

      High-energy proton collision experiments generate a LOT of data, very quickly (I heard it can be at rates around 6 gigabytes per second), and you have to have "storage" that can accommodate it.

      So that's how my company mainly makes money, "storage systems". But that's not the area I work in myself. I'm working on a product called "OnCommand/Insight" (OCI), which is more on the data center management side of things. It inventories all of the important devices in your data center (storage, of course, but also switches, "hosts" (VMs and so on) etc.), and reports not only on the inventory (what devices you have, what's connected to what, like that), but on the performance and utilization of those devices: do you have storage that isn't being used by anyone? Do you have redundant connectivity from hosts to disks? Do you have RAID arrays that need servicing? And so forth.

      Here's a page that describes more comprehensively than I can here what it's all about:
      http://www.netapp.com/us/products/management-software/oncommand-insight/insight-library.aspx

      So, it's great fun. NetApp is a good company to work for, the technology is interesting and pretty deep, and it really makes a big difference to a lot of people, even if it's mostly "back-room". When you hear about "big data", or "analytics" or "cloud", start thinking NetApp!

      Geek ya later!

      Follow me on Twitter @LooseleafLLC
      This article ©Copyright 2013 by Looseleaf Software LLC, all rights reserved. You may link to this page, but may not copy without prior approval.

      The Interesting Difference Between "Exception" and "RuntimeException" (Java)

      Bob Balaban  February 2 2013 06:43:32 PM
      Greetings Geeks!

      I hope everyone who attended Lotusphere IBMConnect '13 had a good time and has returned home safely. I didn't attend (again, I stopped going after the first 17), but I've been hearing that it was pretty good.

      I learned something new about certain types of exceptions in Java programming the other day. Here's the context:
      • I had an Enum class with a few items in it
      • I had a String that came from outside the program that I wanted to convert into an enum'ed constant
      • This was happening inside an existing function where I didn't want to add to the "throws" declaration (because that would cause ripple effects up into the functions that called the function I was modifying)

      Here's my first draft implementation (this isn't the real code, I created a standalone example for ya):

      import
      java.io.*;

      public
      class ExceptionTest
      {
             public enum someEnum
             {
                     firstitem, seconditem, thirditem
             }
             
             public static void main(String [] args)
             {
                     try { func1(args[0]); }
                     catch (IOException ioe) {}
                     
                     try { func2(args[0]); }
                     catch (IOException ioe) {}
             }
             
             public static void func1(String arg) throws IOException
             {
                     someEnum myEnum = null;
                     
                     System.out.println("In func1");
                     if (arg.equals("firstitem"))
                             myEnum = someEnum.firstitem;
                     else if (arg.equals("seconditem"))
                             myEnum = someEnum.seconditem;
                     else if (arg.equals("thirtitem"))
                             myEnum = someEnum.thirditem;
                     else System.out.println("Invalid input: " + arg);
                     
                     if (myEnum != null)
                             System.out.println("myEnum: " + myEnum.name());
             }
      }

      Not too bad, but rather tedious.  Note that myEnum.name() will take the enum's item value and convert it into a String.

      A colleague pointed out to me that the Java Enum class (all "enums" are compiled into instances of the class Enum) has a valueOf() method. You give it a string, it returns the enum item (e.g., someEnum..firstitem). So, it's effectively the reverse of the name() method.

      I noticed from the javadoc that this function can throw an IllegalArgumentException. It'll do that if the string you give it doesn't match one of the enum items. So I changed my "func1" implementation into what I'll show you here as func2():

             public static void func2(String arg) throws IOException
             {
                     someEnum myEnum = null;
                     
                     System.out.println("In func2");
                     try { myEnum = someEnum.valueOf(arg); }
                     catch (IllegalArgumentException iae) {}
                     
                     if (myEnum == null)
                             System.out.println("Invalid input: " + arg);
                     else System.out.println("myEnum: " + myEnum.name());
                     
             }

      Simpler, right? I thought at this point that I had it nailed. But then that same colleague said, "Hey, you know you don't need the try/catch there". I was sure he was wrong (this stuff happens to me all the time). "No, I do need it, because I don't want to add IllegalArgumentException to the function 'throws' list." I was sure that if I took out the try/catch I'd get a compile error saying I needed it.

      Buuuuuuutt noooooooo, I was wrong. And here's what I hadn't realized until then: There are TWO categories of Exceptions. Many classes that extend (inherit from) java.lang.Exception (like IOException, for example), are required to either be declared in a "throws" clause in the function declaration, or "caught" in a try/catch block. The compiler enforces this.

      However, SOME exception classes (such as IllegalArgumentException) extend not java.lang.Exception, but java.lang.RuntimeException (which itself extends Exception). These are special, you don't need to catch them or declare them explicitly. Here's the slimmed down version of func2():

             public static void func2(String arg) throws IOException
             {
                     someEnum myEnum = null;
                     
                     System.out.println("In func2");
                     myEnum = someEnum.valueOf(arg);
                     
                     if (myEnum == null)
                             System.out.println("Invalid input: " + arg);
                     else System.out.println("myEnum: " + myEnum.name());
                     
             }

      No compile problems. AND, if you later add more items to the enum, you don't have to change any of the other code.

      If you run this with a valid input (such as "firstitem"). If you run it with something other than a string that matches one of the enum items, you get this:

      Exception in thread "main" java.lang.IllegalArgumentException: No enum constant ExceptionTest.someEnum.item
             at java.lang.Enum.valueOf(Unknown Source)
             at ExceptionTest$someEnum.valueOf(ExceptionTest.java:1)
             at ExceptionTest.func2(ExceptionTest.java:42)
             at ExceptionTest.main(ExceptionTest.java:15)

      Of course, you still CAN use try/catch here. You might do that if you want to catch the "illegal" argument and re-throw an application-specific Exception, or create a nicer error message. But you don't have to, because derivatives of RuntimeException are treated specially by the Java compiler.

      Now you know!

      Geek ya later!

      Follow me on Twitter @LooseleafLLC
      This article ©Copyright 2013 by Looseleaf Software LLC, all rights reserved. You may link to this page, but may not copy without prior approval.

      "Reflections on Reflecting Light" - Java program

      Bob Balaban  May 9 2012 06:33:45 PM
      Greetings Geeks!

      Today it is my very great pleasure to introduce to you a new guest blogger: my son David. He has graciously allowed me to prevail upon him to write about his recent science fair project, titled "Reflections on Reflecteing Light". That may sound innocuous, but to quote one of my favorite movies, "That's no ordinary rabbit!". David took this project through our local high school's science fair, was invited to the regional fair, and then to the Massachusetts state science and engineering fair, held at MIT. The Java program he wrote is very sophisticated, both in concept and in execution, and David has agreed to post it here.

      David is looking for a software internship for this summer. If you like this sample of his work (and you are in the Boston area), please consider hiring him!  He is a graduating high school senior, and will attend UMass/Amherst in the Fall, majoring in Physics and Computer Science.

      Caveats:

      1) Use the code for anything you like, with appropriate attribution to David Balaban.
      2) If you make any money from this code, you owe some to David.
      3) The entire contents of this page and of the attached code is copyright 2012, Looseleaf Software, Inc. All rights reserved!

      David's email is: david AT Looseleaf.net
      My email is: bbalaban AT gmail

      Here's the abstract:

      The purpose of my project is to determine how distorted an image becomes when it is reflected off of a curved surface and how the distortion changes as the surface becomes more curved. My hypothesis was that parts of the image will move towards the center of valleys while other parts will move away from peaks and move a smaller distance as the surface becomes more curved. To test this hypothesis, I wrote a program that takes an image saved on the computer and converts it to an array of pixels. The program treats each pixel as a beam of light traveling directly towards a mathematically generated curved surface and traces its path as it is reflected. The program ends by reporting statistical information resulting from over one million beams of light and creating a new picture which shows the resulting image from the reflection. My hypothesis was incorrect, because on average the pixels move away from the center of valleys, not towards.

      And here's the code (RAR format): ScienceFair.rar

      Here's an example of one of the images he used as input to the program: Control.jpg

      And here's the distorted result of that same image: Trial1.gif

      Enjoy!

      Geek ya later!

      Happy 5th? Yeah, 5th

      Bob Balaban  April 10 2012 09:11:32 PM
      Greetings Geeks!

      So. March 30, 2007 was my first blog entry. Evidently I missed my own 5th Blanniversary! :-( Better late than never, I suppose. In theory, anyway :-)

      I know I don't post here often, but I do hope that when I do post, it's worth reading. For some people, anyway.

      I'll quote from a nice person who commented on that first post: "energetic,funny,cute--that's what I feel about you from your special explanation about "In Theory".I think you must be very content while writing these words."

      Well, yeah! That does pretty much sum it up. Thanks!

      If you read the original post, you might be interested to know that I'm STILL using that same 7.2 blog template (why change? It works!), and most of the time I'm still using Notes 7, except when I have to do dev work (sit down for beer with me sometime, and I'll tell you why, if you don't already know).

      Any topics anyone wants to hear about from me? Suggestions welcome. I still plan to stick to solid techie topics (GeekStuff), and avoid the fluffy stuff, which you can get anywhere.
      Don't ask for XPages topics, I don't do XPages.

      Thanks to those of you who are still reading after all this time, and thanks to those of you who are newer to this space.

      Geek ya later!

      (Need expert application development architecture/coding help?  Want me to help you invent directory services based on RDBMS? Need some Cloud-fu or some web services? Contact me at: bbalaban, gmail.com)
      Follow me on Twitter @LooseleafLLC
      This article ©Copyright 2012 by Looseleaf Software LLC, all rights reserved. You may link to this page, but may not copy without prior approval.

      "Programming Domino With Java" - Final Flush

      Bob Balaban  January 2 2012 03:56:33 AM
      Happy New Year, Geeks!

      Here's the 7th (and final) installment of the book. Thanks again for all the positive feedback.
      The first installment can be found here
      The 2nd installment is here
      The 3rd is here
      The 4th is here
      The 5th is here
      and the 6th is here

      All of the book content (as is all of the content on this blog) is Copyright 1998 and 2011 by Looseleaf Software, Inc. You may not reproduce or distribute the book's content without permission from me.

      Some Caveats and explanations:

      - This book is now 12 years old. There are lots of things in it that are amusingly dated, even laughable. One example: the absurd emphasis on Java Applets. They died an unlamented death due to bloating a long time ago, but at the time I wrote the book, they were a hot technology.
      - The formatting is not exactly as it first appeared in the book, and some of it is downright awkward. Sorry! The publisher converted the original files to Word for me, perhaps they forgot to select the "high-def" option.
      - The Java package described in the book (lotus.notes) was superseded by lotus.domino in R5 (1999? I think). The methods are all mostly the same. Of course, IBM have added lots more classes and methods than existed when the book was written. Also, the "recycle()" method had not yet been invented in Domino 4.6, so the memory issues that call addresses are not really covered.
      - The figures (screen shots and so on) referred to did not make it into any file format I can deal with, so I had to leave them out. Sorry! The code samples are mostly there, and I'll also post all of the files that were on the CD that came with the original book.

      This Installment

      This hunk is a zip file containing the contents of the CD that was included with the original book.
      Appendix F (previously posted) contains a description of the contents of the CD: Appendix F.pdf

      I have not gone back and checked that the samples still work. If you find that any do not, please let me know.
      The JavaDoc, of course, corresponds to the Notes v4.0 release. "NOI" has come a long way since then, and the whole lotus.notes package was pretty much obsoleted in Notes v5, but they may still be distributing it.

      CD.zip

      Enjoy! Geek ya later

      (Need expert application development architecture/coding help?  Want me to help you invent directory services based on RDBMS? Need some Cloud-fu or some web services? Contact me at: bbalaban, gmail.com)
      Follow me on Twitter @LooseleafLLC
      This article ©Copyright 2012 by Looseleaf Software LLC, all rights reserved. You may link to this page, but may not copy without prior approval.

      "Programming Domino With Java" - Sixth Sip

      Bob Balaban  December 6 2011 11:17:56 AM
      Greetings, Geeks!

      Here's the 6th installment of the book. Thanks again for all the positive feedback.
      The first installment can be found here
      The 2nd installment is here
      The 3rd is here
      The 4th is here
      The 5th is here

      All of the book content (as is all of the content on this blog) is Copyright 1998 and 2011 by Looseleaf Software, Inc. You may not reproduce or distribute the book's content without permission from me.

      Some Caveats and explanations:

      - This book is now 12 years old. There are lots of things in it that are amusingly dated, even laughable. One example: the absurd emphasis on Java Applets. They died an unlamented death due to bloating a long time ago, but at the time I wrote the book, they were a hot technology.
      - The formatting is not exactly as it first appeared in the book, and some of it is downright awkward. Sorry! The publisher converted the original files to Word for me, perhaps they forgot to select the "high-def" option.
      - The Java package described in the book (lotus.notes) was superseded by lotus.domino in R5 (1999? I think). The methods are all mostly the same. Of course, IBM have added lots more classes and methods than existed when the book was written. Also, the "recycle()" method had not yet been invented in Domino 4.6, so the memory issues that call addresses are not really covered.
      - The figures (screen shots and so on) referred to did not make it into any file format I can deal with, so I had to leave them out. Sorry! The code samples are mostly there, and I'll also post all of the files that were on the CD that came with the original book.

      This Installment


      We're done with the main body of text (Chapters 1-14), but there's still more good stuff to post:

      Appendix A "Useful Links and References" Appendix A.pdf

      Appendix B "Notes Object Interface Class Diagram" (yeah, a little out of date now...)
      Appendix B.pdf

      Appendix C "Domino Setup For Writing Java Programs" Appendix C.pdf

      Appendix D "Notes Object Interface Exceptions" Appendix D.pdf

      Appendix E "Creating LSXs" (Note: This was WAY before Lotus released the "LSX Toolkit", known far and wide in song and legend. You can probably still get the toolkit at this site, though I haven't checked lately. It hasn't had a new release in years, and is known to be broken for recent versions of Microsoft Visual Studio)
      Appendix E.pdf

      Appendix F "What's On the CD-ROM" Appendix F.pdf

      Next installment, I'll post the samples and javadoc from the CD

      Enjoy! Geek ya later

      (Need expert application development architecture/coding help?  Want me to help you invent directory services based on RDBMS? Need some Cloud-fu or some web services? Contact me at: bbalaban, gmail.com)
      Follow me on Twitter @LooseleafLLC
      This article ©Copyright 2011 by Looseleaf Software LLC, all rights reserved. You may link to this page, but may not copy without prior approval.

      "Programming Domino With Java" - Fifth Slug

      Bob Balaban  November 28 2011 05:22:21 PM
      Greetings, Geeks!

      Here's the 5th installment of the book. Thanks again for all the positive feedback.
      The first installment can be found here
      The 2nd installment is here
      The 3rd is here
      The 4th is here

      All of the book content (as is all of the content on this blog) is Copyright 1998 and 2011 by Looseleaf Software, Inc. You may not reproduce or distribute the book's content without permission from me.

      Some Caveats and explanations:

      - This book is now 12 years old. There are lots of things in it that are amusingly dated, even laughable. One example: the absurd emphasis on Java Applets. They died an unlamented death due to bloating a long time ago, but at the time I wrote the book, they were a hot technology.
      - The formatting is not exactly as it first appeared in the book, and some of it is downright awkward. Sorry! The publisher converted the original files to Word for me, perhaps they forgot to select the "high-def" option.
      - The Java package described in the book (lotus.notes) was superseded by lotus.domino in R5 (1999? I think). The methods are all mostly the same. Of course, IBM have added lots more classes and methods than existed when the book was written. Also, the "recycle()" method had not yet been invented in Domino 4.6, so the memory issues that call addresses are not really covered.
      - The figures (screen shots and so on) referred to did not make it into any file format I can deal with, so I had to leave them out. Sorry! The code samples are mostly there, and I'll also post all of the files that were on the CD that came with the original book.

      This Installment


      This time I'm posting 3 more chapters:

      Chapter 12: "NOI and Java Beans" (Are they really good for your heart?)
      Chapter 12.pdf

      Chapter 13: "JDBC and NOI"
      Chapter 13.pdf

      Chapter 14: A Look Ahead to Domino 5 (This ought to provide some laughs, eh?)
      Chapter 14.pdf

      Note:
      Chapter 14 is the last real chapter, but there's still plenty more to come. To wit:
      - Appendices A through F
      - Samples, including some javadoc, since I happen to have it anyway

      Release Notes


      Chapter 12: No bugs!
      Chapter 13: No bugs!
      Chapter 14: Also bug-free!

      Enjoy! More soon. Geek ya later

      (Need expert application development architecture/coding help?  Want me to help you invent directory services based on RDBMS? Need some Cloud-fu or some web services? Contact me at: bbalaban, gmail.com)
      Follow me on Twitter @LooseleafLLC
      This article ©Copyright 2011 by Looseleaf Software LLC, all rights reserved. You may link to this page, but may not copy without prior approval.

      "Programming Domino With Java" - Fourth Hit

      Bob Balaban  November 20 2011 01:30:47 PM
      Greetings, Geeks!

      Here's the 4th installment of the book. Thanks again for all the positive feedback.
      The first installment can be found here
      The 2nd installment is here
      The 3rd is here

      All of the book content (as is all of the content on this blog) is Copyright 1998 and 2011 by Looseleaf Software, Inc. You may not reproduce or distribute the book's content without permission from me.

      Some Caveats and explanations:

      - This book is now 12 years old. There are lots of things in it that are amusingly dated, even laughable. One example: the absurd emphasis on Java Applets. They died an unlamented death due to bloating a long time ago, but at the time I wrote the book, they were a hot technology.
      - The formatting is not exactly as it first appeared in the book, and some of it is downright awkward. Sorry! The publisher converted the original files to Word for me, perhaps they forgot to select the "high-def" option.
      - The Java package described in the book (lotus.notes) was superseded by lotus.domino in R5 (1999? I think). The methods are all mostly the same. Of course, IBM have added lots more classes and methods than existed when the book was written. Also, the "recycle()" method had not yet been invented in Domino 4.6, so the memory issues that call addresses are not really covered.
      - The figures (screen shots and so on) referred to did not make it into any file format I can deal with, so I had to leave them out. Sorry! The code samples are mostly there, and I'll also post all of the files that were on the CD that came with the original book.

      This Installment


      This time I'm posting 3 more chapters:

      Chapter 9 (Debugging NOI Agents) - Chapter 09.pdf
      NOTE: When I wrote the book, Eclipse was not yet invented (or at least, widely known). I hadn't yet come up with the now-common "2-Headed Beast" technique of debugging Java agents (Why hasn't IBM created a way to debug agents inside Designer in the 14 years that Java agents have been loose in the world? I don't know, but it's sad). See here for a description of "2-Headed Beast".

      Chapter 10 (Sharing Objects Across Threads) - Chapter 10.pdf

      Chapter 11 (Upgrading Servlets to Agents) - Chapter 11.pdf

      Release Notes


      Chapter 9: No bugs!

      Chapter 10: No bugs!

      Chapter 11: No bugs!

      Enjoy! More soon. Geek ya later

      (Need expert application development architecture/coding help?  Want me to help you invent directory services based on RDBMS? Need some Cloud-fu or some web services? Contact me at: bbalaban, gmail.com)
      Follow me on Twitter @LooseleafLLC
      This article ©Copyright 2011 by Looseleaf Software LLC, all rights reserved. You may link to this page, but may not copy without prior approval.

      "Programming Domino With Java" - Third Gulp

      Bob Balaban  November 14 2011 06:28:59 PM
      Greetings, Geeks!

      Here's the third installment of the book. Thanks again for all the positive feedback.
      The first installment can be found here
      The 2nd installment is here

      All of the book content (as is all of the content on this blog) is Copyright 1998 and 2011 by Looseleaf Software, Inc. You may not reproduce or distribute the book's content without permission from me.

      Some Caveats and explanations:

      - This book is now 12 years old. There are lots of things in it that are amusingly dated, even laughable. One example: the absurd emphasis on Java Applets. They died an unlamented death due to bloating a long time ago, but at the time I wrote the book, they were a hot technology.
      - The formatting is not exactly as it first appeared in the book, and some of it is downright awkward. Sorry! The publisher converted the original files to Word for me, perhaps they forgot to select the "high-def" option.
      - The Java package described in the book (lotus.notes) was superseded by lotus.domino in R5 (1999? I think). The methods are all mostly the same. Of course, IBM have added lots more classes and methods than existed when the book was written. Also, the "recycle()" method had not yet been invented in Domino 4.6, so the memory issues that call addresses are not really covered.
      - The figures (screen shots and so on) referred to did not make it into any file format I can deal with, so I had to leave them out. Sorry! The code samples are mostly there, and I'll also post all of the files that were on the CD that came with the original book.

      This Installment


      I'm including 3 chapters in this post:

      Chapter 6 (NOI Part 5): Chapter 06.pdf

      Chapter 7 (Writing NOI Applications): Chapter 07.pdf

      Chapter 8 (Writing NOI Agents): Chapter 08.pdf

      Release Notes


      Chapter 6, Page 195. Description of Registration.addUserProfile() mistakenly refers to Database.createProfileDocument(). Should be setProfileDocument()

      Chapter 7, no bugs!

      Chapter 8, Page 265. Listing 8.3 ("Waiting for Child Threads, Ex83Multi.java). Sigh. This code has a major race condition in it that I didn't catch until after the book was out. Basically, the problem is that all of the threads could finish before you get to the logic that tests to see if any of them are alive. There's a better and much more robust pattern for doing this stuff that I could write a separate article about, if there's interest.

      Enjoy! More soon. Geek ya later

      (Need expert application development architecture/coding help?  Want me to help you invent directory services based on RDBMS? Need some Cloud-fu or some web services? Contact me at: bbalaban, gmail.com)
      Follow me on Twitter @LooseleafLLC
      This article ©Copyright 2011 by Looseleaf Software LLC, all rights reserved. You may link to this page, but may not copy without prior approval.

      "Programming Domino With Java": Chunk the Second

      Bob Balaban  November 9 2011 04:28:19 PM
      Greetings, Geeks!

      Here's the second installment of the book. Thanks again for all the positive feedback.
      The first installment can be found here

      All of the book content (as is all of the content on this blog) is Copyright 1998 and 2011 by Looseleaf Software, Inc. You may not reproduce or distribute the book's content without permission from me.

      Some Caveats and explanations:

        - This book is now 12 years old. There are lots of things in it that are amusingly dated, even laughable. One example: the absurd emphasis on Java Applets. They died an unlamented death due to bloating a long time ago, but at the time I wrote the book, they were a hot technology.
        - The formatting is not exactly as it first appeared in the book, and some of it is downright awkward. Sorry! The publisher converted the original files to Word for me, perhaps they forgot to select the "high-def" option.
        - The Java package described in the book (lotus.notes) was superseded by lotus.domino in R5 (1999? I think). The methods are all mostly the same. Of course, IBM have added lots more classes and methods than existed when the book was written. Also, the "recycle()" method had not yet been invented in Domino 4.6, so the memory issues that call addresses are not really covered.
        - The figures (screen shots and so on) referred to did not make it into any file format I can deal with, so I had to leave them out. Sorry! The code samples are mostly there, and I'll also post all of the files that were on the CD that came with the original book.

      This Installment


      I'm including 3 chapters in this post:

      Chapter 3 (NOI Part 2) -- was supposed to be in the previous posting, but ooops. Sorry: Chapter 03.pdf

      Chapter 4 (NOI Part 3): Chapter 04.pdf

      Chapter 5 ( Yes! You guessed it...NOI Part 4): Chapter 05.pdf

      Don't worry, the chapter titles get a little more inventive starting with Chapter 7.

      Release Notes


      Chapter 3:
      Page 98, All of the "Item.appendXXX" method names should have capitalized "Value"

      Page 113, discussion of "auto-update", it describes how you can use getNextDocument(), etc. "That'll work okay". It won't work ok, it'll potentially cause an infinite loop. Ooops.

      Page 117, Listing 3.6, the "for" loop starts an index at 0, should be 1. Same for the loop on Page 118 as well.

      Page 119, mentions a property on View named "IsHierarchical". There is no such property (or at least, there wasn't in 4.6)

      Chapter 4:
      No bugs!

      Chapter 5:
      No bugs!

      Enjoy! More soon. Geek ya later

      (Need expert application development architecture/coding help?  Want me to help you invent directory services based on RDBMS? Need some Cloud-fu or some web services? Contact me at: bbalaban, gmail.com)
      Follow me on Twitter @LooseleafLLC
      This article ©Copyright 2011 by Looseleaf Software LLC, all rights reserved. You may link to this page, but may not copy without prior approval.

      "Programming Domino With Java" - First Installment

      Bob Balaban  November 2 2011 05:50:15 PM
      Greetings, Geeks!

      Here's the first installment of the book. Thanks again for all the positive feedback.

      All of the book content (as is all of the content on this blog) is Copyright 1998 and 2011 by Looseleaf Software, Inc. You may not reproduce or distribute the book's content without permission from me.

      Some Caveats and explanations:

          - This book is now 12 years old. There are lots of things in it that are amusingly dated, even laughable. One example: the absurd emphasis on Java Applets. They died an unlamented death due to bloating a long time ago, but at the time I wrote the book, they were a hot technology.
          - The formatting is not exactly as it first appeared in the book, and some of it is downright awkward. Sorry! The publisher converted the original files to Word for me, perhaps they forgot to select the "high-def" option.
          - The Java package described in the book (lotus.notes) was superseded by lotus.domino in R5 (1999? I think). The methods are all mostly the same. Of course, IBM have added lots more classes and methods than existed when the book was written. Also, the "recycle()" method had not yet been invented in Domino 4.6, so the memory issues that call addresses are not really covered.
          - The figures (screen shots and so on) referred to did not make it into any file format I can deal with, so I had to leave them out. Sorry! The code samples are mostly there, and I'll also post all of the files that were on the CD that came with the original book.

      This Installment

      I'm including the Dedication/Preface, and Chapters 1 ("Programmability Overview"), 2 ("NOI Part 1") and 3 ("NOI Part 2"). "NOI" is "Notes Object Interface", what we called the "back-end classes" in the day.

      Preface/etc.: ded_ack_pref.pdf

      Chapter 1: Chapter 01.pdf

      Chapter 2: Chapter 02.pdf

      Release Notes


      Chapter 2:
      Page 15, "Refer to Appendix A for a diagram...", should be "Appendix B"

      Page 33, Listing 2.2, Missing closing curly-brace "}" right before the "catch" statement

      Page 58, description of Database.createCopy() method, should mention that the method does not copy data, only the design of the db

      Chapter 3:
      Page 98, All of the "Item.appendXXX" method names should have capitalized "Value"

      Page 113, discussion of "auto-update", it describes how you can use getNextDocument(), etc. "That'll work okay". It won't work ok, it'll potentially cause an infinite loop. Ooops.

      Page 117, Listing 3.6, the "for" loop starts an index at 0, should be 1. Same for the loop on Page 118 as well.

      Page 119, mentions a property on View named "IsHierarchical". There is no such property (or at least, there wasn't in 4.6)

      Enjoy! More soon.

      (Need expert application development architecture/coding help?  Want me to help you invent directory services based on RDBMS? Need some Cloud-fu or some web services? Contact me at: bbalaban, gmail.com)
      Follow me on Twitter @LooseleafLLC
      This article ©Copyright 2011 by Looseleaf Software LLC, all rights reserved. You may link to this page, but may not copy without prior approval.

      Anyone interested in me posting my 1998 book?

      Bob Balaban  October 29 2011 09:34:15 AM
      Greetings, Geeks!

      In 1998 I published a book titled "Programming Domino 4.6 With Java" (see the Amazon page here). It did fairly well, but of course it's a bit dated now, and it went out of print in 2000.

      While you can apparently get a used copy for $0.59 (plus shipping), I was thinking I might post the PDF version of the whole thing here on my blog (the rights to the book reverted to me when the publisher declared it out of print, so it's legal). There are bugs in there, which I suppose I could publish "release notes" for, too. I think it's still the best detailed description of the Java "back-end classes" for N/D, as that library existed in October, 1997.

      Any interest? If you'd like to see it, post a comment here. If you wouldn't like to see it, post a comment here.

      Geek ya later!