BCmoney MobileTV

In the last post I set the context for the HSVO project and how I winded up a member of it via my contract at NRC, setting the stage to help any potential readers understand all the partners and the objectives they had going in.

Now I’ll talk about the major problems faced on the project (also suffered by Healthcare IT & Medical Schools all over the world and in any shared E-Learning initiatives in general). Hopefully, any time you need to have so many partner systems work together based on the anecdotes shared here, you’ll quickly realize the importance of establishing an Electronic Data Interchange (EDI) amongst your partners, specifically, a simple to parse Canonical Data Model (CDM). These constant communication and data formats must form the foundation for any cross-organizational initiatives like this involving high levels of integration between two or more devices, services, systems, apps and/or courses.

 

How we did it

NRC management was essentially the lead on the project as far as owning the largest piece being delivered (i.e. their mission assigned by the HSVO was to “provide the glue that would join all these separate devices being built by our research partners”). After about 6 months of initial research, the plan formulated by my superiors before I was even hired on by NRC as an Application Developer to implement it, was to extend their existing SAVOIR 1.0 source code. It is common for government organizations like NRC to look to past work and projects/experiments from within their portfolio which can potentially be leveraged or extended beyond their initial expiry date and use cases. Nothing wrong with that, but its not always the best fit, especially when dealing with complex problems which might be better served by green fielding a totally new solution based on Agile development using direct interaction with the customer (in our case, our research partners, but primarily NOSM who the entire project was tailored to in the name of the promises of “E-Learning for Remote Medicine”).

That said, SAVOIR 1.0 was basically a glorified Application Launcher which acted as a sort of dock that could run any application installed on your Operating System with a single-click. Think RocketDock but without any of the slick animations, and, implemented in Java so it could at least work cross-platform on Windows, Unix/Mac, or Linux. It was originally intended to simplify the lives of Architects as part of a separate project that finished up in 2008. Behind its code were also two basic Web Services:

• User Management (called “SAVOIR_UserMgmt“) this allowed the system to keep track of who was using the SAVOIR dock to run their applications and which applications they launched at which time. It also allowed system administrators to turn on an optional desktop-based login popup in Java if desired, to force specific users to login with their username/password before using the applications in the SAVOIR dock; however of course if the applications were installed outside of SAVOIR’s install pack, then those applications could be launched anonymously as usual at any time, by going directly to their “exe” file on the system, or clicking on a shortcut (such as in their Program Files menu).
• Session Management (called “SAVOIR_SessionMgmt“) this allowed a unique identifier to be attached to each running instance of the SAVOIR dock, regardless of whether or not anyone was actually logged in, or if the login feature was turned on by the system administrator for the network on which SAVOIR was running.

We delivered what could really be called SAVOIR 1.5 after approximately my first 6 months on the job, with some heavy modifications to the core Java Swing GUI including the ability to drag & drop to the dock or slide/move applications up and down in launch order, and, the ability to also launch particular websites/webapps in the browser for the first time. This browser launch feature was probably the one we were most proud of, and only became reasonably easy to do in Java 1.6+. Now, any link could be placed on the Dock, either of the form:

• file://path/application.exe (for locally installed apps)
• http(s)://domain:port/path/#hash?param1=abc&paramN=etc (for apps on the web)

This gave the SAVOIR launcher a great deal of flexibility, we even developed a couple canned demos of the new capability which we thought would delight our partners. By being able to popup a simple search window (also in Java) and navigate straight to deep search results of the CMA Guidelines Infobase, PubMed medical journal archive, Wolfram Alpha as a calculation tool, and Wikipedia as a general info resource, as per the user’s selection.

The reception was lukewarm at best though, which came to a bit of a surprise to me after all the hard work myself and the team had put in, doing exactly what our superiors had requested. What I began to learn was that there was a growing disconnect between what our partners (who these partners were is covered in my last post) were expecting and what we were delivering. What our partners were telling us, was that they did not simply want a customized version of SAVOIR 1.0 rather what they wanted was a more unique application with some intelligence that basically not just got out of the way so they could use their applications in a particular order, but they wanted something that could do alot more heavy lifting and facilitate communication between their separate applications.

A brief word on EAI
Enterprise Application Integration (EAI) is the dilemma we face when trying to integrate large complex applications (but the cocepts really apply to applications of any size). With two applications, the integration is easy. Expose one or both as an API and send data, one or two ways, as needed:

It quickly becomes tough to manage as you add in more applications to support/integrate:

If you have any kind of external scalability requirement to support alot of external services/devices, just forget about point-to-point integrations:

Of course the ESB vendors like MuleSoft, WS02, TIBCO and even Apache ServiceMix evangelists promise their ESB products will instantly make your EAI efforts look and feel like this:

Not quite. Even with proper use of JMS for messaging, HTTP for addressing, and an ESB to broker transformations, what we learned the hard way is that despite all the hype of the ESB providers, you can’t expect to just plug in the services to an ESB and walk away laughing, problem solved (its possible that certain members of our team who sha’nt be named bought this notion a little too much though). You simply won’t be able to avoid the need to gain at least a rudimentary amount of working knowledge of each of the given devices or APIs you want to integrate, before you can even try to do anything meaningful with the data inputs they send and outputs they can receive. What that also means is that you’ll need to gain some basic domain knowledge and/or work directly with a domain expert in the initial design stages (akin to Agile development). We lacked that domain expertise at NRC, and at first management also relucted on opening direct lines of communication between our developers and main clients (which from the broader group were NOSM and McGill). At first the reaction was all around frustration, then lots of talk of from the medical experts on our project needing something they were calling “The Eye of Sauron” followed by a rash decision to implement a Rule Engine (in our case Drools) which could live on the ESB to help determine the routing of messages based on business logic defined by our HSVO partners. We struggled with this, not just the implementation and integration of the Rule Engine as yet another endpoint on the ESB and whether that was helping us or hurting us in the long-run. Perhaps most significantly what we struggled with was how to simplify the Domain-Specific Language (DSL) Drools required as input down to a simple enough format that it could be either hand-coded in Excel (one of Drools’ supported input formats) then uploaded periodically before running E-Learning classroom simulations (aka “Scenarios” as the team dubbed them), or, to a consistent enough format that the rules could be automatically generated in the backend by an Authoring Tool (yet another piece of new technology that would have to be built to support this growing monstrosity). Next came discussions about the need for a “Rosetta Stone” behind the scenes of SAVOIR, which I codified as a “Term Dictionary” which was capable of mapping concepts between endpoints (i.e. Blood Pressure in one system may be the input “BP”, but in another system it might be “kPa”; in some it could be split by S-Systolic and D-Diastolic while others it could be together as “S/D mmHg”). Finally, a need for a “Unit Converter” to convert different medical units of measurement (i.e. metric to imperial to bridge gaps from Canada/UK/Ireland to United States, differences between Med School SOAP notes, etc). So much for an Application Launcher huh? We went through several student placements but none of them made a very significant dent in the Rule Engine (aka. Eye of Sauron) piece, or, the Message Translator/Mapper/UnitConverter (aka. Rosetta Stone); so it was up to myself and Roger Sanche at NOSM to come up with a way to get this beast working. Working with the domain experts daily we realized the main problem. Interaction in one application/service/device must be able to fire-off events in a predictable way to the Message Bus and get routed through to the next application/service/device yes, but we are doing this in support of multiple learners at multiple locations, this was the key. It was one of the hardest things I’ve ever done but after many late nights we pulled off a working integration that would show the recipe. We called it the SAVOIR SDK, mostly because we had spent our creative reasoning powers and were too exhausted come up with anything else.

What we realized is that if we could apply Sturgeon’s Law (aka. KISS, DRY, YAGNI) and simply start thinking of each of the complex devices we were trying to integrate as nothing more than differing set of inputs/outputs, with some unique endpoint “addressing needs” based on their own scalability and protocols (i.e. HTTP based web app that can support thousands of users at once, or, a UDP camera that only one person or classroom could use at a time) then we could start to see a different more manageable picture. Finally we could realize something similar to the ESB simplification we were promised; to get SAVOIR 2.0 anywhere near our partners’ desired level of integration, we needed to create just three simple CDM and thin wrappers (SDKs) for our partners to use to send and receive messages in a single consitent format. That simple realization finally got us the big breakthrough we needed, and a matching of Scenario (path to scenario being worked through), sessionID (learner unique identifier), and instanceID (learner’s device’s OS/browser/tab identifier) as attributes of the services we were describing would be the last step to the messaging delivery problem. I’ll be the first to admit, its such a simple concept. To this day I wish I’d bought the infamous EAI book sooner, but it was a chance landing on the book’s website that lead us down the right path. Check out the schema we put together:

• DTD
• XSD
• XML

 

savoir.dtd is the most basic possible summary of the Data Format, our first quickly thrown together CDM prototype.

<!ELEMENT message (service)>
<!ATTLIST message
          id CDATA #REQUIRED
          version CDATA #REQUIRED
          scenario CDATA #OPTIONAL
          sessionID CDATA #OPTIONAL>
<!ELEMENT service (parameter+)>
<!ATTLIST service
          id CDATA #REQUIRED
          command CDATA #REQUIRED
          endpoint CDATA #REQUIRED
          instanceID CDATA #OPTIONAL
          tiimestance CDATA #OPTIONAL>
<!ELEMENT parameter (#PCDATA)>
<!ATTLIST parameter
          name CDATA #REQUIRED
          value CDATA #REQUIRED
          unit CDATA #OPTIONAL
          unit_type CDATA #OPTIONAL>

 

savoir.xsd The more comprehensive XML Schema enabled strict validation of each message.

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">
  <xs:element name="message">
    <xs:complexType>
      <xs:sequence>
        <xs:element ref="service"/>
      </xs:sequence>
      <xs:attribute name="id" type="xs:string" use="required"/>
      <xs:attribute name="version" type="xs:string" use="optional"/>
      <xs:attribute name="scenario" type="xs:string" use="optional"/>
      <xs:attribute name="sessionID" type="xs:string" use="optional"/>
    </xs:complexType>
  </xs:element>
  <xs:element name="service">
    <xs:complexType>
      <xs:sequence>
        <xs:element ref="parameter" maxOccurs="unbounded"/>
      </xs:sequence>
      <xs:attribute name="id" type="xs:string" use="required"/>
      <xs:attribute name="command" use="required">
        <xs:simpleType>
          <xs:restriction base="xs:string">
            <xs:enumeration value="register"/>
            <xs:enumeration value="load"/>
            <xs:enumeration value="status" />
            <xs:enumeration value="info" />
            <xs:enumeration value="start" />
            <xs:enumeration value="pause" />
            <xs:enumeration value="resume" />
            <xs:enumeration value="forward" />
            <xs:enumeration value="backward" />
            <xs:enumeration value="left" />
            <xs:enumeration value="right" />
            <xs:enumeration value="up" />
            <xs:enumeration value="down" />
            <xs:enumeration value="record" />
            <xs:enumeration value="stop" />
            <xs:enumeration value="exit" />
          </xs:restriction>
        </xs:simpleType>
      </xs:attribute>
      <xs:attribute name="endpoint" type="xs:anyURI" use="required"/>
      <xs:attribute name="instanceID" type="xs:string" use="optional"/>
      <xs:attribute name="timestamp" type="xs:dateTime" use="optional"/>
    </xs:complexType>
  </xs:element>
  <xs:element name="parameter">
    <xs:complexType>
      <xs:attribute name="name" type="xs:string" use="required"/>
      <xs:attribute name="value" type="xs:string" use="required"/>
      <xs:attribute name="unit" type="xs:string" use="required"/>
      <xs:attribute name="unit_type" type="xs:string" use="required"/>
    </xs:complexType>
  </xs:element>
</xs:schema>

 

savoir.xml sample XML file

<?xml version="1.0" encoding="UTF-8"?>
<message id="SAVOIR" version="2.0.0"  scenario="http://192.168.2.10/hsvo/scenario/a-bridge-too-far/" sessionID="2n10o4lk2l24l2r39dsa23" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="file:///C:/APPS/SAVOIR.xsd">
  <service id="OpenLabyrinth" command="load" endpoint="http://vp.openlabyrinth.ca/renderLabyrinth/index/" instanceID="129v83a72f1f8ea320w91i9e83212" timestamp="2010-11-01T13:35:45">
    <parameter name="node" value="33" unit="utf-8" unit_type="URL_PARAM"/>
    <parameter name="userID" value="bcmoney" unit="username" unit_type="TEXT"/>
    <parameter name="password" value="***********" unit="hash" unit_type="PASSWORD"/>
    <parameter name="patient" value="Mr. T*********" unit=" " unit_type="TEXT"/>
    <parameter name="HR" value="85" unit="bpm" unit_type="VITAL_SIGN"/>
    <parameter name="RR" value="17" unit="ipm" unit_type="VITAL_SIGN"/>
    <parameter name="BP_S" value="165" unit="mmHg" unit_type="PRESSURE"/>
    <parameter name="BP_D" value="95" unit="mmHg" unit_type="PRESSURE"/>
    <parameter name="IT" value="38.2" unit="celcius" unit_type="TEMPERATURE"/>
    <parameter name="ET" value="19.7" unit="celcius" unit_type="TEMPERATURE"/>
    <parameter name="IV" value="50" unit="ml" unit_type="VOLUME"/>
    <parameter name="DP" value="8.5" unit="mg" unit_type="MASS"/>
    <parameter name="gender" value="Male" unit=" " unit_type="GENDER"/>
    <parameter name="age" value="59" unit="years" unit_type="TIME"/>
    <parameter name="height" value="179" unit="cm" unit_type="LENGTH"/>
    <parameter name="weight" value="88" unit="kg" unit_type="MASS"/>
    <parameter name="BMI" value="27.5" unit=" " unit_type="VITAL_SIGN"/>
    <parameter name="timezone" value="-04:00:00 UTC" unit="AST" unit_type="TIMEZONE"/>
    <parameter name="IncidentOccurred" value="2010-03-29 18:45:00" unit="yyyy-mm-dd hh:mm:ss" unit_type="TIME"/>
    <parameter name="EmergencyCallStart" value="2010-03-29 19:18:46" unit="yyyy-mm-dd hh:mm:ss" unit_type="TIME"/>
    <parameter name="ParamedicsDispatched" value="2010-03-29 19:22:31" unit="yyyy-mm-dd hh:mm:ss" unit_type="TIME"/>
    <parameter name="EmergencyCallEnd" value="2010-03-29 19:25:03" unit="yyyy-mm-dd hh:mm:ss" unit_type="TIME"/>
    <parameter name="HospitalArrival" value="2010-03-29 19:41:07" unit="yyyy-mm-dd hh:mm:ss" unit_type="TIME"/>
    <parameter name="ExamTime" value="00:00:00" unit="hh:mm:ss" unit_type="TIME"/>
    <parameter name="DI_Costs" value="458.25" unit="$" unit_type="MONEY"/>
  </service>
</message>

That example shows a large example from the production server (from 5 years ago), and shows the full details available. Many of the fields are optional, so here’s more of a barebones example XML file:

<?xml version="1.0" encoding="UTF-8"?>
<message id="SAVOIR" version="2.0" sessionID="123456789abcdef" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://hsvo.org/schema/SAVOIR.xsd">
  <service id="BMI" endpoint="http://localhost/bmi/bus-interface/HSVOSavoirInterface.php" command="status" instanceID="PHPSESSID::vq3vnlrkuet09uuj4nlqp7b3f7">
    <parameter id="bmi" value="24" unit="" />
    <parameter id="height" value="1.75" unit="m" unit_type="LENGTH" />
    <parameter id="weight" value="75" unit="kg" unit_type="MASS" />
  </service>
</message>

I also (somewhat cheekily perhaps, at least according to my superiors who strongly preferred SOAP Web Services) created a JSON representation for the client-side. I felt this was out of necessity as storing XML in a Cookie doesn’t quite make as much sense as a simplified JSON version, and we needed cookies to persist sessions beyond accidental tab/window closes so it seemed like a useful addition. At the time I was surprised on how much resistance there was internally to doing things RESTfully (i.e standard formula of using JSON over HTTP via GET/POST/PUT/DELETE) rather than following strict XML-based SOAP Web Services (ws-*) methodologies, even where the technology wasn’t a natural fit.

savoir.json JSON-equivalent representation

{
    "message": {
        "service": {
            "id": "BMI",
            "endpoint": "http://localhost/bmi/bus-interface/savoir-wrapper.php?bmi=24&height=1.75&weight=75&mode=json",
            "parameter": [
                {
                    "id": "bmi",
                    "value": "24"
                },
                {
                    "id": "height",
                    "value": "179",
                    "unit": "cm",
                    "unit_type": "LENGTH"
                },
                {
                    "id": "weight",
                    "value": "88",
                    "unit": "kg",
                    "unit_type": "MASS"
                }
            ]
        }
    }
}

 

Example implementations
Once we had these file formats and parsers for data exchange standardized, everything else started to come together. With this simple yet robust CDM in hand we could then pass just about any type of data as parameters, from just about any service/device and instantly know the address of it thanks to its required endpoint attribute, be able to message it in real-time (via JMS, HTTP, TCP, UDP, FTP, etc) using its unique identifier, and stay up-to-date with the status of all its main parameters. What this gave us was simplicity in integrating new devices, as we also setup a registration mechanism which allowed us to “register” new devices by following this exact same XML format at discovery time. Device Registration, CHECK… Rule Engine message format consitency, CHECK… Authoring Tool capable of saving to the CDM, CHECK!!! While Roger and I finalized the OpenLabyrinth to SAVOIR deep integrations to enable any stage of the Virtual Patient pathways to be able to reach out to any device, Yonghua You (NRC) took over the Front-End UI and built out the Authoring Tool prototype I had started, porting it to Flex his preferred language, and using a useful Authoring Tool library that was available on that platform and at the time far surpassed anything available in the open source JavaScript world. Keep in mind this was all taking place shortly after the launch of Yahoo! Pipes, and to our knowledge nothing like that had ever been done to this scale in the medical industry before (at least not with this much modularity and reusability baked in, Pipes/YQL together was the only thing that even compared). Alas, we had the rest of our HSVO partners on board in a big way and it was starting to look good for continued funding.

Here are modified versions of the first three example reference implementations of the SAVOIR SDK we built. One was AJAX-based (HTML5/CSS3/JavaScript), one was PHP-based, and one was of course Java-based to be used within our middleware itself as a Message Router endpoint that would talk to services/devices via their choice of JMS, HTTP, TCP, FTP, etc.

File Launcher (JAVA)
This one was easy, I ported a simplified version of the code from the original (scrapped SAVOIR 1.5) concept and made it into a service reference implementation. It seemed like a no-brainer, as even though that “run via web or OS” capability was not the sole end-goal of the HSVO, as we learned the hard way, we still knew everyone liked the idea of being to launch anything on their OS or on the Web, on command, at any time exactly when desired. It was more like a fundamental building block. This was considered a service because it didn’t do much on its own, just listened for commands and fired off “run” events to either the Browser listener code or the OperatingSystem listener code.

-or-

Video Player (HTML5/AJAX)
This acted as a nice device reference implementation and was based on JW Media Player and its API. Initially it simulated the ECG device which was to be delivered and integrated at a later date by Cork University’s Medical School partners in Ireland. Basically it just played a sine-wave type of ECG simulation on repeat, until paused, stopped or closed. Being so heavily invested in Online/Mobile Video research and pushing the boundaries of what’s capable in the browser, I convinced the HSVO partners on the value of having both a pure JS Bus Interface SDK example and a baisc video player (complimentary to the more complex C++ based 3D Camera Array) to integrate alongside any E-Learning scenarios they were developing (if images say 1000 words, a well-done short video can truly tell a story to and share experiences with the student, many people use YouTube videos to learn how to do DIY everything from home repairs to pulling their kids’ loose teeth safely and everything in between).

-or-

BMI Calculator (PHP)
This would become the service reference implementation for how to expose your APIs in a manner that would facilitate integration to HSVO following the SAVOIR SDK. It simply calculated Body Mass Index (BMI) based on inputted height and weight, but had the ability to authenticate users and keep track of a particular UserID’s BMI over time.

-or-

 

Conclusion
In the end, we made our best go at an incredibly complex and ambitious project. Looking back, with the ability to integrate such a diverse spectrum of services/devices/apps, what we created was really an Internet of Things (IoT) platform before all this “IoT marketing hype” arose. We’re seeing every company starting to get into IoT this year (and the trend will probably continue to rise until “Connected Things” are as pervasive as the internet itself). Unfortunately, the HSVO group disbanded shortly after my contract ran up as the funding for NEP was completed at the end of 2010. Over the following calendar year most HSVO members would move on to new projects. Lots of experience was gained and lots of lessons were learned. Either way, the HSVO project was one of the largest, most expensive/well-funded, and most complex projects I’ve ever been a part of. To this day it is without a question definitely still the most widely geographically disbursed project I’ve ever been a part of, where I worked with developers from China, India & Signapore to researchers in Ireland to my own immediate team in North America, spread out from East to West coast of the United States and Canada.

As for the NRC, although it has suffered some budget cutbacks since my time there, and its scope fluctuates heavily between political changes in the federal government, I have to say NRC is an important tool for serving Canadians. They remain solely capable of doing the collaborative research that the general public simply cannot, due to financial, intellectual and geographical situations. Quite simply, employees there are capable of spending the in-depth research time, using top industry tools and connecting with industry thought leaders. All things which are simply out of reach for the average citizen or even for most private businesses due to their constant need to just “Keep The Lights On” (KTLO) by constantly keeping revenues coming in and doing tried and tested things to make money via sales of products or services, rather than experimentation and invention. With a shift to providing their “research as a public service” and their spaces as common research spaces rather than focusing too much on just private enterprises, I strongly believe NRC can continue to flourish and make their importance to Canadians felt. I see a bit of it happening as the place I worked in Fredericton has dedicated the whole first floor to being a “business incubation space” that no longer limits participation to big businesses; a step in the right direction, but where is the “come play with our technology & ask our experts questions space”? Something like that could go a long way…

My group at NRC had several publications, conference appearance credits and full coverage in conference proceedings related to the innovative work around SAVOIR and the HSVO project, here are a few I was credited or acknowledged in:

1. Health Training with SAVOIR and the RSM: https://www.computer.org/csdl/proceedings/skg/2009/3810/00/3810a262-abs.html
   IEEE – Semantics, Knowledge and Grid, International Conference proceedings (2009)
   Zhuhai, China
2. HSVO – a functional XML specification for integrating simulation devices: http://yadda.icm.edu.pl/yadda/element/bwmeta1.element.baztech-3f13e5e8-c7e1-4daa-84e9-486f54f07470
   Bio-Algorithms and Med-Systems (2010)
   Vol. 6, no. 11
3. Simulation Integration for Healthcare Education, Training and Assessment:
   https://www.researchgate.net/publication/224204377_Simulation_Integration_for_Healthcare_Education_Training_and_Assessment
   Digital Information Management (ICDIM) – Fifth International Conference (2010)