Today, something unexpected happened. I had the (somewhat unplanned and impromptu) pleasure of showing the ropes to the “new recruit” at work, a student here for a work term over summer break.
Now, we’re not necessarily doing that much coding here yet, as we’re still in the process of bringing back large portions of IT functionality in-house. We do, however, do a lot of software configuration, release management, testing/QA tasks, and, we are ramping up to use a major enterprise CMS to be able to create front-end content quickly (HTML/JS/CSS backed by JSP & EJB following OSGI structure).
I’ve always wondered in the back of my mind, if I were “in charge”, how would I more gently introduce the younger generation to the world of “enterprise programming”? Certainly the “enterprise world” is often significantly different, if not completely far-removed, from the real-world of cutting edge software development based on agile methodologies and lightweight web frameworks, co-developed with the customer in real-time, or implemented competitively overnight at a weekend hackathon. It is also far-removed from the naiively specialized world of “academic coding”, where “programming problems” (albeit sometimes very tricky ones) are assigned with a very clear set of up-front requirements and well-defined metrics for acceptance, where every assignment is given a certain amount of time to complete and graded for completeness and of course for “originality” or “ability-to-follow-the-book-without-copying” (where copying any minor component is seen as the devil’s work, labelled plagiarism, and ostracized).
Enterprise application development on the other hand, often times has no clear-cut requirements, no well-defined acceptance criteria (other than customer happiness) and is both behind schedule and over-budget before coding even begins. That thing about the no copying? Yeah that’s tossed out the window in favour of cutting corners and “getting it to market” as quickly as possible, often at the expense of quality (or in some cases even the development team understanding the solution, the most recent case that comes to mind is this hilarious StackOverflow verbatim copy “programming faux pas” from a Nissan connected car developer). All that being said, enterprise application development isn’t that hard, just more complex and frustrating than greenfielding, open source work, or even consulting. So it turned out to be a good opportunity to take a stab at it, as the student in question only had a year of Computer Science so far and despite some exposure to Java had not much in the way of Web development yet as those courses were coming later in the program. He did however, have a healthy interest in the Gaming industry, an industry which is increasingly finding an audience and monetization options for its wares on Mobile and Web platforms.
“The only thing constant is change”
This blog post will summarize my recent Raspberry PI 2 – Canakit unboxing & configuration. I purchased the kit as a sort of “from-me-to-me” post-holiday gift as I have been thinking about doing so for quite some time and when the Raspberry PI 2 kits fell in price due to rumors of the pending release of the new Raspberry PI 3, I realized the time had finally come to dive into the world of PI.
The first thing you’ll notice if you’re looking to make the same sort of “pre-RPI3 release” purchase, is that this kit does not include the extra attachments required to do DIY electronics projects, and you would have to buy a separate components kit later or find the components piecemeal elsewhere. In hindsight, I would suggest to go with the full kit if you are even slightly thinking you might want to get into the electronics experiments. If you just need a really cheap computer or have a plan for a basic DIY Home Theater or DIY ROM/Emulator Gaming Console, then you can stick to the first cheaper starter kit and have pretty much everything you need (except possibly a few peripherals like console-specific USB controllers if you’re doing gaming).
I had no idea what I wanted to do, so I started out with the basic kit, which I got slightly cheaper on Amazon in a post-holiday sale than what the same kit is advertised for elsewhere.
Unboxing the Hardware
As soon as you take everything out of the box, ensure you have all the components needed. The official hardware guide shows you everything you need to begin (all of which should be included in your kit).
Ensure that you have:
- Raspberry PI (any model will do, but I personally chose 2… with 3 Bluetooth & WiFi are onboard so don’t need to take up USB slots)
- TV or Computer Monitor to hookup to initially (won’t need this later once we are done configuring)
- HDMI cable to plug into both the Raspberry PI and the TV/Computer Monitor
- USB Keyboard to type into the command-line (you’ll be typing alot initially!)
- USB Mouse to navigate to the required applications (and check out the menus and various apps installed)
- AC Adapter (power supply to turn it on)
- 8+ GB MicroSD card or SD card (depending on the Raspberry PI model you chose)
Without an OS, the Raspberry PI is just a curious box of chips and hardware with so much potential waiting to be tapped, but largely nothing more than a fancy paper weight. The first thing you’ll want to do is pick an OS and install it onto an SD card (or MicroSD card) so you can begin experimenting.
As the instructions on the official Raspberry PI site state, your best bet as a beginner will be Raspian (a Linux distro derived from Debian but designed and streamlined specifically for the Raspberry PI). I had trouble with the Operating System that came with the Canakit “Complete Starter Kit” and had to flash (wipe out and reformat) the provided MicroSD card in order to put a later and greater version on to the card. Once I did that, I was finally able to get through the boot process and get it installed. After you follow the steps in the official Software guide you should have done the following:
- (optionally) Flashed the provided SD or MicroSD card to start fresh
- Downloaded the ZIP file for NOOBS “OS installer” (offline version, or files for your chosen OS directly)
- Extracted the ZIP file to your computer to make it available to card
- Setup the NOOBS OS installer (or chosen OS directly) to your card
- Connected all the Raspberry PI hardware
- Gone through the NOOBS installer wizard on your Raspberry PI
- CHANGED THE DEFAULT USERNAME & PASSWORD (can’t stress this enough, do this before you connect it to the internet or even before opening up your device to Bluetooth connections for the first time)
- Connected to your local (home/office) WiFi on the PI (if you have a WiFi dongle or PI model with built-in WiFi)
- Tested your Bluetooth connectivity (if you have a Bluetooth dongle or a PI model with built-in Bluetooth)
- Restarted your PI and ensure the OS comes up successfully and saved all your configuration changes (if not troubleshoot)
CAUTION: Many in the PI community online have mentioned they corrupted their SD card during poweroffs before the PI had completely halted. For that reason you also have to be extra careful if you’re thinking of doing “portable/mobility PI” projects where you’d use an external battery pack or power source to enable the PI to be used on-the-go. When this happens you’ll need to reinstall everything and potentially lose all your files, so be sure to only power down once you have fully shutdown your PI. Thanks to it’s low power usage and heat radiance, most people just leave it running 24×7 once setup, but you don’t need to do this, just take care with the shutdowns and reboots. If you have alot of time-consuming changes or important work you’re doing with the PI that you’d be devastated to lose, remember to back everything up to another device and/or on a file-hosting service. I use a combination of backing up indivudal key files to my laptop, backing up the whole image to an external hard-drive, and all my code changes go to a Git repository (i.e. local, GitHub, BitBucket, etc). I don’t do the files or image that often, but at least once a month or more when I’m particularly active with it and alot of changes are made. Code will be saved for every commit though.
There are some things you’ll still want to configure, but you could stop here if you just wanted to make sure everything in your kit works correctly. For practical software and hardware/electronics development projects, you should proceed with the following useful configurations: Read the rest of this entry »
In the last post I setup the context for the HSVO project and how I winded up a member of it via my contract at NRC, setting the stage to help you understand all the partners and the objectives they had going in.
Now I’ll talk about the major problems faced on the project (also suffered by Healthcare IT & Medical School E-Learning initiatives in general). Hopefully you’ll quickly realize the importance of a Canonical Data Model (CDM) for any cross-organizational initiatives like this involving high levels of integration.
How we did it
NRC management was essentially the lead on the project as far as owning the largest piece being delivered (i.e. to provide the glue that would join all these separate devices being built by our research partners). After about 6-12 months of initial research, the plan formulated by my superiors before I was even hired on by NRC as an Application Developer to implement it, was to extend their existing SAVOIR 1.0 source code. It is common for government organizations like NRC to look to past work and projects/experiments from within their portfolio which can potentially be leveraged or extended beyond their initial expiry date. Nothing wrong with that, but its not always the best fit, especially when dealing with complex problems which might be better served by green fielding a totally new solution based on Agile development with direct interfacing with the customer (in our case, our research partners, but primarily NOSM who the entire project was tailored to in the name of the promises of “E-Learning for Remote Medicine”).
That said, SAVOIR 1.0 was basically a glorified application launcher which acted as a sort of dock that could run any application installed on your Operating System with a single-click. Think RocketDock but without any of the slick animations, and, implemented in Java so it could at least work cross-platform on Windows, Unix/Mac, or Linux. It was originally intended to simplify the lives of Architects as part of a separate project that finished up in 2008. Behind its code were also two basic Web Services:
- User Management (called “SAVOIR_UserMgmt“) this allowed the system to keep track of who was using the SAVOIR dock to run their applications and which applications they launched at which time. It also allowed system administrators to turn on an optional desktop-based login popup in Java if desired, to force specific users to login via username/password before using the applications in the SAVOIR dock; however of course if the applications were installed outside of SAVOIR’s install pack, then those applications could be launched anonymously as usual at any time, by going directly to their “exe” file on the system, or clicking on a shortcut (such as in their Program Files menu).
- Session Management (called “SAVOIR_SessionMgmt“) this allowed a unique identifier to be attached to each running instance of the SAVOIR dock, regardless of whether or not anyone was actually logged in, or if the login feature was turned on by the system administrator for the network on which SAVOIR was running.
We delivered what could really be called SAVOIR 1.5 after approximately my first 6 months on the job, with some heavy modifications to the core Java Swing GUI including the ability to drag & drop to the dock or slide/move applications up and down in launch order, and, the ability to also launch particular websites/webapps via the browser for the first time. This browser launch feature was probably the one we were most proud of, and only became reasonably easy to do in Java 1.6+. Now, any link could be placed on the Dock, either of the form:
- file://path/application.exe (for locally installed apps)
- http(s)://domain:port/path/#hash?param1=abc¶mN=etc (for apps on the web)
This gave the SAVOIR launcher a great deal of flexibility, we even developed a couple canned demos of the new capability which we thought would delight our partners. By being able to popup a simple search window (also in Java) and navigate straight to deep search results of the CMA Guidelines Infobase, PubMed medical journal archive, Wolfram Alpha as a calculation tool, and Wikipedia as a general info resource, as per the user’s selection.
The reception was lukewarm at best though, which came to a bit of a surprise to me after all the hard work I had put in, doing exactly what my superiors had requested. What I began to learn was that there was a growing disconnect between what our partners (covered in my last post) were expecting and. What our partners were telling us, was that they did not simply want a customized version of SAVOIR 1.0 rather what they wanted was a more unique and application that basically not just got out of the way so they could use their applications in a particular order, but they wanted something that could do alot more heavy lifting and facilitate communication between their separate applications.
With two applications, the integration is easy. Expose one or both via API and send data one or two ways as needed:
Not quite. Even with proper use of JMS for messaging and an ESB to broker transformations, what we learned the hard way is that despite all the hype of the ESB providers, you can’t expect to just plug in the services to an ESB and walk away launghing, problem solved (its possible that certain members of our team who sha’nt be named bought this notion a little too much though). You simply won’t be able to avoid the need to gain at least a rudimentary amount of working knowledge of each of the given devices or APIs you want to integrate, before you can even try to do anything meaningful with the data inputs they send and outputs they can receive. At first the reaction was frustration, then a rash decision to implement a Rule Engine (in our case Drools) which could live on the ESB to help determine the routing of messages based on business logic defined by our HSVO partners. We struggled with this, not just the implementation and integration of the Rule Engine as yet another endpoint on the ESB and whether that was helping us or hurting us in the long-run. Perhaps most significantly what we struggled with was how to simplify the Domain-Specific Language (DSL) Drools required as input down to a simple enough format that it could be either hand-coded via Excel (one of Drools’ supported input formats) then uploaded periodically before running E-Learning classroom simulations (aka “Scenarios” as the team dubbed them), or, to a consistent enough format that the rules could be automatically generated in the backend by an Authoring Tool (yet another piece of new technology that would have to be built to support this growing monstrosity). So much for an application launcher huh? We went through several student placements but none of them made a very significant dent in the Rule Engine piece, so it was up to myself and Roger Sanche at NOSM to come up with a way to get this beast working. So that interaction in one application/service/device would finally be able to fire-off events in a predictable way to the Message Bus and get routed through in support of multiple learners. It was one of the hardest things I’ve ever done but after many late nights we pulled off a working integration that would show the recipe. We called it the SAVOIR SDK, mostly because we had spent our creative reasoning powers and were too exhausted come up with anything else.
What we realized is that if we could apply Sturgeon’s Law (aka. KISS, DRY, YAGNI) and simply start thinking of each of the complex devices we were trying to integrate as nothing more than differing set of inputs/outputs, with some unique endpoint “addressing needs” based on their own scalability and protocols (i.e. HTTP based web app that can support thousands of users at once, or, a UDP camera that only one person or classroom could use at a time) then we could start to see a different more manageable picture. Finally something similar to the ESB simplification we were promised started to emerge; to get SAVOIR 2.0 anywhere near our partners’ desired level of integration, we needed to create just three simple CDM and thin wrappers (SDKs) for our partners to use to send and receive messages in a single consitent format. I’ll be the first to admin, its such a simple concept. To this day I wish I’d bought the EAI book sooner, but it was a chance landing on the book’s website that lead us down the right path:
Read the rest of this entry »
Today marks the 5-year anniversary of the conclusion of my 2 year on-site contract for NRC which ran from November 3rd, 2008 to November 3rd, 2010. As I believe all the NDAs on the project we worked on have expired, I’ll be looking back at what that project was all about, what its challenges were, and (in a separate follow-on post) how we solved them.
With over 20 research facilities in nearly as many different cities across the country, the National Research Council of Canada (NRC) is our nation’s largest government-sponsored, citizen-operated science & technological research organization.
How I got there
Stephen MacKay (then Research Group Leader at NRC-IIT Fredericton) would become the first representative of NRC I would meet in-person during my early career. It was my final summer of freedom, towards the end of my “startup run” after Grad School in Japan, where I had spent a year in between the conclusion of my Sony internship (2007-10-12) up to my eventual hiring at NRC (2008-11-03). For that year, I had been a self-starter entrepreneur living on the thinnest of shoestring budgets. I had been trying my best to setup a business around online video that I was convinced would change the way we monetize content and behavior online more fairly in favor of content creators, and attempting to evangelize a model where site loyalty would be rewarded properly (as I still think it should be, although making a go of it I’ll admit requires not just initial funding but you must guarantee a certain scale which is the real trick, if that original revenue-share idea is to be sustainable). Think about that “not yet” failed business/technology as Blockchain but the rewards of a “BitCoin” don’t go out for solving a cryptographic hashing algorithm, rather for duration of video viewing and in particular ecosystem ad/partner economic interaction. Apologize for the digression, but it’s important to set the tone though, as I had just given it my all and built a product still in search of an audience (hah even to this day).
In any case, I met Stephen MacKay at the 2008 CNSR Conference which by chance took place in Halifax, NS allowing me to attend on my limited “wannabe entrepreneur” budget. I had already spent the last of my “petty cash” booking a June flight to Toronto, to speak at MobileMonday Toronto, an effort which I had hoped would expose me to some potential wealthy investors in the big city. Stephen was interested in my work and saw my potential, we exchanged cards and he said to look him up and that there might be some opportunities to work with NRC coming up in the near future. I continued my long 18-20 hour days working on “the dream” throughout that summer until my 25th birthday in October, by which time I had promised my parents and most importantly myself that I would have either secured funding, or, bitten the bullet and started looking for a full-time job.
So it was with great resignation that I finally desperately reached out to Stephen to follow-up on our meeting at CNSR to see what opportunities he mentioned might be available. I had at first just attempted to gain an audience to pitch the business idea and early prototype to NRC as a potential for their “new business incubation lab”. Stephen however urged me to look into some of the recent full and part-time contract postings and working with NRC that way, as it was unlikely that they could take on any funding relationship with such an early stage startup (to this day I don’t think they take enough risks on startups but prefer helping fairly established incumbents build out their products or research new ones, but that’s another story). I found the Application Development position profile to be quite interesting and relevant to the work I had been doing. They wanted someone who knew multimedia/video-conferencing (had been in deep research on online & mobile video for over 2 years), who knew SOA and how to integrate web services (had been integrating YouTube, GoogleSearch, Flickr, PayPal and several other key APIs to my startup), and who could program in Java (focused on Java for 4 years in University and 1-2 years of practical on-the-job work afterwards).
In fact, when I really think back though it was actually one of my would-be interviewers Bruce Spencer, my eventual supervisor, whom was the first person at NRC that I corresponded with. I had briefly contacted him in 2006 while working on my thesis at the International University of Japan. That work focused on Semantic Web technologies and how they might benefit the growing MobileTV (now more commonly known as OTT) market. Bruce was a leading academic in Atlantic Canada for accomplishing intelligent content recommendations in the RACOFI project that lead to the InDiscover music recommendation service which got sold to Bell Labs in 2007. This was what led me to email him via my 4th year “CS Advanced Topics – Semantic Web” teacher at Acadia University (Dr. Andre Trudel) who was my first go-to for Semantic Web questions, as I worked through my thesis in Japan. So quite a long-winded explanation, but that’s how I wound up at NRC for 2 years of my life. The rest will tell the story of what happened while I was there.
According to their own website, their purpose is to be:
“the Government of Canada’s premier research and technology organization (RTO).”
They intend to accomplish this by:
“Working with clients and partners, to provide innovation support, strategic research, scientific and technical services to develop and deploy solutions to meet Canada’s current and future industrial and societal needs”
In many ways in fact NRC really acts as the public-facing version of the Military-focused government research department known as the Defence Research and Development Canada which of course researches and develops technologies and solutions for our navy, military and airforce in cooperation with established military-industrial provider companies (Aerospace, Transport, Armour, Ballistics/Weapons, Threat Detection, etc).
Within that operating model there are a few other similar departments (check out the full list of Canadian government departments) but based on my understanding the different the two which stand out differ as follows:
- Innovation, Science and Economic Development Canada (ISED) – focuses on investments in domestic companies and research projects at a large-scale, particularly those at late-stages to assist with implementation and real-world roll-outs, to realize immediate benefits to Canadians (i.e. jobs-based investments, or practical short-term ROI projects needing a help to finalize projects already underway)
- International Development Research Centre (IDRC) – focuses on investing in projects beyond our borders with large-scale benefit potentials, particularly those within developing countries
- Natural Sciences and Engineering Research Council of Canada (NSERC) – focuses on working with academics and students at at post-secondary education institutions (Universities, Colleges, etc) and helping find companies to encourage joint-investment from into academic research projects with potential to deliver innovations to Canadians.
See slides 22-29 for a nice summary of the NRC’s role in HSVO:
Read the rest of this entry »
While backing up some files to an external hard drive from my older one, I recently came across an old Cover Letter which had accompanied my resume in a job application I sent to Research In Motion (RIM) back in Summer of 2008.
It’s interesting to look back at, because I recall back then being very frustrated with the state of the Mobile industry in North America (particularly in Canada). These feelings were only magnified by my time spent in Japan 2006-2008, a country which at that time was a clear leader in Mobile technologies and in the global consumer electronics in general. Since then, Korea and China (two other countries I was fortunate enough to have spent some time in during my Graduate school vacations in between terms) have now caught up in terms of innovation and even surpassed Japan’s leading Mobile technology companies in sales as well.
Back then companies (again particularly China & Korea but many European firms as well) were sending some of their top experts and technologists to Japan to do market research with and/or attempt to poach talented Japanese engineers from, the likes of world leading Japanese tech companies: Sony-Ericsson, Panasonic, Sharp, Toshiba, Hitachi, Mitsubishi, Fujitsu, Fuji-Xerox, Konica-Minolta, Nintendo, Softbank, NTT, KDDI, etc. The goal was of course to glean as much information and consumer insights as possible from the country which boasted the fastest home fiber internet speeds, mobile internet speeds, mobile data usage, and mobile revenue per unit (ARPU) in not just gaming which usually comes to mind when thinking of Japan, but all application sectors.
My experience in Japan indeed taught me a thing or two about “sticky” services, particularly the infamous “iMode business model” by Takeshi Natsuno of NTT DoCoMo which succeeded by providing a cohesive ecosystem of applications and a flat-rate (about $40 USD/month) unlimited data service, which drove subscriptions through the roof. On top of this, standards and specifications which were simple to follow for developers and which reduced page-size for web content with cHTML then later WAP/WML helped grow the service’s offerings in an organic way. Only now are we starting to see the same sorts of initiatives by Google [LINK] & Apple [LINK] in North America. Over here, very little regard has been made for how to optimize mobile services for users, which is why our Mobile industry is only now catching up to and finally surpassing where Japan was 9-10 years ago. Indeed, we constantly hear reports about the growth of Online/Mobile Video (i.e. streaming ad-supported content like YouTube, Vimeo, etc), On-Demand/IPTV (i.e. rentals or purchases on iTunes, GooglePlay, etc), and OTT (i.e. subscription services like Netflix, Hulu & Amazon Prime Instant Video). However, MobileTV via OneSeg had already reached millions of users whereas SMS texting was just starting to take off in North America (in any meaningful way that resembles its adoption level today).
BC$ = Behavior, Content, Money
The goal of the BC$ project is to raise awareness and make changes with respect to the three pillars of information freedom - Behavior (pursuit of interests and passions), Content (sharing/exchanging ideas in various formats), Money (fairness and accessibility) - bringing to light the fact that:
1. We regularly hand over our browser histories, search histories and daily online activities to companies that want our money, or, to benefit from our use of their services with lucrative ad deals or sales of personal information.
2. We create and/or consume interesting content on their services, but we aren't adequately rewarded for our creative efforts or loyalty.
3. We pay money to be connected online (and possibly also over mobile), yet we lose both time and money by allowing companies to market to us with unsolicited advertisements, irrelevant product offers and unfairly structured service pricing plans.