Creating IT Roadmaps, Gathering Data

This is a second post in a series of posts describing a technology roadmapping exercise I am completing. All the posts in this series can be found under my roadmap label for this blog. This post focuses on the how, why and where I am gathering data, with beginnings of how I am organizing and visualizing the data.

1. Narrow the subject area and context

This roadmap will focus on adults engaging in continuing professional development and life-long learners focusing on legal education for lawyers, legal assistants, notaries and self represented litigants. In general, the audience is focusing on accessing legal materials and related learning resources published from a number of online sources, both public and private.  The context for access is usually for researching a subject of personal or professional concern over a (long and short) period of time. The assumption being that the longer the duration the greater the depth. This does not mean that short bursts of access is not seeking depth of learning.

The main threat is within two areas. Firstly, in published materials. Not in the published materials being replaced, but the customers are expecting them being available on a new device which eases access geographically and 24x7, and allows greater collaboration around the published materials so they are more relevant and up to date. Customers will increasingly seek published materials being made available in this way. Second, is with online programs, courses, workshops, etc. Blended and online learning is growing and this eases the need to travel and allocated set blocks of time to attend learning events.

2. Know your audience

The audience are adult learners with a post-secondary level of education. Their learning styles are going to be constructivist with a strong influence from connectivist approaches. Increasingly these learners are looking for alternate ways to access learning materials. These alternatives are both geographic (reducing the need to travel and access from any device any time) and the ability to access learning resources 7 x 24. When a learner leaves working on one device the next device they resume their learning has knowledge of where they left off.

Understanding the technology adoption rates for your audience is very important. The challenge is finding data defining technology adoption rates for specific audiences and the adoption rates for the different demographic groups within the audience. If you have the resources doing surveys targeted toward your audience can be very helpful. Otherwise, staying aware of technology trends and bookmarking or tagging technology adoption is a good way to gather data. I have often tag resources related to technology adoption, they fall within my "roadmap" delicious tag, follow it here; http://delicious.com/prawstho/roadmap
 
3. Acknowledge that roadmaps are visual tools

Within this roadmap there are a number of different attributes that need to be represented in a single (well, potentially multiple) visual(s). As my research of these attributes deepened they began to fall into three main categories;

Pedagogical - events, ideas, new theories, approaches that relate to teaching and learning.

• emerging and existing learning theories
• emerging approaches to online learning and teaching
• social and collaborative technologies well suited to learning

Technological - current and emerging technologies well suited to and influencing adult learning.

• personal devices and browsers
• internet and technology platforms 
• application software well applied to learning

Sectoral - subject or business sector attributes to be considered or will influence the roadmap.

• strategic plan (known initiatives)
• financial & economic
• jurisdictional issues
• threats

Suggested Reading
http://www.downes.ca/me/mybooks.htm
http://www.nmc.org/horizon-project/horizon-reports
http://criticaltechnology.blogspot.com/search/label/roadmap

Embedding Google Docs

If you want to publish (and embed) a Google doc into a blog post this is easily possible as Google docs provides the embed code. The process of embedding informs you that the underlying data will be made public and read only. Which is kind of the point of blogging about some data you have created. Here is a chart I put together for a roadmapping exercise I am currently completing.



What are my data sources for putting together this graph? It is an accumulation of a number of things; 25 years working in technology and being an educator graduate level studies in Education with a focus on Information Technology, constant monitoring of RSS feeds, blogs, online publications, a deep curiosity of the subject of educational technology and the reading a number of reports on the subject, in particular; Horizons Reports and the exemplary work of Stephen Downes.

Learning Architect

Clive Shepherd gets it! What he is speaking of is closely aligned with my idea of a Learning Systems Architect from a while back. I would say the difference between the two is my role is more technical. I really need to read the book if I am get a complete sense of Clive's Learning Architect. From what I have read so far by browsing his companies site, reviewing his new books index, and listening to the embedded video the Learning Architect designs the pedagogical approaches and recommends the technology platforms to best support the learning. The Learning Systems Architect I speak of works with Subject Matter Experts to design the pedagogical approaches and implements (builds if necessary) the technology platforms to best support the learning. I would see the Learning Architect and the Learning Systems Architect working back-to-back; where the Learning Architect is facing toward the learner and the Learning Systems Architect is facing toward the technology. Regardless of how you see things, if you are into adult education this is a good video to watch.

On a closing note, I really appreciate the way he classifies learning into four approaches; formal, non-formal, on-demand and experiential. The Learning Architect role can be read about in his new book, "The New Learning Architect". Even a browse through the index and what technologies fall into the four approaches can provide insight into learning in the near future. Thanks Clive!

Rackspace Step 5: Updating the DNS

It's been a while since I posted on my work with moving all my sites over the rackspace, it's been summer and the start of the school year for my kids. The task I intermittently focused on through the summer was to move my domain name hosting over to rackspace. Its great that rackspace also provides a DNS based cloud service, and I like the management console available to manage your DNS.

 
Moving DNS servers may not be so simple
Usually you would think that changing DNS servers would be a simple, and it should be. Depending on where you start and who "controls" the ability to update, things may not go as smoothly as you would like. I mention this because without a good move of your DNS your site may disappear from the internet for a period of time. What I want to say is, "When moving your DNS it is important that you monitor the move closely". This is what happened to me and a similar series of events could happen to you;

1. I logged into my previous providers domain hosting console and changed the domain name server for the domain I was moving. I was prompted the save was successful.
2. I went back to the console to see what name servers were assigned to the domain, it was still the old names. I figured this was OK because name server changes need to be updated through-out the internet to truly complete.
3. A couple days later I logged into the domain hosting console to check the name associated with the name server of the domain. It was still set to the old name server. Naturally, I tried again to update it myself. And again I got a confirmation of the change.
4. I got busy and a few days later I checked the names again and my DNS was still pointing at the old name server. I wrote an email to tech support, sent it off and waited.
5. Almost immediately, I got confirmation of my query and was assigned a tracking number for the issue. A few days later nothing, so I phoned... I did speak to someone and they confirmed they had made the change, to the correct domain name. I was adamant about this and they confirmed the correct domain name.
6. The next morning I logged into my domain hosting console and discovered they had made the name server changes to the incorrect domain. 

There is really no point in going any further with this description, and eventually I got it all cleared up. Needless to say, all this was only confirming I was doing the right thing to be moving away from netnation as my hosting provider. Don't get me wrong, netnation has provided me with many years of very stable hosting. Its just my needs have changed and the cost savings provided by cloud based services are too strong to ignore. The main lesson learned is when making changes to things DNS related you need to monitor it very closely, particularly when their are intermediaries involved...

Suggested Reading
http://en.wikipedia.org/wiki/Domain_Name_System
http://www.rackspace.com/cloud/blog/2009/06/04/dns-the-overlooked-cloud-service/
http://www.rackspace.com/knowledge_center/index.php/Managing_DNS

Creating Information Technology Roadmaps, Getting Started

Creating technology roadmaps can be hard. Mostly because you are trying to predict the future. And predicting the future is, well, unpredictable. So coming up with a technology roadmap for a specific subject or practice area narrows the horizon and could increase success. Gaining as much knowledge of the narrowed area by reading, reviewing and referencing as much existing information and related predictions will help greatly. Essentially, you want to gather all the applicable technology and subject area information you possibly can regarding the present and future and try to gestalt a technology roadmap.

The important factors in creating technology roadmaps are;

1. Narrow your subject area and context

This is important mostly due to narrowing the number of attributes influencing the future. The subject area is somewhat self explanatory, is it; higher education, medical, financial, legal, etc. Context would be; mobile technology for adult customers, wealth management for families, etc.

One important attribute here would be to identify any serious threat(s) to the financial health of your organization due to a disruptive technology or competitor. These unforeseen threats rarely occur if you have been doing regular roadmapping... for they should identify the threats...

2. Know your audience

The audience who will reference the roadmap is important for they will read it based on their decision making needs. The audience can be as varied as; senior management, customers, business partners even competitors.

3. Acknowledge that roadmaps are visual tools

People have become used to roadmaps being visual tools, invest the time in finding a visual representation that suits your audience. Engage your audience early, present a visual framework and get feedback. Improve the visual. This has two benefits; it assists in the audience learning how to use the roadmap and assists the creator in understanding the audience and the issues of why they need the roadmap.

4. You don't know the destination, only important attributes of the journey

When predicting the future with a technology roadmap there is no destination other than the many factors that influence the decisions you make on the journey. The technology roadmap will provide a topographical map and the roadways that are available to you when making decisions. It is assessing the current location and the things of importance around you (which will change through time) that will determine which route to take.

5. Sometimes the journey is the destination

It is having to make the decisions about the journey that are the technology decisions required by the organization. Using the roadmap to know where you are and the current surroundings is the what the technology roadmap is for. It helps in making the technology decisions right in front of you, no more. Really that is what is needed anyhow.

6. The roadmap should align with the organizations vision and strategy

The roadmap should be derived from the organizations vision and strategy. If their is no vision or strategy this should be done before the roadmapping exercise has begun.

7. The technology roadmap will influence the organizations tactical plan

Also derived from the organizations visions and strategy are the tactical plan. The tactical plan and the roadmap work together to drive the individual projects tasked with implementing the vision and strategy.

Step 1. Start writing openly about these seven factors and how they apply to your roadmapping exercise. Be open and transparent about your thinking an seeking feedback is very important. Using an internal (or privately external) blogging approach and allowing people to comment would be a great way to be open, transparent and solicit input.

Step 2. Begin to gather all the technology roadmap material you can. Search high and low, contact your vendors, contact your peers, investigate industry publications, look for other technology roadmaps. Leave no stone unturned.

Step 3. begin to create a visual representation of what you are finding. Be creative, seek different sources for inspiration. Publish the visual frequently to begin soliciting feedback, and developing a shared understanding. It is the feedback and shared understanding that will improve the accuracy of the roadmap.

For a growing list of references on this subject feel free to follow my roadmap tag in delicious;
http://www.delicious.com/prawstho/roadmap

Follow-up Posts
If you have read this far you may be interested in the follow-up posts I have written that actually implement what I have described here;

• Creating IT Roadmaps; Gathering Data
• Creating IT Roadmaps; Pedagogical Trends
• Creating IT Roadmaps; Technological Trends

MVC in a three-tier architecture - TRANSLATED

A month back I wrote a post on architecting web and mobile based applications. In the post I spoke very technically about the MVC pattern and three-tier architectures. One of the comments I got on the post was from a very bright friend of mine who also works on educational technology and professional development, only from a senior management perspective. His comment was in really wishing he knew what I was talking about. And after reading the post again, I agreed that for a non-technical person the post would have been difficult to understand, I also felt there was good value in translating it for the non-technical person. Explaining the MVC pattern and three-tier architectures in this way would have great value to those who want (even need) to understand the web, mobile technologies and how these are put together. So this post attempts to answers that need... and if it works out, I may rewrite a number of my posts for the non-technical person.


Model-View-Controller

The Model-View-Controller (MVC) is a design pattern used to design the user interface and activities of a software application. In other words, what does a web page or mobile application look like and how does it work. How is the software designed so pictures, words, links and buttons show up, and what happens when someone clicks on a button or a hyperlink. The individual items of Model, View and Controller each serve a purpose; the Model is the business and related data and processes, the View is what is displayed to the end user and the Controller handles the events between what is displayed and how the business responds.

• The View is what is rendered (drawn) on the screen. How it is rendered is where the software part comes in. The technology (programming) behind rendering a View (or screen full of information) includes many options and approaches. Beyond a lot of formatting and graphic design, what makes the work behind the View is that there are many screens to that the software has to render a view. These screens come from different types of computers (PCs, Mobile, Tablets, Etc.), different browsers (firefox, safari, chrome, ie), different sized screens, Etc. Writing software for these different screens takes work and isolating the code into that responsible for only rendering the View greatly simplifies what can become complicated.
• The Model is the business logic or software that collects and retrieves any data required for the View. When data needs to be retrieved from or saved to the data storage it is the model which is responsible. The model can contain a lot of complicated business logic. As an example, someone submitting a credit card transaction may have only a few data fields and one button click on the view but the amount of different business activities (or software modules) that get utilized to complete the credit card transaction can be numerous and span different computers and businesses.
• The Controller responds to user events. The events can trigger changes or activities from both the View and the Model. As a simple example; a user enters their user credentials (user name, password) and clicks the log in button. The Controller then initiates a call to the Model which in turn executes the software required to find the user name and confirm the password... the Model then returns a call to the View with the pass or fail of the log in and the View re-renders itself with either a logged in user interface or the "forgot password?" user interface.

There you have it, simply put; the View is responsible for drawing the screen, the Model is responsible for retrieving and adding / updating information and the Controller is responsible for managing the events between the View, the end-user and the Model. Once you feel comfortable with your understanding of this MVC design pattern, move of to the next section describing the three-tier architecture. Hold off asking yourself how the Model-View-Controller (MVC) fits in the three-tier architecture for now. That will be discussed later.

Three-Tier Architecture

A design / concept of the three-tier architecture was created over 20 years ago, and was initially adopted when building client-server software. The idea being the personal computers on the network were the clients and the big computers on the network were the servers. And the clients made requests to the servers, the servers talked among themselves (business logic servers and database server) figured things out and then the servers responded back to the clients. This basic idea continues. Now the clients are web browsers and other devices, and the servers are web site servers, business logic servers and database servers.

Why so many servers? Performance, security, reuse, maintainability and understandability. There are actually other reasons to implement a three-tier architecture. I see these as the primary reasons;

1. Performance, security - Any interactive website with a significant amount of traffic could not be hosted on a single computer for performance reasons. The software needs to be constructed in a way where it can span multiple servers. The best way to have software span multiple servers is to modularize the software based upon its activity. Within a three-tier architecture modules would be constructed for the specific activities of user interface rendering, business logic, database reads and database writes. Each of these modules would be optimized for its activity and implemented across servers best suited to the modules performance needs. Another powerful reason for separating modules and hosting them on different servers is for security reasons. The closer a software module is to the storage of a piece of content (data, rich-media, documents) the stronger the security needs to be. It is also important to mention the idea of a cache, when building high performance web sites it is important to utilize a cache. The main idea of a cache is that it takes frequently requested content and makes it more quickly available to the web site.
2. Reuse, maintainability - modularizing software enables reuse and increases maintainability. The idea being that many modules can call one module to perform the same activity. As an example; saving both a bill to address and a ship to address is almost exactly the same activity, this should be done through a single module. This also eases maintainability, if you need to fix or update the address saving abilities of the software it only has to be done in one module.
3. Understandability -  understanding how a website application is put together, particularly when it spans multiple servers, becomes increasingly important as time passes. People change, business approaches change, features get added and updated. Having the IT Team easily understand how to enhance and maintain a website is greatly improved when the software is well organized and built in a modular way.

Building software in three-tiers provides the flexibility to organize the software to meet performance and business needs while it is operational. How the software is going to perform under user load is not always known until after the release of the software onto the internet. This is more true for web based software as performance will be determined by user traffic and users are unpredictable. Business needs will also change, so being able to alter the software with little to no impact to existing software modules is more easily done within a three-tier architecture. Well designed three-tier architectures are more easily understood and maintained than other approaches.

The MVC implemented in a three-tier architecture 

Figure 1. The MVC / 3-Tier Hybrid

How do these two come together. From a MVC perspective the View and Controller exists in the presentation tier and the model spans the business and data tiers. From a three-tier perspective this means the Model is broken into modules which are optimized based on their activity and for their ability to be reused and altered for new business opportunities. The Model will span the business and data tiers. The Controller and View exist in the presentation tier and this is also optimal for it allows the software developer to build, render and respond to user interfaces best designed for the different devices (internet browsers and mobile devices).

So how is all this best described from a non-technical perspective. We will start at the top and work our way down and back to the top again. Our scenario will include three activities. Arriving at a web site, logging into the site and adding a ship to address to a persons profile.

Scenario: A user wants to update their profile so they have separate ship to and bill to addresses.
Note: this scenario may not be as all websites handle these activities, it serves as an example to explain the MVC pattern and three-tier architecture.

Arriving at a web site
The user arrives at a website and the sites main page is displayed. The main page is rendered in the following manner;

1. Software in the Controller will determine the type of device browsing the website. Once the device / browser type is determined the Controller requests the correct View to render itself. Specific Views are built to service the different device and browser types.
2. The rendering of the View will make requests of the Model to fetch data (text, images, etc.) from the business and data tier modules. These modules will mostly fetch data from the Reads side of the data storage (see Figure 1. The MVC / 3-Tier Hybrid). For performance reasons, some websites will have these reads come from the cache. Many Views will also use style sheets and templates to help in their rendering.
3. All this activity takes only moments in a well designed website and once completed the View will be rendered and the user can view and interact with its content.

Logging into a site
After review of the sites main page, the user logs into their account. The process is managed is as follows;

1. The user types their email and password into the required text fields and clicks the associated button. This click event is intercepted by the Controller and decides the appropriate action. In most cases, some software (usually javascript) is run within the Controller to check the username and password are complete and follow some basic rules. If all is good the Controller will pass the email and password onto the Model as parameters for the email and password verification request.

2. The Model will take these parameters and make a request to a module built to handle user related requests to verify the email and password are valid.
3. The module dedicated to user requests will query the Reads side of the database modules to fetch the users password based on their email address provided in the parameter(s). 
4. If the password matches the module will pass TRUE back up to the Model, and the Model will request the View re-render itself with a valid login. This re-rendered View will often have additional menus for the user to perform additional actions not available to the non-logged in user.
5. If the password doesn't match the module will pass FALSE back up to the Model, and the Model will request the View re-render itself with an invalid login. This re-rendered View will again prompt the user for their email and password with the additional prompt of "Can't access your account?".

Adding a ship to address
Once the user has successfully logged in they will have the ability to edit their profile. The process of displaying their profile page and adding a ship to address would be as follows;

1. The profile View would make requests of the Model to fetch all the data required to render the persons profile data. This data fetching would occur via business and data tier modules built to handle the user related requests.
2. These requests would occur on the Reads side of the database modules and once completed the Model would request the View to re-render itself with all the users profile data.
3. Texts fields for the users ship to address would be presented and the user would complete the required fields. The user would then click a button to update their profile and save this new address to the database.
4. The click event is intercepted by the Controller and the appropriate action would be performed. Again, in most cases, some software (usually javascript) is run within the Controller to check the data entered follows some basic rules for accuracy.
5. The Controller would make the request of the Model to save the ship to address. The Model would make requests of the business and data tier modules specifically built to save address data.

6. The request to save this new address would occur on the Writes side of the database modules. Database writes are different than reads and this is explained in greater detail in a related post titled, "separation of database reads from writes". 
7. If the saving of data is successful two activities will occur; first, a confirmation will be sent back up to the Model, and the Model will request the View re-render itself with a confirmation of the data being successfully saved. Second, the data saved to the write side of the databases will be synchronized with the Reads side making it available for any subsequent read request.
8. If the saving of data is unsuccessful a return-code will be sent back up to the Model, and the Model will request the View re-render itself with correct error message. The user would correct the errors and the round trip of saving the data would occur again.

Why the hybrid?
Because building websites can be complicated, particularly when the site engages users and have a lot of content that can be targeted toward and created by users.  Why the hybrid of MVC and three-tiers? Two main reasons; First, because the MVC pattern does a great job of simplifying and managing the development of user interfaces over multiple devices and browsers, but it doesn't do a good job of defining how to build scalable server infrastructures. Second, The three-tier architecture does a great job of simplifying and managing the development of high performing, scalable, extensible and maintainable server infrastructures, but doesn't do a good job of defining how to build user interfaces across multiple devices and browsers. With a MVC three-tier hybrid you can utilize the best of both approaches without compromising either.

Inspired Permaculturist

I've been wanting to add Leigh to my Inspired Learner series for a while. I've known Leigh for a number of years and have come to appreciate the alignment in our shared love to be in the back-country, our personal values and our slightly skew view to the world. This post is mostly about how I see Leigh as a very inspired learner, and when it comes to permaculture how he uses social media very well to deepen and enhance his learning. I believe one of the strongest elements of Leigh's work is that he has developed an exceptional understanding of how to use video to capture and explain his work. Here is one of Leigh's videos in how to build a composting hot water system, it is one video from a series of videos he has created on the subject.

The amount of time a person puts into their reflective efforts is rewarded with a deeper understanding of their subject. When writing about or putting together a piece of content (like a single or series of videos), the accompanying reflection while creating this content greatly enhances their personal learning. The idea of reflective activities deepening learning is not new [Garrison, D. R. (1997). Self Directed Learning: Toward a Comprehensive Model.] And the idea that participatory video deepens learning is also gaining more and more acceptance, with an increasing body of research and new approaches becoming available with increasing frequency. If you want further information on how to use video, YouTube has a creators corner website to help out in learning how to be a good videographer. 

So back to Leigh; he has been building up a comprehensive set of content, sources and references to support his learning in and around permaculture.His approach to this I consider exemplary.

• You can see with his use of youtube that he is building a good set of video resources; http://www.youtube.com/user/leighblackall#grid/user/3E9C6C0FC24AAB39
• You can see with his community building and course facilitation he is engaging others; http://groups.google.com/group/permaculture-design-course
• And if you follow along with his tagging, he is creating his own folksonomy with what he has found in his readings about permaculture and related topics; http://www.delicious.com/leighblackall

I believe Leigh Blackall is an inspired learner who uses many of the available social media technologies and techniques to deepen his learning by engaging others and reflecting upon what he finds along the way. Leigh is an Inspired Learner.

Learner Satisfaction for Educational Resources

I like this image. It makes sense to me. The ideas that products and premises is what attracts customers (and learners) and that processes and people is what keeps learners returning is simple and seems true to me. The attributes attached to each of these that create customer (or learner) satisfaction seem simple and well defined. For further information regarding this model review the five pillars of service quality.

I believe these apply very well to building adult education / professional development resources. As more educational resources go online being mindful of the learner is what will differentiate the resources. I see each of these four applied to learner satisfaction in the following ways;

• Products - the products (learning resources) need to be of high quality with good learner value. The brand (institution) associated with the products is as important as is product availability. All products should provide 7 X 24 access and this will grow in importance as more people study through-out their waking hours. Having access to high quality resources will attract people to the learning resources.
• Premises - the premises (whether online or physical) need to be easily accessible and very understandable. Access is also about finding what you need, quickly. I increasingly believe that good (subject specific) search is one of the keys to accessibility. Once the resources have been found they should be very usable and this usability should be intuitive. The resources should be interesting and engaging not only from a content perspective, also from a rich-media perspective. Usability studies would be important here. The premises (again, whether online or physical) should be very well serviced. This means they are clean and well organized, guides (virtual or otherwise) are available to assist learners find their way. The premises is often updated taking advantage of the latest technologies that will assist the learner pedagogically.
• Processes - the processes need to be complete from the learner perspective. The learning journey needs to be well defined and communicated and include the assessment / accreditation part of the journey. The learning processes need to be implemented so they suit (and adjust to) each individual learner and their learning style. The learning environments need to be continuously improved as more learners engage and complete processes. Intelligence needs to be built into the system so outcomes, learner success and depth of learning can be measured. These measures will offer guidance and the ability for processes to be improved and refined. A well articulated learning process combined with solid assessment / accreditation and continuous improvement will keep learners engaged and retain many of them as they continue on their life-long learning journey.
• People - People are key to retaining and engaging the learner. All people who engage with the learner (regardless if they are faculty, staff or peers) need to be very aware of peoples individual and group needs and adjust the products and processes to accommodate. With modern learning environments this takes technical skill, and this technical skill should not be limited to hardware and software it should also include pedagogy. For in my mind, good pedagogy is a technology. All people involved in learning must work as a team for everyone's interests, and everyone's interests must be well articulated. Everyone benefits when people learn... not only the learner, also the faculty and staff.

Is all this possible? Absolutely!

With three of my last projects that fall within technology and learner (customer) satisfaction this is how we addressed the policies within these four areas;

• We were always looking for the products that could have improved quality and would further enhance brand. Having products available 7 x 24 helped the learner greatly as we increased quality and access. Federated and faceted search assists greatly with easing access to products.
• Most of the projects focused on online and blended learning where we focused on user experience and ease of access. Having access improved within our partnership model(s) also allowed the learners access to products of a higher quality they didn't previously have access.
• Taking the learning beyond the content and building systems that assist in the measurement of learning and provide opportunities for peer mentorship is where we increase the focus as we build learning environments. Building learning environments where people join community and return on a regular basis enhances life-long learning.
• Flattening the organization, empowering front-line staff and taking a stewardship approach to learning engages people. I believe the best learning environments are peer based learning environments that use inquiry models as their foundation and leverage the technologies to encourage peoples participation and engagement.

Separation of database reads from writes

More than once I have found myself in discussions regarding the correct way to architect web applications. In a recent discussion the concepts of MVC and REST were in the mix as the discussion focused on mobile devices. In this previous post I spoke of using REST as the preferred way to build the middle tier; particularly, when the application is targeted toward mobile devices. My main rationale for this is how mobile apps are mostly stateless and have less requirements for interactions to be transactional.

CRUD (Creates, Reads, Updates & Deletes)
This previous post generated some interesting discussion from within Google+ and got me thinking about being really clear in how you architect a mobile application using REST / SOA / MVC. To start it is important understand the MVC pattern and n-tier applications and how these two fit in regards to designing and architecting a web / mobile application. Particularly important is to think about is how you architect your CRUD (Creates Reads Updates and Deletes). These need thought because not all web / mobile activities are equal from a transactional and security perspective. So lets consider each of these functions from the web / mobile application development perspective;

• CREATE - When creating (adding) data to a database particularly from a web page or mobile device performing this through a RESTful API would not be preferred, particularly when a development language / environment is available. Not to say you couldn't do it through an http POST, it's just that a fully formed development language is going to provide transactional qualities and a more robust set of language features for creating data. Also important when creating data is security and ensuring that context and permissions are enforced when creating data. Ensuring security is also easier to do with fully formed development language.
• READ - When reading (fetching) data from a database this is where the most options exist as reading data is a simple operation from a transactional, security and state perspective. And if the reads are open to the public this would really be well implemented in REST. Reads are simple from a transactional perspective for they require no record locking or transactional requirements for they are reads (of course there could be exceptions to this, but in general reads are simple from a database perspective). Reads are simple from a security perspective for databases are good at securing things particularly from a read-only perspective. And if there is private data this can be easily secured at the database level. Reads in there nature are stateless as once the data has been retrieved there is no need to maintain a connection (tightly coupled or otherwise) to the data on the source.
• UPDATE - updates are similar to CREATES in they effect change on the database and could require record locking, have transactional qualities or reference to existing data. From a RESTful perspective an UPDATE could be done through a PUT, but if the UPDATE is complicated it would be better done through the use of a robust development language / environment, rather than a RESTful approach.
• DELETE - if you are deleting a small discreet amount of data that has no dependencies a DELETE could be done through a RESTful call. This is seldom the case with web or mobile applications. Data often has other related data in the database and sometimes it is better to mark data as deleted rather than delete it completely. The status of a DELETE action is also important as it needs to complete or may leave the database in a false state. Implementing the DELETE with the ability to finalize the action by receiving a status would ensure the data remains accurate.

Action	SQL	REST	Best Practice
CREATE	INSERT	POST	Use robust programming environment rather than use of RESTful calls. Inserting data requires confirmation of success allowing the software to continue appropriately.
READ	SELECT	GET	Reads are the actions which could best be implemented using a RESTful approach, particularly if the reads can service multiple needs and access approaches.
UPDATE	UPDATE	PUT	Again, use a robust programming environment. Updates can be complicated and require reference to existing data.
DELETE	DELETE	DELETE	Use REST only when deleting discreet amounts of data.

All this discussion regarding CRUD and REST leads to a couple of simple principles when deciding what goes where from a RESTful API perspective;

1. Reads / Reporting of public data are the best candidates for REST.
2. Creation / Updates of small / discreet amounts of data which fit well within a single concise message can be candidates can be candidates for REST.
3. Larger amounts of data with some complexity are best left within a robust development environment.

Separate the reads from the writes

Most web applications (mobile or otherwise) display a massive amount of information. Getting this information from its data storage via code (software) to be displayed is where most of the activity happens. The two activities of reading data and writing (creating, updating and deleting) data are fundamentally different. With reading, the application is fetching data with the assistance of indexes, joins and optimized execution paths. When writing, the application is allocating storage, performing duplicated activities (transaction logging), assigning locks to data elements, confirming data integrity, cascading activities, updating indexes and other activities. It could be considered that database reads and database writes are opposites. And the needs of the physical servers and the application software need to be optimized for these two opposite database activities. This can be done by physically separating the databases that service these two different activities. All the writes go to a single database server, which then replicates the written data to the database server(s) dedicated to servicing the reads. The code written for the reads can also be further optimized with caching techniques.

Make a RESTful API available
To add functionality, enable the ability for people to use your data in new and interesting ways, to build traffic and to increase your partner access it is a good idea to publish data via a RESTful API. Of course you need to be mindful to what data is made available via the REST approach. If you are going to make data public be sure it should be read by the public. This kind of data is vast and could be all public government data, all product data, social information, etc. In general, all data available in a RESTful API is read data. As described above in the CRUD description, read data is most appropriate for REST.

Related Readings
http://www.infoq.com/news/2009/07/CRUDREST
http://en.wikipedia.org/wiki/Create,_read,_update_and_delete
http://buytaert.net/scaling-with-mysql-replication
http://newitup.com/archive/2011/01/26/cqrs-in-phases-phase-1---separate-your-reads-from.aspx 

MVC in a three-tier architecture

The increase of web and mobile software development can get one thinking about how to best design and architect an application for these platforms. The two proven approaches that find themselves into this discussion are the Model View Controller (MVC) pattern and the three-tier architecture. This post is meant to describe these two approaches and how I believe they should work together and why.

Model-View-Controller

The MVC Pattern

The MVC pattern provides very well when building applications that are to be consumed by many different platforms (it also provides well when you are building for a single platform). When targeting platforms that are mobile devices, tablets and different browsers on different operating systems the MVC pattern is a proven approach. The MVC pattern separates different parts of rendering the User Interface (UI) into modules, so the code for rendering the actual interface is separated from the code that manages the data which is also separated from the code that handles the user events.

Three-Tier Architecture

The 3-tier Architecture

The three-tier architecture separates the deployment of software components into three logical layers. These layers include;

• Presentation Tier - responsible for rendering the User Interface
• Business Tier (or Logic Tier) - responsible for processing the business rules or logic
• Data Tier - responsible for interacting with the data storage system

This separation is for a number of reasons which include maintainability, reuse and deployment. When software is modularized and separated into these three-tiers, the modules can be deployed to different server infrastructures for security, scalability and performance. And the deployment approach can change as the need requires. The modules are also easier to maintain and reuse when they are each built for a discrete purpose.

The MVC implemented in a three-tier architecture

The MVC - Three Tier Hybrid

When the MVC and three-tier approaches are brought together the View and Controller are considered the presentation tier, the Model exists in the business tier (and has access to many business and data tier modules). To a certain extent the Model could span both the business and data tiers. It is this authors view that the Model would not exist in the layer of the data tier that is specifically Data Access Layer (DAL) code including SQL or code optimized for DAL within a cached environment.
Some important aspects of this diagram include is the separation of data reads from writes. Writes take considerably more data storage resources than do reads, so separating these into different physical data stores can have huge benefit. It should also identified that the modules for the business and data logic can be numerous. And these exist for maintainability and scalability purposes. It is also reasonable that the Model may access the Data Tier directly without going through a Business Tier module.

Why the hybrid?
The MVC pattern does not describe how to best design the data access and how to manage the complexity that can occur within the business and data tiers. As the internet has become more open and further entrenched in business processes, the demand to access the business tier without the presentation tier has increased. The creation of Business Tier modules allows for other approaches (such as REST) to access business logic from other sources, enabling business-to-business interactions and innovations.

Follow-up post
if you would like to read a non-technical post describing the MVC pattern within a three-tier architecture, follow this link; http://criticaltechnology.blogspot.ca/2011/10/mvc-in-three-tier-architecture.html 

Related reading

• http://stackoverflow.com/questions/4577587/mvc-vs-3-tier-architecture
• http://stackoverflow.com/questions/698220/mvc-vs-n-tier-architecture
• http://en.wikipedia.org/wiki/Multitier_architecture#Comparison_with_the_MVC_architecture

Realities of performance art

This spectacular photograph taken by Bill Pusztai captures so much of the time between dances, and from my perspective many aspects of Bowen Island Blacksheep Morris. I'll take you through the photo, left to right, and describe what I see from my perspective as a dancer, performer and friend to many in the picture. My thoughts may not be correct for others, but these are my perceptions.

• Banjo Jim (far left, holding the banjo) - his smile comes from a deep satisfaction. Davie Days is becoming one of our favorite venues to perform, it is one of the places where the audience is as colorful as our dance kits. Jim loves to perform live and the positive energy from the audience is reflected in Jim's smile. This is one of the main reasons we perform, the joy of entertainment, the deep satisfaction which comes from performance art, sharing oneself with your smaller and larger community and the amazing friendships and camaraderie.
• Gerald - I suspect Gerald is checking his hand for blisters. He always gives 200% during these performances and is most often to perform while injured. Which happens often when performing with sticks bashed by fellows at least 6 foot tall and over 200 lbs. Yes folks, Morris dancing takes some work. We practice (almost) weekly, we work on new dances most of the time and come up with (or learn) a new dance twice a year.
• Squire Bob - Melodian player extraordinaire. Bob is always playing to the crowd, so no surprise his back is to the photographer as he is turned toward the audience. He provides the musical heart and fills the role of our Squire like no other. Without Bob there would be no Blacksheep Morris. Bob also incarnates the tradition of Morris... he has been performing Morris for more than 35 years and there is no corner of this tradition he is not familiar.
• Peter - dancer, musician and fool... and master of none! (well, maybe the fool). Morris is as much about the music as it is the dance. And in either capacity I believe we all feel we could improve. The unique part of Morris dance is the relationship between the dance and the music and which takes the lead. For without one, you could not have the other... which one sets the tempo? which captures the most attention? From afar it is the music, it draws you in. Once engaged the visuals, and rhythm, of the dance take over. In the end, the music has to lead. This can be difficult when the dancers also use percussion instruments.
• Graham - dancer, founder and drill sergeant. Without Graham we would likely have no discipline. The Blacksheep are about as disorganized as you can get when it comes to pulling off any kind of performance. Graham keeps us focused during our practices and works really hard at this in the hopes to bring some rigor to our performances... but, then of course it is very very hard to herd sheep. And Graham is no sheep dog.
• Brian (far right, holding the drum)- mainland sheep, occasional sheep, wise sheep. I am always very excited when Brian joins us in either practice or performance. In fact, if any of the off island sheep show up I am excited. The sheep welcome all who choose to join and participate (regardless of gender, we are a mixed side). And many of those from off island bring a knowledge of music, morris and dance that makes us stronger.
• All other sheep (not shown and too many to list here. You know who you are!) - if Squire Bob is the Heart of the Blacksheep then the whole herd is the Soul. It's amazingly eclectic mix of individuals, and those words were carefully chosen. We like that we have no theme within our outfits, many border morris sides have a chosen colour scheme, we have none. Individual freedom to create is desired. Eclectic mix, just get to know us. You may be surprised that such a diversity of backgrounds can so deeply enjoy each others company. 

So there you have it. What is it that happens between performances. Even though Morris dancing takes some work we do it for the deep satisfaction, camaraderie, love, friendship and connection to community. The music fills our hearts and grounds us into the tradition of Morris dancing. This perfomance art is as much about the music as it is the dance; amazingly, this eclectic mix of individuals finds the balance between discipline and disorganization to actually perform. We pride ourselves on being a mixed side where the Soul of our performances lies within every member of the side.


Thank-you Sheep... for the love and friendship, you all deepen and enrich my life!

Sharing the trial with mountain goats

Lisa, Lucas, Kai and Peter bag
Quiniscoe Mountain

My family and I spent the last week in Cathedral Lakes Provincial Park. What stood out for me the most is how our two young sons Lucas (6) and Kai (5) went on long hikes and bagged a few peaks each over 2500 m. They were pretty flipped-out in getting to the top of Quiniscoe Mountain (2551 metres). And to be honest, I am one proud father to think of the effort, endurance and goodwill they both showed in getting to the top of this mountain.

Quiniscoe Mountain seen
from Quiniscoe Lake

The Cathedral Lakes Provincial Park is an outstanding place to bring a family and both the Ranger (John Henry) and all the staff at the Cathedral Lakes Lodge were outstanding. During our stay in the park, we spent the first three nights camping and the last two nights in the lodge. During our last night of camping the temperature dropped to -2 degrees Celsius. All good as we had brought good sleeping bags and fleecy jammies... The food prepared in the lodge was outstanding and it was good to meet others who had spent their days hiking in the park and exploring its many lakes and mountain peaks. We even took the time to do some trout fishing, and Lucas caught his first lake trout. We cleaned it and cooked it on an open fire... yum. Unfortunately my presentation of the cooked trout wasn't the greatest and the boys refused to eat any. Note to self, when having the boys try a new food, presentation is important.

Another significant event was sharing the trail with two mountain goats. We were very fortunate to sit ourselves just off the trial and allowed these amazing creatures to pass.The fun part was having the goats peer back at us from over the small bluff after they had passed.

What were my big lessons while on this family trip?

• Lucas and Kai love wildlife... they always want to see creatures, big and small, in their natural environment.
• Sometimes, children need a firm reminder that they are capable and need to push though to achieve their goals (even if they have been set by their parents)
• Durvla Murphy is correct in that we need to be reminded we are still animals and spending time in the solitude of the mountains is time well spent.

...To me that perfection of stillness is the grace of the mountains poured into one's soul.....I know and have always known that we 20th century humans need to escape at intervals from that alien world which has so abruptly replaced the environment that bred us. We need to be close to, and opposed to, and sometimes subservient to, and always respectful of the physical realities of the planet we live on. We need to receive its pure silences and attend to its winds, to wade thru its rivers and sweat under its sun, to plough thru its sands and sleep on its bumps. Not all the time but often enough for us to remember that we are animals. Clever animals yet ultimately dependent, like any other animals on the forces of Nature... - Dervla Murphy

Mobile devices and good architecture

I'm in the midst of a project that includes a significant mobile device component. The build out requires super engaging user experience (UX) for the interactions on the mobile devices. The project also includes a companion web portal where people will have further insight into their mobile device interactions. The project is very social and has a good possibility in innovating some monetary processes. Very exciting!

The other day I attended a meetup with focus on sproutcore. Great introduction to the product with some very useful insight into how the framework was put together and how it should be utilized. What was cool about the whole meetup is I posted a photo from the AppNovation offices where the event was being hosted. I posted directly from my Android phone to Google+ with no effort (nice feature) and added a comment. The really cool part was the discussion that occurred between myself and two guys I worked with on a really innovative client-server C++ / Sybase SQL Server application 20 years back. Weird thing is we are all still coding in different capacities. Anyhow, the really interesting conversation was around the MVC pattern and business logic in the browser. We all agreed this was not the right way to go, and the sproutcore site was wrong in promoting business logic in the browser as a marketing message.

So why isn't business logic in the browser a good architecture?
Because pushing it out to the browser means it can no longer run on the server. Business logic can be reused for other services (think RESTful) where browsers are consumers of the logic (not the producers), business logic is intellectual property and shouldn't always be exposed in the browser, business logic often needs data and implements business decisions, this should stay on the server. Don't get me wrong a framework that supports HTML5 and implements the MVC pattern is a good architecture. Just pushing the business logic out to the browser is not good architecture. Now we may be getting into the definition of business logic, and if the Model within the MVC is considered the business logic and is only responsible for acting as a proxy for the data and business decisions through a web service API like REST then we are good.


So what does this have to do with mobile devices and good architecture?
HTML5 and the MVC pattern are very good practices when developing on mobile devices. This is mostly because of HTML5's ability to target multiple devices and the MVC being a very mature and proven pattern. The issue then becomes how should the application be architected? I'd suggest the following practices;

1. Use a HTML5 / MVC approach when building the User eXperience (UX)
2. Keep the Model thin and it should act mostly as a proxy to the server hosted web services
3. Build the server hosted services using a well thought out RESTful design, ensure these services could also be consumed by non-mobile devices
4. Be mindful of which features are required for mobile and which are better implemented within a browser based portal (consider the difference between the Google+ mobile app and the Google+ browser portal)
5. Separate the database reads from the writes within the RESTful API (this will greatly help scalability)

Practices for IT stewardship

I completely agree! Information Technologies role is shifting into stewardship away from control. There are many forces at play that are leading to this end and this linked article does a good job of identifying these forces. As the article points out, much of the change will be a gradual cultural change and there are many practices that can encourage the change. This is what I see as important to facilitate the change from an IT perspective;

1. Define IT stewardship - doing IT right the first time
Every organization needs to create a shared understanding of IT Stewardship. I agree with the Merriam-Webster definition of Stewardship.

the conducting, supervising, or managing of something; especially : the careful and responsible management of something entrusted to one's care

It's the idea of being entrusted, responsible and careful with IT so it supports and empowers all those who use it. It's how IT professionals get out of the way and facilitate the organizations innovation so it doesn't have to look elsewhere. This also means doing the right thing the first time, instead of compromising the future with a quick fix or half-baked approach. Most seasoned IT professionals know the right answer when selecting and proposing approaches, they need to trust themselves and gain the trust of others.
2. Empower the middle tier - they should be interpreting and contributing to the strategic planning, writing the business cases and executing on plans, discussing with front line staff all customer issues, working diligently to make senior executives irrelevant or at least let them know there is a solid group of up and comers!
3. Encourage the front line staff - they know the heartbeat of the organization from the customers perspective, this is every organizations most important resource.
4. Engage your customers - all you design should be done with the assistance of your customers

That's a good bet

Just viewed this in my LinkedIn and couldn't help but smile... so the originator thought they would win the bet for a few hundred dollars; well it turns out, with the wonder of social media, it has turned it into a few thousand. And it wouldn't surprise me if it pays off his mortgage by the time its all done. I'd hedge its about to go viral. I certainly hope we get a follow-up post to see how the bet was actually paid off...

And in the time it has taken me to compose this post the number of likes has gone from 5,908 to 6,186... When will people realize the real value of social media is making bets with your Luddite friends in social media's power as a communication and networking tool.

Federated & Faceted Search

In the last few months CLEBC has shipped a new search engine. The upgrades have been significant and these improvements include;

• performance and currency has been greatly improved - search results occur quickly and the indexes are updated within 12 hrs of new content being added to any information silo.
• navigation made more intuitive - navigational facets provide standard approaches that are currently being used for search and faceted search.
• alignment of taxonomies across all information silos - the top level taxonomy has been reconciled across all information silos for better browsing of results across silos.
• search applied across all silos including the CLEBC store and CanLii - search results include references across all silos with the inclusion of the CLEBC store (to include the coming learning events) and CanLii to include broader results.
• ability to both search and discover (with the ability to narrow the search criteria within both of these approaches) - the use of faceted search is powerful when it provides the ability to both directly inquire and discover information.
  

All these new search features are due to using both a federated and faceted search approach. The rest of this post describes these two concepts as they relate to search and why they are important to legal search.

Federated Search
Federated search spans multiple data sources, where the search indexes include results from these many different, and related, information sources. CLEBC already has many high quality content sources and many of these sources are related through practice areas, taxonomies and other common and legal attributes. The design process of this federated search included analysis of each information source and reconciled the common attributes to provide a unified search result across all sources. The result of this federation is to have results from all information silos appears as one result set.

The effectiveness of federated search lies in how single search terms can span multiple information sources and how the results can be organized by date, relevance and information source. Finding related and relevant results across information silos covers more contexts a provides greater insight from different perspectives.

Faceted Search

Faceted search is best understood in a comparison of discovery based searching vs. direct searching. This is the difference between "I'd like to find a beach resort in the British Virgin Islands" (discovery) and "I'd like the contact information for the Long Bay resort in the British Virgin Islands" (direct). With direct searching you know exactly what you want you just need further details; with discovery searching you don't really know what you want and like to browse to find it. A search facet provides this discovery. As  show with the image on the right there are two facets, one for Content Type and the other for Practice Area. As you search for a general (or more specific term) the contents of the facet changes also with the number of results within each term to the right of the faceted list. This allows discovery where you can drill down on terms until you find specifically what you want... discovery based searching.

Special Thanks
Many great projects happen in the open and with many peoples contribution. In this project I would like the thank Susan Munro, Laura Selby and all of CLEBC who provided guidance and insight in how legal search could be improved. I'd also like to thank LexUM for their expertise in building faceted and federated search.

This search patterns book from O'Reilly is an essential read from understanding search. One of the authors, Peter Morville, has put together an excellent collection of flickr images showing the different search patterns.