Workflow is becoming more and more popular within larger organisations trying to improve internal processes or integrate operations between various internal and external systems, but what exactly is it ? There are many systems on the market with the word 'workflow' within the title and all offering different features and solutions. So, can they all really solve the workflow problem.

I guess a good starting point is to decide on what we mean by workflow. Lets start by looking at this from a business perspective. Within any large organisation, there are various departments involved in processes designed to meet a business objective. Along the way there are many systems (I.T. and others) designed to make the process faster, with the ultimate business aim of needing the fewest number of people to carry out a business objective. These processes can be specific to a department/role or may involve many different departments and staff to achieve a single end to end process.

As described earlier, there are lots of supporting tools available to make the work easier depending upon the business type. Ranging from simple word processors to help produce documents/correspondence through to custom applications to perform a specific business service.

For me, the idea of workflow is to manage these individual business processes ensuring that work is moved through the organisation as efficiently as possible. For any workflow system there are going to be two types of operations that can take place.The first is an activity that is managed automatically by I.T. based systems, the second is an activity that required a member of staff's involvement.

If you have a clearly defined and simple process, you can probably achieve everything you need in a single application, however, as the business becomes more complex and fluid, this definition becomes more and more fuzzy. Additionally, investment in legacy and custom applications cannot simply be thrown away just because the system cannot participate with these disperate business processes.

So the perfect workflow solution needs to be able to support the following:

• Define end to end business processes quickly and simply.
• Allow integration with various internal and external systems and process source/targets.
• Provide for Human interaction within the process.
• Provide information on how processes are performing to measure the effectiveness of a business process.
• Must be reliable, perfromant and scalable. If your business is relying on this this is a must have.

Ok, thats my starter for 10 on what we need from workflow. Now, how to acheive all this quickly, simply and preferably without massive investment.....

Sharepoint, Biztalk & K2

I recently was involved in the constuction of a large workflow based environment. The underlying technologies used were:

• Biztalk, to provide the integration with various internal and external systems.
• Sharepoint providing front-end services and document management.
• K2 Blackpearl providing the Workflow design and hosting.

This system should have been perfect. Biztalk is a fantastic tool to transfer pocess requests between back end and remote systems, its scalable, extremely flexible and you can get really complex systems working really quickly. Sharepoint is a really great portal for collaboration, document management and housing custom data capture forms and applications. K2 Blackpearl is a little less familiar, but provides a workflow engine that easily integrates with both Sharepoint and Biztalk and as it is hosted seperately is very flexible.

So, did it work ? well, yes and no. The technology was sound, the overall architecture was sound. As a platform, this would be a great basis for providing a general purpose workflow solutions. However, where things went wrong on this particular project was a lack of understanding of both workflow and the tools by the management and analysts signing off the requirements. Once you start to break the workflow model to shoe-horn badly fitting requirements in, then you will start to hit problems. Alongside this, many developers on the project had come from standard ASP.NET web site development into Sharepoint with no real grasp of the differences. As such a lot of the Sharepoint development did not take advantage of the power of Sharepoint as the underlying code was simply traditional ASP.NET dropped into a Sharepoint site.

Amazingly, these same people who had effectively mis-used the tools, started trying to blame the tools. I guess this is pretty standard behaviour (thus the old adage), but does present a real problem for anyone wanting to implement real workflow. I guess the real lessons are:

• Ensure you have a good understanding of the Business Objectives before coming up with the product set.
• Ensure that the designers of the system fully understand the underlying products  - at a detailed level.
• Ensure you have good change management and keep you business and managers up to date with the product sets.
• Ensure your developers are fully conversant with the tools before letting them loose....Or have a really good technical lead.

Ok, I admit it, I think Sharepoint is a really great product. It can help in so many business areas or be expanded to produce some really great applications. One of the biggest problems I find is that I.T. themselves often don't understand how useful it is so don't generally put it forward as a solution to business. The irony is, Sharepoint within an IT department is a god-send, and once you start to use it and see its benefits then it can really help you solve your business needs.

It seems to be a strange thing that I.T. departments are often the worst places for adopting technology...go figure ?

Let me give an example. I did some work recenlty in a team that developed Workflow and document management solutions for business. And often complained that business were not using their software to its true potential. Yet the same people complaining about business were managing their own processes using whiteboards and e-mails without ever thinking about using their own software ?

Ok, back to Sharepoint. Right, how many of you out there find the following familiar : During a project, all informtion is stored on a network drive in a complex directory structure. Project issues and tracking is done via a shared spreadsheet buried in a directory somewhere. The project plans are stored on the PM's own area who spend half their day telling people what tasks they should be doing and running round with printouts checking progress with the team.

Right same scenario but after installing sharepoint and Project Server :  An MS Project team site is set up with several documentation libraries to store project information (which is all versioned, change controlled and searcheable). The issues and activities logs (Sharepoint lists) are visible to all team member with individual alerts available. The Project plan is managed by the PM but stored in Sharepoint using a central resource pool (so that a developer can work on multiple projects without 2 PM'sgetting into a fistfight), once in Sharepoint, each user will see their own task list (derivable from all plans to which they are allocate), and they can send online updates of progress, whcih can automatically keep the plans updated with reality (you can enforce PM approval here).

So, for little overhead you now have a streamlined IT process with minimal process overhead, and your managers can get on with managing rather than admin. Plus, you now understand how Sharepoint works and can start to find benefits elsewhere in your organisation.

Win Win scenario.

Recently, while working within a large organisation, I came up with a concept for a new language which followed on from C# and F#. This new language was formed during a phase in the development where we were bogged down by policy and an increasing supply of management. This new language was called M Flat, and was cynically based around a design for a new language based on management principles. I posted this on a development sharepoint site, and was suprised as how fast contributions came in from other developers, which were just as cynical.

Here are a few examples of the language features described:

• New Try_Catch_Blame block: following the principle that whenever an error occurs, the main thing to do is to find someone to blame.
• Continuous Interupt cycle: During the execution of any code, a seperate process continuously interupts what is going on to get a status update.
• Obfuscation Semantics: language syntax designed to sound impressive without acutually conveying any real meaning.

And so on....shame I don't still have the stuff as there were pages and pages, even down to a section on design patterns.

Anyway, the point is, in one organisation nearly every developer offered the same cynical view of management. Hopefully, this is specific to this organisation, but I dont think so. If not, then what is happening ? Have we stopped producing good management skills, or is I.T. just getting too complex for the non serious techies to cope with ? or are management right not to trust technical people and if so who can they trust ?.

Personally, I'm edging towards the complexity thing. Not too many years ago, most of my managers came from an IT background, and could still remember their programming days. This meant that they could participate in IT discussions and, more importantly, they could spot when us techies were pulling a fast one or more likely could spot developers who were out of their depth.

Now, things have changed, its hard enough for us techie types to keep up to date, you just get the hang of ASP.NET & C#, then we get WCF/WPF/WF, you just get the hang of that then your project switches to use Sharepoint or Biztalk and your learning again, and again. Don't get me wrong, this is what keeps me interested. But what about your poor manager, or developer who can't cope and wants to switch to management for an easier life.

Lets be honest, if you don't spend 26 hours a day learning, your out of date. A manager who last coded in Turbo pascal 10 years ago has no chance of joining in in any meaningful technical discussions and has no idea if what they are being told is true , false or just meaningless gibberish (because they've been let down too often by the developers who can't cope). So the only option is to add more and more process and control in the hope that this will replace the lack of general understanding.

Even more worrying, is that this trend is spreading into Architecture. As a metaphor, would you employ an architect to design your house if they didn't understand about stress, materials, or what bricks were ? No ? then why trust someone with a £10,000,000 database project if their technical knowledge is restricted to Powerpoint and Excel ?

Don't get me wrong, there are still some good ones out there....it just seems that they are getting harder to find.

Am I wrong ? Hope so. 

I guess as a consultant, I'm used to working exclusively on my laptop and trying to use it to do everything. For a while I've used VHD's to provide various O/S and development environments which is wonderfull - except I always find them really slow. Now, I know that my laptop might not be top of the range or be jam packed with RAM but my solution of dual booting has always provided good performance, its just not as flexible as VHD's.

Enter Windows 7.

Recenlty I upgraded to Windows 7 and started re-installing my development tools. My biggest problem area was with Sharepoint. Previously I have managed to install this on Vista but always with limitations. Instead, I noticed the information about the ability to boot from VHD, starting with Windows 7 and Windows 2008 r2. For me, this is fantastic, I simple created a new VHD from Windows 7 (making it bootable) and installed 2008 r2 onto this.

Now I can simply boot from my 2008r2 VHD to do my development work and get much better performance while still providing the benefits of using VHD's.

Wonderful.

Over the years there have been many changes in I.T. As I'm now reaching that age where the phrase 'grumpy old man' could soon be applicable, then this section will be dedicated to various rants and observations which I'm hoping people out there can either relate to, or can confirm that I've simply reached a certain age where there is nothing left but cynicism.