It’s All About the Database

The key component to any content management/archive/workflow system is the database. Many times this key component is overlooked at the time a new imaging system is put in place, often the database is placed on an existing server which already has many critical duties or it is a new install placed on the same server as the imaging system software. These choices might be fine for a proof of concept, but as a system takes on more data and more users the database becomes the performance bottle neck.

As with any database, issues will rarely arise within its first year of usage, the data is slow to grow and performance of queries is quick and responsive since the load put on the system is minimal. During this time with minimal data the number of active users in the system at any one time is low as well. There are more inserts taking place than data retrievals. But the point of putting in one of these systems is not that the amount of data and the number of users will remain small for any length of time; most organizations bought their software and underwent the installation and configuration process with the goal of creating a vast repository that would make access to their content easy and quick for a large community within their organization.

It is when the system has finally been adopted fully and has enough content to be useful that the undersized database server will start to become a problem for the return of data to the users. I have seen systems where the input of new data is scheduled to go in during the off hours to compensate for the inability of the database server to perform inserts as well as retrievals during business hours. By this time the money for the original project is long since gone and often times the stake holders have forgotten about the decision to use a less than desirable database server configuration as a short term solution during the software’s “Pilot phase”.

No where does this problem raise its head more often than in a workflow system configuration. The audit history of actions builds quickly, as does the number of transactions moving through the system. Users will become frustrated when trying to check the history of transactions and they encounter long delays or retrieve no data at all. Timed events which cannot traverse the entire queue within the time period allotted will impact the system and keep the processor and memory continually strained at high limits. This can go from slow response to actually a big problem very quickly. A workflow system is almost always put in to make processing business needs faster (among other things such as accountability and tracking of the state of a current request).

On top of which this shortcoming is going to happen exactly at the systems busiest time of the year (naturally more data input and retrievals), just when the business unit is least able to afford it to be offline. The opportunity for moving to a new database server will not be available until the busy time has passed and the system has less demand put upon it. The opinion of the business group about their system, IT support staff, and vendor can all be called in to serious question since their perception is that the software used to work and it doesn’t anymore.

At the time of implementation, the high end server, dedicated to the system’s databases and probably the most expensive server by far, should not be seen as a “nice to have” option. You want to put in a total solution which will be robust enough to handle all foreseeable needs over the next few years. Ideally, the system will need little maintenance other than normal database maintenance procedures if the servers are sized correctly. The system needs to be looked at from the perspective that it will be more than a success for the immediate user base and will need to be able to archive other departments’ documents, manage other departments’ content, and bring more business workflows online.  It really comes down to planning to succeed.

Jeff Doyle

Senior Systems Engineer

Image Source Inc.

jeffdoyle