Category Archives for "ILINX Content Store"

Migrating 85 million documents from Oracle 10g with no data loss—a process, not an anomaly

ImageSource has been successfully executing migrations, large and small, to and from content repositories since 1994. Recently, we completed a mammoth project for a global financial information services company in which we migrated 85 million documents from a deprecated Oracle 10g IPM system to ILINX Content Store in 9 months with 99.99995% accuracy. That’s zero loss of data, except a few hundred docs that were already corrupt in the source system!
The speed and accuracy of this project can be attributed to 3 important factors that made the content migration successful:

  1. We assembled a strong technical team.
    The team for this project consisted of an Oracle 10g technical subject matter expert, a seasoned PMP Project Manager with years of experience managing Oracle 10g migrations and a SQL database expert. This team provided all the expertise required to plan, execute and audit this size of migration.
  2. We used time-tested methodology from years of experience performing similar data migrations.
    Given the mission critical and high transaction volume nature of the legacy Oracle 10g system, maintaining business continuity was critical. A detailed Migration Plan was developed addressing all content to be migrated, associated requirements (e.g. retention of annotations, excluding content that had or would meet document retention\destruction requirements during the duration of the migration, etc.), and auditing requirements. A detailed audit and migration report was performed and delivered to provide an accounting for every single document in the source system.
  3. We utilized powerful, configurable technology.
    We leveraged multiple tools and techniques in the execution model (e.g. temporary migration environment that included replicated data, multiple instances of ILINX Export, etc.) to be able to complete the migration. By creating a temporary migration environment, we were able to leverage the power of ILINX Export, ILINX Import and a multi-instance model to perform the migration in record time. The numbers speak for themselves: 85M docs, 9 months, zero data loss.

We’ve used a similar services formula for migration execution large and small—your migration doesn’t have to be massive like the one mentioned above to give you great ROI. That, and the right software mix, will eliminate headaches from the equation the next time you have to migrate content.
If schedules or other factors outside your control don’t allow time for a full migration prior to going live with your new system, we have a feature in ILINX Content Store that will allow customers to turn off their normal end user access to your legacy system on day one of using the new system and still pull content from the legacy system through the ILINX interface, read more about that here. This enables significant benefits that include smarter resource allocation within your operating constraints and migration flexibility. Contact us for more information.

Gene Eckhart, PMP
Project Manager
ImageSource, Inc

Transferring ILINX Release Configurations When Upgrading

Starting with ILINX Capture v6, the Release configurations are stored within the ILINX database. In ILINX Capture v5x, the ILINX Release configurations were stored in XML files on a disk. ILINX Capture called ILINX Release using a SendAndReceivedReply IXM. The change to store the settings within the ILINX database is very useful for a number of reasons: Release settings are part of the batch profile allowing for simpler migrations between environments, Release is much easier to configure, all configurations are in the database, etc. However, this change can create some extra work when upgrading from ILINX Capture 5x to ILINX Capture 6x. Because of the different architecture, ILINX Release needs to be completely reconfigured for the existing batch profiles. In addition, the Release XML doesn’t change, but there is a shortcut that can be taken. After you have upgraded ILINX Capture to v6, you’ll notice a new IXM in the palette: ILINX Release IXM Icon

The existing ILINX workflow will likely have a SendAndReceiveReply IXM on the map that the 5x version of ILINX Capture used to call ILINX Release. Most likely, it would look like this:
SendAndReceiveReply_IXMTo configure ILINX Release for ILINX Capture 6x, the SendAndReceiveReply IXM will need to be removed from the map and a Release IXM must be dragged onto the workflow map in its place. Once the new Release IXM is on the map, it will need to be configured. This is where the shortcut can be taken. Instead of having to manually enter in the correct URLs, map the metadata values, and configure any other settings, do this:
Configure and save Release with some place holder settings: I normally leave the settings at default and enter in the bare minimum:

  • Job Name
  • User Name
  • Password
  • Batch Profile
  • Release Directory

Once ILINX Release configuration is saved and the workflow map is published, there will be a new entry in the ILINX Capture database Capture WorkflowAppSettings table. The CaptureWorkflowAppSettings.SettingsXML column is where the Release configuration is stored. Now it’s time to update the SettingsXML column with the XML from the ILINX Release 5x job settings file. The Release job should be on the ILINX Release 5.x server at c:ProgramDataImageSourceILINXReleaseSettingsJobs. The only caveat here is to be sure to place single quotes around the XML content. Here is what the SQL update statement would look like:

update [ILINX CAPTURE DATABASE].[dbo]. [CaptureWorkflowAppSettings] set SettingsXml = ‘COPY AND PASTE ALL TEXT FROM 5.4 OR PRIOR RELEASE JOB SETTINGS FILE HERE’
where settingsID = ‘APPROIATE ID HERE’

Following this procedure can save some time if upgrading an ILINX Capture 5x system that has a lot of batch profiles. A lot of the time spent on the upgrade could be in the ILINX Release configuration. If I was upgrading a system with only a few batch profiles, I would probably just reconfigure them. If I was upgrading a system with a lot of batch profiles, I would go through the above steps to save some time.

John Linehan
Sr. Systems Engineer
ImageSource, Inc.

Implementing SQL FILESTREAM Part II

Last month I wrote about enabling SQL FILESTREAM with ILINX Content Store. After discussing this with a few people, I think I should share some more information and reiterate a couple points.

For Existing Applications:
As I mentioned before, the decision to enable FILESTREAM should be done during the planning phase. If you perform this process on an application with a lot of content, it can be a very time costly endeavor with a big performance impact to the server. Also, after the move from BLOB to FILESTREAM, you could have a fragmented database. The BLOB to FILESTREAM process can definitely be done on an existing system, just be sure to plan accordingly and allow for sufficient time.

After step #10 of my previous blog post (all the data is copied and you have deleted the BLOB column), you will notice that the database file size hasn’t decreased. This is remedied easily enough be executing a DBCC CLEANTABLE command. The DBCC CLEANTABLE command will reclaim the space from the dropped variable length column. For example, if your database is named ILINX_CS and your application is named Sample Application, the query to do this is:

DBCC CLEANTABLE ('ILINX_CS','[dbo].[Sample Application]',10000)Continue reading

Storing content outside of SQL Server for ILINX Content Store using SQL FILESTREAM

By design, ILINX Content Store stores documents within the SQL database as BLOBs. There are many advantages to this design (security, performance, etc.) but sometimes there is a reason to store the documents outside of the SQL database. SQL Server has a method to do this called FILESTREAM. FILESTREAM integrates SQL Server with the NTFS file system by storing varbinary(max) data outside of the SQL database. FILESTREAM uses the NT system cache for caching file data: this helps reduce any effect that FILESTREAM data might have on Database Engine performance. The SQL Server buffer pool is not used; therefore, this memory is available for query processing.

One of the main reasons to implement FILESTREAM would be because your documents are generally larger than 1MB in size, storing them outside the database can have a performance advantage. If these are TIFF documents, then this 1MB threshold would be on a per-page basis. This is due to how ILINX Content Store stores TIFF documents. By design, ILINX Content Store splits multipage TIFFs into single pages to allow for users to perform actions on single pages of a document: things like a reorder of pages, single page delete, or rotation.Continue reading

ILINX 6.X Database Lookup IXM

ILINX 6.X is an easy to configure and easy to use software package to scan, index, and provide workflow. The workflow steps are based on IXM (ILINX eXtension Modules) that are very similar to a programming language. There are several different types of IXM’s available out of the box. The following is a quick listing by name of the out of the box IXM’s:

5

By using the IXM’s, the designer of a workflow can have a batch move through single or multiple steps to perform any required task.

In addition to the IXM’s there can be actual code executed through a Client Side Extension or through a Server Side Extension. So there is little that cannot be accomplished using the ILINX Capture workflow IXM’s.

This week I would like to concentrate the discussion on a single IXM Database Lookup. The Database Lookup IXM is one of the most powerful when it comes to interacting with entities outside of ILINX. It not only allows ILINX to perform a database lookup and return column values to the Batch Profile or Document fields, but it also allows for the update of a database table’s columns.Continue reading

1 2 3 8