Blog

Plus16 Webinar Transcript "Avoid the Document Tsunami Creating Information Management Policies in SharePoint"

By Anthony Baratta on November 4, 2016

Good morning / afternoon. Welcome to the Plus16, our monthly live webinar series. Thank you for taking the time to join us. We know that you are busy – as always we intend to pack meaningful data into an efficient 30-minute window. My name is Anthony Baratta and I’m CTO for 2Plus2 Partners. Today we’re talking about the Creating Information Management Policies in SharePoint. Drop any questions into the chat as we go through the presentation, we’ll save time for Q&A at the end. We will be recording this session and sending you a link. Additionally, we always post transcripts of our webinars on our blog.

Creeping out of the Dark Ages and into the Light

Tsunami 01 - Lets Get Started .jpgAs any business grows digital documents tend to grow exponentially along with it. In the “dark ages”, your only choice was a file system (local or networked) and organized, hopefully, with a pyramid style file folder hierarchy. You were lucky to have any search capability that didn’t break down as your digital pile of files grew out of control.

With the advent of SharePoint 2010, the addition of Document Centers, Meta Data and enhanced content types, we had the first set of SharePoint tools for re-organizing your now flood of digital documents. These tools were updated and expanded over the last two SharePoint revisions to give you more control to help you contain your continually rising tide of digital documents.  The challenge for any business using SharePoint today is where to start and how to build your flood control system and additionally to maintain it as the document deluge continues.

SharePoint uses the term “Information Management Policy” as the catch-all for several features that when combined in a sensible manner can help you, unlike King Canute, stem the digital tsunami lurking on your network. The challenge is that certain design decisions and feature setups have to be done close together and in parallel in order to build a working Information Management Policy.

Parts of the Whole: The process is not linear

Tsunami 02 - Parts of the Whole.jpgSo what are these parts that comprise the whole of your SharePoint Information Management Policy? They are:

  • Content Syndication Hub
  • Custom Content Types
    • Custom Meta Data
    • Retention Rules
      • Workflow 
        or
      • File Move Actions
  • In-Place Records Management
    or
  • Record Center
    • Records Center Connection
    • Drop off Folder
    • Meta Data Routing Rules
    • Target Folders
  • Setting Document Libraries & Document Centers with the new default Content Types
  • Corporate wide adoption

The design pattern for setting up your companies Information Management Policy is not a linear process, and circles back on several of these parts in order to fully complete all the necessary steps to have a working policy. Additionally, some of the design decisions need to be well thought out and incorporated into your company’s DNA in order to ensure internal compliance with the new rules.

Side Note: While Microsoft offers global document deletion policies for your SharePoint Online system, we are discussing a more finessed way to manage your digital documents than just a blanket deletion of aged out documents.

Analysis Paralysis: It starts with defining content types

Tsunami 03 - Analysis Paralysis.jpgThe first part of building your Information Management Policy is probably the biggest hurdle to overcome, designing your default content types. The challenge is designing something basic that suits your company that can be expanded, if needed. Simple is better because you can easily overdesign any new content types your company might utilize therefore slowing or even preventing adoption.

Can you under design? Not really, you’ll need to create at least one custom content type in order to access the last update date and document creation date of any document. Microsoft does not allow you to access these two attributes via the default Document Content Type. However, without at least one column of Meta Data for the user to categorize their document, your Record Center Drop-off Library (called a Content Organizer) will not have anything to act on in order to route documents into the Record Center Document Libraries.

There are two main design philosophies when it comes to building content types, going wide: large number of very specific document content types or going deep: utilizing multi-column meta data to tag and categorize your documents. As with most things, you are going to need to do a bit of both to get what you want.

Lastly, decisions we’ll be discussing later will most likely cause you to revisit the design decisions we are discussing next – hence the feedback loop and feeling of a never ending cycle of discovery and change as you walk through this process.

One more important point before we move on: You’ll want to design all your custom content types within your Content Syndication Hub. We’ll discuss later why this is important, but in order to get your content types adopted within all your site collections you need to start with a central location and make sure you can “push / pull” your customizations completely throughout your SharePoint installation.

Going Wide: Don't over think it, a content type per doc type

Tsunami 04 - Going Wide.jpgBuilding new content types for each document type you have for your company is probably the path of least resistance.  The content type itself becomes the categorization label, and you don’t need much more than selecting the “Invoice” content type to know what you have at a later date. Additionally, one of the biggest benefits of going with multiple content types is you can attach a default template to the content type and ensure that anyone creating a new document is using the approved document template, if they create the new document via SharePoint and not cloning an existing document.

The downside if multiple content types is that each content type will have its own set of information policy rules. If you want to standardize the retention rules among your content types, you’ll have to touch each one in turn to update / synchronize them.

If your company has a set of documents that are highly structured and your employees need to be able to access the most current template / version for that starting document, you’ll want to create a content type for each of your needed templates. Examples are: Invoices, Expense Reports, and company correspondence letters that need letter head and default company information.

Going Deep: Create attributes for each doc type

Tsunami 05 - Going Deep.jpgThe opposite of building a multitude of content types is to design your content type to support multiple documents types. Your focus is on a single or limited set of content types. The challenge here is to include enough custom columns and multiple choices per column to ensure proper cataloging of your documents. The choices can be embedded into a custom column, selected from a custom list, or from managed meta data provided at the Site Collection or Tenant/Farm Level. When designing your content type customizations, reusability and sustainability should be the focus. Can your content type customizations scale as your business grows and your document needs change over time?

The downside of going deep is you will have to incorporate all the document properties / categories into a one or a couple of content types. New categories will need to fit into the existing columns or will require their own columns and meta data, expanding the complexity each time.

If you have a limited set of documents, or are only interested in utilizing the meta data for necessary “actions” on documents, going deep might be a great option for your company.

In-place or Record Center: Determine the set-up that works for you

Tsunami 06 - In-Place or Record Center.jpgWe need to step out of our focus on the discussion of the content types to briefly discuss the different record management options within SharePoint, because this decision will affect how we run our retention rules.

The two main options are In-Place Record Management and building one or more Record Centers. These two management philosophies are not mutually exclusive, but being feckless with your decision will cause confusion and directly affect your company’s ability to support an audit or eDiscovery process.

The choices are pretty self-explanatory: In-Place Record Management allows the Site Collection Admin, or the Business Rules built to manage the documents, to set a target file as a record at its current location. The file is not moved anywhere, but instead becomes read-only and nonvolatile.  The document is only removed from its current location when the business rule allows it, e.g expiration of the document and deletion. This is a good choice for team and project documents where archiving the whole sub-site / site collection is needed at a pre-determined end date of the collaboration.

A Record Center is akin to a basement storage location, where a project manager or department employee will bring a box full of stale documents to the Record Center Manager, have the set cataloged and receive a document ID for tracking the document set for future reference. By design the target documents are moved from their development or public viewing location to the Record Center by a document processing rule or manual archival process, and set as read-only in a giant digital warehouse for eventual eDiscovery or automated digital shredding when a pre-determined time has expired.

While your decision on in-place records or building a record center is not an either-or decision, you do need to understand your company’s document archival and expiration requirements to make the best decisions going forward. These requirements and design decisions will loop back on your content type decisions and cause you to revisit them for changes and updates.

The end result is you have two life cycles or 3 states of a document: in use, archived, and expired. An in use document is needed by a team or department on a regular basis. An archived document is a document that has been declared a record and is no longer changeable. An expired document is a document that is no longer needed by the business, even for reference, and can be destroyed. Whether you archive the document in-place or move the document to a record center for archival, the timing of those actions is dependent upon your company and regulatory needs.

Retention Rules - The First Bastion: Fortification, a projecting portion of a rampart

Tsunami 07 - Retension Rules.jpgNow that you have your custom content types setup, or at least in the design phase of your content types, the retention rules for each of your content types are the next set of decisions that need to be made. First off, what are you required to do with your business documents? For example, your business tax documents might need to be retained for 7 years, Employee records for 10 years, research and reporting documents that have to do with regulatory oversite might be 20 years. These required retention rules might require you to re-examine your content types and rework them to better support the different retention rules needed.

This is an example of one of the feedback loops that will require consent refinement of your content types and which exacerbates the analysis paralysis discussed previously. The retention rules will be specific to each of the content types you created earlier and specific rule requirements may necessitate the creation of additional content types.

However, retention rules are not just for clearing out expired documents. They also give you a chance to clean up existing document libraries and document centers by moving “stale” documents to a new location (records center) or identifying those documents that are candidates for in-place records management. We’ve discussed the pros and cons of Record Centers and In-Place Record management previously because it pertains directly to your design decisions for retention rules; however, the discussion we need to have next is whether to use the file move feature or the execute workflow feature of the Retention Rules.

Side Note: There is an option to work with Custom Retention Rules, but the Custom Rules need to be built via code and uploaded to the Web Farm as a new solution / feature. Custom Retention Rules are not currently available for SharePoint Online.

Execute Workflow Retention Rules: If flexibility is the most important thing

Tsunami 08 - Execute Workflow.jpgExecuting a workflow when a retention rule is activated gives you the most flexibility for decisions on what to do with a specific document. You can setup an approval workflow, email notifications on actions taken, reset the document life cycle, and even move the file to a holding area for action. The limitations are only the available workflow actions, which are quite numerous. Third party tools can extend the available workflow actions and increase your company’s file management options.

So what do you need to do with each content type? What do you *want* to do with each content type? If you don’t need user notification, document metadata analysis or approval / intervention when a document status changes you don’t need to utilize a workflow.

Content Organizer File Retention Rules: Use a drop-off folder to keep it simple

Tsunami 09 - Content Organizer.jpgThe other option for retention rules is to automatically move a target document to a drop off or content organizer folder. This is the simplest option for document life cycle actions. A document might hit the 5-year mark since it’s last update and qualifies for moving to a Content Organizer. The retention rules assigned to that document’s content type make sure that happens without having to keep track manually of document ages and locations. No user intervention is expected when this rule activates.

This rule is simple because all the decisions are made by the content organizer. However, you can increase the complexity of the file move rule by assigning different content organizers to each content type.

Content Organizer Metadata Decisions: Routing Rules Rule

Tsunami 10 - Meta Data.jpgThe culmination of all your content type design work will end up being utilized by your Content Organizer, if you decide to forego In-Place Record Management and migrate stale documents to a Record Center. The Content Organizer holds all the rules needed to route documents to their archival location using the meta data designed and implemented earlier. HR documents can be routed to a specific HR Library or Folder, Employee Expense Reports to their own Library or Folder, any of the meta data values of the target content type can be utilized for document routing. As mentioned earlier, if you went with a simple but wide content type solution, just the content type itself can be used to route the target document.

The power of the Content Organizer is creating a series of hierarchal rules that will be used to identify and route a document. The rules are ranked so that you can go from highly specific to more general document types and route accordingly. Lastly, if a document is not acted upon within the rule set defined, a notice can be sent to a Record Center Admin (or Admins) for manual routing.

Again, design decisions on the content type meta data may need to be revisited once you get to this phase in your Information Management Policy construction because no plan survives first contact with the enemy.

Propagating Design Decisions: Content Syndication Hub Focuses Logic

Tsunami 11 - Content Syndication Hub.jpgNow it’s time to propagate your design decisions through your SharePoint Site.

There are two major parts to ensuring your new Information Management Policies are propagated through your SharePoint environment. The first is utilizing a Content Syndication Hub, the second is making your new content types are the default choice when creating / uploading new documents. 

The first part is relatively simple: as we mentioned earlier make sure all your content type customizations start within a Content Syndication Hub. This special site collection is needed to ensure that your customizations are pushed to all the site collections within your SharePoint installation. Additionally, changes / updates you make to your existing content types are propagated too. Update a template attached to your Employee Expense Report content type, and relatively shortly all site collections will have the updated template.

The second part is unfortunately not so simple, setting your custom content types as the default for document libraries and document centers. Since the underlying structure of SharePoint has changed and is changing still, and SharePoint Online has closed off all the formerly acceptable ways to ensure custom features are part of any new site collection or sub-site built, you are now solely dependent upon the CSOM (Client Site Object Model) for ensuring custom features are propagated to new site collections / sub-sites. You can use CSOM via a local Web Application, an Azure Cloud Web Application, or even just PowerShell scripts available to your Admin. Either way, you’ll need to make sure your request and build process incorporates the scripts needed to customize your site collections and sub-sites to ensure compliance and default use of your custom content types.

There is no magic bullet in this area, the latest versions of SharePoint have definitely increased the day-to-day administrative demands in order utilize some of the more powerful features of the platform.

Corporate-wide Adoption Voila: Information Management Policy

Tsunami 12 - Adoption.jpgSo we’ve reached the end of our construction list and probably have cycled several times through each of the steps to create a solid information management policy that works for your business. Now you need to evangelize and train your users to utilize the new features. Support them as they grow to understand the need for identifying the correct document type, update the custom content types based on real world feedback, and remind them where their file “disappeared” to when it was identified as stale and migrated to the Record Center.

Fortunately, SharePoint’s search features are not constrained by site collection boundaries and your employees should be able to find all their documents – both active and archived no matter where they live.

Thank You. Any Questions?

Thank you for your time today.

We hope you will join us next month Tuesday, December 6th at 10 am PT, 1 pm ET.

The topic for our December webinar is Managed Navigation versus Structured Navigation

Anthony Baratta
Anthony Baratta – Chief Technology Officer
Anthony helps the company realize it's mission - Real results. Every time. Fluent in technology, Anthony breaks down complex problems into scalable solutions and manageable automated tasks.