By Cathy Dew
If you use SharePoint primarily as a place to store, create, collaborate on, and otherwise manage documents, then you may be pushing your SharePoint document libraries to the limit, literally. SharePoint has a 5,000 “item limit threshold” for document libraries and lists, and while this threshold doesn’t mean what you think it might mean—SharePoint doesn’t cap your library at 5,000 documents, in other words—it can still pose a challenge to organizations that need to manage scores of documents.
In our next series of articles, we’ll discuss the best practices in storing, searching, and managing large document libraries. In this first part, we will discuss the soft and hard limits associated with list and libraries and what you can do to work with them instead of against them.
Understanding the 5,000 Item Limit Threshold
Even if you don’t think your business has a ton of documents, understanding the so-called “limits” of SharePoint document management is important. After all, storing 5,000 documents in a single document library is not an unusually large amount—especially for a business. A single individual freelance writer can easily build up a document library of that size! An enterprise that employs dozens or hundreds of people on a project is going to rapidly surpass 5,000 SharePoint documents being stored in a shared document library. In other words, SharePoint wouldn’t be a prevalent platform for enterprise document management if it imposed such hard limits on its users.
There is an actual physical limit to how many items you can store in your SharePoint document libraries and lists. That limit isn’t 5,000 items per library, but 30 million. Now that sounds more reasonable, doesn’t it? (There are other limits to SharePoint’s document storage capabilities, such as a maximum file size of two gigabytes and a maximum bulk upload of 100 items, but those boundaries are part of a different conversation.
So, what is the 5,000-item limit then? This threshold refers not to how many documents your libraries can store, but how many documents they can display at any one time. No view of your SharePoint document libraries can show more than 5,000 items at any one time. So, the 5,000-item threshold is going to have more of an impact on your ability to browse, search, and find documents than on your ability to store them.
SharePoint has a good reason for limiting the view to 5,000 items. While 5,000 items sound like a small number in the grand scheme of an organization that produces dozens or hundreds of documents each day, it’s still a lot of data to process and display. If SharePoint had to display 100,000 files every time you opened a document library, that would involve a lot of processing and querying. By capping the view limit at 5,000 items, SharePoint cuts down on the resources it needs to use for this process. This throttling of resources allows for faster functionality, better performance, and makes it possible for multiple users to access and browse the same library at any given time.
Working with the 5,000 Item Limit Threshold on Your SharePoint Document Libraries
Now we understand why SharePoint throttles queries for more than 5,000 files and how that rationing of resources can benefit end users. However, the 5,000-item threshold can still be a barrier for effective and efficient searching of document libraries. If you have thousands upon thousands of documents stored in your document library, SharePoint’s 5,000-item limit won’t prevent you from finding what you need, but it can make the process slower and more difficult.
There are a few solutions to this problem that you might try to streamline your SharePoint document management processes. Which solution is right will depend on your business, what kind of SharePoint environment you are building, and how much time you want to invest in making SharePoint documents more easily discoverable. Below, we’ve listed some of your options, as well as their pros and cons.
Option 1: Raise the list view threshold limit
If you are using SharePoint Online, you can skip right to option 2. SharePoint will only let you increase the limit of the library or list view threshold if you are operating in an on-premise server environment. Even for server environments, Microsoft cautions against using this solution, as it will invariably cause performance decline for your SharePoint document libraries.
However, if you do want to try to tweak the list view threshold limit, you will need to have an administrator login to Central Admin, access the “Manage Web Applications” menu (found under “Application Management”), and select the application to alter the list view threshold. Once you find the appropriate web application, click on the arrow for General Settings (in the ribbon) and then click “Resource Throttling.” There, you will be able to edit the value next to “List View Threshold” to increase the limit from 5,000 to whatever you want it to be.
Another option with on-premise is to provide a daily time window that allows increases in the list view threshold limit. Actions that you want to have more flexibility are managing indexes, managing folders, deleting websites. Giving these administrative functions a time period with higher limits can prevent these actions from failing and impacting your user base during the admin operation.
Option 2: Create multiple different document libraries
If you want to be able to browse an entire document library in a single view, but also want to avoid bogging down your servers, the simplest option might just be to create multiple document libraries for different departments, projects, or purposes. Having one central document library where you toss all your SharePoint documents doesn’t really make any sense anyway since the library will just be a grab bag of files that have nothing to do with one another. Breaking your libraries apart into different categories or business functions can help solve the list view threshold dilemma while also leading to smarter organization habits for your business.
Option 3: Utilize metadata and column indexing
Creating multiple document libraries should usually be your first step after hitting the 5,000-item threshold. However, at a certain point, you’ll run out of meaningful ways to break up your lists and libraries. If you’ve organized libraries by department, category, or function and still have more than 5,000 items stored in most of them, then your next step should be to rely on metadata.
In a lot of ways, metadata is the key to effective SharePoint document management. Proper use of metadata makes files more easily searchable and discoverable, which in turn can improve efficiency and productivity within your business. Metadata can also help you solve the 5,000-item threshold if it is starting to get in the way of intuitive document library usage.
In every document list or library, you have the option to “index” the list or library columns. Indexing a column essentially makes that column and the information stored inside it more accessible and usable in a search query than it would be otherwise. You can use indexed columns to filter your views, which makes it easier to get under the list view threshold and find the specific documents you need at any given time.
When you apply the indexing to meta data columns (e.g. columns that are bound to term store data sets) you get a huge increase in search performance and the user experience. And by also enabling the meta data navigation you can quickly filter down the document library to the most interesting files.
There’s one catch with using column indexing as a workaround for the list view threshold. Unfortunately, once you’ve surpassed 10,000 items in a SharePoint document library, SharePoint automatically locks some of the privileges and operations for that repository. Most crucially for this current conversation, SharePoint prohibits you from adding or indexing new columns while there are more than 10,000 items in the library. As such, if you want to go the indexing route, you will first need to get your document library below the list view threshold. For this reason, assigning metadata and column indexing is best-done before a library becomes overcrowded. This factor also encourages the use of multiple document libraries to reorganize your files and eliminate some of the bloat from existing repositories.
Option 4: Use the Document Center
While SharePoint document libraries can store up to 30 million files, the fact is that they weren’t really designed to manage bulk quantities of documents. If your SharePoint document management setup is straining the thresholds and boundaries of your document libraries—even after reorganization and column indexing—it might be a good idea to consider using a Document Center site.
The Document Center site template is geared specifically toward businesses with hundreds or thousands of items to store and manage. A Document Center will give you a lot of the same features as a document library (such as versioning, content types, and metadata navigation), but merely gives you more leeway for storing large numbers of documents. You can create your Document Center as either an authoring environment (for creating and editing documents) or a content archive environment (for storage of view-only documents).
You might even consider using a Document Center site as a supplement to your existing document library setup. Having a content archive for old files that need to be retained but don’t see regular use can help free up space in your document libraries and might help you stay beneath the list view threshold.
Call Us for Help
Managing thousands, hundreds of thousands, or even millions of SharePoint documents is a complicated, daunting scenario. If your business has a lot of files to store and manage and is looking for the best solution to ensure simplicity, efficiency, and discoverability, then 2Plus2 can help. Our team is familiar with all the ways to avoid the list view threshold (among other common SharePoint document management pitfalls) and can help you implement the solution that is best for your business. Go online to schedule a free consultation with our team or call 510-652-7700 today.
Sources: