Updated December 9, 2010
Quality control measures should be implemented at each stage of the process so that the review is consistent, accurate and defensible. Quality control measures may avoid unnecessary duplication of the review and aid in meeting deadlines.
The review platform should allow reviewers to mark a document for further review when they are unsure about how to code it. This ensures that reviewers are not forced to make an uninformed decision. They also should be able to easily revisit a document and change the coding in the event that it becomes clear that their original decision was not correct, although sometimes the review tool and/or workflow make this difficult.
The review platform should also be customizable enough to allow for quality control restrictions that can be applied to families of documents such as threaded e-mails and attachments. For example, the protocol of the review might be that if any e-mail attachment is considered privileged, then that makes the whole family group (e-mail and attachments) privileged. If such protocols do exist they should be captured and executed automatically by the software to ensure consistency.
Quality control monitoring may include a second-level review of all or a percentage or a technically valid sampling (by hash code, Bates number, or other means) of the documents in the review collection. Second-level review is performed by more experienced attorneys who are very familiar with the matter under review. This typically is done with regard to documents coded as privileged or protected. In many law firms, all documents initially deemed privileged are reviewed by at least one, if not two additional and highly experienced attorneys.
In addition to or in lieu of a second-level review, the lead attorney or a specified quality control team may randomly review the team’s coding to check for inconsistent application of the review protocol. Both of these steps provide for validation of coding decisions and can bring to light inconsistent coding patterns that then can be addressed with the entire review team. Both should be done on a daily basis in order to identify problems with the review process or with an individual reviewer’s understanding of the process.
Discussing quality control provisions with your consultant/vendor is advised as they may be able to apply some automated procedures to ensure consistency or highlight areas of concern.