How can richness be estimated at the start of an active learning project?

Estimating richness at the start of an active learning project is crucial for determining the amount and quality of information contained within a sample set of documents. Coding a sample set of documents allows you to assess the complexity and diversity of the data you are working with. This process involves reviewing a small subset of the total documents to identify various categories, themes, or concepts that exist. By doing so, you can gain insights into the statistical distributions of these categories, which helps in understanding the overall richness of the information present in your larger dataset.

This coding exercise provides a tangible basis for measuring richness because it directly involves analyzing the content to see how varied and informative the data is. As you code more documents, you can better predict how much effort may be required to achieve reliable results. Additionally, the insights gained from this process can inform strategies for the active learning process, such as how to prioritize which documents to label next based on their potential value to model improvement.

Other options, while they may contribute to understanding aspects of the project, do not directly measure the richness of the content in the same way. For example, reviewing past projects could provide insights into methodologies or outcomes, but it doesn’t assess the richness of the new data set directly. Surveying reviewer efficiency