Lay Use of Reports Published by the National Academies

Overview

Advocates for open access have argued that the public needs scientific information. Institutions have invested in making such information publicly available. Yet evidence that the public uses scientifically based information is absent outside of patients seeking treatment information. For the first time, we reveal in detail, at scale and across science how the public uses scientifically based information, specifically reports of the National Academies of Sciences, Engineering and Medicine. Our results reveal adults across the country seeking the highest quality information to improve how they do their job, to help family members, to satisfy their curiosity and to learn. The picture contrasts starkly with the dominant narrative of a misinformed and manipulated public targeted by social media.

Approach

The public information ecosystem has broadened in recent years leading to concerns over misinformation and manipulation. However, exclusive focus on the very real dysfunction serves to obscure the fact that the broadening has also made accessible to the general public the highest functioning elements of the ecosystem, that the public seeks out such information and uses it. Attending to such use is important to justify the investment in producing and making freely available high quality, scientifically based information. We deploy BERT, a highly proficient machine learning text classification model, to reveal how the general public makes use of National Academies’ reports. We use BERT to classify 1.5 million comments left by US based users downloading National Academies’ reports. Downloaders were responding to a prompt asking how they intended to use the report. We created an initial classification scheme using keywords and then refined it in creating a ground truth training set for fine tuning the BERT model. We used the fine tuned model to classify comments into 64 use categories. The results provide the first detailed empirical evidence of public use of open access scientifically based information outside medicine. We find that research, teaching or studying accounts for about half of report use. The other half involves varied informational, professional and personal purposes detailed in our taxonomy and including, for example, veterans submitting disability claims to the VA, local disaster resilience planning committees, retired people and others seeking personal edification. We are able to quantify the magnitude of different types of use and types of report downloaded.

The Public Uses National Academies Reports

Investigators

Professor Diana Hicks – School of Public Policy (Georgia Tech)

Professor Omar Asensio – School of Public Policy (Georgia Tech)

Read the paper here

Hicks, D., Zullo, M., Doshi, A., & Asensio, O. I. (2022).  Proceedings of the National Academy of Sciences119(9), e2107760119. https://doi.org/10.1073/pnas.2107760119