Daejeon, July 27–31
Legislative transparency remains a cornerstone of democratic accountability. Yet, media coverage of legislative activities is inherently selective, creating an "information asymmetry" where certain activities receive disproportionate attention while others remain invisible (Shoemaker & Vos, 2009). This gap has widened as newsroom budgets have shrunk: the number of statehouse reporters has declined sharply in recent decades, and those who remain are more likely to be students or assigned only part-time, resulting in coverage that skews toward the final passage of major bills rather than the broader legislative process (Howe et al., 2024; Enda, Matsa, & Boyles, 2014; Weiss, 2015). In California alone, thousands of hearings occur annually, but only a fraction receive media coverage. This selective filtering, driven by both editorial gatekeeping and resource constraints, raises critical questions for digital humanities scholarship: What makes a legislative hearing newsworthy? Can computational methods predict which activities will attract media attention?
This research presents a machine learning framework for predicting California legislative bill coverage, using Digital Democracy's AI-generated "tip sheets" and articles from CalMatters (calmatters.org), a leading non-profit news agency focusing on California. By combining semantic similarity-based dataset augmentation with diverse feature engineering, we achieve 89.91% accuracy in identifying newsworthy activities over the total lifetime of a bill. This work demonstrates how natural language processing and machine learning can bridge the gap between vast legislative information and limited media resources, contributing to more civic engagement and government transparency.
Media gatekeeping theory (White, 1950; Shoemaker & Vos, 2009) explains how journalists determine newsworthiness, with recent work updating traditional criteria for the digital age (Harcup & O'Neill, 2016). Computational journalism has transformed news production, with researchers exploring how algorithmic methods can analyze large datasets and identify patterns that would otherwise remain hidden (Cohen et al., 2011; Anderson, 2013). Lewis and Westlund (2015) explore how big data transforms journalistic epistemology, expertise, and ethics, noting that algorithmic approaches to newsworthiness prediction raise questions about editorial judgment and professional autonomy. Bandari et al. (2012) reported 84% accuracy for predicting popularity of a news article on Twitter based on features of the article. Howe et al. (2024) demonstrate a complementary approach: AI-generated tip sheets that enable local reporters to cover state legislative news from afar, effectively expanding coverage capacity without requiring physical presence at legislative meetings.
Recent computational political science advances enable large-scale legislative text analysis (Grimmer & Stewart, 2013; Wilkerson & Casas, 2017), including parliamentary debate sentiment analysis (Abercrombie & Batista-Navarro, 2020; Patz et al., 2025) and legislative text segmentation (Siqueira et al., 2024). However, predicting which legislative activities will receive media coverage remains underexplored, leaving a significant gap between media studies research on gatekeeping and computational approaches to legislative analysis.
This research uses Digital Democracy, a platform providing AI-generated summaries of California legislative activities (Ruprechter et al., 2018). Figure 1 shows the platform interface, where AI-generated "tip sheets" aggregate hearing transcripts, voting records, stakeholder testimony, and engagement data into structured datasets, providing rich information for computational analysis.
We address two problems: (1) bill-to-article mapping to expand training data, and (2) newsworthiness prediction. Figure 2 illustrates our framework.
We have identified news articles from CalMatters tagged with key words including bills being discussed in the California Legislature. However, tagging is an imperfect reporter-driven manual process and we find that some other news articles also cover pending bills, but are not tagged accordingly. Thus our first challenge is to construct a corpus of articles larger than the set tagged by CalMatters.
Starting with 424 bill/article pairs from CalMatters covering the 2023-2024 legislative session, we develop a semantic similarity engine using sentence transformers (all-MiniLM-L6-v2), generating 384-dimensional embeddings (Reimers & Gurevych, 2019), providing us with another 1,400 articles auto tagged with bills of the same session that we can use for prediction training.
The legislature discussed over 3,000 distinct bills in the 2023-2024 session. We have about 10,000 tip sheets from the Digital Democracy project, each in the form of a json file that contains various aspects of a bill discussion taking place as part of a committee hearing or a floor session of the California legislature. See Table 1 for the specific parts of the tip sheets that we use in feature selection.
The semantic similarity approach computes cosine similarity between all bill/article pairs. We test multiple similarity thresholds (0.5-0.8), manually validating 100 samples at threshold 0.6 on a 3-point scale: (3) Highly Related, (2) Somewhat Related, (1) Not Related. This validation protocol addresses legislative data annotation challenges (Darji et al., 2024) through systematic human oversight of automated discovery methods.
We use a simple binary label for data points. The positive class means bill information corresponds to a CalMatters news story at a later date, hence the bills in this class can be called “newsworthy”. The negative class means there was no publication. Each data point consists of one or more legislative events or tip sheets each with their own metrics. For the positive class, we aggregate the information from all the hearings prior to the article publication. For each data point in the positive class, we randomly select a bill for which we have tip sheet information but no article publication as a negative class data point. We also match the number of hearings or tip sheets for the negative counterpart with the positive data point. This one-to-one correspondence creates a balanced data set while keeping the positive and negative classes comparable.
We engineer diverse features(Table 1): (1) TF-IDF textual features from bill summaries, hearing transcripts, and engagement data, (2) temporal features like number of hearings, days since first hearing, cumulative newsworthiness which are necessary because typically multiple legislative events occur before an article is published, and thus we must combine all historical tip sheets and associate them as a group with the article (3) sentiment scores, (4) engagement metrics (speakers, duration), and (5) contextual metadata (committee, dates). Nine algorithms are evaluated, and their accuracies, precision, recall, and F1 scores are measured.
| Feature Category | Feature | Description |
| Text Features (TF-IDF) | Transcripts | Real-time debate and testimony on bills generated by CalMatters |
| Bill Analysis | Official staff-generated analysis of the bill including listing of supporting and opposing organizations. | |
| High Engagement | A measure of intensity of debates and back-and-forth questioning | |
| Non-Text Features | Sentiment Score | Emotional tone of the hearing speeches |
| Engagement Metrics | Average number of speakers during the hearing and average duration of the discussions | |
| Committee | Name of the policy committee assigned to the bill (i.e. Health, Education, Agriculture, etc.) | |
| Temporal Features | Aggregation method | Aggregates all tip sheets in data set prior to the prediction date of the article |
| Hearing count | How many times the bill was discussed formally | |
| Benefit | Captures the escalation of the bill's importance over time |
Manual validation achieved 93% accuracy (93/100 samples rated ≥2), with 44% "Highly Related," 49% "Somewhat Related," and 7% "Not Related" (Figure 3). The 95% confidence interval (86.1%-97.2%) validates threshold 0.6 for automated discovery.
The semantic engine discovered 1,400 additional pairs, expanding the dataset by a factor of 3.3 (424→1,824 samples). Figure 4 shows XGBoost accuracy on newsworthiness prediction improved from 78.5% (original) to 89.91% (expanded), an 11.41 percentage point gain, validating that AI-discovered pairs contribute meaningful signals.
XGBoost achieved 89.91% accuracy, 90.81% precision, and 90.36% F1-score (Figure 5). LightGBM (89.75%) and CatBoost (89.49%) have comparable F1 scores, while linear models (Logistic Regression: 78.00%, SVM: 73.5%) and simpler models (Random Forest: 75.73%, Naive Bayes: 61.36%, KNN: 61.13%) lagged significantly. The 12.36-point gap between XGBoost and Logistic Regression demonstrates that for newsworthiness, complex non-linear patterns do better.
Figure 6 and Table 2 show temporal prediction performance. The 30-day window achieves 78.68% accuracy with 81.88% precision, an 11.23-point drop from full-timeline but valuable for planning. The 60-day window (78.53%) shows minimal drop. Version-based analysis (Figure 7) using Logistic Regression shows small progressive improvements from 73.60% (Version 1) to 76.97% (Version 5).
This research demonstrates legislative newsworthiness can be predicted with 89.91% accuracy, validating that editorial judgment exhibits systematic patterns (Bandari et al., 2012). Early prediction accuracy (78.68% at 30 days, 74.16% at Version 1) is promising in that it enables proactive resource allocation.
| Prediction Window | Accuracy | Precision | Recall | F1-Score | Samples |
| Full Timeline | 89.91% | 90.81% | 89.91% | 90.36% | 1,824 |
| 30 Days | 78.68% | 81.88% | 75.98% | 78.82% | 2,476 |
| 60 Days | 78.53% | 81.72% | 75.84% | 78.67% | 2,476 |
| Next Hearing | 69.52% | 73.45% | 65.12% | 69.05% | 2,051 |
In general we observe higher precisions compared to recall and this is preferred since it minimizes false positives that waste journalistic resources (Cohen et al., 2011; Lewis & Westlund, 2015). The semantic augmentation approach (93% precision, 3.3X expansion) demonstrates that pre-trained models enable high-quality data expansion addressing legislative annotation challenges (Darji et al., 2024).
We rely on CalMatters publications as a proxy for what is newsworthy or publishable in general. CalMatters is the biggest and most prominent news agency that is focused solely on state-wide issues and California legislature, but still a single outlet is probably not representative of the broader media landscape. The other shortcoming may be the binary labeling of our corpus based on “published” / “not published” which we use as a proxy for “newsworthiness” which is likely an oversimplification. Future work should incorporate multiple sources, develop nuanced coverage measures, expand to other jurisdictions, and explore newsroom integration while preserving editorial autonomy (Lewis & Westlund, 2015).
Abercrombie, Gavin / Batista-Navarro, Riza (2020): “Sentiment and position-taking analysis of parliamentary debates: A systematic literature review”, in: Journal of Computational Social Science 3, 1: 245–270.
Anderson, C. W. (2013): “Towards a sociology of computational and algorithmic journalism”, in: New Media & Society 15, 7: 1005–1021.
Bandari, Roja / Asur, Sitaram / Huberman, Bernardo (2012): “The pulse of news in social media: Forecasting popularity”, in: Proceedings of the 6th International AAAI Conference on Weblogs and Social Media.
Cohen, Sarah / Hamilton, James T. / Turner, Fred (2011): “Computational journalism”, in: Communications of the ACM 54, 10: 66–71.
Darji, Harsh / Mitrović, Jelena / Granitzer, Michael (2024): “Challenges and considerations in annotating legal data: A comprehensive overview”, in: arXiv preprint arXiv:2407.17503.
Enda, Jodi / Matsa, Katerina Eva / Boyles, Jan Lauren (2014): America’s shifting statehouse press: Can new players compensate for lost legacy reporters? Pew Research Center.
https://www.pewresearch.org/journalism/2014/07/10/americas-shifting-statehouse-press/ [last accessed 07.05.2026].
Grimmer, Justin / Stewart, Brandon M. (2013): “Text as data: The promise and pitfalls of automatic content analysis methods for political texts”, in: Political Analysis 21, 3: 267–297.
Harcup, Tony / O’Neill, Deirdre (2016): “What is news? News values revisited (again)”, in: Journalism Studies 18, 12: 1470–1488.
Howe, Paul / Robertson, Craig / Grace, Lindsay / Khosmood, Foaad (2024): “Exploring reporter-desired features for an AI-generated legislative news tip sheet”, in: #ISOJ Journal 14, 1.
Lewis, Seth C. / Westlund, Oscar (2015): “Big data and journalism: Epistemology, expertise, economics, and ethics”, in: Digital Journalism 3, 3: 447–466.
Patz, Leonard / Beyer, Michael / Späth, Johannes / Bohlen, Leon / Zschech, Peter / Kraus, Mathias / Rosenberger, Johannes (2025): “Analyzing German parliamentary speeches: A machine learning approach for topic and sentiment classification”, in: arXiv preprint arXiv:2508.03181.
Reimers, Nils / Gurevych, Iryna (2019): “Sentence-BERT: Sentence embeddings using Siamese BERT-networks”, in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing.
Ruprechter, Thomas / Khosmood, Foaad / Kuboi, Takashi / Dekhtyar, Alexander / Guetl, Christian (2018): “Gaining efficiency in human assisted transcription and speech annotation in legislative proceedings”, in: Proceedings of the 19th Annual International Conference on Digital Government Research: 1–2.
Shoemaker, Pamela J. / Vos, Tim P. (2009): Gatekeeping theory. London: Routledge.
Siqueira, Fernando / Pressato, Daniel / Pereira, Felipe / da Silva, Nilton / Souza, Ewerton P. / Dias, Marcelo / de Carvalho, André (2024): “Segmenting Brazilian legislative text using weak supervision and active learning”, in: Artificial Intelligence and Law: 1–82.
Weiss, Stephanie (2015): “140 characters of news”, in: State Legislatures 41, 7: 42.
White, David Manning (1950): “The ‘gate keeper’: A case study in the selection of news”, in: Journalism Quarterly 27, 4: 383–390.
Wilkerson, John / Casas, Andreu (2017): “Large-scale computerized text analysis in political science: Opportunities and challenges”, in: Annual Review of Political Science 20: 529–544.