Arts, Faculty of
Vancouver School of Economics
DSpace
UBCV
Hackinen, Bradley Alexander
Doctor of Philosophy - PhD
University of British Columbia
Politicians and regulators rely on external expertise when setting policies, providing an opportunity for interest groups to influence government policy with persuasive information. This process forms the foundation of many models of lobbying and special interest politics, but is very difficult to study empirically. I use data from U.S. Federal rulemaking process to create new measures of influence, revealing that U.S. corporations use charitable donations to non-profits to influence regulators, and opening up possibilities for future research. In Chapters 1 and 2, I present co-authored research on the relationship between strategic advocacy in rulemaking and corporate philanthropy. For-profit corporations and non-profit entities are active in the rulemaking process and are arguably expected to provide independent viewpoints. However, non-profits also receive grants from firms with an interest in the outcomes of the regulatory process. Chapter 1 shows that corporations give large amounts of money to non-profits that participate in the rulemaking process and that, shortly after a firm donates to a non-profit, the non-profit is more likely to comment on rules for which the firm has also provided a comment. Chapter 2 extends the study of firms and non-profits to include the text content of regulatory comments, and the regulators response in the final rule. We find that, when a firm comments on a rule, the comments by non-profits that recently received grants from the firm’s foundation are systematically closer in content similarity to the firm’s own comments than to those submitted by other non-profits commenting on that rule. And, when a firm comments on a new rule, the discussion of the final rule is more similar to the firm’s comments when the firm’s recent grantees also comment on that rule. These patterns suggest that corporations strategically deploy charitable grants to induce non-profit grantees to make comments that favor their benefactors. In Chapter 3, I present a new technique for estimating influence of comments on final rules that improves on the approach used in Chapter 2. These new estimators can also control for each author's writing style with fixed effects.
Essays on Strategic Advocacy andInfluence in RulemakingbyBradley Alexander HackinenB.Sc., The University of Victoria, 2012A THESIS SUBMITTED IN PARTIAL FULFILLMENT OFTHE REQUIREMENTS FOR THE DEGREE OFDOCTOR OF PHILOSOPHYinThe Faculty of Graduate and Postdoctoral Studies(Economics)THE UNIVERSITY OF BRITISH COLUMBIA(Vancouver)September 2019© Bradley Alexander Hackinen 2019The following individuals certify that they have read, and recommend to the Faculty of Graduate andPostdoctoral Studies for acceptance, the dissertation entitled:Essays on Strategic Advocacy and Influence in Rulemakingsubmitted by Bradley Alexander Hackinen in partial fulfillment of the requirements forthe degree of Doctorate of Philosophyin EconomicsExamining Committee:Matilde Bombardini, Vancouver School of EconomicsCo-supervisorFrancesco Trebbi, Vancouver School of EconomicsCo-supervisorMarit Rehavi, Vancouver School of EconomicsUniversity ExaminerWie Cui, Peter A. Allard School of LawUniversity ExaminerDecio Coviello, HEC MontrealExternal ExaminerAdditional Supervisory Committee Members:Kevin Milligan, Vancouver School of EconomicsSupervisory Committee MemberiiAbstractPoliticians and regulators rely on external expertise when setting policies, providing an opportunityfor interest groups to influence government policy with persuasive information. This process formsthe foundation of many models of lobbying and special interest politics, but is very difficult to studyempirically. I use data from U.S. Federal rulemaking process to create new measures of influence,revealing that U.S. corporations use charitable donations to non-profits to influence regulators, andopening up possibilities for future research.In Chapters 1 and 2, I present co-authored research on the relationship between strategic advocacyin rulemaking and corporate philanthropy. For-profit corporations and non-profit entities are activein the rulemaking process and are arguably expected to provide independent viewpoints. However,non-profits also receive grants from firms with an interest in the outcomes of the regulatory process.Chapter 1 shows that corporations give large amounts of money to non-profits that participate in therulemaking process and that, shortly after a firm donates to a non-profit, the non-profit is more likelyto comment on rules for which the firm has also provided a comment. Chapter 2 extends the study offirms and non-profits to include the text content of regulatory comments, and the regulators responsein the final rule. We find that, when a firm comments on a rule, the comments by non-profits thatrecently received grants from the firm's foundation are systematically closer in content similarity tothe firm's own comments than to those submitted by other non-profits commenting on that rule. And,when a firm comments on a new rule, the discussion of the final rule is more similar to the firm'scomments when the firm's recent grantees also comment on that rule. These patterns suggest thatcorporations strategically deploy charitable grants to induce non-profit grantees to make commentsthat favor their benefactors.In Chapter 3, I present a new technique for estimating influence of comments on final rules thatimproves on the approach used in Chapter 2. These new estimators can also control for each author'swriting style with fixed effects.iiiLay SummaryThis dissertation explores how corporations and other interest groups influence public policy by com-municating with policymakers during the design process. It focuses on U.S. federal regulators, who arelegally required to accept public comments on proposed regulations and respond to the comments inthe final version. The first two chapters present co-authored work. We discovered that large U.S. cor-porations use charitable donations to non-profits to influence regulators. Non-profits are more likely tocomment on the same regulation as their donors in the year after receiving a grant, and the commentsthey submit are unusually similar in content to their donor's comments. We find suggestive evidencethat the strategy is effective by comparing the text of the resulting regulation with the commentsthemselves. In the third chapter, I present new and improved techniques for estimating how commentsinfluence regulations that will be useful for future projects.ivPrefaceThis dissertation is original, unpublished work.Chapters 1 and 2 are based on a working paper titled Hall of Mirrors: Corporate Philanthropy andStrategic Advocacy, co-authored with Marianne Bertrand, Matilde Bombardini, Raymond Fisman,and Francesco Trebbi. The paper was the result of collaborative discussions between all the authors.My coauthors provided data on corporate donations to non-profits and assisted with implementing theco-commenting regressions and writing the working paper. I provided data on the rule-making process,handled all the data linking, and took the lead on developing and implementing the text analysis. Icontributed to writing the working paper in the areas where we describe institutional features of theregulatory system, associated data collection, and the text analysis algorithm.Chapter 3 is my own independent work.vTable of ContentsAbstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiLay Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ivPreface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vTable of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viList of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiList of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiDedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiIntroduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Corporate Philanthropy and Strategic Advocacy . . . . . . . . . . . . . . . . . . . . . 71.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.2 Institutional context: Rulemaking process . . . . . . . . . . . . . . . . . . . . . . . . . . 121.3 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121.3.1 Charitable giving by foundations . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.3.2 Comments and rulemaking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.3.3 Basic data facts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141.4 Evidence based on charitable giving and non-profit commenting on regulations . . . . . 151.4.1 Co-commenting specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161.4.2 Regulation specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181.5 Getting paid not to comment: The role of hush money . . . . . . . . . . . . . . . . . . . 181.6 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201.7 Figures and Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 Text Similarity and Regulatory Influence . . . . . . . . . . . . . . . . . . . . . . . . . . 272.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272.2 Quantifying the similarity in content across regulatory comments . . . . . . . . . . . . . 272.2.1 Sample construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31vi2.2.2 LSA implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322.2.3 Similarity measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332.2.4 Rule similarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332.3 Comment impact analysis: Evidence from final rule citations . . . . . . . . . . . . . . . 332.4 Concluding remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362.5 Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373 Estimating Influence from Bag-of-words Text Features . . . . . . . . . . . . . . . . . 433.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433.2 Modeling textual influence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473.2.1 Bag-of-words influence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473.3 Estimating influence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483.3.1 Frequency-based estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483.3.2 Token and Document vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493.3.3 Monte Carlo experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523.4 Panel Estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 633.4.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 633.4.2 Panel estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 663.4.3 Modeling author writing style . . . . . . . . . . . . . . . . . . . . . . . . . . . . 673.4.4 Fixed effects estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 683.4.5 Monte Carlo Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 703.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83AppendicesA Regulation comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89A.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89A.2 Collecting metadata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89A.3 Identifying org comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89A.4 Collecting comment text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90B Additional tables and figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91viiList of Tables1.1 Co-commenting - Time-invarying specification . . . . . . . . . . . . . . . . . . . . . . . . 231.2 Co-commenting - Recent donation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241.3 Commenting on regulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251.4 Hush money . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262.1 Similarity of comments - Any donation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372.2 Similarity of comments - Recent donation . . . . . . . . . . . . . . . . . . . . . . . . . . 382.3 Sentiment - Any donation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392.4 Sentiment - Recent donation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402.5 Rule-comment similarity - Any donation . . . . . . . . . . . . . . . . . . . . . . . . . . . 412.6 Rule-comment similarity - Recent donation . . . . . . . . . . . . . . . . . . . . . . . . . 423.1 Summary of Estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79A.1 Organization name extraction accuracy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90A.1 Top Agencies by Number of Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91A.2 Co-commenting in time-varying sample - Contemporaneous and lagged donations . . . . 92A.3 Co-commenting in time-varying sample - Future donations . . . . . . . . . . . . . . . . . 93A.4 Similarity - Contemporaneous and lagged donations . . . . . . . . . . . . . . . . . . . . . 94A.5 Similarity - Sector Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95A.6 Similarity - Future Donation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96A.7 Rule-comment similarity - Robustness . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97A.8 Citations of Firm in Rule Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98A.9 List of Agencies on regulations.gov (A-F) . . . . . . . . . . . . . . . . . . . . . . . . . . 99A.10 List of Agencies on regulations.gov (F-Z) . . . . . . . . . . . . . . . . . . . . . . . . . . . 100viiiList of Figures1.1 Rulemaking process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211.2 Co-commenting and charitable donations . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.1 Single-sample estimation: This grid of plots summarizes the performance of the main linear estimatorswhen the same input documents are used for generating the rule and estimating influence. Each dotrepresents the mean performance on one docket across 100 samples with random βs. The solid horizontalline shows the average performance across all dockets. The horizontal axis is the number of commentsin the docket. The vertical axes show four measures of estimator quality: Correlation is the Pearsoncorrelation coefficient for βd and βˆd, log Mean Squared Error (MSE), Bias (mean βd− βˆd), Attenuation(equal to 1−α, where α is the slope coefficient in a regression of the form βˆd = αβd + ε), and Coverage(the fraction of estimates for which the estimated 95% confidence interval for βˆd includes the true βd).Vectors are generated from document-level LSA (k = 200). . . . . . . . . . . . . . . . . . . . . . . 583.2 Split-sample estimation: This grid of plots summarizes the performance of the main linear estimatorswhen the tokens of each input document are split into separate gen and est samples. The gensamples are used to construct the final rule, while the est samples are used to estimate influence.Each dot represents the mean performance on one docket across 100 samples with random βs (withthe same gen/est split). The solid horizontal line shows the average performance across all dockets.The horizontal axis is the number of comments in the docket. The vertical axes show four measures ofestimator quality: Correlation is the Pearson correlation coefficient for βd and βˆd, Log Mean SquaredError (MSE), Bias (mean βd − βˆd), Attenuation (equal to 1 − α, where α is the slope coefficient in aregression of the form βˆd = αβd + ε), and Coverage (the fraction of estimates for which the estimated95% confidence interval for βˆd includes the true βd). Vectors are generated from document-level LSA(k = 200). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 593.3 Split-sample vector performance comparison: This grid of plots summarizes the performance of threedifferent types of vectors in the VFGLS estimator when the tokens of each input document are split intoseparate gen and est samples. The gen samples are used to construct the final rule, while the estsamples are used to estimate influence. Each dot represents the mean performance on one docket across100 samples with random βs (with the same gen/est split). The solid horizontal line shows the averageperformance across all dockets. The horizontal axis is the number of comments in the docket. Thevertical axes show four measures of estimator quality: Correlation is the Pearson correlation coefficientfor βd and βˆd, Log Mean Squared Error (MSE), Bias (mean βd − βˆd), Attenuation (equal to 1 − α,where α is the slope coefficient in a regression of the form βˆd = αβd + ε), and Coverage (the fraction ofestimates for which the estimated 95% confidence interval for βˆd includes the true βd). . . . . . . . . 60ix3.4 Split-sample vector performance comparison by k: This grid of plots summarizes the performance ofthree different types of vectors in the VFGLS estimator when the tokens of each input document aresplit into separate gen and est samples. The gen samples are used to construct the final rule, whilethe est samples are used to estimate influence. Each dot represents the mean performance averageperformance across all dockets for vectors truncated to length k. The vertical axes show four measuresof estimator quality: Correlation is the Pearson correlation coefficient for βd and βˆd, Log Mean SquaredError (MSE), Bias (mean βd − βˆd), Attenuation (equal to 1 − α, where α is the slope coefficient in aregression of the form βˆd = αβd + ε), and Coverage (the fraction of estimates for which the estimated95% confidence interval for βˆd includes the true βd). . . . . . . . . . . . . . . . . . . . . . . . . . 613.5 Split-sample weighting performance comparison: This grid of plots summarizes the performance of fourtoken weighting schemes in the VFGLS estimator when the tokens of each input document are split intoseparate gen and est samples. The gen samples are used to construct the final rule, while the estsamples are used to estimate influence. Each dot represents the mean performance on one docket across100 samples with random βs (with the same gen/est split). The solid horizontal line shows the averageperformance across all dockets. The horizontal axis is the number of comments in the docket. Thevertical axes show four measures of estimator quality: Correlation is the Pearson correlation coefficientfor βd and βˆd, Log Mean Squared Error (MSE), Bias (mean βd − βˆd), Attenuation (equal to 1 − α,where α is the slope coefficient in a regression of the form βˆd = αβd + ε), and Coverage (the fractionof estimates for which the estimated 95% confidence interval for βˆd includes the true βd). Vectors aregenerated from document-level LSA (k = 200). . . . . . . . . . . . . . . . . . . . . . . . . . . . 623.6 This grid of plots summarizes the performance of the panel estimators when the rules are generatedwith no style effects. Each dot represents the mean estimate for one commenter across 1000 sampleswith random βs. The solid horizontal line shows the average performance across all commenters. Thehorizontal axis is the similarity between the commenter's style and the regulator style, as measured bythe correlation between x¯i and y¯. The vertical axes show four measures of estimator quality: Correlationis the Pearson correlation coefficient for βd and βˆd, log Mean Squared Error (MSE), Bias (mean βd−βˆd),Attenuation (equal to 1− α, where α is the slope coefficient in a regression of the form βˆd = αβd + ε),and Coverage (the fraction of estimates for which the estimated 95% confidence interval for βˆd includesthe true βd). Vectors are generated from document-level LSA (k = 100). . . . . . . . . . . . . . . . 743.7 This grid of plots summarizes the performance of the panel estimators when the rules are generated withstyle effects. Each dot represents the mean estimate for one commenter across 1000 samples with randomβs. The solid horizontal line shows the average performance across all commenters. The horizontal axisis the similarity between the commenter's style and the regulator style, as measured by the correlationbetween x¯i and y¯. The vertical axes show four measures of estimator quality: Correlation is the Pearsoncorrelation coefficient for βd and βˆd, log Mean Squared Error (MSE), Bias (mean βd− βˆd), Attenuation(equal to 1−α, where α is the slope coefficient in a regression of the form βˆd = αβd + ε), and Coverage(the fraction of estimates for which the estimated 95% confidence interval for βˆd includes the true βd).Vectors are generated from document-level LSA (k = 100). . . . . . . . . . . . . . . . . . . . . . . 75x3.8 This grid of plots shows the point estimates for all 1000 samples for three randomly selected commenters(indicated by colour) using panel VOLS. Each horizontal axis indicates the true value being estimated,while the vertical axis indicates the estimate. Points along the 45 degree line are correctly estimated.In the top row, rules are generated with no style effects, while in the bottom row the rules are generatedwith style effects. In the left column the estimator has no fixed effects, while in the right column theestimator uses style fixed effects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76xiAcknowledgmentsI would like to thank to the Izaak Walton Killam Memorial Fund, Social Sciences and HumanitiesResearch Council, and Vancouver School of Economics PhD program for providing funding that sup-ported this research. I am grateful to my two excellent supervisors Matilde Bombardini and FrancescoTrebbi for helping to set me on this research path, and for the support, guidance, and motivation theyprovided along the way. Kevin Milligan was a valuable committee member, and provided both usefulperspective and detailed feedback at each stage of developing this research. I was lucky to meet manywonderful people in the PhD program who made the good times better and the hard times much morebearable. Most of all, I would like to thank Amelja Zoehner for her patience and support throughoutthe process, and Arthur and Fiona, who arrived part-way through the PhD to remind me that thereare more important things than research.xiiTo Amelja, Arthur, and FionaxiiiIntroductionIn the U.S., corporations and other special interest groups spend billions of dollars annually on campaigndonations, lobbying, and other more secretive activities in the hope of shaping public policy at federal,state and local levels (Drutman, 2015; Bertrand et al., 2018, 2014). There are also are strong revolvingdoor patterns in many industries (where people move back and forth between private and public-sectoremployment), and becoming a lobbyist after public service is the expected career path for members ofcongress and their staff (Vidal et al., 2012). If we want to understand why we see the policies we do,or how to improve them, we need to understand how these activities influence government decision-making. Many researchers have explored these questions of political influence, but the literature isbiased toward theoretical analysis and restricted by the scarcity of good data.In this dissertation I explore what can be learned about political influence using large-scale dataanalysis of U.S. federal rulemaking. The U.S. federal rulemaking process provides an unusually trans-parent view of interactions between special interest groups and government. Under the AdministrativeProcedures Act of 1946, regulators who make new rules (regulations, interpretive guidance, and andother policies with legal weight) must provide a mechanism for public input and justify their decisionsin relation to that input. The most important format for feedback is the notice-and-comment processfor new rules. In this process, regulators publish a proposed version of the new rule, provide a way forthe public to submit written comments, and then summarize those comments and respond the mainarguments in the text of the final rule. This process is now fully digital, and both comments andregulatory documents can be downloaded in bulk. U.S. federal agencies have published about 2,500rules per year over the last two decades, and received millions of public comments. I have collectedand organized a near-universal sample of comments and regulations for U.S. federal rulemaking since2003 containing 6 million public comments, and hundreds of thousands of documents published byregulators. The scale, detail, and breadth of this data across agencies, industries, and commenterscould transform our understanding of political influence.To capitalize on the potential of rulemaking data we must overcome two challenges. First we mustlink documents to each other to form a usable database, identify the authors of comments, and linkcomments to rules. Second, we must construct a measure of influence from the text of the documents.My work in this dissertation addresses both challenges.I have largely overcome the challenge of linking the documents by using several custom machine-learning based algorithms. In my data, comments are linked to regulatory documents, proposals arelinked to rules, and I have identified about 1 million comments submitted by specific organizations.Chapter 1 shows how useful this linked data can be. With Marianne Bertrand, Matilde Bombardini,Raymond Fisman, and Francesco Trebbi, I study how firms use corporate philanthropy to influenceregulators. We show that non-profit organizations often comment on the same regulation as their donor1firm, and that the probability of this type of co-commenting is much higher in the year following adonation.I have also made progress on the challenge of extracting useful information about influence fromthe text of the documents. In Chapter 2 I use Latent Semantic Analysis (an existing topic modelingalgorithm) to measure similarities between text in different documents. I show that comments betweenfirms and non-profit organizations are more similar when the firm has donated to the non-profit. Andby comparing the text of comments to the final rule, we find patterns that suggest the strategy ofdonating to non-profit organizations may indeed help firms shape public policy.In Chapter 3 I show how influence can be measured more directly and precisely with new estimatorsthat combine ideas from topic modeling and econometrics. The estimators overcome several challengesthat make large-scale analysis of rulemaking data difficult or impossible with existing techniques, andimprove on the LSA similarity measure developed in chapter 2 in terms of precision and interpretability.Another benefit is that the new estimators naturally extend to a panel data setting and allow for theseparation of fixed writing style from regulation-specific comment content.Together, these chapters represent a significant step forward in the study of political influence usinglarge-scale data from the rulemaking process.Related LiteratureThe research in this dissertation contributes to a growing part of the political economy literature thatuses empirical techniques to study how private firms and other special interest groups influence publicpolicy in the U.S. and other advanced economies. The defining idea behind Political Economy isthat there is no benevolent and omniscient government agent to turn to for policy design. Instead,government policy is the product of imperfect institutions and actors, who are often motivated by selfinterest. When we study influence, we hope to understand how actors outside the government affectdecisions made inside the government.Theoretical foundationsThe theoretical foundation of the literature on influence is very general, particularly early theoreticalresearch that explores lobbying and special interest politics. Part of this literature evolved out of PublicChoice theory, which aims to understand how large-scale empirical patterns (like industry-specific tradebarriers or economic subsidies) can be explained by the outcome of interactions between rational agentswho represent special interest groups and government. The workhorse model that developed out ofthis literature is the common-agency model of lobbying (Rodrik, 1995; Grossman and Helpman, 1994;Persson and Tabellini, 2000). This model takes the existence of different organized groups as given.Each group has a corresponding local public good which benefits only the group, and the governmenttaxes all players to provide public goods to each group. Groups can be either organized to lobby or not.Lobbying influence is modeled as a quid-pro-quo exchange, where interest groups bid to provide rentsto the government in return for increased provision of their preferred local public good. The modelis simple to work with, and generates the result that public goods are over-provided to organized2(lobbying) groups, and under-provided to the disorganized groups. Because it is simple and agnosticabout institutional details, the model is also easy to apply in many different contexts. The commonagency model gives us the intuition that industries that are better organized politically (or have moreconcentrated ownership) will obtain more preferable outcomes from the rulemaking process.The common-agency model is closely related to the idea of regulatory capture (Stigler, 1971),where a particular firm or industry becomes so effective at influencing regulators that policies aredesigned primarily for industry benefit. Theories of regulatory capture emphasize the principle-agentrelationship between the main government agent and the agency tasked with regulating a particularindustry, and how the agency has an incentive to collude with industry instead of maximizing thegovernment's objective. Like the common-agency model, regulatory capture does not depend on aparticular mechanism of influence, and the literature in this area considers a variety of mechanisms, fromsimple bribery (Dal Bó, 2006), to career incentives that accompany revolving door relationships (Cohen,1986; Makkai and Braithwaite, 1992; Lucca et al., 2014), to more subtle forms where agency viewsabout optimal policies are shifted through repeated interactions with industry or industry propaganda(Shapiro, 2012).An important mechanism of influence that appears in many models is strategic information provi-sion. Bribery and simple quid-pro-quo exchanges are theoretically simple, but do not provide satisfyingexplanations for many of the activities that special interest groups engage in such as lobbying, com-menting on rulemaking, or media campaigns that appear to be primarily about shaping the informationavailable to policymakers. There is a large literature that seeks to understand this type of commu-nication from a game-theoretic perspective (Grossman and Helpman, 2001). The general assumptionis that special interest groups are also domain experts with private information that is relevant forpolicy design. The government can improve policy if special interest groups truthfully report theirinformation to the policymaker. But special interest groups have an incentive to mislead the govern-ment into designing policies in a way that maximizes the interest group's private benefit instead of thegovernment objective. When communication is costly, we call it a signaling model, and the cost ofcommunicating makes the choice to send a message meaningful in a way that can convey informationin itself. When communication is free, we refer to the setting as a cheap talk model, and successfultransmission of information is more complex.The literature on cheap talk is enormous, and has some surprising twists and turns. In the canonicalmodel by Crawford and Sobel (1982), the interest group (sender) communicates to the government(receiver) about a state of the world that is represented by a point on the one-dimensional unit interval.The authors show that the amount of information that can be transmitted depends on the alignment ofpreferences between the interest group and government. When the interest group has similar preferencesto the government they can transmit high quality information in equilibrium, but as their preferencesdiverge, the precision of the information that they can transmit decreases. As intuitive as it is, thisresult turns out to be quite sensitive to the setup of the model. Battaglini (2002) shows that in a modelwhere multiple senders communicate with a receiver in a multi-dimensional, unbounded state-space,it is possible to convey information perfectly regardless of the preferences of senders or receiver. Asa rejoinder, Takahashi and Ambrus (2008) recover some of the intuition of the Crawford and Sobel3(1982) with an extension that adds bounds back into the multi-dimensional state-space. When thestate-space is bounded, some messages are no longer credible, and the relative preferences of interestgroups and government matter once more. Similar sensitivity to modeling assumptions can be seen inmodels with different timing, presence of reputation, or other variations (for example, see Aumann andHart (2003); Ottaviani and Sørensen (2006); Chakraborty and Harbaugh (2010)). Cheap talk modelsalso always have multiple Nash equilibria, adding another degree of freedom in the form of equilibriumselection. One takeaway from this literature is that the details matter in these strategic informationtransmission games, and that it is hard to predict what types of communication will be possible andwho will benefit from them without extremely precise information about the institutional structureand payoffs to all players.Empirical lobbying studiesThe research in this dissertation is closely related to empirical studies of lobbying. These studies aimto understand how interest groups use lobbying to influence policy, who lobbies who, how effectivelobbying is at achieving interest group objectives, and the role of lobbyists as intermediaries betweeninterest groups and government. De Figueiredo and Richter (2014) provide an excellent overview ofestablished results.Ideally, empirical research can answer questions that theoretical analysis cannot, especially in areaswhere it is not clear what assumptions are reasonable. In practice, empirical research faces its ownlimitations. One constraint is that there are only a small number of good data sources. The mostpopular data comes from the U.S. federal Lobbying Disclosure Act, which requires that lobbyists recordhow much clients pay them and what issues they lobby on. Some states and cities release similar data.Lobbying expenditure is most useful when examining the market for lobbying services. For example,Vidal et al. (2012) use the retirement of politicians to show that lobbyists are valued primarily fortheir political connections and they lose business when their contacts in government leave office. And,Bertrand et al. (2014) find that lobbyists move from topic to topic according to who they know ingovernment, not what they know about policy issues. However, lobbying expenditures are hard toconnect to an outcome. Lobbyists only need to report what branch of government they are contactingand a general topic (sometimes a bill). The data does not reveal anything about the specific messagesconveyed from interest groups to politicians, and it is hard to attribute changes in policy to specificlobbying interventions.Some researchers have used structural modeling as a substitute for more complete data. In tradepolicy there is a strand of research which investigates the relationship between political influence andtrade protections (Goldberg and Maggi, 1999; Bombardini, 2008). In this context, trade models clearlysuggest that protected industries benefit at the expense of unprotected industries. More recently,Huneeus and Kim (2018) look at the proximity of politicians to corporate headquarters and feed thisinstrument into a structural monopolistic competition model to estimate how lobbying affects thedistribution of firm sizes. In this model, the shift in firm sizes results in a large decrease in aggregateproductivity. These structural approaches are valuable, but depend heavily on modeling assumptionsthat are difficult to verify. For example, Huneeus and Kim (2018) assume that the effects of lobbying are4zero-sum, and only reallocate benefits between firms. But perhaps the information conveyed throughlobbying also has benefits (as suggested by some cheap talk models), such as reducing the probability ofthe government implementing economically disastrous policies. If the benefits of lobbying are included,this might substantially change the implied effects.Another approach is to code approximate data about the goals of interest groups and policy out-comes. Baumgartner et al. (2009) undertake a large study of congressional lobbying where they man-ually assign groups as either seeking to maintain or seeking to change the status quo on issues, andalso determine whether the policy did in fact change. They find little correlation between lobbyingeffort and whether the status quo changes, and argue that this is actually an intuitive outcome whenconsidering that the status quo already reflects the relative power of different groups.The emperical literature on lobbying makes it clear that more detailed data on interactions betweeninterest groups and government would be very valuable. Rulemaking offers an under-exploited sourceof data of this type.Empirical rulemaking studiesThe literature on rulemaking itself is quite small, and most of the most of the research comes fromlaw and political science. Golden (1998) provides an early analysis of public comments, focusing oneleven randomly selected rules from the EPA, NHTSA and HUD. Golden finds that most commentscome from business interests, and that nine out of the eleven rules are changed at least a small amountin response to content in comments. Wagner et al. (2011) perform a remarkably thorough analysis ofthe rulemaking process for the EPA's Air Toxic Emission Standards. By tracking contacts betweenthe EPA and interest groups over each rulemaking cycle (from pre-proposal to final rule), the authorsshow that the process is dominated by input by industry groups, but that public interests are moreactive early in the process. Comments come primarily from industry, and the EPA is more likely toreduce the stringency of rules in response to comments than to increase stringency.SusanWebb Yackee and co-authors advanced the study of rulemaking with a series of related studies.Yackee (2005) provides the another analysis of whether interest groups are able to affect policy throughthe notice-and-comment process, showing that regulators often adapt rules to address commenterrecommendations, particularly when there is consensus among commenters. Yackee and Yackee (2006)shows that most comments come from business interests, and that these interests are more successful inaffecting policy. McKay and Yackee (2007) explore the role of interest group competition in rulemaking,showing that agencies respond most the the group that dominates the commenting process. Naughtonet al. (2009) emphasizes the importance of interest groups that engage early in the process when therule is less well defined. Haeder and Yackee (2015) show how commenters can also be influential bycommunicating with the Office of Management and Budget which oversees parts of the rulemakingprocess on behalf of the president.Daniel Carpenter and Brian Libgoeber also have several working papers that explore rulemaking.Libgober and Carpenter (2018a,b); Libgober (2018) use stock price event studies of U.S. banks aroundthe time of rule announcements to estimate the financial effects of rules promulgated under the Dodd-Frank Act, and argue that the observed patterns imply the effect of comments is large (worth billions).5In related work, Libgober and Rashin (2018) analyze the content of comments to determine whatstrategies commenters are using to influence regulators. They find that most commenters provide pureinformation, with a smaller fraction communicating legal threats.Together, these existing empirical studies of rulemaking represent a path-breaking exploration ofinfluence in the rulemaking process. The main limitation is that they are all based on relativelysmall sets of rules and comments that were manually coded by the authors. My dissertation focuseson expanding the analysis to the universe of comments and rules. Expanding the data has severaladvantages: First, it allows us to more fully map the territory - it is not clear whether each resultabove holds in general, or only for specific industries and time periods. Second, having a large samplewith a panel structure allows more flexible specifications including using high-dimensional fixed effectsto control for confounders, and enough precision to extract information from those estimates. Finally,broader data allows us to ask basic questions like how the relationship between comments and rulesdiffers across industry, agency, and political cycle, as well as painting a more complete picture of howeach firm or interest group interacts with multiple agencies.A path forwardUltimately, it would be nice to be able to explore questions of policy design. For example, how can theinstitutions through which interest groups interact with government (such as congressional lobbying orthe notice-and-comment process for rulemaking) be designed to maximize the information that specialinterest groups provide to government and minimize zero-sum rent-seeking? Or, given the institutionsthat exist, how will other economic policies feed back into the political system? Some researchers andpublic intellectuals are concerned that lax U.S. anti-trust policy has allowed large firms to grow inpolitical as well as economic power, at the expense of more equal democratic representation (Zingales,2012). Should we be concerned about feedback from economic power to political influence? Finally,global challenges like climate change require rapid and large-scale social changes to address. Corporateinterests appear to have played a large part in downplaying climate risk and delaying governmentaction (Oreskes and Conway, 2011; Mayer, 2017). Are there institutional changes that would helpimprove how environmental policy is determined? Answering any of these questions requires a betterunderstanding of how interest groups affect government policy  not just in broad strokes, but detailsabout who gains and loses, and how these outcomes change under different economic and institutionalconditions. Large-scale analysis of rulemaking data provides a promising path forward.This dissertation is structured as follows. Chapter 1 presents the results of a co-authored studyusing rulemaking data to examine how non-profits advocate on behalf of their corporate donors. Chap-ter 2 extends this analysis to the text content of comments and rules using existing topic-modellingtechnology. Chapter 3 proposes novel estimators for measuring the effect of comments on rules fromtext data.6Chapter 1Corporate Philanthropy and StrategicAdvocacy1.1 IntroductionEconomists and political scientists have long studied  both theoretically and empirically  the roleinterest groups play in the formation of laws and regulations. In the U.S., as in many democracies,there are well-established channels through which interest groups can try to influence the laws andrules that may impact their communities, their businesses, or society at large. Through means such aslobbying, grassroots campaigns, testimonies, or public advocacy, interested parties inform politiciansand bureaucrats of the costs and benefits of government action.While interest groups may have expertise on topics of direct relevance to them, they may alsobe tempted to present information that is tainted by their self-interest. This logic is at the core ofthe literature on informational lobbying (Grossman and Helpman, 2001).1For example, oil companyrepresentatives may have expertise in drilling, but also a strong incentive to minimize, say, the predictedenvironmental costs of Arctic oil exploration. Government officials must thus weigh both the qualityof information and its impartiality, based in part on its source. As such, lawmakers and rulemakersmay view information provided by for-profit corporations as less credible if that information is notcorroborated by other groups with non-aligned (i.e., neutral or opposing) interests.Non-profit organizations often fall into the role of interests that are non-aligned with business.Some non-profits  such as research groups, universities, and think tanks  are providers of nonpar-tisan, technical expertise and are commonly expected to offer more neutral input into the lawmakingand rulemaking process, with a focus on cost-benefit analysis and broader societal interests. Othernon-profits  such as human services organizations, environmental protection groups, social welfareorganizations, and advocacy groups  may have opposing interests to business, to the extent that lawsor regulations that benefit their members (or those on whose behalf they advocate) adversely constrainbusiness profits. Non-profit organizations are therefore expected to play an important balancing rolein the informational lobbying process.This role may be subverted, however, by the financial links between corporations and non-profits: inexchange for donations, a non-profit may (consciously or otherwise) take a perspective that is favorableto its benefactor's bottom line. If politicians and bureaucrats are more likely to implement a proposalwhen it is supported by interest-group diverse coalitions (as suggested theoretically in the strategic1By informational lobbying we refer to the broad literature on information transmission which encompasses cheaptalk and costly signalling models in the context of lobbying, for example Potters and Van Winden (1992), Austen-Smith(1993), Austen-Smith (1995) and Lohmann (1995).7advocacy literature, e.g., Krishna and Morgan (2001), Dewatripont and Tirole (1999), Dewatripontand Tirole (2005), and empirically in Lorenz, 2017) and if such ties are undisclosed, such coalitionbuilding via corporate giving may distort the outcome of the political process away from the publicgood and towards private interests.2The goal of this chapter is to provide systematic evidence establishing this to be an empiricallyrelevant phenomenon. The context of U.S. Federal Regulation, with its far-reaching economic implica-tions and its carefully documented record of communications between organizations and governmentagencies, offers an ideal setting to establish such evidence.There exists anecdotal evidence that these concerns are well-founded. Across a range of issues andregulatory agencies, researchers and journalists have documented cases of companies using charitablecontributions to co-opt ostensibly neutral and even non-aligned non-profits. Notably, Peng (2016)describes the efforts of telecommunications firms to win merger approvals in front of the Federal Com-munication Commission (FCC), in part by assembling diverse and vocal coalitions of supporters. Pengquotes Crawford (2013) on the Comcast-NBCU merger, in which [t]he company encouraged letters tothe FCC from more than one thousand non-profits...including community centers, rehabilitation cen-ters, civil rights groups, community colleges, sports programs, [and] senior citizen groups. For theAT&T/T-Mobile merger, Peng similarly documents letters of support addressed to the FCC from non-profits that, at first glance, would appear to have little interest or expertise in telecommunicationspolicy, including a homeless shelter in Louisiana, a special needs employment agency in Michigan, andthe Gay & Lesbian Alliance Against Defamation (GLAAD). The non-profits were all AT&T Founda-tion grantees (in the case of the homeless shelter, the donation had come in just five months beforethe merger was announced). In no case did the non-profit disclose its AT&T funding in its letter tothe FCC. In at least one case, the comments did not appear to represent the views of the non-profitmembership. According to Peng, GLAAD's president and six board members resigned when its mergerendorsement made headlines and revealed that the organization had received AT&T funds.Journalists and medical experts have documented similar persuasion-via-donation in public healthdebates. Jacobson (2005), for example, describes a (no-strings attached) $1 million donation fromCoca-Cola Foundation to the American Association of Pediatric Dentistry (AAPD), accompanied by ashift in the tone of AAPD statements on sugary beverages, from describing soft drinks as a significantfactor in tooth decay, to describing the scientific evidence of the relationship as unclear.3Similarconcerns have been raised with respect to the role of donations from corporations to university research2Implicitly we are presuming that Coasian bargaining in the political sphere does not already lead to efficient policies.To the extent that, for example, it is difficult to contract across multiple regulatory agencies and/or pieces of legislation(let alone make outright side payments), one may think of the government as aiming to set optimal policy on a rule-by-rulebasis, assigning winners and losers in each instance. See, e.g., Acemoglu (2003), for a discussion.3A more direct link to policy can be found in the soda industry's efforts against New York City's ban on large sugarydrinks in the 2010s. In his decision to strike down the Bloomberg administration policy, the presiding judge cited amicusbriefs filed by two New York non-profits (the local chapter of the NAACP and the Hispanic Federation), which arguedthat the ban would disproportionately affect ethnic and racial minority groups. Both non-profits were recipients of fundsfrom Coca-Cola and PepsiCo. See Minority Groups and Bottlers Team Up in Battles Over Soda. The New YorkTimes March 12, 2013. Aaron and Siegel (2017) show that 95 national public health organizations received fundingfrom Coca-Cola and PepsiCo during 2011-2015; the study does not look, however, at the effect on organizations' publiclystated positions.8hospitals.4Investigative journalists have also documented many instances of companies influencing the policystatements of neutral non-profits that ostensibly provide evidence-based analysis on matters of publicinterest. Confidential memos and documents suggest that some think tank reports are discussed withcorporate donors before the research is complete, with donors potentially shaping the final reports,so that the resulting scholarship can be used to corroborate their separate lobbying efforts. In her2017 book Dark Money, journalist Jane Mayer , provides one prominent example, documenting howthe philanthropic activities of the billionaire industrialist brothers Charles and David Koch furtheredtheir efforts to influence political discourse: [The Koch brothers] subsidized networks of seeminglyunconnected think tanks and academic programs and spawned advocacy groups to make their argumentsin the national political debate. [...] Much of this activism was cloaked in secrecy and presented asphilanthropy, leaving almost no money trail that the public could trace. But cumulatively it formed,as one of their operatives boasted in 2015, a 'fully integrated network.' Raising concerns about suchpractices in general, Senator Elizabeth Warren, also a commercial law professor, observed that, [t]hisis about giant corporations who figured out that by spending, hey, a few tens of millions of dollars, ifthey can influence outcomes here in Washington, they can make billions of dollars.56In this chapter we show that the patterns discussed in these anecdotes hold more broadly in a settingin which we can plausibly draw a strong circumstantial connection between corporate donations andthe participation of non-profits in the political and regulatory process.We focus on the formation of federal rules and regulations. Federal agencies in the U.S. are legallyrequired to publish proposed rules in the Federal Register and accept public comments on those pro-posals before rules are finalized and comments discussed.78While there is no legal requirement foragencies to act on feedback received in comments, the agencies themselves often attribute changes be-tween proposed and final rules to arguments made via rulemaking. As emphasized by Sunstein (2012),4For example, Harris, Gardiner Top Psychiatrist Failed to Report Drug Income. The New York Times October 3,2008; Charles Piller and Jia You Hidden conflicts? Pharma payments to FDA advisers after drug approvals spark ethicalconcerns Science News July 5, 2018. See also Ross et al. (2000).5https://www.nytimes.com/2016/08/08/us/politics/think-tanks-research-and-corporate-lobbying.html6Warren also commented on the use of these practices in the rulemaking context, which we focus on in the empiricalanalysis below: Unlike congressional action, agency rules are constrained by well-established judicial review standardsthat seek to determine whether the agency's action is supported by the evidentiary record and the authority delegated toit by Congress. Rules must be supported by substantial evidence; agency actions must not be arbitrary and capricious.But corporate players are savvy. They have learned that those same judicial review standards can be used to suffocatenew rules. They play a sophisticated game leveraging their own expertise and paying outside experts with purportedlyindependent credentials to produce long, detailed comments filled with data and analyses, all selectively produced toserve their own interests. Discussing fixes, she also writes: Another [principle] would be to help agencies and courtsdistinguish between legitimate, high-quality data and research, on the one hand, and bought-and-paid-for studies on theother, by requiring disclosure of financial arrangements and editorial relationships associated with regulatory comments.See https://www.theregreview.org/2016/06/14/warren-corporate-capture-of-the-rulemaking-process/ (accessed October31, 2018).7The Administrative Procedures Act of 1946, 5 U.S.C. 553(c) states: . . . the agency shall give interested persons anopportunity to participate in the rule making through submission of written data, views, or arguments with or withoutopportunity for oral presentation. After consideration of the relevant matter presented, the agency shall incorporate in therules adopted a concise general statement of their basis and purpose. https://www.law.cornell.edu/uscode/text/5/553.Accessed October 31, 2018.8There are some exceptions for urgent actions or cases in which the change is so trivial that the agency does notexpect comments, but in general, agencies which fail to publish a sufficiently informative proposal or fail to follow thecommenting procedure can have their regulations vacated in court.9public commentary is also a valuable source of feedback to preempt regulatory mistakes when thestakes are high and the issues novel. Regulations.gov provides the largest single source for commentinformation on proposed rules, and was rolled out in 2003 when most agencies started a systematiceffort to digitize the commenting process. By 2008, 80% of all proposed rules provided a regulations.govlink for commenting, and the fraction is about 90% as of 2018.For the purpose of this chapter, we use regulations.gov to build a comprehensive dataset includingthe majority of the comments submitted in the rulemaking process since 2003. For each comment,we know the specific proposed rule the comment is in response to, as well as the content (text file)of the comment and the identity of the commenter. We may thus connect specific organizations tocommentary on the same proposed regulation and its final discussion (we refer to a sequence of rulepostings from proposal to final version as a regulatory stream or docket).We complement the commentary data with information on corporate foundations and their bene-ficiaries, using data on charitable donations by foundations linked to large corporations through taxforms filed to Internal Revenue Service (IRS). The combination of these datasets allows us to explorewhether (i) non-profits that benefit from corporate philanthropy are more likely to comment on thesame rule as their benefactors; (ii) conditional on both providing feedback on the same regulation, thenon-profits' comments are unusually similar to that of their benefactors; and (iii) co-comments by acorporate foundation's grantees lead to discussions of the rule by the regulator that use language thatis more similar to the language contained in the company's comments. By exploiting the particulartiming of corporate donations and comments, as well as the inclusion of firm-grantee pair fixed effects,we argue that we can plausibly draw a compelling link from funding to co-commentary and commentoverlap.9Our sample of firms is comprised of the companies that have appeared at any point in the 1995 to2016 lists of Fortune 500 or S&P 500 (or both) for which we identify a corporate foundation, and oursample of non-profits is the set of all grantees that received at least one donation from these foundationsover the period 1998-2015. Organizations (firms or non-profits) are linked by name via a fuzzy matchto 981,232 rulemaking comments made on all proposed regulations on regulations.gov during the years2003-2017. The main sample for our analysis is comprised of the 414 corporations with charitablefoundations and 11,746 grantees that commented at least once during this period.We show that non-profits are more likely to comment on the same regulation as their benefactors,and that this co-commentary is most strongly associated with donations in the year preceding thecomments. We consider several specifications: first, a cross-sectional regression that reveals strongcorrelations between donations and commenting on the same regulation independant of any particulartiming. Second, we break the commenting information into annual bins and show that there non-profitsare particularly likely to comment on the same regulation as their donor when they have received adonation in the year prior to the comment. This result is robust to the inclusion of pair-wise firm-grantee fixed effects which control for any fixed relationship between the donating and commenting of9Of course, this does not obviate the possibility that non-profits have time-varying policy preferences, and corporategifts coincide with shifts in these preferences. While we cannot rule out this possibility (a critique that applies evento the Coca-Cola/AAPD example mentioned above), our approach does help to rule out the possibility that latent,time-invariant shared interests drive both donations and comment overlap.10firms and non-profits. Our analysis implies that a donation in the preceding year is associated withnearly a doubling in the likelihood of co-commentary. We also run a similar regression at the regulationlevel and show that non-profits are more likely to comment on a particular regulation when they havereceived money from one of the firms that also comment. The regulation specification allows us toinclude regulation fixed effects which control for the fact that some regulations attract much morecommenting interest than others.Finally, we explore whether corporations use charitable donations to encourage otherwise opposingvoices to remain silent (rather than encouraging non-profits to provide supportive commentary). Whileit is challenging to devise a decisive test to detect the omission of comments that might otherwise havebeen made, we provide suggestive evidence, based on an extension of our main results on donationsand co-comment frequency, that hush money may not be of first-order importance in our setting.More specifically, we show that the link between co-commentary and donations is strongest in areas inwhich a non-profit most commonly provides comments, the opposite of what one might expect if hushmoney played a dominant role.Our findings first and foremost provide a contribution to the literature on the mechanisms bywhich interest groups seek to influence government policy (for canonical early contributions see, forexample, Grossman and Helpman, 1994, 2001 and for a more recent discussion Baumgartner et al.,2009; Bertrand et al., 2014; Drutman, 2015). We differ from much of this prior work in our focuson influence via expert commentary rather than financial contributions and, much more importantly,in documenting one mechanism by which private interests may cloak biased advice by inducing itsprovision by a non-obviously aligned party. This has implications for how we model the process ofgovernmental information acquisition (Austen-Smith, 1993; Laffont and Tirole, 1993), and is also ofdirect policy relevance. Our results suggest that calls for restrictions on financial relationships amongthose aiming to influence government policy may be well-founded, and that at a minimum potentialconflicts-of-interest statements should be required for any organization providing input on governmentregulations (Peng, 2016). Our work is also related to prior research that has shown the value ofcoalitions of diverse interest groups in the adoption of government policy. In particular, studying billsintroduced in Congress between 2005 and 2014, Lorenz (2017) shows that bills supported by interest-diverse coalitions are more likely to receive committee consideration; in contrast, Lorenz (2017) findsno association between committee consideration and lobbying coalitions' size, or their interests' PACcontributions. Generalizing beyond the lawmaking process, this work complements our findings inthat it suggests that corporations can expect some return for the type of charitable investmentswe uncover in this chapter. Other papers that have focused on returns to lobbying instead includeBombardini and Trebbi (2011, 2012); Kang (2016); Kang and You (2016). Finally, this chapter expandson earlier work highlighting how corporations may strategically use their corporate philanthropy as anundisclosed tool of political influence. Bertrand et al. (2018) show that corporations allocate more oftheir charitable giving to congressional districts that are more relevant to the corporations due to thecommittee assignments in the House of Representatives of their elected representatives. We identify inthis chapter another, independent, category of strategic CSR (Baron, 2001) in the government arena.111.2 Institutional context: Rulemaking processThe rulemaking process of U.S. federal agencies provides a context in which we may observe both thepresence and the content of communication by different entities with an interest in influencing thepolicymaker. While informational lobbying at the federal or local level does not come with statutoryrequirements of disclosure of the content or even the exact target of communication,10the rulemakingprocess consists of a series of codified procedures that regulate the activity of federal agencies in theproduction of rules under the Administrative Procedure Act of 1946 (APA). The subject of policydeliberation is a rule designed to implement, interpret, or prescribe law or policy, according to theAPA. The process of rulemaking may be set in motion by Congress passing a new law requiringimplementation or by an agency itself, upon regularly surveying its area of legal responsibility andidentifying areas that need new regulations.11Figure 1.1 sketches the process of informal rulemaking. It starts with a Notice of Proposed Rule-making (NPRM) including the objective of the rule and how it would modify the current Code ofFederal Regulations. The NPRM is published in the Federal Register, at which point the agency spec-ifies a period of 30 to 60 days during which the public can submit comments on the proposed rule.After comments have been received and additional information collected, the agency may proceed topublish a Final Rule in the Federal Register or issue a Supplemental Notice of Proposed Rulemakingif the initial rule was modified substantially, in which case further comments are invited. This notice-and-comment procedure is meant to include the general public and all interested parties in the craftingof the new rule. Importantly, the agency also publishes in the Federal Register a discussion of thegoals and rationale of the policy, and how the comments were incorporated into the final rule in theSupplementary Information section of the final rule.Occasionally, the process of rulemaking requires merging or splitting specific elements of a rule,issuing interim versions of the rule if the process is delayed, and more generally adapting to otherexternal factors, including further direction from the legislative branch, and so forth. These variousadditional documents are typically filed in dockets maintained by the regulatory agency.Upon finalization of the rule, comments represent part of the official record, and rules can bechallenged judicially on procedural or substantive grounds based on comments filed by entities thatparticipated in the rulemaking. Judicial review is an important constraint to rulemaking activity in theUnited States in that it effectively forces regulators to attend to opinions expressed via commentary.1.3 DataThis section introduces our sources and provides a brief overview of the data. For further detailswe refer to Appendices A and B. We begin by describing the data on charitable giving by corporatefoundations, followed by the data on public comments on rulemaking. The starting point for our10Under the Lobbying Disclosure Act of 1995, lobbying registration and reporting forms only require lobbyists to listthe topic and the agency lobbied (e.g., Trade, the Senate of the United States), in addition to clients and payments. SeeVidal et al. (2012); Bertrand et al. (2014).11Agencies may decide to engage in rulemaking under the recommendation of congressional committees, other agencies,or following a petition from the general public.12sample is the set of corporations that have appeared at any point during the period 1995 to 2016 inthe Fortune 500 and/or S&P 500 lists, which counts 1398 firms.121.3.1 Charitable giving by foundationsData on charitable donations by corporate foundations come from FoundationSearch, which digitizespublicly available Internal Revenue Service (IRS) data on the 120,000 largest active foundations in theU.S. We find 629 active foundations that can be matched by name to 474 of the initial list of 1398firms.13As noted in Brown et al. (2006), larger and older companies are more likely to have corporatefoundations, which results naturally from the fixed cost of establishing a foundation.14Each charitable foundation must submit Form 990/990 P-F Return of Organization Exempt FromIncome Tax to the IRS annually, and this form is open to public inspection. Form 990 includes contactinformation for the foundation, as well as yearly total assets and total grants paid to other organizations.Schedule I of Form 990, entitled Grants and Other Assistance to Organizations, Governments, andIndividuals in the United States, specifically requires the foundation to report all grants greater than$5,000. For each grant, FoundationSearch reports the amount, the recipient's name, city and state,and a giving category created by the database.15While the IRS assigns a unique identifier (Employer Identification Number, EIN) to each non-profit organization, FoundationSearch does not report this code, so we rely on the name, city and stateinformation to match a grantee to a master list of all non-profits. This list, called the Business MasterFile (BMF) of Exempt Organizations, is put together by the National Center for Charitable Statistics(NCCS) primarily from IRS Forms 1023 and 1024 (the applications for IRS recognition of tax-exemptstatus). The BMF file reports many other characteristics of the recipient organization, includingaddress, assets and a non-profit sector called the National Taxonomy of Exempt Entities (NTEE).The results of the matching between all public charities, private foundations or private operatingfoundations (designated as 501(c)3 organizations for tax purposes) in the BMF and the recipients ofcharitable giving by 2014 Fortune 500 and S&P 500 companies is reported in Bertrand et al. (2018).1.3.2 Comments and rulemakingThe source of data on comments, proposed, final and interim rules, as well as discussion of final rules isregulations.gov, a website through which the majority of U.S. federal agencies collect public commentsin the notice-and-comment phase of rulemaking.16The website regulations.gov API provides a searchfunction for document metadata.12The initial number of firms is 1434, but we combine firms that merge during the sample, hence obtaining a smallertotal number.13The 629 foundations we find are linked to 474 corporations, since there are instances of multiple foundations associatedwith the same corporation.14They also find that state-level statutes  in particular laws relating to shareholder primary and the ability of firmsto consider broader interests in business decisions  predict establishment of a foundation. Various endogenous financialvariables are also predictive of foundation establishment. The analysis in Brown et al. (2006) is cross-sectional, so theirvariables are absorbed by the various fixed effects in many of our analyses.15The 10 broad categories are: Arts & Culture, Community Development, Education, Environment, Health, Interna-tional Giving, Religion, Social & Human Services, Sports & Recreation, Misc Philanthropy.16For the complete list, see Appendix tables A.9 and A.10.13Our research sample consists of all comments posted to regulations.gov in the years 2003-2017. Weuse a custom machine learning tool to extract organization names from the comment metadata. Thealgorithm identified 981,232 comments that appear to be authored by organizations (as opposed toprivate individuals) and downloaded the full text of the comments. We are particularly interested incomments submitted by non-profits and by corporations that we observe in our FoundationSearch sam-ple. The comments are linked to corporations' and grantees' names through a custom name matchingtool that implements multiple types of fuzzy matching and manual corrections.17The unit of observation is what regulations.gov refers to as a docket. This is a way for agenciesto organize comments that relate to a particular topic. Most straightforwardly, one may think of aone-to-one correspondence between a rule and a docket. Conceived in this way, as mentioned above,a docket will contain all comments that pertain to all versions of that rule. An example of a simpledocket is FNS-2006-0044 from the Food and Nutrition Service (FNS) which contains a proposed rule(06-09136) and its corresponding final rule (E8-21293) on Fluid Milk Substitutions in the SchoolNutrition Programs. All comments in this docket therefore are easily linked to this regulatory stream.There are more complex cases in which a docket contains multiple proposed rules and notices (see,for example, docket EPA-HQ-OAR-2008-0699, the Environmental Protection Agency's review of theNational Ambient Air Quality Standards for Ozone). We associate all comments to the same docketgiven the homogeneity of the topic. The only exception is when we turn to examine the wording of thediscussion of final rules as a function of corporate and non-profit comments. There, we will considereach rule within the docket separately to ensure a finer connection between comments by corporationsand the exact wording of the final rule in the docket under discussion, as the multiple rules associatedwithin a docket are discussed and published separately in the Federal Register. We will elaborate onthis distinction in Section 2.3, which discusses those results.1.3.3 Basic data factsRecall that our sample starts with the set of companies that appeared at least once in the Fortune500 or S&P 500 lists between 1996 and 2015. Of the 1398 firms in that sample we find 909 that havecommented at least once in the period 2003-2016.18This is the sample of firms that forms the basis ofour regressions. We have a total of 22,654 firm comments over 5,792 dockets. Of these 909 firms 414have a foundation.In terms of non-profits we start from the 225,180 entities that received at least one grant fromany foundation in our sample over the period 1998-2015. Our sample consists of the 11,531 of thesegrantees that comment at least once at any point during the period starting in 2003. We have a totalof 318,841 comments in 8,729 dockets from those grantees.There is vast heterogeneity among firms in their activity in the commenting phase. The mostactively commenting firm, Boeing, provided comments on 1284 dockets. On average each firm commentson 18 dockets, but the distribution is skewed: the median firm comments on 6 dockets, while the firmsat the first and third quartile comment on 2 and 17 dockets, respectively. The distribution of comments17Available from the authors upon request.18We only consider comments starting in 2003 because this is when the comments database is complete.14among grantees is even more skewed. On average each grantee comments on almost 5 dockets, butthe median is 1 and the third quartile is 3 dockets. The most active grantee (Center for BiologicalDiversity) comments on 905 dockets.Tables A.1 reports the agencies that receive the highest number of comments from grantees andfirms.19At the top of the list for grantees are the EPA (Environmental Protection Agency), the FAA(Federal Aviation Administration) and the FDA (Food and Drug Administration). The top three agen-cies as recipients of grantees' comments are the FWS (Fish and Wildlife Service), the NOAA (NationalOceanic and Atmospheric Administration), and the HHS (Health and Human Services Department).It is worth noticing that the EPA, the FAA and the FDA feature in the top 10 agencies for granteesas well.Finally, we provide some information on the prevalence of commenting behavior of grantees andtheir co-commenting with firms in our sample. In our regressions we will often focus on recentdonations, defined as donations from a firm to a grantee that occur in the same year, or one year priorto a public comment on a rule. Consider the set of all firms-years where the firm has commented atleast once and donated recently. We can break the recipients of these recent donations in to a set ofnested groups with increasingly close ties to the firm.Firms donate to an average of 327 non-profit grantees. Of these, an average of 54 grantees eversubmit a comment in our sample. Within these commenters, 28 non-profits ever comment to one ofthe same agencies as the firm (not necessarily at the same time or in the same year), 8 ever co-commenton a regulation with the firm, and 1.4 co-comment with the firm that year.In terms of expenditures, the average total amount spent on donations over a two year period is$26 million dollars, 26% of which go to grantees that ever comment. Within commenting grantees,expenditures are biased towards the grantees with closer commenting ties to the firm. Thus, granteesthat never comment to the same agency as the firm receive an average of $103,412 each, and granteesthat comment to one of the same agencies as the firm, but never on the same regulation, receive$156,348 each, while those that ever co-comment with the firm receive an average of $240,515 each andgrantees that co-comment with the firm that specific year receive $206,994 each.1.4 Evidence based on charitable giving and non-profit commentingon regulationsThis section focuses on the link between firms and non-profits through charitable grants, and estab-lishes a relationship between firm-grantee financial ties and their tendency to comment on the sameregulations.We denote firms/foundations by f ∈ F and grant-receiving non-profits (grantees) by g ∈ G. LetDfgt be an indicator function that takes a value of 1 if we observe a donation from firm f to grantee g inyear t, and 0 otherwise. The indicator function Cfrt is equal to 1 if firm f comments on regulation r inyear t, and 0 otherwise (throughout this section and the following one, regulation or rule will referto a docket). The indicator function Cgrt is defined similarly and is equal to 1 if grantee g comments19Agency acronyms are listed in Appendix tables A.9 and A.10.15on regulation r in year t, and 0 otherwise. A graphical representation of this configuration is describedin Figure 1.2.We adopt two types of specifications: co-commenting specifications and a regulation specification.1.4.1 Co-commenting specificationsWe begin by relating the event of a firm and a grantee commenting on the same regulation to a financialtie between the two in the form of a charitable donation. The indicator function CCfgrt is equal to 1when donor f and grantee g comment on the same regulation r at time t, so that CCfgrt = Cfrt×Cgrt,and 0 otherwise. Our first specification explores a time-invariant link between co-commenting anddonations, aggregating co-commenting to the firm-grantee pair, so that we define a new indicatorCCfg which is equal to 1 if we observe any co-commenting from firm f to grantee g in our sample, and0 otherwise. That is, CCfg = I (∑r∑tCCfgrt > 0). Similarly, the indicator variable Dfg indicateswhether we observe any donation from f to g in our sample.We first consider the following time-indepentent specification that relates the presence of co-commenting by firm f and grantee g to the presence of a donation within the same pair:CCfg = β0 + β1Dfg + δf + δg + εfg. (1.1)The specification includes firm fixed effects δf to capture the potential bias resulting, for example,from the higher probability that large and profitable firms both donate to charities and comment onmultiple regulations. Similarly, we include grantee fixed effects δg, to control, for example, for the factthat charities that are more successful at fundraising may on average have more resources to devoteto commenting on various regulations. A positive coefficient β1 would indicate that firm-grantee pairsthat are connected by donations are also more likely to comment on the same regulations.The results are reported in Table 1.1. The different columns of Table 1.1 include different setsof fixed effects (and clustering dimensions) of increasing levels of stringency. Of particular interest iscolumn (4), the most conservative specification, that includes both grantee and firm fixed effects. Thefirm fixed effects may account for the average propensity of firms to comment and to donate, whichmay depend on size and sector. The grantee fixed effects can capture the average level of commentingactivity of the grantee, which may in turn be related to its size and overall resource endowment. Acrossall specifications in Table 1.1,we can see that a grantee and a firm are more likely to comment on thesame regulation when we observe any donation from the firm to the grantee. The magnitude of thiseffect is large. The baseline probability of co-commenting for a firm and a grantee is 2.16%, meaningthat of all the possible pairs of grantees and firms only 2.16% comment on the same rule at any pointin time. This probability increases by 4 to 8 percentage points when we observe a donation connectingfirm and grantee. Put differently, the presence of a donation is associated with a two- to four-foldincrease in the probability of co-commenting.Of course, this cross-sectional pattern of co-commenting may stem from the fact that firms con-tribute to non-profits sharing similar objectives and views, or that, more simply, operate in similarsectors. For instance, the Bayer Science & Education Foundation associated with Bayer US, a phar-16maceutical company, may be more likely to donate to healthcare-related research non-profits, andboth Bayer and healthcare-related non-profits may be more likely to comment on healthcare-relatedregulation than an average organization.Our second specification addresses this concern, and further allows us to control for the generaltendency by some firms to comment on certain issues and to contribute to non-profits that operate inrelated areas. It does so by focusing on the timing of donations. In particular, we examine whetherco-commenting is more likely in the year immediately following the presence of a donation. For this,we turn to the following panel specification, which exploits time variation in both co-commenting anddonations:CCfgt = β0 + β1Dfgt−1 + δfg + δt + εfgt (1.2)where CCfgt = I (∑r CCfgrt > 0) indicates whether firm f and grantee g comment on the sameregulation at time t, and Dfgt−1 is equal to 1 if we observe a donation from f to g in the concurrent (t)or preceding (t − 1) year of the comments, 0 otherwise. This specification includes firm-grantee fixedeffects δfg and time fixed effects δt. Therefore, β1 is estimated only employing within-pair variationover time in donations and co-commenting. In particular β1 will detect whether, controlling for theaverage tendency of a certain firm f to co-comment with and donate to a specific non-profit g, weobserve co-comments occurring immediately after a donation from f to g has been made.Given the coarseness of the data along the time dimension (we only observe year of comment), itis possible for a comment to be made in, say, January of 2006 and a donation in June 2006; hence wecan only be certain that the lagged-year donation took place prior to co-commenting. In Table 1.2,we report results in which we create a dummy that is equal to 1 if we observe a donation at either tor t − 1, and 0 otherwise.20 Our preferred specification in Table 1.2 is column (5), where we includefirm-grantee pair fixed effects. This specification exclusively exploits variation within a firm-granteepair in donations and in co-commenting. The δfg pair fixed effects control not only for the higherprobability of donation and co-commenting for firms and grantees in the same sector, but also for thegeneral ideological alignment of firm and grantee that may result in both donations and co-commentingon similar topics.We find a robust association between donations in year t− 1 and the likelihood of co-commentingin year t. The magnitude of effects is large in this panel specification. Co-commenting is obviouslymore sparse in equation (1.2) than equation (1.1): of all firm-grantee-year triples only 0.163% featureco-commenting. In column (4) of Table 1.2, the presence of a recent donation is associated with aquadrupling of the probability of co-commenting. In column (5) of Table 1.2, the presence of a recentdonation is associated with a 81% increase in the likelihood of co-commenting, even after controllingfor the general propensity of a specific firm to give to and as well as co-comment with a specificgrantee. The example provided in the introduction, which described AT&T Foundation grantees suchas GLAAD or a homeless shelter commenting on the AT&T/T-Telecom merger close on the heals ofreceiving donations, provide an illustration of the behavior implied by this statistical evidence (see20In Appendix Table A.2 we separate contemporaneous and lagged donations and find that lagged donations stronglypredict co-commenting, while contemporaneous donations are a weak predictor of co-commenting.17Peng (2016) for other illustrations).As a further robustness exercise, in Appendix Table A.3 we augment our preferred specificationwith a dummy for whether firm f donated to g in year t+1. In column (5) of that table, with the mostrestrictive set of fixed effects (i.e. pair fixed effects), we find that donations made immediately afterthe commenting period are not associated with co-commenting, whereas only immediately precedingdonations are. This pattern further confirms that co-commenting seems to be more prevalent after weobserve a donation from firm to grantee.1.4.2 Regulation specificationIn the specifications we have considered thus far, we have aggregated co-commenting across differentrules within a fg pair or fgt pair-year. We now present an alternative approach that links commentingby a grantee to donations received by a firm that also comments on the same rule r. The followingregulation specification relates the probability of commenting by a grantee on a regulation r todonations received:Cgr = β0 + β1I∑fDfg × Cfr > 0︸ ︷︷ ︸DonorCommentgr+ δg + δr + ηgrwhere Crg is equal to 1 if g comments on regulation r (0 otherwise) and DonorCommentgr =I(∑f Dfg × Cfr > 0)is equal to 1 if g receives a donation from any firm that comments on r, and 0otherwise. This specification includes regulation fixed effects δr, which capture how certain rules aresubject to more intense commenting, and grantee fixed effects δg, that account for factors like resourcesand size of the non-profit, which may make g both more visible and more likely to comment on anyregulation.Table 1.3 reports estimates of β1 under different fixed effects and clustering options. Our preferredspecification in column (4) has docket and grantee fixed effects, as well as two-way clustering on theseattributes. When considering all the possible combinations of grantees and rules, we find a commentin 0.039 percent of the cases. It is not surprising that this number is small, since the universe ofall possible grantee-rule pairings involve non-profits, like the Red Cross, that we would not expectto comment on, say, financial regulation. Starting from this baseline probability of commenting on aspecific rule, we find that the probability that the non-profit comments is three to five times higherwhen a donor firm commented on the same rule, a result that accords with our previous results underspecification (1.2).1.5 Getting paid not to comment: The role of hush moneySections 1.4-2.3 focused on the role of donations from corporations to non-profits in generating addi-tional messages that are more similar to the donor's position. In our final set of results, we examinewhether corporations also use donations for a distinct strategic purpose: to silence opposing opin-ions. It is plausible to envision an informational lobbying environment in which agents supporting a18specific action opposed by a counterparty may be motivated to suppress these opposing voices (andcompensate the counterparty for its silence). For example, in a discussion of the strategies employedin the multi-year campaign of the tobacco industry Lando (1991) writes: The tobacco industry hasbeen effective in purchasing what has been described as 'innocence by association'. Tobacco industrysponsorship of sports events is notorious. The industry has also contributed substantially to the arts,to women's groups, and to organizations representing minorities. These types of pernicious industryactivities have been successful in buying the silence or the tacit support of some groups that have suf-fered a disproportionate share of the tobacco burden. Payment in exchange for inaction and silence iscommonplace in the market (e.g. noncompete, nondisclosure agreements, non-disparagement clauses,etc.) and such private agreements or clauses do not represent per se invalid contracts or violations offree speech. They may be, however, private agreements that are undisclosed to regulators, who mayinterpret the silence of some parties to the regulatory process as informative.21The role of such negative strategies is thought to be crucial to the success of special interestgroups in politics. Blocking unfavorable bills from ever seeing the light of day (or committee discharge)in the U.S. Congress is as much a part of lobbying as facilitating the passage of bills favorable toan industry. Similarly, interest group comments in rule making often involve aim to kill unfavorableprovisions or stalling the implementation of rules. (Nothing happening is almost always the desirablepolicy outcome for incumbent industry, see Baumgartner et al., 2009.)To test for the presence of hush money in rule making, we propose an extension of our empiricalframework in Section 1.4. In particular, we modify the regulation specification in Section 1.4.2 asfollows:Cgr = β0 + β1DonorCommentgr + β2DonorCommentgr × ShareCommentsgR + δg + δr + ηgr (1.3)where DonorCommentgr is equal to 1 if grantee g received a donation from a firm that also commentedon the same regulation, and 0 otherwise. ShareCommentsgR is the number (or share) of commentsfrom g that are directed at rules under agency R over the entire sample. This new variable captureshow common it is for grantee g to comment on rules from agency R.To understand the intuition behind this test, observe that certain non-profits may have specificexpertise or focus in a specific area of regulation, which we approximate by the identity of the agencyoverseeing the rule (e.g., the Sierra Club commenting on rules proposed by the EPA).22InteractingShareAgencygR with the donation from a commenting firm, DonorCommentgr, aims to establishwhether such donations have a differential effect on the likelihood of commenting for grantees thattypically comment on rule considered by agency R, versus grantees that normally do not comment onrules by R. We argue that this interaction is useful for assessing the potential role of hush money,as within the set of issue experts (high ShareCommentsgR), it more likely that donations are madewith the aim of inducing silence and muting commentary. A plausible null hypothesis supporting thepresence of hush money is therefore β2 < 0, as charitable donations may be more likely to be hush21Absence of a signal is in fact informative in games of incomplete information in which Bayesian rationality is assumed.For an applications to elections see Kendall et al. (2015).22A similar approach was followed to define issue expertise of individual lobbyists from federal lobbying reports inBertrand et al. (2014).19money for grantees that routinely comment on rules from R.Our results based on this specification and reasoning suggest that hush money is not a commonstrategy in our setting. In Table 1.4 we present several specifications accounting for the nonlinearityin equation (1.3), adding increasingly conservative sets of fixed effects across the six columns. Theevidence points clearly in the direction of donations increasing co-commenting from grantees thatroutinely comment on rules from the regulator proposing r. The coefficient β2 > 0 is systematicallypositive and highly statistically significant, indicating that firms are more likely to induce  rather thanstifle  comments from such grantees. While this does not completely rule out the existence of hushmoney, it suggests that it is at a minimum less prevalent than the co-commenting behavior documentedin Sections 1.4-2.3.1.6 Concluding remarksPoliticians (and voters) are frequent targets of messages aimed at persuading them of the merits ofspecific policy positions. While in most cases the identity of senders is disclosed, allowing an assessmentof the bias and interests of the originators of the message, in other cases their identity may be obscured,and deliberately so. These situations range from the use of dark money in U.S. electoral politics in theaftermath of the Supreme Court's decisions of Citizens United v. Federal Election Commission andMcCutcheon v. Federal Election Commission to the circulation of white papers by think tanks andnon-profits.In such circumstances, a common trait identified by the qualitative literature reviewed in this articleis the reliance on independent arms-length organizations to extend the credibility of the positions heldby special interest groups. While in most cases such overlap of intent and opinion is genuine, one hasto be careful in assessing those cases where such support is offered in close proximity to monetarydonations from corporations to advocate non-profits. Such transfers, often in the form of charitablegrants, are virtually undetectable by private citizens and civil servants without access to detailedtax forms. Thus, these transfers represent potential forms of distortion that cannot be weighted andassessed in decision making.In order to provide a quantitative and systematic perspective on this issue, this chapter studiesthe interaction of non-profit organizations and large corporations within the United States federalregulatory environment. We offer systematic empirical evidence which underscore new findings in theliterature on corporate philanthropy and special interest politics. The chapter presents evidence thatthe charitable grants of corporate foundations reach targeted non-profits just before those same non-profits engage in public commentary. In the next chapter, we extend this analysis to the text of thecomments and rules.201.7 Figures and TablesFigure 1.1: Rulemaking processProposed Rule Final Rule Discussion + Regulatory Text Firm Non-profitNotice & Comment Period (30-60 days)commentcommentcommentIndividual21Figure 1.2: Co-commenting and charitable donationsProposed Rule Final Rule Discussion + Regulatory Text Firm Non-profit$$$commentcommentNotice & Comment Period (30-60 days)22Table 1.1: Co-commenting - Time-invarying specificationDependent variable Grantee g and firm f comment on same regulation×100Mean 2.16(1) (2) (3) (4)Grantee g received 8.156*** 6.044*** 6.276*** 4.033***donation from firm f (0.393) (0.506) (0.215) (0.484)Fixed EffectsFirm Y YGrantee Y YSE Clusters Grantee Firm Grantee Firm+GranteeObservations 11,111,716 11,111,716 11,111,716 11,111,716Notes: The dependent variable is equal to 100 if grantee g and firm f comment on thesame regulation in any year between 2003 and 2016. The independent variable is equalto one if grantee g received a donation from firm f in any year between 2003 and 2016.Standard errors are clustered at the level indicated in each column under SE Clusters.*** p<0.01, ** p<0.05, * p<0.123Table 1.2: Co-commenting - Recent donationDependent variable Firm f and grantee g commented on the same regulation in year tMean 0.163(1) (2) (3) (4) (5)Firm f contributed 1.153*** 0.960*** 0.927*** 0.728*** 0.132***to grantee g (0.028) (0.065) (0.110) (0.121) (0.037)in year t or t− 1Fixed effectsYear Y Y Y Y YGrantee Y YDonor Y YGrantee-Firm Pair YSE Clusters Grantee Firm Grantee+Firm Firm×GranteePairObservations 136,400,199 136,400,199 136,400,199 136,400,199 136,331,013Notes: The dependent variable is equal to 100 if grantee g and firm f comment on the same regulation inyear t. The independent variable is equal to one if grantee g received a donation from firm f at year t ort− 1. Standard errors are clustered at the level indicated in each column under SE Clusters. *** p<0.01,** p<0.05, * p<0.124Table 1.3: Commenting on regulationsDependent variable Grantee g commented on regulation r × 100Mean 0.039(1) (2) (3) (4)Grantee g received donation 0.210*** 0.157*** 0.181*** 0.122***from any firm commenting on r (0.012) (0.011) (0.010) (0.014)Fixed effectsGrantee Y YRegulation Y YSE Clusters Grantee Grantee Regulation Grantee+RegulationObservations 144,628,498 144,628,498 144,628,498 144,628,498Notes: The dependent variable is equal to 100 if grantee r comments on regulation r. The inde-pendent variable is equal to one if grantee g received in any year 2003-2016 a donation from a firmthat commented on r. Standard errors are clustered at the level indicated in each column under SEClusters. *** p<0.01, ** p<0.05, * p<0.125Table 1.4: Hush moneyDependent variable Grantee g commented on regulation r × 100Mean 0.039(1) (2) (3) (4) (5) (6)DonorCommentgr 0.086*** 0.058*** 0.042*** 0.035*** 0.024 -0.000(0.014) (0.007) (0.009) (0.008) (0.017) (0.010)DonorCommentgr 0.150*** 0.167*** 0.150***×NumberCommentsgR (0.027) (0.009) (0.028)DonorCommentgr 2.560*** 2.540*** 2.517***×ShareCommentsgR (0.149) (0.185) (0.232)Fixed effectsGrantee Y Y Y YRegulation Y Y Y YSE Clusters Grantee Grantee Regulation Regulation Grantee Grantee+Regulation +RegulationObservations 144,628,498 144,628,498 144,628,498 144,628,498 144,628,498 144,628,498Notes: The dependent variable is equal to 100 if grantee r comments on regulation r. The DonorCommentgr is equal toone if grantee g received in any year 2003-2016 a donation from firm that commented on rule r. Standard errors are clusteredat the level indicated in each column under SE Clusters. *** p<0.01, ** p<0.05, * p<0.126Chapter 2Text Similarity and Regulatory Influence2.1 IntroductionOur findings on the link between corporate donations and co-commentary frequency by advocate non-profits point to potential corporate influence over non-profits in their regulatory feedback. In thischapter we extend our analysis of the relationship between comments and donations to the content ofthe comments and rule documents.Incorporating text data allows us to examine whether, conditional on co-commentary frequency,the content of comment-pairs from firms and non-profits linked via charitable donations tend to bemore similar, relative to other comments on the same proposed rules. Using established methodsof natural language processing, we generate pairwise measures of textual similarity between any twofirm-non-profit comments on a given rule. Co-comments by non-profits contain textual material thatis more similar to comments by their corporate benefactors relative to other co-comment pairs and,importantly, the timing of this relationship parallels that of our first set of findings  co-commentsin the year immediately following a donation are most similar. We also investigate the semanticorientation of the comments and show that the co-comment similarity for firm-grantee pairs does notresult from comparably-worded comments that express opposing sentiments. Specifically, we find thatco-comments by firm and grantees that are connected by an immediately preceding charitable donationdo not express adversarial views on the same regulatory matter.We also show that the co-commenting relationship matters for the final rules. Focusing on allcomments made by corporations in our dataset, we show that, if a grantee (and particularly onereceiving a recent donation) also commented on the proposed regulation, the language of the discussionof the final rule is more closely aligned with that of the corporation's comments. This result survivesthe inclusion of both firm fixed effects and rule (docket) fixed effects, and also holds when we measurea firm's influence based on whether it is cited by the regulators in their discussion of the final rule.2.2 Quantifying the similarity in content across regulatorycommentsIn chapter 1, we explored non-profit's propensity to comment on regulation. However, a crucial impli-cation of our thesis that non-profits may act as strategic advocates for their corporate donors is that thecontent of the message delivered by non-profits to regulators may be affected by financial connections.In particular, upon receipt of (a) charitable grant(s), comments targeted to federal regulators by non-profits should be closer in content to the messages sent by their corporate benefactors (relative to the27counterfactual of no corporate donations). To provide evidence in this direction, we build a portfolioof circumstantial findings with the intent of discriminating among alternative theoretical mechanismsbased on how well they match the empirical regularities that we present.To build intuition (and without intent to claim any deliberate deception by the parties involvedin this particular instance), consider the example of Bank of America's donation of $150,000 to theGreenlining Institute in 2010. While Bank of America is the second largest bank in the United Statesby total assets and is a central player in housing finance in the country, the Greenlining Institute isa non-profit focused on improving access to affordable housing and credit to low-income families andminorities (African American, Asian American, and Latino, in particular). In 2011 both organizationscommented on the Office of the Comptroller of the Currency's Credit Risk Retention (CCR) docket,23as part of one of the regulatory rulemaking streams initiated under the Dodd-Frank Act of 2010 (TitleIX, Subtitle D, Section 941). CCR, also known as the skin in the game rule, imposed a 5 percentretention requirement on all mortgage loans originated by lenders in the United States to moderateoriginate-to-distribute moral hazard problems pervasive in the build-up to the 2008 financial crisis.The main comment submitted by Bank of America24remarks that, in relation to relaxing thedefinition of qualified mortgages exempted from retention requirements on the issuing bank's balancesheet (i.e. of mortgages deemed safe enough not to warrant the restriction): ...the PCCRA provisionwill cause some borrowers to be unable to obtain a loan at all. In the currently tight private residentialmortgage market, borrowers already must provide significant down payments. The Greenlining Insti-tute provides a similar assessment in its comment,25suggesting that by raising the barrier to affordablehome ownership with an unreasonable 20% down payment requirement, we will not only keep familiesfrom rebuilding after foreclosure, but we will prohibit an entire generation of first time borrowers fromowning a home, despite lower home prices across the country. In sum, both organizations appear toadvocate openly for laxer definitions of the CCR exemptions, limiting the rule's bite, and allowing as-sets with substantially lower quality and higher risk to be exempt  an effort that ultimately succeededin entirely defanging the rule.26In this section, we provide a framework for examining the content and textual similarity of commentsfiled by non-profits and firms, and show that, upon receipt of a donation from a firm's foundation,comments by a non-profit are more similar to those of its donor, suggesting that the Bank of America-Greenlining example holds more broadly in the data.We compute approximate measures of semantic similarity of pairs of public comments using LatentSemantic Analysis (LSA) with bag-of-words features. LSA is an established technique borrowed fromthe natural language processing (NLP) literature, and it has been shown to perform well on a variety ofdifferent document classification and retrieval tasks.27LSA requires the conversion of text documentsinto vectors of word counts and applying term frequencyinverse document frequency feature extraction23Docket ID OCC-2011-000224Document ID OCC-2011-0002-014125Document ID OCC-2011-0002-035326For a discussion, see Floyd Norris for the New York Times, Oct. 23, 2014, Page B1 Banks Again Avoid Having Any`Skin in the Game ', available at https://www.nytimes.com/2014/10/24/business/banks-again-avoid-having-any-skin-in-the-game.html27See Dumais et al. (1988) and Deerwester et al. (1990). For a more recent discussion of latent semantic analysis, seeDumais (2004).28within each regulatory docket r. Following this preparation phase, one can compute document-levelsingular vectors from a singular value decomposition of the text matrices and take the cosine similarityof any pair of document vectors. This approach provides a similarity score Sfgr normalized by thestandard deviation in each docket r and distributed between -1 and 1 for every pair of texts formedby a comment by firm f and a comment by grantee g within a given docket. To further demonstratethe validity of our approach, we show in Appendix B that our measure performs well in a classifica-tion task of separating documents from different regulations and in clustering comments from similarorganizations.Using this comment-pair similarity score as the outcome, we consider a specification of the form:Sfgr = β0 + β1Dfgr + δf + δg + δr + εfgrwhere the coefficient of interest is β1 and Dfgr is indicator variable that equals 1 if firm f donates tograntee g, 0 otherwise. As the timing of such donations is a useful discriminant for interpretation of ourfindings, we will be careful in constructing Dfgr under different time horizons. The dataset we exploitfor this analysis includes all possible firm-grantee pairs of comments conditional on commenting on adocket r.We begin by exploring the sign and magnitude of the estimated coefficient β1 when the donationindicator variable takes the value of one in the event of any grant from f to g over our entire time period.Table 2.1 reports estimates for β1 across a set of four specifications with an incremental inclusion offirm, grantee, and docket fixed effects. Coefficients are clustered at the docket, firm-grantee, or doubleclustered at both levels depending on the specification. The estimates of β1, which capture the increasein units of standard deviations of similarity across comment pairs within each r, range from 0.25 to 0.09in the most restrictive specification (all significant at least at the 1 percent level). This indicates thatpairs of comments made by firms and their grantees are more similar relative to a baseline similarityobtained by pairing comments at random within a docket.As with our results on comment propensity in Section 1.4, the presence of a donation at any pointin our sample period may proxy for some average similarity in the interests and beliefs of a firm and itsgrantee. Table 2.2 thus focuses on donations that take place in either the year in which the commentsare filed (year t) or in the previous fiscal year (t − 1). The point estimates are smaller in magnitudeacross comparable columns in Tables 2.2 and 2.1, but statistically indistinguishable. In separatingexplicitly contemporaneous donations and those made in the fiscal year immediately preceding thecomments, as reported in Appendix Table A.4, we observe that precision and magnitude of the effectcome from the donations made at time t − 1. The estimates, which capture the increase in units ofstandard deviations of similarity across comment pairs within each docket, range from 0.17 to 0.08 inthe most restrictive specification.Appendix Table A.5 addresses the concern that the timing of donations may be spuriously relatedto some underlying tendency of firms and grantees working in related areas of interests, by controllingin our most restrictive specifications also for North American Industry Classification System (NAICS) 6sector code of the firm interacted with the IRS's National Taxonomy of Exempt Entities Classification(NTEEC) code of the non-profit. As can be seen in the table, the estimated coefficient β1 remains29precisely estimated and within the confidence intervals of our baseline estimates across specificationswhen accounting flexibly for such industry pair controls. Finally, notice that the reduction in samplesize for this table results from missing sector information for some firm-grantee pairs, and that thissample shift also does not affect the point estimates relative to the baseline specifications. Moreprecisely, we estimate a β1 of 0.073 in column (4) of Table 2.2 and of 0.074 in column (1) of AppendixTable A.5, and a β1 of 0.079 for Dfgr at time t− 1 in column (4) of Appendix Table A.4 and of 0.072in column (3) of Appendix Table A.5.We also present a placebo exercise that underscores the very specific timing of the link from donationto comment similarity. In particular, we modify our definition of donations to focus on the periodimmediately after the regulatory commenting phase. Appendix Table A.6 reports these results. As canbe seen in the table, across specifications with incremental sets of fixed effects and industry controls, theestimated coefficient β1 appears insignificant and smaller in magnitude relative to our base estimates.28This placebo exercise is informative along several dimensions. As the donation is close in time to thecommentary activity, but statistically and economically insignificant, these findings further assuage theconcern that our results may be spuriously driven by some underlying tendency of firms and granteesoperating in related areas. The systematic timing of excess similarity between comments' texts justfollowing the disbursement of a charitable grant offer intuitive support to the logic of some form ofsuasion being exerted by the donor over the grantee.As a final check, we investigate whether firm-grantee co-comments differ in their sentiment. We doso to assess the possibility that firms and grantees may employ a similar terminology while nonethelessdelivering adversarial messages to regulators.Our test is based on an analysis of comment sentiment, which relies on established NLP scholarship.Semantic orientation exercises are common in the NLP literature (e.g., the unsupervised classificationof book reviews as positive or negative), including application to economics and finance, for examplein the classification of monetary policy announcements as hawkish or dovish, in the study of thetone of financial news, or in partisan speech (Lucca and Trebbi, 2009; Tetlock, 2007; Tetlock et al.,2008; Gentzkow et al., 2016).29Using these tools, our goal here is to rule out the possibility that thecomments of non-profits receiving grants use similar words which express views that are nonetheless inopposition to their corporate benefactors. This specifically rules out the possibility that donations byfirms may reach non-profits intervening on the same issues as the donor (and therefore using similarterminology), but expressing systematically antagonistic views.Tables 2.3 and 2.4 maintain the same design and structure of fixed effects as Tables 2.1 and 2.2, butreplace the similarity score Sfgr with a semantic orientation concurrence score Wfgr as our dependentvariable. The construction of this variable relies on polarity scores defined for each comment basedon the popular AFINN sentiment lexicon, with valence scores ranging between -5 (negative) and 5(positive) for each labeled word. For each comment we construct the sum of valence scores divided bythe number of words with non-zero valence scores. Wfgr is defined as the negative absolute difference28In the last column of the table, we also include donations at t or t− 1, and show that only pre-comment donationsmatter, relative to donations at t+ 1.29In general, by semantic orientation we refer to the direction (polarity) of words, phrases or longer pieces of text in asemantic space or context (e.g., friendly/adversarial, dovish/hawkish, positive/negative) calculated based on a referencelexicon of words or n-grams over which directionality is carefully labeled by a pool of researchers.30between this measure for the pair of comments from firm f and from grantee g on rule r. Theinterpretation of the coefficient of interest β1 on Dfgr is therefore the effect of a charitable donation onthe alignment of sentiment across firm and non-profit (i.e. the excess comovement of sentiment in thetwo comments relative to any randomly generated pair of firm and grantee comments on that rule).30The results in the tables do not support the view that donations reach grantees expressing opposingviews to the firm providing the grant relative to a random grantee. If anything, the evidence points inthe opposite direction: the coefficient β1 is consistently positive in sign, indicating that firm-granteecomments are more aligned in sentiment. This relationship is significant in the specifications that linkfirms and non-profits by the existence of a donation at any point during our sample period (Table2.3). The coefficient β1 is positive, though significant in only one out of four specifications, for fg pairslinked by donations at year t−1 or t (Table 2.4). These findings are inconsistent with firm and granteecomments carrying antagonistic messages.In sections 2.2 and 2.3 of the chapter we compare the content of firm comments with grantee com-ments and regulator discussion text. In the first case, our goal is to capture similarities between inthe policies advocated for (or against) in by different commenters. In the second, it is to measure howmuch attention the regulator has paid to different comments. Complete solutions to these problems (inthe sense of replicating what a literate and informed human could deduce from reading the text) arecurrently beyond the frontier of natural language processing (NLP) technology. Instead, we approxi-mate these notions with a simple and robust method of text analysis called Latent Semantic Indexing(or sometimes Latent Semantic Analysis) with bag-of-words features. The basic recipe is as follows:After extracting and cleaning the comment text (to remove headers, page numbers, etc), each commentis converted into a vector of word counts. Very rare and very common words are dropped completely,and the remaining counts are weighted by a standard term-frequency-inverse-document-frequency (tf-idf) function to emphasis the words that are most useful in distinguishing between documents in eachregulation. These weighted count vectors are then summarized by computing document-level singularvectors from a singular value decomposition of the feature-document matrix (this is the latent partof LSA, and generally improves the performance beyond using the raw feature vectors). Finally, thepairwise document similarity is computed as the cosine similarity between the document LSA vectors.The rest of this section explains these steps in greater detail, and describes a docket classification testwe conducted to verify that the measure is informative.2.2.1 Sample constructionWe perform our analysis at the docket level. For each docket where at least one firm or one granteecomments, we load all organization comment text documents (initially treated as separate even if theyare from the same author), and also discussion text from all linked rule documents. If there are atleast three documents in total, we process the text and perform LSA to compute similarity measures.Comment text is cleaned in by a Python script that attempts to identify and remove addressesand other header material that appear before the body text, tail sign-off and other material that appear30In addition, a standardization within rule as for the variable Sfgr is employed for Wfgr, which allows to readcoefficients in units of standard deviations of sentiment alignment across comment pairs within each r.31after the body text, as well as repeated headers and footers (including page numbers) that appear onmultiple pages. The script does not always succeed in removing the desired material (the commentsare too varied in format to cover every possible case), but it is intended to remove some noise from thedata.Regulator discussion text is identified in the following way: First we load all rules that follow oneor more comments in the docket (see appendix X on Federal Register document linking) and constructa separate discussion text document for each Federal Register rule document. We immediately dropAgency, Action, Dates, Summary, Addresses, Contact sections, as well as all appendices and tablesof contents. Then we search for the strings comment and letter in all paragraphs and footnotes,and count a paragraph or footnote as discussion text if it appears under the same 2-level header as aninstance of those strings. In other words, if the word commenters appears in the third paragraph underthe heading SUPPLEMENTARY INFORMATION: V. Discussion of Final Rule, every paragraph andfootnote located under that heading will be included.2.2.2 LSA implementationLSA is essentially the application of singular-value decomposition (SVD) to a document-feature matrix.We follow a standard approach in constructing this document-feature matrix from word counts, anduse the Gensim31python package for efficient implementation of these steps. First, each documentis converted to lower case and words are stemmed (meaning removing common prefixes and suffixes,including pluralization so that House and houses both become hous). This step increasing theprobability that closely-related words will be matched across documents. Next we identify everysequence of alphanumeric characters that are unbroken by white-space or other punctuation (except -)as a word and count the number of occurrences of each word in each document. We drop all words thatappear in more than 70% or less than 20% of documents (this seemingly arbitrary step is important forgood results with LSA and the numbers were chosen based on experiment in a docket classification testtask). Finally, we re-weight the word counts in each document using term-frequency-inverse-documentfrequency (tf-idf) weighting with the following formula:wij = fijln(Ddi)where fij is the count of word i in document j, diis the number of documents containing word i,D is the total number of documents in the docket. The matrix of wijentries then form a (W × D)feature-document matrix M (where W is the number of distinct words).Recall that SVD decomposes the matrix M into the product of three matrices: M = UΣV ∗whereU is (W × W ) and V is (D × D). We use an algorithm32 that can compute the first k singularvalues and associated columns of U and V . If k < min(W,D) then the resulting decompositionforms a rank-k approximation of M . The word latent in Latent Semantic Analysis refers to theidea that compressing the full feature-document matrix to a lower-dimensional approximation squeezes31https://radimrehurek.com/gensim/index.html32https://pypi.org/project/sparsesvd/32synonyms into the same singular vectors and improves overall quality of the document model. Inpractice, researchers have found that values of k around 200-400 appear work well in large samples ofdocuments. However, k is bounded above by the number of separate documents D, and we have manydockets with fewer than 300 comments. As a general solution, we choose k according to the followingformula:k = min(D − 2, 50)So the LSA vectors have higher rank in large dockets, but we keep the maximum value a bit lowso that the approximations are not wildly different in dockets of different sizes. Our object of interestis the resulting (D × k) matrix V . We describe each row as a document LSA vector.2.2.3 Similarity measuresOnce the document LSA vectors are computed, estimating the similarity between comments from firmsand grantees is straightforward. We compute organization-level vectors by summing the LSA vectorsfor all documents associated with that organization, and define the pairwise comment similarity ascosine similarity of the organization-level vectors.2.2.4 Rule similarityEstimating the similarity between the rule discussion and an organization's comment(s) is only slightlymore complicated. In the case that there are multiple rules linked to a docket, we first constructall the comment-rule pairs and keep only those for which the comment was posted before the rulewas published. Then we perform the same summing procedure to aggregate document LSA vectorsassociated with multiple sources of comment text submitted by the same organization, and computesimilarity with the rule as the cosine similarity between the rule LSA vector and the organization-levelvector.2.3 Comment impact analysis: Evidence from final rule citationsWhile the preceding sections focus on the frequency and similarity of firm-grantee comments, we nowturn to examining whether firms' comments  and the similar comments made by grantees  commentshave an impact on rulemaking. As it is typically very hard to assess the effects of advocacy on policyoutcomes (and in general of informational lobbying on government policy choices), we will focus hereon a newly devised approximation for such outcomes by asking how the final rule was shaped by thecommentary. In particular, we aim to establish that when a firm comments on a rule, the publisheddiscussion of the rule by the regulator is closer in content similarity to the firm's comments when thefirm's grantees also comment on that rule.It is important to clarify that the final regulatory text itself is written with a terminology andstructure that makes it very different from comments submitted or the explanation of the rule itselfoffered by the regulator in the preamble to the rule. The final regulatory text is designed to formulate,33amend, or repeal sections of the Code of Federal Regulations (5 U.S.C.  551(5)). The discussion ofthe rule itself offers a justification and analysis of the regulator's decision making process and intendedscope or interpretation of the regulation.33In fact, the discussion of the rule tends to be longer andreveals arguments in favor of or against specific choices that may have been brought forward by, forexample, the comments from various entities, firms and grantees, in persuading the regulator. Wetherefore focus on this part of the final rule.As an example consider the concern expressed by Wells Fargo, one the U.S. largest depository in-stitutions, on a specific regulatory burden that appeared implied by the proposed rule version of the socalled Volcker Rule of the Dodd-Frank Act of 2010. The Volcker Rule aimed at prohibiting depositoryinstitutions from engaging in the use of part of its depository funding for speculative trading (propri-etary trading).34Wells Fargo expresses concern that the proposal requires transaction-by-transactionoversight: We also do not believe that the Proposed Rule's transaction-by-transaction approach, whichwould require analyzing permitted customer trading, market making, underwriting and hedging activi-ties on a transaction-by-transaction basis, is the best way for the Agencies to implement the ProposedRule...35The OCC addresses this concern directly and concedes some changes to the rule: A numberof commenters expressed general concern that the proposed underwriting exemption's references to a'purchase or sale of a covered financial position' could be interpreted to require compliance with theproposed rule on a transaction-by-transaction basis. These commenters indicated that such an ap-proach would be overly burdensome. . . .A general focus on analyzing the overall 'financial exposure'and 'market-maker inventory' held by any given trading desk rather than a transaction-by-transactionanalysis. Importantly, also the Black Economic Council, a recent Wells Fargo grantee, is found toexpress concerns on the same rule on grounds of excessive complexity.36We begin by defining Sfr the similarity score between the discussion of docket r and firm f 'scomment. In contrast to the score constructed in Section 2.2, Sfr measures the similarity between acomment and the discussion of the rule in a docket, rather than the similarity between the texts of twocomments on a rule. Sfr is designed as a proxy for the salience and effectiveness of the firm's commentin shaping the regulator's decisions. As with the previous similarity measure Sfgr, we normalize Sfrby the standard deviation in each docket r, so that Sfr is distributed between -1 and 1 for every pairof texts.37Dropping time subscripts, let us posit Sfr as function of the commenting effort of the firm and of33The discussion of the rule is found in the Supplementary Information section, which is partof the preamble to the final rule and typically constitutes its most important component. Seehttps://www.federalregister.gov/uploads/2011/01/the_rulemaking_process.pdf34Docket ID OCC-2011-001435Document ID OCC-2011-0014-0285)36Document ID OCC-2011-0014-0024)37As in some cases multiple rules may be included in a docket by regulators (including amendments, notices, etc.) andeach regulatory stream can be linked to a final rule, our approach here is to take for each firm and docket the closest insimilarity to the firm's comment vector. This is meant to more accurately represent the dimension of the docket the firmmore closely commented about. Our results are similar when removing the lowest similarity score within a docket-firmgroup and then taking the mean similarity or when keeping only dockets with exactly one rule document. See OnlineAppendix for these robustness checks.34grantees connected to the firm by donation:Sfr = β1∑gCCfrg ×Dfg + β2∑gDfg + β3∑gCCfrg + δf + δr + εfrFocusing on the extensive margin of commenting behavior, we can replace all sums with indicatorfunctions and also include firm and docket fixed effects:Sfr = β1I(∑gCCfrg ×Dfg > 0)+δf + β2I(∑gDfg > 0)︸ ︷︷ ︸Firm FE+δr + β3I(∑gCCfrg > 0)︸ ︷︷ ︸Docket FE+εfr (2.1)The variable of interest is I(∑g CCfrg ×Dfg > 0), which is equal to 1 if we observe a donationby the firm to a grantee co-commenting on the same rule, and 0 otherwise. If there is excess similaritybetween rule discussion and a firm's comment when grantees connected to the firm by donation alsocomment on that rule, we expect β1 to be positive. As we established in the previous two sections, suchcomments by non-profits occur around the time of firm donations and appear to exhibit a systematicallyhigher textual similarity to the comments filed by the grantee's benefactors. Here, we aim to establishthat corporate benefactors appear to gain in terms of Sfr, a proxy that at a minimum captures havingthe attention of the regulator, but could conceivably correlate with influence in shaping the final ruletext or keeping certain provisions out.Let us also clarify that in specification (2.1) the coefficient on the term I(∑gDfg > 0)cannotbe separately identified from a firm f fixed effect, since it counts whether the firm ever donates toany grantee. Also the coefficient on the term I(∑g CCfrg > 0)cannot be separately identified froma docket fixed effect, as it counts the average level of commenting by grantees for that rule (onlyfirms commenting on the rule are included in the estimation and all grantees commenting on r are, bydefault, co-commenters of every firm also commenting on r). As β2 and β3 allow us to measure thedirect effects of each element to the main interaction term I(∑g CCfrg ×Dfg > 0), we include firmand docket fixed effects in our key specifications. We also experiment by removing each set (or both)in order to estimate these direct effects.As in Section 2.2, we begin by exploring the sign and magnitude of coefficient β1 when the donationindicator variable takes value 1 if there is any grant from f to g over our entire time period, and 0otherwise. Table 2.5 reports estimates for β1 across a set of five specifications with an incrementalinclusion of firm and docket fixed effects for specification (2.1) in columns (1) to (4) and a specificationwith the continuous variable∑g CCfrg × Dfg in column (5). Coefficients are clustered by firm ordocket, or double clustered at both levels depending on the specification. In columns (1) to (4), theincrements expressed in terms of increases in units of standard deviation of similarity within eachdocket range from 4.5 to 23.7 percentage points, indicating that comments made by firms on rules thatalso received comments from their grantees appear closer in content to the final rule discussion.As the presence of any donation over time is a less accurate indicator of a direct connection betweenfirms and grantees than recent donations, Table 2.6 looks at donations that take place in either theyear in which the comment is filed (year t) or in the previous fiscal year (t−1). In this specification, the35point estimates of β1 appear more precise and quantitatively sizable, with 0.173 of a standard deviationhigher similarity for comments filed by firms with co-commenting grantees who were recipients of theirdonations in our preferred column (4). Similar results are obtained focusing on the intensive margin,as reported in column (5).Appendix Table A.7 further probes our results on rule-comment similarity by adding controls forthe log number of pages of commentary filed on r by f which, even controlling for firm and docketfixed effects, turns out to be a strong predictor of similarity between rule discussion and commentby the firm. The effect of this control is intuitive, in the sense that carefully articulated commentsmay capture more of the attention of the regulator and translate in higher Sfr. The coefficient onI(∑g CCfrg ×Dfg > 0)based on donations at t or t− 1 remains positive and statistically significantin all specifications in Appendix Table A.7. Contrasting these estimates with those based on the samevariable constructed with donations at any time, included in columns (2) to (4), shows that the increasein similarity is driven by the co-commenting of a grantee that received a donation in the current orprevious year, i.e., recent donations. When both variables (constructed with recent donations versusdonations at any point in time) are included in columns (3) and (4), it is evident that recent donationscarry the relevant variation.38Importantly, the content of the messages simultaneously communicated by non-profits and by cor-porations appears systematically closer in terms of textual and semantic similarity in presence of acharitable contribution provided immediately before those comments are filed. While circumstantial,the evidence seems to point to potential concerns in the assessment of prima facie independent infor-mation on the part of targeted regulators, who may be unaware of the philanthropic grants that realizein the backdrop and may interpret similar comments stemming from different segment of the publicspectrum as indicative of merit.2.4 Concluding remarksIn this chapter we use text analysis to flesh out the relationship between charitable donations fromcorporations, and the commenting behavior of non-profits in federal rulemaking. First, we comparecomments submitted by corporations and non-profits according to the similarity of their text, and findthat non-profits receiving donations submit comments that are unusually similar in content to thoseof their benefactors. This result strengthens the interpretation that firms are able to use donations toamplify their messages to the regulators. Second, we use textual similarity between the commentingfirm and final rule discussion to gauge influence of comments over policymakers. It appears that theco-commenting patterns of firms and non-profits can offer additional visibility to the messages sentby the firms themselves measured in terms of comment similarity to the final rule or even likelihood38In online Appendix table A.8 we also replaced similarity to the final rule discussion with indicators or log 1+ countsof the number of times that a firm is cited in the final rule discussion. We obtain similar qualitative results as in theanalysis in this section. Specifically, when focusing on an indicator variable for being cited or not for a firm, our resultsindicate a positive but imprecise relationship when controlling for docket and firm fixed effects, but when focusing onnumber of times the firm is cited, the presence of recent donations to co-commenting non-profits is positive, significantat standard confidence levels, and robust to firm and docket fixed effects, and controlling for log pages of commentssubmitted and the any donations over time.36of citation of a donor firm. As rates of return for political influence activities are extremely complexto measure, this is an area of statistical investigation requiring further study. Its exploration remainsopen to future empirical research. The availability of a large set of public comments by non-profitsand by corporations on a diverse set of rules and regulations, ranging from banking to environmentalregulation, makes for a rich and virtually untapped empirical environment.2.5 TablesTable 2.1: Similarity of comments - Any donationDependent variable Similarity of comments by grantee g and firm f on same regulation(1) (2) (3) (4)Grantee g received 0.249*** 0.161*** 0.140** 0.088***donation from firm f (0.068) (0.027) (0.065) (0.022)Fixed EffectsDocket Y YFirm Y YGrantee Y YSE Clusters Docket Docket Firm+Grantee Firm+Grantee+DocketObservations 301,602 301,602 300,817 300,792Notes: The dependent variable is a similarity index between the comment of firm f and the commentof grantee g on regulation r, divided by the standard deviation of similarity of all comments relative tor. The independent variable is equal to one if grantee g received a donation from firm f between 2003and 2016. Standard errors are clustered at the level indicated in each column under SE Clusters. ***p<0.01, ** p<0.05, * p<0.137Table 2.2: Similarity of comments - Recent donationDependent variable Similarity of comments by grantee g and firm f on same regulation(1) (2) (3) (4)Grantee g received donation 0.172*** 0.158*** 0.041 0.073***from firm f at t or t− 1 (0.058) (0.037) (0.039) (0.023)Fixed EffectsDocket Y YFirm Y YGrantee Y YSE Clusters Docket Docket Firm+Grantee Firm+Grantee+DocketObservations 301,602 301,602 300,817 300,792Notes: The dependent variable is a similarity index between the comment of firm f and the commentof grantee g on regulation r, divided by the standard deviation of similarity of all comments relative to r.The independent variable is equal to one if grantee g received a donation from firm f in the year when thecomment appears or the year before. Standard errors are clustered at the level indicated in each column underSE Clusters. *** p<0.01, ** p<0.05, * p<0.138Table 2.3: Sentiment - Any donationDependent variable Sentiment similarity of commentsby grantee g and firm f on same regulation(1) (2) (3) (4)Grantee g received 0.030* 0.030*** 0.020** 0.017***donation from firm f (0.017) (0.009) (0.008) (0.006)Fixed EffectsDocket Y YFirm Y YGrantee Y YSE Clusters Docket Docket Firm+Grantee Firm+Grantee+DocketObservations 309,033 308,576 308,184 307,719Notes: The dependent variable is the difference between the sentiment score assignedto the comment of firm f and the comment of grantee g on regulation r as describedin Section 2.2, divided by the standard deviation of this measure within rule r. Theindependent variable is equal to one if grantee g received a donation from firm f between2003 and 2016. Standard errors are clustered at the level indicated in each column underSE Clusters. *** p<0.01, ** p<0.05, * p<0.139Table 2.4: Sentiment - Recent donationDependent variable Sentiment similarity of commentsby grantee g and firm f on same regulation(1) (2) (3) (4)Grantee g received donation 0.027 0.032*** 0.009 0.009from firm f at t or t− 1 (0.019) (0.012) (0.015) (0.012)Fixed EffectsDocket Y YFirm Y YGrantee Y YSE Clusters Docket Docket Firm+Grantee Firm+Grantee+DocketObservations 309,033 308,576 308,184 307,719Notes: The dependent variable is the difference between the sentiment score assigned to thecomment of firm f and the comment of grantee g on regulation r as described in Section 2.2,divided by the standard deviation of this measure within rule r. The independent variable isequal to one if grantee g received a donation from firm f in the year when the comment appearsor the year before. Standard errors are clustered at the level indicated in each column underSE Clusters. *** p<0.01, ** p<0.05, * p<0.140Table 2.5: Rule-comment similarity - Any donationDependent variable Similarity of rule discussion and commentby firm f on same regulation r(1) (2) (3) (4) (5)At least one grantee 0.237*** 0.173*** 0.177*** 0.045co-commenting and receiving (0.065) (0.038) (0.059) (0.056)donation from firm fin any yearLog number of grantees 0.055co-commenting and receiving (0.040)donation from firm fin any yearFixed EffectsDocket Y Y YFirm Y Y YSE Clusters Docket Docket Firm+Dock Firm+Docket Firm+DocketObservations 5,538 5,145 5,367 4,965 4,965Notes: The dependent variable is a similarity index between the comment of firm f and the discussion ofregulation r, divided by the standard deviation of similarity of all comments relative to r and discussion ofregulation r. The independent variable is equal to one if there is at least one grantee g co-commenting onregulation r and receiving a grant from firm f in any year. Column 5 reports the coefficient on the logarithmof one plus the number of such grantees. Standard errors are clustered at the level indicated in each columnunder SE Clusters. *** p<0.01, ** p<0.05, * p<0.141Table 2.6: Rule-comment similarity - Recent donationDependent variable Similarity of rule discussion and commentby firm f on same regulation r(1) (2) (3) (4) (5)At least one grantee g 0.274*** 0.261*** 0.189*** 0.173***co-commenting and receiving (0.101) (0.052) (0.066) (0.062)donation from firm fin year t or t− 1Log number of grantees 0.112**co-commenting and receiving (0.052)donation from firm fin year t or t− 1Fixed EffectsDocket Y Y YFirm Y Y YSE Clusters Docket Docket Firm+Grantee Firm+Docket Firm+DocketObservations 5,538 5,145 5,367 4,965 4,965Notes: The dependent variable is a similarity index between the comment of firm f and the discussion of regulationr, divided by the standard deviation of similarity of all comments relative to r and discussion of regulation r. Theindependent variable is equal to one if there is at least one grantee g co-commenting on regulation r and receivinga grant from firm f in year t or t− 1. Column 5 reports the coefficient on the logarithm of one plus the numberof such grantees. Standard errors are clustered at the level indicated in each column under SE Clusters. ***p<0.01, ** p<0.05, * p<0.142Chapter 3Estimating Influence from Bag-of-wordsText Features3.1 IntroductionOne of the most important research areas in Political Economy is the study of how special interestgroups influence government policymaking. Quantifying the degree of influence that firms and other or-ganizations have on policy is a necessary prerequisite for exploring many important research questions.However, quantifying influence is generally very difficult because data on interactions between interestgroups and government rarely include detailed information about messages conveyed by interest groupsto government, or policy outcomes chosen by the government after these messages are received.Digitization of text documents is opening up a new frontier in the study of influence. Many actionsby governments, courts, interest groups, firms, and individuals are recorded as text documents, andare increasingly available for download in bulk. In some cases, such as in the context of U.S. federalrulemaking explored in chapters 1 and 2, it is possible to obtain text documents describing the advocacyof interest groups and the actions taken by policymakers. This type of data can be used to producenew estimates of influence. However, the task is challenging, and no existing approach is completelysuitable in this context. In this chapter I present a novel approach to estimating the influence thatovercomes several key challenges with analyzing influence in the rulemaking.My new approach bridges a large family of common text analysis techniques with a simple model ofinfluence that allows me to derive linear estimators of the influence of each comment on the final policy.It has several advantages. First, the process is completely automated, as opposed to hand-coding someor all of the data. This makes feasible to analyze the thousands of rules and millions of commentsthat can be obtained from rulemaking data. Second, the estimators make no assumptions about thedimensionality of the topic or policy space, which can be very complicated in regulatory discussions.Third, the estimators can be extended to operate in panel data settings where the same interest groupscomment on multiple different rules. Panel data offers increased precision by aggregating estimatesfrom across rules, and more importantly, it allows for the possibility of controlling for the author'swriting style which could otherwise bias estimates. I demonstrate how this can be done with my newestimators using the appropriate fixed effects.These advantages are best understood in the context of the existing techniques for measuringinfluence. There are three competing approaches (broadly defined). The first is hand coding. Inprinciple, it is possible for a researcher to read the text describing each interest group's advocacy andcode the group's preferences on each issue they raise. Then, the researcher could also read the text43describing the policymaker's decision and determine which groups obtained their preferred outcomes.Yackee (2005) performs such an analysis on 1,444 comments submitted to several regulators regarding40 randomly selected proposed rules. The effort involved in constructing the data for this paper musthave been substantial. However, Yackee still faced constraints on how many comments and rules couldbe reviewed and to what degree of detail. Yackee coded the top five issues mentioned by commentersfor each rule, but was unable to cover all issues raised by commenters. When coding the regulatorresponse, she simplified all debates to a single more-or-less regulation dimension. These choices aresensible given the finite budget researchers face when hand-coding data. But the approach rulesout estimating commenter-level influence, potentially discards many influential arguments that wereonly offered by a small number of commenters, and aggregates policy outcomes that may be viewedvery differently by the actors involved. Yackee was also only able to analyze a tiny fraction of theproposed rules published during her study period. To the extent that we are interested in rare eventsor heterogeneity across firms, industries and agencies, manual coding leaves a lot on the table.A second approach is to automatically classify texts describing interest group positions along one ormore policy dimensions, and then relate these positions to policy outcomes. Klüver (2009) experimentswith two algorithms for placing comments from interest groups into a space of policy preferences andcompares them to manual coding. Unfortunately, this type of coding only solves half the problem.After classifying interest group preferences, it is still necessary to compare these preferences with thefinal outcome. While one could imagine also automatically classifying outcomes in a compatible way,in all the examples I am aware of this second part is done manually. In contexts where the number ofoutcomes is small, this part of the manual coding might be relatively easy. But federal rules are complexand multidimensional, and it can be laborious to characterize the outcome for even a single rule, muchless thousands. The Wordfish algorithm explored in Klüver (2009) also assumes that preferences areonly differentiated along a single dimension. This single-dimension assumption makes sense in contextslike legislative voting in a two-party system, but the interest groups that participate in the notice andcomment process are very diverse and I am skeptical that their preferences are well approximated bypositions on a line.A third point of comparison is the pairwise similarity approach introduced in chapter 2. Thisapproach is built on top of Latent Semantic Analysis, a a simple type of topic model. Topic modelsidentify commonalities across documents in terms of topicsgroups of words that frequently occurin the same documentsand represent each document as weighted mixture of these topics. Thisstructure makes it straightforward to define and compute measures of similarity between documentsthat describe how much overlap two documents have in the topics they cover. We use the similaritybetween comments and the comment discussion as a measure of attention paid to each commenter. Theprimary difference between our pairwise similarity approach and the other automatic coding algorithmsdescribed above is that we do not attempt to place comments and rules in a policy space. Insteadwe skip this intermediate step and simply compare comments directly to rules. Our argument is thatrule discussions that contain many of the same words as a comment are likely to be summarizingor responding to that comment, and that this similarity therefore gives us a measure of regulatorattention for each comment. One advantage is that no manual coding is required and we can compute44the similarity between every comment and every rule in the sample with only a few hours of computerprocessing. Another advantage is that we don't have to assume that the policy debate around each ruleis single-dimensional, or that the dimensions are the same across rules. Pairwise measures thereforerequire fewer assumptions about the policies and preferences of commenters. The similarity approachtherefore overcomes many of the substantial problems with previous approaches. However, it still hasdrawbacks. One is that the similarity measures produced by topic models are hard to interpret in termsof influence. Another is that LSA provides no notion of precision, so we cannot distinguish betweenvery confident and very uncertain pairwise estimates, or test whether differences between estimates arestatistically significant. Finally, the pairwise measures fail to solve the problem of what I think of asstyle bias.One of the central challenges of working with natural language is that there are many ways to repre-sent the same idea. One way to think about these different texts with similar meanings is to distinguishbetween the style of a text, and the substancethat is, the underlying meaning. Ideally, when wemeasure influence we would like to detect the similarity in the fundamental ideas, arguments, or poli-cies described in different documents. When we accidentally detect similarity in writing style instead,we mis-measure influence. In particular, we are likely to over-estimate the influence of commenterswho writing in the same formal, jargon-filled writing style as the regulator, and underestimate theinfluence of commenters who use more casual language. In section 3.4 I run Monte-Carlo tests whichsuggest that this type of style bias is a substantial problem when estimating influence. Unfortunately,topic modeling does not provide any tools to differentiate between similarity in writing style and morefundamental similarity of meaning. However, with panel data (that is, multiple comments from eachauthor), it is possible to differentiate between the average writing style of each commenter and thecontent specific to each rule. The logic behind this decomposition is similar to that used in other eco-nomic settings where we want to distinguish between fixed and varying components of panel data: thelinearity of the estimators allows us to use high-dimensional fixed effects to control for each author'saverage comment. This is another important justification for focusing on linear models beyond theircomputational tractability, as the same procedure would likely lead to biased estimates in non-linearmodels due to the incidental parameters problem.All estimators described above, including my new estimators, use a bag-of-words approach torepresenting text. Bag-of-words refers to the way that documents are represented entirely by thenumber of times each word (or other discrete feature) occurs in the text. All document structure andcontextual information is stripped away, and each document is reduced to a simple vector of wordcounts. The advantage of this approach is that many operations on text can be implemented usinglinear algebra. It is also easier to write down a process for generating new text documents, since sincethe only thing that needs to be defined is the distribution of word frequencies.The alternative approach is to look at sequences of text. Cutting-edge natural language processinguses deep neural networks to model sequences of text and combine information from many words andthe way words relate to each other. For example, Devlin et al. (2018) construct a model that readscomplete sentences and can be used to improve performance on a suite of natural language processingtasks. There are two main reasons why I do not attempt to use these approaches. First, the models are45very computationally expensive to train and run. Second, the outputs are much harder to interpret.Sequence modeling techniques generate context dependent representations of text that come from avery complex algorithm. The data generating process is an unpredictable black box, and we endup back in the position of having measures that don't have a clearly defined scale or meaning. I'moptimistic that there are long-term solutions to this problem, but developing them is beyond the scopeof this chapter.I take advantage of a technology known as word vectors or word embeddings which existssomewhere between bag-of-words and full sequence modeling. Word vectors aim to overcome some ofthe limitations of using words alone to represent text. Even in two long documents discussing the sametopic, it will often be the case that the most important words are only used a small number of times,and the set of key words used in each document may not overlap. For example, one document mayuse the words house, housing, and homes, while another document discusses houses, dwellings,and apartments. Clearly both documents discuss houses, but the token frequency vectors wouldbe orthogonal. This problem is referred to as synonymy in the text analysis literature. Synonymycan be alleviated to some degree by simplifying or stemming words when tokenizing. For example,house, housing, and houses can all be stemmed to hous. But the problem of linking house toapartment cannot be solved this way. A popular solution to this approach is to instead representeach word by a real valued vector, where the vectors are generated by an algorithm that uses the waywords co-occur in text to infer which words are similar and give them similar vectors (Mikolov et al.,2013; Pennington et al., 2014; Joulin et al., 2016; Bojanowski et al., 2016). Each word has a contextindependent representation, but that representation summarizes contextual information learned fromthe text. Thus, house and apartment are located nearby in vector space because they often appearnear the same words. The estimators I propose use a general set of word vectors that could comefrom any algorithm. I show in Monte Carlo tests that vectors generated with LSA improve estimatorperformance relative to random vectors.The chapter is structured as follows: First, I present a very simple model of textual influence. Thepurpose of this model is to provide a clear interpretation for the coefficients representing influence, andto provide an explicit data generating process to motivate the estimators. Next, I derive estimatorsbased on cross-sectional data (one comment per author and a single rule) and present evidence of theireffectiveness from Monte Carlo tests where synthetic rules are generated from real comments. Finally,I extend the model of influence to account for commenter and regulator writing style and derive panelversions of each estimator that can control for writing style. I run Monte Carlo tests to evaluate howwell each estimator performs when writing style is an important factor in the data generating processand find that including fixed effects is very important when rules are generated using commenter andauthor writing styles.463.2 Modeling textual influence3.2.1 Bag-of-words influenceThe model takes a common approach to describing text. There are D input documents i = 1, 2, ..., D.These documents have been broken down into unique tokens (which could be n-grams, words,short phrases, or any other discrete features). There are T unique tokens across all the documents,denoted t = 1, 2, ..., T . Each document is represented as a column vector of token counts cd =[cd1 cd2 cd3 · · · cdT]Twhere cdt is the number of times token t occurs in document d. The totalnumber of tokens in each document is denoted nd. Each document has a corresponding token-frequencyvector that is equal to the token count vector divided by the number of tokens in the document,fd := cd/ndI construct the matrix F by stacking the document token-frequency vectors horizontally:F(T×D):=[f1 f2 f3 · · · fD]I model influence in the following way. After receiving the comments, the regulator generates a ruler from a weighted mixture of the input documents. Specifically, the regulator draws nr tokens fromthe input documents according to the following sampling procedure: For each token, the regulatorindependently selects an input document d with probability βd, and then copies one random tokenfrom that document into the rule. I interpret the vector βd as the degree of influence document d hadon the final rule.Under the procedure described above, the token counts in the rule are a random variable withmultinomial distributioncr ∼ mult(nr, pi)where,pi =D∑d=1βdcd/nd = FβOverall, this is a very simplistic model of both text generation and influence. The regulator simplyaccepts whatever comments are submitted and then mixes them together to generate the rule. However,I will address several of the most obvious weaknesses throughout the chapter. First, I will show that wecan relax the assumption that the regulator copies words exactly from the comments to the rule, andinstead allow for a regulator that paraphrases text from the comments with their own words. Second,I will show that we can extend the model to represent a regulator who mixes in their own writing styleand filters out the writing style of other authors. These two modifications make the model much morerealistic in terms of how we might actually expect a regulator to write a rule using comments as input,while maintaining the simplicity and robustness of the linear estimator framework. Nevertheless, the47model leaves several questions unanswered. One loose end is how the regulator chooses the numberof tokens in the rule. I see this as a minor point, since the estimates are not directly affected bythis parameter once everything is converted to token frequencies (though longer documents do resultin more precise estimates because of reduced sampling error). A more important consideration iswhere the influence parameters come from and to what extent we can think of the regulator as arational decision-maker. Ideally we would be able to map the influence parameters to the equilibriumstrategies of some game theoretic communication game played by the commenters and regulator andthen interpret the estimators as a reduced form. At the moment however, I am agnostic about whatgame is actually being played, and I think there is value in first building some descriptive statisticsthat might (if interpreted carefully) point towards certain communication models to focus on. Thesimple pi = Fβ model therefore serves as a foundation to build on, both in terms of adding linguisticrichness, and also game theoretic considerations in the future.3.3 Estimating influence3.3.1 Frequency-based estimatorsSetupGiven the setting described above it is possible to estimate the measures of influence β from theobserved token-frequency vectors f1, f2, ..., fD and the observed final rule document token frequenciesusing a linear estimator.First, note that from the properties of the multinomial distribution,E[cr|β] = nrpi = nrFβV ar(cr|β) = nr(diag(pi)− pipiT ) = nr(diag(Fβ)− (Fβ)(Fβ)T )If we define the empirically observed vector of token frequencies in the rule asg := cr/nrand the difference between the realized and expected value of g asε := g − E[fr|β] = g − Fβthen we can write the observed rule token frequencies g in the familiar form:g = Fβ + ε48OLS estimatorThe last equation suggests that we can estimate β with a least-squares estimator, the simplest beingOLS:βˆOLS = arg minβ‖g − Fβ‖2However, ε is not i.i.d. and instead has a correlation matrix that depends on β. OLS will be consistentbut not efficient.FGLS estimatorThe alternative is to construct an iterative FGLS estimator. Let Σ(β) be the variance-covariancematrix of ε. Then,βˆFGLS = arg minβ∥∥∥Σ−1/2(g − Fβ)∥∥∥2Σ(β) := V ar(g|F, β) = 1nr(diag(Fβ)− (Fβ)(Fβ)T )where we can use OLS to obtain the first estimate of β. FGLS sometimes performs poorly in finitesamples, so it is not clear which estimator will perform better in practice.Identification conditionsThe rank condition for OLS is Rank(F ) = D, implying two identification conditions:1. (a) The input documents must have unique token frequencies(b) The number of unique tokens must be larger than the number of documents (T ≥ D)3.3.2 Token and Document vectorsMotivationEstimating influence directly from token frequencies is probably not a good idea in practice, for tworeasons. First, the FGLS estimator is difficult to compute. Σ(β) has dimensions (T × T ), where T isoften on the order of hundreds of thousands of unique tokens. This puts the memory requirements forstoring this matrix somewhere on the order of hundreds of gigabytes.The second reason relates to the problem of synonymy raised in the introduction. Suppose thatwhen the regulator is copying tokens from the comments into the rule, they replace each word with asynonym or closely related word. Then the similarity of the token frequencies between comment andrule could be very low even when influence is high. The frequency-based estimators will be biaseddownwards. This seems like a realistic departure from the original model and something worthwhileto address.Employing pre-trained token vectors can solve both problems simultaneously. Suppose we have aset of pre-trained token vectors so that each token t has a vector representation vt ∈ Rk. Then we49can represent documents as weighted sums of the token vectors rather than the raw token frequencies.Best practices in the text analysis literature generally recommend token vector sizes with k in the lowhundreds. Thus, the first benefit of using token vectors is to reduce the size of Σ(β) from (T × T ) to amuch more smaller (k × k) matrix. The second benefit is difficult to show analytically, but at least asimportant. Each token vector algorithm will generate different token vectors, but correlations betweenthe occurrences of different tokens will be captured in the values of the vectors, so that tokens thatoccur in similar contexts (i.e., house and apartment) have similar vectors. It follows that documentswill also be assigned similar vectors when they discuss similar topics, even if they do not use exactlythe same words. The resulting estimator will be much more robust to changes in specific word choicewhile still detecting similarity of meaning and topic.SetupI assume that each token has an associated column vector vt ∈ Rk. The full set of token vectors isrepresented by the matrix V :V(k×T ):=[v1 v2 v3 · · · vT]Each document d is represented by a document vector xd, constructed by taking the sum of alltoken vectors, weighted by the frequency of their occurrence in the document.xd :=T∑t=1fdtvdt = V fdWhen we stack the input document vectors to formX(k×D):=[x1 x2 x3 · · · xD]and we can write the vector transformation succinctly as X := V F .I define the final rule document vector similarly, asy := V gToken Vector EstimatorsWe can derive vector OLS (VOLS) and vector FGLS (VFGLS) estimators from those in the previoussection by pre-multiplying g = Fβ + ε by V . This gives the base equation,y = Xβ + ωwhere ω := V ε is the difference between expected and observed document vectors. Then the VOLSestimator is:50βˆV OLS = arg minβ‖y −Xβ‖2And the VFGLS estimator is defined by the equations:βˆV FGLS = arg minβ∥∥∥Ω−1/2(y −Xβ)∥∥∥2Ω = V ΣV TThe new rank condition Rank(X) = D requires Rank(V ) ≥ D, and implies two additional identi-fication conditions:2. (a) There must be at least D unique token vectors(b) The token vectors must have dimension k ≥ DThese estimators are computationally feasible because it is possible to compute Ω in parts withoutever computing the full Σ matrix.Token weightsAnother lesson that can be learned from best-practices in text analysis is that not all tokens are equal.Most tokens are not very informative about the topic. For example, the results should not be driven byhow often input documents use the words the, and, or this. In our simplistic model of influence,these are equally informative about which documents have been mixed into the rule. But in practice, wewould like to de-emphasize uninformative words and emphasize the importance of matching importantkey words.LSA addresses this problem by computing a separate token weight for each token that reflectshow informative it is in distinguishing between documents. Document vectors are then constructed asweighted means of token vectors, which are often further normalized to unit length before comparison.By analogy, it seems intuitive to apply a similar weighting scheme when constructing document vectors.For example, one could scale each vector by a pre-determined weight that relates to the frequency ofthe token. Scaling the vectors does not affect the derivation of the estimators, so at first glance thisappears to be a tidy solution. However, this type of weighting is actually canceled out by the FGLSestimator since the optimal weighting matrix re-scales all tokens to have equal variance. Instead, wecan use token weights to scale the loss function as in weighted least squares.Suppose we have a desired token weight wt for each token t, and diagonal weighting matrix.W(T×T ):= diag([w1 w2 w3 · · · wD])Because W is (T × T ) we need to apply the weights to the loss before reducing the estimators totheir document vector form. This suggests the following weight matrix for the VFGLS estimator:Ω(β) := V ar(y|F, β) = VW−1/2ΣW−1/2V T51Which is equivalent to scaling the variance of each token by 1/wt. In the final estimate, highweight/low variance tokens are given more importance when estimating influence.3.3.3 Monte Carlo experimentsThe previous sections describe several estimators and consider how to implement those estimators withgeneral sets of token vectors and token weights. However, it is not clear how the choice of estimator,vectors, and weights affect the quality of estimates. In this section I present the results of severalMonte Carlo experiments which are designed to a) test the effectiveness of different estimator types,and b) provide guidance on choices of vectors and weights.Comment sample and setupAll of the results presented in this section are based on a common sample of real comments to theOffice of the Comptroller of the Currency (OCC) from regulations.gov. I constructed the commentsample as follows: First, I started with all comments to the OCC on regulations.gov (9311 commentsin 101 dockets). Then, I sampled a smaller number of comments from each docket, where the numberof comments samples was equal to the total number of comments in the docker raised to the 2/3 power.For example, if a docket had 39 comments in it, I took 392/3 ≈ 12 random comments to represent thisdocket. The purpose of this sampling procedure was to scale down the size of dockets with thousands ofcomments while keeping most of the comments in small dockets. The final sample has 1170 commentsin 80 dockets. The number of unique tokens is T = 19, 044.The comments were used to generate a series of synthetic rules according to the sampling proce-dure described in 3.2: Comments were first broken down into tokens (in this case single words withstandardized capitalization) and converted to vectors of token frequencies. Then, for each docket Igenerated 100 random β vectors. For each β, I constructed a synthetic rule from the real commentsusing β as the sampling weights.Each vector β was generated independently according to the following procedure:1. Draw element β0 value from uniform(0,1)2. Draw each element βd for d = 2, ..., (D − 1) in sequence from uniform(0,1−∑d−1i=1 βi)3. Set elements βD = 1−∑D−1d=1 βd4. Randomly shu e the order of the elements of βThus, each β is guaranteed to sum to one and will tend to have at least a handful of influentialcomments, even in very large dockets where the average influence of each comment must necessarilybe tiny.Single-sample and Split-sample estimatesAs I discussed in section 3.3.2, one of the main advantages of using appropriate token vectors is thatthey should help to improve an estimator's performance when the regulator is not using exactly the52same words as the commenter. But in order to observe this benefit, we need to generate synthetic ruleswith words that are related to, but not exactly the same, as the comments used to estimate influence.One way to achieve this property is to split every comment's tokens into two separate samples and useone sample for generating the rule, and the other sample for estimating influence. Then the estimatornever sees the true generating tokens, just a set of documents that are correlated with the generatingdocuments because they came from the same comment. The correlations come in two types. First,words that are more common than average in a particular comment will also be more common in boththe generating and estimating samples. Second, the comment is likely to use a variety of synonymsand closely-related topic words that co-occur with higher frequency than in the general corpus. Whenthese are split between the generating and estimating samples, it provides an opportunity for tokenvectors to improve the quality of the estimates. When the vectors do a good job of capturing thesemantic similarity between tokens, the differences between the vectors in the two samples should besmaller and the performance of the estimators should be higher. This is a much more realistic test ofthe estimator's performance. I refer to this approach as split-sample (as opposed to single-sample),and I use it for every Monte Carlo test except the first where I am establishing baseline estimatorperformance.EstimatorsDefinitions I compare the performance of four estimators: LSA,Cosine, VGLS, VOLS, VFGLS:1. LSA: This is a reproduction of the docket-level LSA similarity measure used in Chapter 2. Themeasure is computed from the raw token counts as follows: First, the comment token countsand rule token counts are concatenated into a single term-document matrix. Tokens that appearin less than 20% of the documents or more than 70% of the documents are dropped. Then thecounts are transformed with TF-IDF weighting such thatweightedcountdt =log(1 + cdt)log(1 + documentcountt)Where documentcountt is the number of unique documents that contain token t. Finally, thematrix of weighted counts is decomposed into token and document vectors using SVD. Theestimates are computed as the cosine similarity between each comment document vector and therule document vector. One major limitation of this measure is that when LSA is performed at thedocket level, the maximum dimension of the vectors is constrained by the number of comments,which is often quite small.2. Cosine: This is the simple cosine estimator often used for comparing the similarity of documentvectors in text analysis, whereβˆd =y · xd‖y‖ ‖x‖I include it as an intermediate step between LSA and the new linear estimators. The cosinemeasure is very similar to LSA except that it constructs document vectors from weighted meansof the token vectors within the documents. This means that the cosine measure can use token53vectors that were pre-trained on a much larger set of documents than those in the docket (in thiscase all the comments in the sample). On the other hand, it might turn out to be the case thatvectors trained on the docket alone are better at capturing the relationships between tokens thatare distinct to a particular rule, in which case docket-level LSA may perform better.3. VGLS: Vector Generalized Least Squares with the optimal weighting matrix derived in section3.3.2. This estimator is not feasible in practice because the optimal weighting matrix is a functionof β. However, it is a useful reference point when evaluating the FGLS estimator because it showsan upper limit of how well we can expect FGLS to perform when the procedure for estimating Ωworks very well.4. VOLS: Vector Ordinary least squares with no weighting matrix.5. VFGLS: Vector Feasible Generalized Least Squares. The OLS estimate for β is used to estimatethe optimal weighting matrix using the formula derived in section 3.3.2. There is a small addi-tional complication because β is strictly positive and βˆV OLS may have negative elements becauseof estimation error. These negative elements can cause the estimated weighting matrix to benon-positive semi-definite. My solution in these cases is to use a linear combination of βˆV OLSand 1/D to construct the weighting matrix,βˆ = (1−√λ)βˆV OLS +√λ1Dwhere λ solves (1− λ)min(βˆV OLS) + λ 1D = 0. This ensures min(βˆ) > 0.Single-sample estimation results The first Monte Carlo test establishes the performance of thefour estimators when the rule is generated from the same comments that are observed (single-sample).Figure 3.1 shows the results. In these plots, each dot represents the mean performance of the estimatorfor 100 randomly generated rules using one docket's comments. The 80 dockets in the sample result in80 data points in each plot. The x-axis is the number of comments in the dockets, so that it is possibleto see how performance varies across dockets and docket size. Each column is a different estimator,and each row is a different measure of performance. Correlation between βd and βˆd gives a measure ofhow much information the estimates extract about influence, ignoring issues of bias and whether thecoefficients can be interpreted directly. Mean squared error (MSE) captures the overall accuracy. Biasindicates whether the estimates are on average too high or low. Attenuation indicates how estimatesare biased towards zero for larger values of βd. Coverage shows the fraction of 95% confidence intervalsthat cover the true β. To make comparing across plots easier, the mean score among all the docketsin each plot is represented as a solid horizontal line.The main result is that all versions of the linear estimator perform better than both LSA and cosinesimilarity. Both LSA and cosine estimates have large biases, high mean-squared error, and of course,no standard errors for which to compute coverage. On important reason for the low performance ofthese estimators is related to the issue of interpretation raised in the introduction to this chapter.In particular, the relationship between these estimates and the true level of influence (defined by the54weight put on the comment when constructing the rule) is unknown. These results show that theseestimates  and cosine estimates in particular  are badly calibrated: an estimate of βˆd = 0.5 shouldnot be interpreted directly as the rule putting 50% weight on comment d.However, even when we ignore bias, the correlation between βd and βˆd still much is lower for cosinesimilarity and LSA than other estimates. This means that the problem with these estimators is notsimply that they are biasedthey also convey less information about the degree of influence than thenew estimators. There are several factors that likely contribute to this result.First, influence is assumed to be mutually exclusive (the elements of β sum to one) and as thenumber of comments in the docket increases, an increasingly large fraction of the comments haveessentially zero influence on the rule. The benefit of using the linear estimators instead of LSA or cosinesimilarity increases they compute all elements of β simultaneously, implicitly incorporating informationabout the correlations between comments. When two comments are similar, their estimated influenceis reduced (even though the estimated β is not constrained to sum to one). The LSA and cosinemeasures just look at pairwise comment-rule relationships and ignore this additional information. Thisdifference is particularly important for large dockets where most comments have negligible influenceon the rule, but still have some average degree of similarity with the rule text.The linear estimators also automatically weight vector components by the inverse of their variance.This places more weight on the informative dimensions of the document vectors. LSA attempts todo something similar by dropping extremely frequent and extremely rare tokens and using TF-IDFweighting for the rest, based on the assumption is that only tokens that occur with moderate frequencyare informative about topic, and that the importance of a particular token is inversely related to thenumber of documents in which it appears. But in this test, the regulator does not differentiate betweencommon and rare tokens when constructing the rule, and all are informative. The number of documentscontaining the token is probably helpful in guessing which tokens are informative, but it is a fairly adhoc approach relative to the linear estimators' inverse variance weighting.Given the fact that LSA uses this additional weighing scheme it is surprising that the cosineestimates have such similar correlations with the true β. One possibility is that TF-IDF weightingdoes improve the LSA estimator, but the cosine estimator benefits instead from using token vectorsthat have been trained on a larger set of documents. LSA is based on dimensionality reduction of theterm-document matrix, which is only beneficial when there is a very large number of documents and inthis context there are many dockets with only a handful of comments. This gives very little informationto learn about the co-occurence of different tokens and construct high quality token vectors. More testswould be required to distinguish how different weighting these different factors influence the estimates.Overall, the test results suggest that the linear estimators are a substantial improvement over LSA,both in terms of interpretability and accuracy. VFGLS is particularly effective, with the low mean-squared error and good coverageperforming essentially as well as the infeasible VGLS estimator thatwas included for reference. As expected, VOLS is unbiased, but has the higher MSE, and relativelypoor coverage because it (incorrectly) assumes that the errors are i.i.d.Split-sample estimation results In the second Monte Carlo test I run the same estimators (withthe same betas) on the split-sample comment tokens. The results are presented in figure 3.2. As55expected, all the estimators have higher MSE and the estimates are attenuated. Coverage is also muchworse. However, the new linear estimators still produce useful information even in this much hardertask, and still perform substantially better than LSA or cosine similarity. In the following Monte Carlotests I examine whether the choice of vectors and weights can improve performance on this task usingthe VFGLS estimator.VectorsDefinitions I consider three types of vectors: Random: Completely random vectors for each token, drawn from a standard multivariate normaldistribution Doc: LSA vectors computed on the whole multi-docket comment sample, using the usual document-level token frequencies. Par: LSA vectors computed on the whole multi-docket comment sample, but splitting commentsinto paragraphs and treating each paragraph as a separate document. This is an experimentalapproach to see if LSA vectors computed this way are higher quality. It is inspired by Word2Vecand other algorithms that use a small neighborhood around each word to infer the word's vector.Additionally, for each type of vector I run the estimator with several truncations at k = 50, 100, 200, 400.Results Figures 3.3 and 3.4 show the results of the vector experiments. The first shows docket levelscatter plots as before. The second figure averages docket scores together, but splits the estimates byk. The docket sample is restricted to those with fewer than 25 sample comments to avoid violating thek ≥ D identification condition. Overall, random vectors perform surprisingly well. We can expect thatas k → T , V becomes full rank and the estimator converges to the token frequency-based described insection 3.3. This pattern is apparent in figure 3.4 where MSE decreases with k. But it is interestinghow good the performance is with relatively low values of k relative to T .At the same time, it is clear that the LSA vectors (particularly the doc vectors) improve perfor-mance and reduce attenuation as hypothesized. The benefit of using LSA over random vectors appearsto peak at a moderate degree of k. This suggests that LSA is successfully extracting a low-dimensionallatent representation of the tokens that successfully removes much of the noise in the raw tokencounts. Given that the vectors do seem to matter, it will be worthwhile to experiment with a broaderset of vector types in the future.Weighting schemesDefinitions I test several weighting schemes that have been explored in the LSA literature (seeAnaya (2011)).1. None: Vectors are normalized by their standard deviation, but no differential token weights areapplied.562. Idf: Inverse document frequency weights, defined as follows:wt =1log(1 +∑Dd=1 1{cdt>0})3. Ent: Log-entropy weights, as defined in (Pincombe, 2004):wt = 1 +∑Dd=1 sdtlog(sdt)log(D)sdt =cdt∑Dj=1 cjt4. Norm: Weights are set such that the column vectors of F have the equal euclidean length.wt =1√∑Dd=1 c2dtIn each case, I apply the weights to tokens to the count matrix before computing LSA vectors as wellas using them to adjust the estimator as in section 3.3.2.Results Adjusting the token weights has minimal impact on estimator performance in this setting.One might expect that rare words are more informative and that increasing the weight on these wordswould improve the estimates. However, it seems that the optimal weighting matrix in the VFGLSestimator already accounts for this factor and is difficult to improve upon. I believe this is a smallcontribution to the NLP literature.57Figure 3.1: Single-sample estimation: This grid of plots summarizes the performance of the main linear estimatorswhen the same input documents are used for generating the rule and estimating influence. Each dot represents the meanperformance on one docket across 100 samples with random βs. The solid horizontal line shows the average performanceacross all dockets. The horizontal axis is the number of comments in the docket. The vertical axes show four measures ofestimator quality: Correlation is the Pearson correlation coefficient for βd and βˆd, log Mean Squared Error (MSE), Bias(mean βd − βˆd), Attenuation (equal to 1− α, where α is the slope coefficient in a regression of the form βˆd = αβd + ε),and Coverage (the fraction of estimates for which the estimated 95% confidence interval for βˆd includes the true βd).Vectors are generated from document-level LSA (k = 200).58Figure 3.2: Split-sample estimation: This grid of plots summarizes the performance of the main linear estimators whenthe tokens of each input document are split into separate gen and est samples. The gen samples are used to constructthe final rule, while the est samples are used to estimate influence. Each dot represents the mean performance on onedocket across 100 samples with random βs (with the same gen/est split). The solid horizontal line shows the averageperformance across all dockets. The horizontal axis is the number of comments in the docket. The vertical axes showfour measures of estimator quality: Correlation is the Pearson correlation coefficient for βd and βˆd, Log Mean SquaredError (MSE), Bias (mean βd− βˆd), Attenuation (equal to 1−α, where α is the slope coefficient in a regression of the formβˆd = αβd + ε), and Coverage (the fraction of estimates for which the estimated 95% confidence interval for βˆd includesthe true βd). Vectors are generated from document-level LSA (k = 200).59Figure 3.3: Split-sample vector performance comparison: This grid of plots summarizes the performance of threedifferent types of vectors in the VFGLS estimator when the tokens of each input document are split into separate genand est samples. The gen samples are used to construct the final rule, while the est samples are used to estimateinfluence. Each dot represents the mean performance on one docket across 100 samples with random βs (with the samegen/est split). The solid horizontal line shows the average performance across all dockets. The horizontal axis is thenumber of comments in the docket. The vertical axes show four measures of estimator quality: Correlation is the Pearsoncorrelation coefficient for βd and βˆd, Log Mean Squared Error (MSE), Bias (mean βd− βˆd), Attenuation (equal to 1−α,where α is the slope coefficient in a regression of the form βˆd = αβd + ε), and Coverage (the fraction of estimates forwhich the estimated 95% confidence interval for βˆd includes the true βd).60Figure 3.4: Split-sample vector performance comparison by k: This grid of plots summarizes the performance of threedifferent types of vectors in the VFGLS estimator when the tokens of each input document are split into separate genand est samples. The gen samples are used to construct the final rule, while the est samples are used to estimateinfluence. Each dot represents the mean performance average performance across all dockets for vectors truncated tolength k. The vertical axes show four measures of estimator quality: Correlation is the Pearson correlation coefficientfor βd and βˆd, Log Mean Squared Error (MSE), Bias (mean βd − βˆd), Attenuation (equal to 1− α, where α is the slopecoefficient in a regression of the form βˆd = αβd + ε), and Coverage (the fraction of estimates for which the estimated95% confidence interval for βˆd includes the true βd).61Figure 3.5: Split-sample weighting performance comparison: This grid of plots summarizes the performance of fourtoken weighting schemes in the VFGLS estimator when the tokens of each input document are split into separate genand est samples. The gen samples are used to construct the final rule, while the est samples are used to estimateinfluence. Each dot represents the mean performance on one docket across 100 samples with random βs (with the samegen/est split). The solid horizontal line shows the average performance across all dockets. The horizontal axis is thenumber of comments in the docket. The vertical axes show four measures of estimator quality: Correlation is the Pearsoncorrelation coefficient for βd and βˆd, Log Mean Squared Error (MSE), Bias (mean βd− βˆd), Attenuation (equal to 1−α,where α is the slope coefficient in a regression of the form βˆd = αβd + ε), and Coverage (the fraction of estimates forwhich the estimated 95% confidence interval for βˆd includes the true βd). Vectors are generated from document-levelLSA (k = 200).623.4 Panel Estimators3.4.1 MotivationThe main advantage of panel data is that it allows one to distinguish between fixed and transientcomponents of the data. For example, when estimating the effect of a new labour policy on wages,panel data allows researchers to separately control for each worker's average wages using a fixed effectsmodel. When we observe organizations submit comments on multiple rules, we can pursue an analogousestimation strategy to control for each author's average writing style. By style I mean broadly anyaspects of word choice that a) are correlated across comments by the same author, and b) do notsubstantially inform the meaning of the comment on a particular rule. For example, consider thefollowing two excerpts that come from comments submitted to the Environmental Protection Agency(EPA) regarding the Clean Power Plan, a major regulation that would have limited CO2 emissionsfrom power plants. The first is from an individual TK who opposes the proposed rule because it willincrease electricity prices for seniors on fixed incomes:I am a retired veteran. Over 40 years we have been trying to fight climate change! TheEPA has done its share, to both fight and deter thru reglations. Some good, some bad. Iwill not be alive to see the current results of EPA's efforts; however, my primary concern isthat the rule as proposed will result in significant electricity rate increases and additionalenergy costs for all users. The costs always seems to fall most heavily on we, the elderly,the poor, and those of us on fixed incomes. The Administration is blowing smoke, to makeus believe that in 30 years we can overcome the pollution problem, much less the climatechange, they dream of!The second is from a comment submitted by the Thomas Jefferson Institute for Public Policy:Energy prices and stagnant incomes are straining the budgets of Virginia's lower- andmiddle-class families. Virginia households with gross annual incomes below $50,000, repre-senting 41% of Virginia's households, spend an estimated average of 20% of their after-taxincome on energy. Energy bills for the 185,000 poorest households earning less than $10,000represent 71% of their family incomes, before accounting for any energy assistance programs.Increased energy costs are competing with other necessities for lower- and middle-incomefamily budgets across Virginia.The impacts of increased energy costs are falling disproportionately on Virginia's elderlySocial Security recipients, representing 27% of the state's households. In 2012, SocialSecurity recipients in Virginia had average household Social Security incomes of $16,928.Some 21% of Virginia households had retirement income averaging $28,357 before taxes.Unlike young working families, many fixed income seniors are limited to cost-of-livingincreases that do not keep pace with energy prices. Maintaining the relative affordability ofelectricity and natural gas prices, and increasing low-income energy assistance, are essentialto the wellbeing of millions of Virginia's senior and lower-income citizens.These Virginia citizens cannot afford this rule and, as discussed below, it produces nobenefits worth this cost and is not authorized under the Clean Air Act.63The two comments make similar arguments, but use many different words. The first is written ininformal language with non-standard spellings of thru and reglations, the use of I as a pronoun,and the vernacular blowing smoke. In contrast, the second comment is more formal in tone, containsno obvious spelling mistakes and uses many numerical quantities. Intuitively the two excerpts seemquite different, and we can expect these differences to be measurable in terms of token frequencies.Now consider an example of the EPA discussing how the plan will affect electricity prices:Under Option 1, average nationwide retail electricity prices are projected to increaseby roughly 6 to 7 percent in 2020 relative to the base case, and by roughly 3 percent in 2030(contiguous U.S.). Average monthly electricity bills are anticipated to increase by roughly3 percent in 2020, but decline by approximately 9 percent by 2030. This is a result of theincreasing penetration of demand-side programs that more than offset increased prices toend users by their expected savings from reduced electricity use.Like the Thomas Jefferson Institute, the EPA uses formal language and many numerical quantities.None of the informal spellings and vernacular used in the first comment are present. Thus, it is likelythat the second comment will be estimated to be more similar to the EPA's rule text (and thereforemore influential) than the first, even though the two comments make essentially the same argument.Furthermore, these differences are likely to persist across all comments submitted by these authors,leading to biased influence estimates for every rule.Comments also often contain boilerplate text that are unrelated to the content of the message.Many commenters begin with a paragraph describing their organization such as these examples fromcomments to the OCC:The National Community Reinvestment Coalition (NCRC) is an organization of morethan 600 community-based organizations with a mission to promote access to basic bank-ing services, including credit and savings, to create and sustain affordable housing, jobdevelopment, and vibrant communities for America's working families. NCRC has been anoutspoken advocate on these issues for more than 20 years.The American Bankers Association represents banks of all sizes and charters and is thevoice of the nation's $14 trillion banking industry and its 2 million employees. Learn moreat www.aba.com.Comments also often contain extraneous text such as email footers like this confidentiality notice fromCommunity National Bank & Trust:CONFIDENTIALITY NOTICE: This message contains information from CommunityNational Bank & Trust which may be confidential and privileged. lf you are not an intendedrecipient. please refrain from any disclosure, copying. distribution or use of this informationand note that such actions are prohibited. If you have received this transmission in error,please notify by e-mail techsupport@communitynational.net and destroy all copies of theoriginal message. Thank you.64Ideally these pieces of unrelated text would be removed from comments before estimating the similarity,but in practice it is difficult to reliably detect which text is unrelated to the current rule. If thesepieces of extraneous text are correlated with the regulator's average writing style (either positively ornegatively), they will also bias individual influence estimates.Existing pairwise similarity measures cannot solve this problem. As an illustrative example, supposethe rule document vector yr can be decomposed into a fixed style component y¯ and a rule-specificsubstance component y˜r, and each comment document vector xir can be similarly decomposed intofixed style component x¯i and a rule-specific substance component x˜ir:yr = y¯ + y˜rxir = x¯i + x˜irHere the style component contains token frequencies that reflect specific word choices that mightidentify the commenter, such as the use of I or we, vernacular terms, choices of specific synonyms,as well as portions of extraneous text that appear in every comment. The substance component onthe other hand, contains all the deviations from the average style, and therefore the information in thecomment that is specific to a particular rule.Suppose we compute a pairwise similarity measure s(yr, xir) between each rule r and commentsubmitted by author i. For the linear measures discussed in this chapter,s(yr, xir) = s(y¯, x¯i) + s(y˜ir, x˜ir) + s(y¯, x˜ir) + s(y˜ir, x¯i)If s(y¯, x˜ir) and s(y˜ir, x¯i) are small on average (which seems likely given that the style and substancecomponents will look very different), then we can focus on the similarity between styles and thesimilarity between substances:s(yr, xir) ≈ s(y¯, x¯i) + s(y˜ir, x˜ir)Now consider two commenters, a and b. Commenter a is very influential, but has a very differentwriting style to the regulator so s(y¯, x¯a) = 0 and s(y˜r, x˜ar) = β. Commenter b is not at all influential,but has a similar writing style to the regulator so s(y¯, x¯b) = β and s(y˜r, x˜br) = 0. Then, when thepairwise estimates are computed, s(yr, xar) ≈ s(yr, xbr) ≈ β, and this relationship holds for every rule.So there is no observable difference between a and b using pairwise measures. Subtracting the meanpairwise similarity, for example, will not help at all.The solution I propose here is to subtract the means of the document vectors (y¯ and x¯i) beforecomputing similarity. Then we isolate s(y˜ir, x˜ir), the overlap in substance between the rule and com-ment. This operation is analogous to subtracting each worker's mean wage before estimating the effectof the policy in the wage example above, and, as in the wage example, a consistent estimator can alsobe constructed using the appropriate fixed effects.The remainder of this section takes a systematic approach to deriving a computationally feasibleestimator based on this intuition.653.4.2 Panel estimationSuppose there are I authors i = 1, 2, ..., I who comment on R rules r = 1, 2, ..., R, with a constantdegrees of influence captured by the vector β = [β1, β2, . . . , βI ]T. Let xir ∈ Rk be the vector for thedocument submitted by author i on rule r, and yr ∈ Rk be the rule document vector for r. Then wecan stack the observations as follows:y =y1y2...yR X =x11 x21 · · · xI1x12 x22 · · · vI2............x1R x2R · · · vIR ω =ω1ω2...ωRSuch thaty = Xβ + ωWe can use the fact that we know the distribution of each rule's token sampling error r to motivatea feasible generalized least squares estimator,βˆPFGLS = arg minβ∥∥∥Ω−1/2(y −Xβ)∥∥∥2Because the token sampling error is independent across rules, the variance-covariance matrix of ωis block diagonal:Ω = blockdiag(Ω1,Ω2, . . . ,ΩR)Where each block Ωris the same variance-covariance matrix as the cross-sectional estimator for thatrule:Ωr = VW−1/2ΣrW−1/2V TΣr =1nr(diag(pir)− pir · piTr )As in the single-rule case, we can construct a feasible version of this GLS estimator with a two-stepestimation procedure where Ωˆ is constructed from a first-stage, unweighted estimation of βˆ.One consequence of moving to a panel estimator is that identification is easier. Recall that forindividual dockets, the estimators required that the dimensionality of the document vectors k waslarger than the number of documents D, and no two documents could have the same text. With paneldata this restrictions are relaxed. The rank condition for the panel estimator isRank(X) = IThis rank condition is much easier to meet, since:1. Columns of X will be unique even if commenters occasionally submit the same text for one rule(the rank condition only requires that they do not submit the same text for every rule)662. The token vectors only need to have dimension k such that R∗k ≥ I. So it is possible to estimatethe influence of 100 commenters using 50-dimensional document vectors as long as there are atleast two rules in the panel.3.4.3 Modeling author writing styleSuppose each comment token frequency vector fir is generated as the sum of a style component f¯i anda substance component f˜ir with the following properties:fir = f¯i + f˜irEr[f˜ir] = 0In other words, each commenter i has a baseline token frequency f¯i representing their averagecommenting style, and a rule-specific deviation f˜ir from that baseline frequency that accounts for thesubstance of each comment. Over enough rules, the rule-specific component averages to zero. I treatthis property as an assumption, but it can also be thought of as a definition - if a commenter repeatsthe same content in every rule, the regulator is likely to discount it as repetitive and uninformative.The regulator generates the token counts cr for each rule r using only the substance component f˜irof each comment, plus an additional rule style component p¯i:cr ∼ mult(nr, pi)Where I redefine pi aspi = θ +I∑i=1βif˜ir = p¯i + F˜rβAs before, let us define gr = cr/nr as the observed token frequencies in the rule document. Then:E[gr|F˜r, β] = p¯i + F˜rβV ar(gr|p¯i, F˜r, β) = 1nr(diag(pi)− pipiT ) = 1nr(diag(p¯i + F˜rβ)− (p¯i + F˜rβ)(p¯i + F˜rβ)T )ε := gr − E[gr|F˜r, β] = gr − p¯i − F˜rβand we can write:gr = p¯i + F˜rβ + εrFor simplicity I assume that the matrix of token vectors V is constant for all rules. Then we canpre-multiply all terms by V and write the document-vector version as67yr = y¯ + X˜rβ + ωrAnd if we stack the observations for each rule, we can writey = Y¯ + X˜β + ωwherey =y1y2...yR Y¯ =y¯y¯...y¯ X˜ =X˜1X˜2...X˜R ω =ω1ω2...ωRWe have now a data-generating process that captures the intuitions about writing style explored atthe beginning of this section. Each commenter i has a fixed writing style represented by their averagecomment token frequency vector f¯i. The regulator ignores this component of the comment text, andonly uses deviations from the average comment to construct the rule. Similarly, the regulator has afixed writing style represented by p¯i which they add to the text of every rule. The challenge is toestimate the influence vector β given that neither f¯ nor p¯i are directly observed.3.4.4 Fixed effects estimationThere are two ways to implement a fixed effects estimator that will estimate β. One is to estimate Y¯ andX˜ separately by taking the mean of y and subtracting the mean of X, and then run the regression. Thisdemeaning is a linear transformation that can be represented by a centering projection matrix P whichremoves the means across rules. The second is to add commenter-by-vector-component fixed dummyvariables and estimate β in one step. The two approaches generate identical estimates, but it turnsout that the dummy-variable estimator is more computationally efficient (at least when implementingthe full FGLS estimator).First, suppose we follow the demeaning approach. Then we estimate Y¯ or X˜ras follows:Y¯ = (I − P )yX˜ = PXThen, we can convert the regression equation to one based purely on observables by pre-multiplyingwith P to give,Py = PXβ + PωI call the FGLS estimator based on this equation the demeaning estimator:βˆDemean = arg minβ∥∥∥Γ−1/2P (y −Xβ)∥∥∥268Γ = PΩPWhere Ω is the optimal weighting matrix from the panel estimator above.Alternatively, one can simultaneously estimate Y¯ and X˜ with dummy variables. Let yj ,Xj indicatethe set of R observations in the panel data that share a common vector component j = 1, 2, . . . , k (Itmight be helpful to imagine sorting the rows in the data by vector component instead of regulation, sothat each block of the data has one observation for each regulation from a common vector component).Then we can replaceY¯j = 1αjX¯j =[1 1 · · · 1]diag(γj)To getyj = 1αj +Xjβ −[1 1 · · · 1]diag(γj)β + δjNote however, that 1αj and[1 1 · · · 1]diag(γj)β are perfectly collinear. So we can actuallyjust simplify the regression toyj = Xjβ + 1µj + ωjwhere µj = αj + diag(γj)β.In other words, commenter and regulator styles cannot be separately identified from the data.Conversely, adding vector component-specific fixed effects is sufficient to control for both commenterand regulator fixed components (style). Thus, the dummy estimator can be written as,βˆDummy = arg minβ∥∥∥Ω−1/2(y −Xβ −Mµ)∥∥∥2where M = 1R ⊕ Ik using Kronecker product notation.Note that P projects onto the orthogonal complement ofM . Therefore, by the Frisch-Waugh-Lovelltheorem (Frisch and Waugh, 1933), the demeaning and dummy estimators will give the same estimatesfor βˆ. However, they are not computationally identical for VFGLS estimation because the demeaningestimator requires computing (PΩP )−1/2, while the dummy estimator only requires computing Ω−1/2,which is much easier. To see why, note that Ω is block diagonal. ThereforeΩ−1/2 = blockdiagonal(Ω−1/21 ,Ω−1/22 , . . . ,Ω−1/2R )Thus, the weighting transformation can be applied to each set of rule observations independently,just as in the single-rule estimators. In contrast, PΩP is a dense kR×kR matrix that may be difficult tohold in memory, much less compute a square root factorization and solve for an inverse transformation.On the other hand, the dummy estimator requires an additional step of computing Ω−1/2M andadding an additional k variables to the OLS optimization. The block-diagonal structure of Ω also helps69when computing Ω−1/2M . Specifically,Ω−1/2M =Ω−1/21Ω−1/22...Ω−1/2RGiven a reasonable value of k, the additional burden of estimating µ should not be that high. Thedummy estimator is likely to be much more computationally efficient than the demeaning estimator.3.4.5 Monte Carlo ExperimentsHere I present the results of Monte Carlo tests with panel estimators. The main goal of these testsis to examine how the existence of commenter and regulator style affect the performance of the basicpanel estimators, and to determine whether the fixed-effects estimation approach described above com-pensates for this more realistic data generating process. To provide a point of comparison, I constructsynthetic rules with and without style components, and run both FE and non-FE estimators on bothsets of synthetic rules. This allows me to examine how robust each estimator is to misspecification.To make the comparison as realistic possible, I again use real comments submitted to the OCC, anduse real rules published by the OCC to choose the regulator's average writing style. The results of thetests suggest that style can generate huge biases in influence estimates. However, the tests also showthat fixed effects are an effective way of removing style bias.Practical considerationsThe derivation of the panel data estimators makes two assumptions that are not completely realistic.First, the derivations assume a balanced panel. In contrast, real commenting data almost always formsan unbalanced panel (most commenters do not comment on every rule). I address this issue in oneof two ways, depending on the type of estimator. When there are no fixed effects, I treat missingvalues as zeros. When fixed effects are used, I substituting missing comment token frequencies withthe commenter's average style f¯i. In effect, the commenters are assumed to submit perfectly genericcomments (relative to their own commenting style) on every rule. When regulators only pay attentionto deviations from average style, submitting a generic comment and not submitting at all becomeequivalent.The second problem is that p¯i+F˜rβ may be negative for some tokens. The negative values representa fundamental tension between the assumptions that p¯i + F˜rβ is a valid probability vector and theassumption that E[F˜r] = 0. The latter implies that F˜rβ will often have negative components, andthere is no guarantee they will be smaller in magnitude than the corresponding (positive) componentsin p¯i. For example, this problem will always occur when a commenter occasionally uses a token that theregulator never uses. Then the comments lacking this token will have f˜rt < 0 while p¯it = 0. Improvingthe modeling of this style component might be a useful direction for future work. In the meantime, Itake the expedient approach of replacing negative values with zero. One limitation of this approach is70that it adds noise to the generated rule and likely decreases the apparent effectiveness of the estimators.DataThe sample of comments used in these tests consists of all comments submitted to the OCC by orga-nizations that submit at least 5 comments in the years 2003 to 2017. The minimum comment countensures that commenter style is identified and can be reasonably estimated, and also keeps the size ofthe regression matrices manageable. The input data consists of 1289 comments from 108 unique com-menters in 82 dockets. To estimate the OCC's average writing style, I use the set of all rules publishedby the OCC to the federal Register since 2000 (92 documents). For each rule, I extract the portionof the rule text that contains the agency's discussion of the comments using the same technique as inChapter 2.Test procedureI construct Monte Carlo influence estimates according to the following procedure:1. I split each comment into gen and est token samples, as in the split-sample tests described inSection 3.32. I compute a style vector f¯i for each commenter by taking the mean token frequency across thegen comment token samples3. I compute the regulator's style vector p¯iby taking the average token frequency across all discussiontext from OCC rules4. I compute V with document level LSA using both comment gen tokens and the OCC discussiontext5. I draw 1000 random βs using the procedure described in Section 3.3.36. For each random β:(a) I construct two synthetic rules for each docket in the comment sample: One where the ruleis a simple mixture of input comments such thatpir = Frβ, the other where the rule is amixture of the regulator's base style and the deviation from commenter's average styles suchthat pir = p¯i + F˜rβ.(b) I run each estimator on each synthetic rule, with and without style fixed effects, to generatea total of four estimates. For two of these estimates, the data generating process matchesthe estimator's modeling assumptions, for the other two estimates, the estimators are mis-specified - either using style fixed effects when they are not necessary, or omitting themwhen they should be included.71EstimatorsI compare the performance of four estimators: Cosine, VOLS, VFGLS:1. Cosine: The cosine estimator, extended to panel data as follows:βˆd =y ·Xi‖y‖ ‖Xi‖Where Xi is the formed by concatenating the document vectors xi1, xi2, . . . , xiR. The fixed effectsversion is implemented by subtracting the observed mean token frequencies for each commenterfrom each of their comments before computing βˆd (the cosine equivalent of the demeaning esti-mator derived above).2. VOLS: Panel Vector Ordinary least squares with no weighting matrix.3. VFGLS: Panel Vector Feasible Generalized Least Squares. The VOLS estimate for β is usedto estimate the optimal weighting matrix using the formula derived in section 3.4. Each Ωriscomputed as1nrV (diag(pˆir)− pˆir · pˆiTr )V TFor the non-fixed effects version of the estimator, pˆi is computed the same way as for the crosssectional estimators (pˆi = Fβˆ). However, for the fixed effects version, the formula for pˆi is,pˆi = p¯i + F˜ βˆwhere p¯iand F˜ are estimated from the observed data. I apply the same correction to the estimatedβ strictly positive as in the cross-sectional Monte Carlo analysis.ResultsFigures 3.6 and 3.7 summarize the results of the Monte Carlo test. Figure 3.6 shows the results whenthe rules are generated without style effects. Each column contains estimates from a single estimator,and each row shows a different measure of the quality of the estimates. Each point represents theaverage performance for a specific commenter.The x-coordinate of each point is determined by the similarity between the commenter's averagecomment vector x¯iand the regulator's average rule vector y¯. In other words, commenters with a similarwriting styles to the OCC will appear to the right in each plot, while commenters that have verydifferent writing styles from the OCC will appear on the left. When rules are generated with styleeffects, it seems likely that the this correlation between commenter and regulator style will affect theperformance of the estimators, and separating the data along this dimension allows us to inspect forthese types of patterns visually. Because this style similarity is fixed across all sample estimates, thex-coordinate for each commenter is constant across all plots. Most commenters have a correlationof between 0.05 to 0.1 with the style of the OCC. Note that this is a highly selected sample, sinceonly organizations that are very active in the regulatory process comment at least five times. Manually72exploring the comments suggests that the sample is dominated by banks and organizations representingbanks. Less frequent commenters would probably have writing styles that are, on average, less similarto the OCC.The main result of these Monte Carlo tests is that generating rules with style effects makes a largedifference to the influence estimates and it is very important to include fixed effects to compensate.VFGLS is particularly sensitive to omitting fixed effects when they are required, and this type ofmisspecification results in extremely high mean-squared error. On the other hand, the performancepenalty for including fixed effects when they are unnecessary is negligible. The story for VOLS issimilar, though the differences are not as large. For simple cosine estimation, fixed effects actuallyhelp even when there are no style effects. Overall, the results suggest that if there is any doubt aboutwhether the true data generating process includes style effects, fixed effects estimators should be used.Figure 3.8 illustrates how writing style and fixed effects change the estimates. Here I show theindividual VOLS point estimates βˆi plotted against the true values of βi for three random commenters.In the top-left quadrant the rules are generated without any style effects, and the estimator does notinclude fixed effects. Here the estimator is correctly specified and the estimates line up along the45-degree line where βˆi = βi. In the top-right quadrant, the estimator includes unnecessary style fixedeffects, but they do not have a large effect on the estimates. In the bottom row, the rules are generatedwith style effects and the results are quite different. First, when the estimator lacks fixed effects(bottom-left quadrant), each commenter's estimates are offset by a individual bias that shifts all oftheir estimates up or down by a constant amount. These biases are very large relative to the estimatesthemselves. For example, the green commenter is estimated to be substantially more influential thanthe other two commenters, regardless of the true value of βi. For VFGLS these biases are so largethat the estimates often fall far outside of the range of the plot. Given that the actual correlationsbetween commenter and regulator style are only on the order of 0.05-0.1, this pattern suggests thatthe non-FE VOLS and VFGLS estimators exaggerate similarity in unpredictable ways. However, whenfixed effects are included (bottom-right quadrant), all of the commenter-specific biases are removed,leading to accurate estimates.3.5 Concluding RemarksThe goal of this chapter is to provide improved techniques for measuring political influence from textdata. I explore a class of estimators that are based on a novel combination of existing techniques:bag-of-words features for representing documents, word embeddings, and generalized least squaresestimators. These estimators can be applied in situations that competing approaches such as manualcoding or Wordfish cannot, and improve on the LSA algorithm we applied in 2 in several ways.I use Monte Carlo tests to demonstrate that the estimators are unbiased and have correct coveragewhen the rules are generated exactly as assumed. While LSA generates estimates that are correlatedwith the true degree of influence, the LSA estimates are also biased upwards, less precise, and provideno standard errors. I also test a more realistic variation of the rule-generation process that attemptsto mimic a regulator who samples from comments, but replaces many of the words with synonyms andrelated terms instead of using exactly the same language. I find that the estimators still perform much73Figure 3.6: This grid of plots summarizes the performance of the panel estimators when the rules are generated withno style effects. Each dot represents the mean estimate for one commenter across 1000 samples with random βs. Thesolid horizontal line shows the average performance across all commenters. The horizontal axis is the similarity betweenthe commenter's style and the regulator style, as measured by the correlation between x¯i and y¯. The vertical axes showfour measures of estimator quality: Correlation is the Pearson correlation coefficient for βd and βˆd, log Mean SquaredError (MSE), Bias (mean βd− βˆd), Attenuation (equal to 1−α, where α is the slope coefficient in a regression of the formβˆd = αβd + ε), and Coverage (the fraction of estimates for which the estimated 95% confidence interval for βˆd includesthe true βd). Vectors are generated from document-level LSA (k = 100).74Figure 3.7: This grid of plots summarizes the performance of the panel estimators when the rules are generated withstyle effects. Each dot represents the mean estimate for one commenter across 1000 samples with random βs. The solidhorizontal line shows the average performance across all commenters. The horizontal axis is the similarity between thecommenter's style and the regulator style, as measured by the correlation between x¯i and y¯. The vertical axes show fourmeasures of estimator quality: Correlation is the Pearson correlation coefficient for βd and βˆd, log Mean Squared Error(MSE), Bias (mean βd − βˆd), Attenuation (equal to 1 − α, where α is the slope coefficient in a regression of the formβˆd = αβd + ε), and Coverage (the fraction of estimates for which the estimated 95% confidence interval for βˆd includesthe true βd). Vectors are generated from document-level LSA (k = 100).75Figure 3.8: This grid of plots shows the point estimates for all 1000 samples for three randomly selected commenters(indicated by colour) using panel VOLS. Each horizontal axis indicates the true value being estimated, while the verticalaxis indicates the estimate. Points along the 45 degree line are correctly estimated. In the top row, rules are generatedwith no style effects, while in the bottom row the rules are generated with style effects. In the left column the estimatorhas no fixed effects, while in the right column the estimator uses style fixed effects.76better than LSA on this more realistic task.The proposed estimators can also be extended to control for commenter and regulator writing stylewhen panel data is available with authors submitting comments on multiple rules. Intuitively, thepresence of writing style that is separate from the information content of documents is likely to be alarge source of bias when estimating influence since commenters with writing style that is similar to theregulator will appear more influential than they actually are. Existing tools for text similarity analysisdo not have a good way of dealing with the problem. In the last section of the paper I derive a morerealistic model of rule generation that incorporates commenter and regulator writing styles and showthat it motivates adding a particular set of fixed effects to the estimators. I then present Monte Carlotests that use real comments and regulatory text to generate synthetic rules with realistic commenterand regulator writing styles. These tests demonstrate that style biases have a large detrimental impacton the performance of the proposed estimators, but these biases are effectively eliminated by theproposed fixed effects specification. I also show that including fixed effects when there the documentshave no consistent writing style is relatively costless, suggesting that the fixed effects estimators shouldbe the default choice when panel data is available.When thinking about the big picture of how to measure influence from text data, I see two broadpaths. The first is to constrain ourselves to traditional econometrics and statistics, applied to discretetext features such as bag-of-words tokens. This approach sacrifices the realism of how we treat textbecause it ignores much of the nuance and contextual information contained in the sequential structure.It is possible to capture sequential information in the tokens (by using bi-words for example), but longersequences become increasingly rare and unlikely to appear in multiple documents, even when they havesimilar or exactly the same meaning as other related sequences. As a thought experiment, considerthat we could use the entire document's text as a single unique feature - but then every document iscompletely orthogonal and we have no way to think about similarity. On the other hand, we could alsorepresent each document as a count of individual letters, but this would also not be very informative.The optimal token size is somewhere between individual letters and full documents, but there are reallimitations to how effectively discrete features can represent text information. Topic modeling andword embeddings improve text analysis measures by capturing some of the relatedness between wordsor tokens, but this approach still strips away a huge amount of information from the ordering (forexample, who he refers to in a paragraph).The second path forward is to embrace new NLP approaches based on neural networks. This is anexciting frontier, and researchers are regularly improving performance on a variety of benchmark tasks(see Radford et al. (2019); Vaswani et al. (2017); Devlin et al. (2018) for a sample of recent research inthis area). These approaches use very flexible models that can capture much more information aboutthe context of language features, at the cost of computational complexity and interpretability. Thegeneral recipe for applying these models is to train the model to predict omitted tokens on a largecorpus of text. The model learns to construct an internal representation of the text that maximizespredictive accuracy. Then supervised learning is then used to train another classifier to solve the mainproblem (sentiment or topic classification for example) using the model's internal state as input insteadof the raw text. Neural network models are therefore a natural extension of token vectors: the model77generates a vector for each token in the sequence that captures some lower-dimensional representationof the text and assigns similar vectors to similar tokens. The difference is that the vector assigned toeach token is context-specific. This is what makes higher accuracy possible. But it also means that itis much harder to write down a simple model of how a regulator might generate text, and it is alsoharder to see how to model the covariance structure of token vectors. These are open problems forfuture research. In the meantime, more traditional approaches, like those outlined in this chapter,remain useful.78Table 3.1: Summary of EstimatorsEstimator OLS version GLS versionFrequency βˆOLS = arg minβ‖gr − Frβ‖2 βˆFGLS = arg minβ∥∥∥Σ−1/2r (gr − Frβ)∥∥∥2Σr =1nr(diag(pir)− pir · piTr )Vector βˆV OLS = arg minβ‖yr −Xrβ‖2 βˆV FGLS = arg minβ∥∥∥Ω−1/2r (yr −Xrβ)∥∥∥2Ωr = VW−1/2ΣrW−1/2V TPanel βˆPV OLS = arg minβ‖y −Xβ‖2 βˆPV FGLS = arg minβ∥∥Ω−1/2(y −Xβ)∥∥2Ω = blockdiag(Ω1,Ω2, . . . ,ΩR)FE βˆV OLSFE = arg minβ‖y −Xβ −Mµ‖2 βˆV FGLSFE = arg minβ∥∥Ω−1/2(y −Xβ −Mµ)∥∥2Ω = blockdiag(Ω1,Ω2, . . . ,ΩR)79ConclusionThe overarching goal of this dissertation is to advance the study of political influence by providing newdata from the U.S. federal rulemaking process, new tools to analyze this data, and new empirical resultsgenerated by applying the tools to the data. The rulemaking process is an exciting area to explorebecause agencies record a huge amount of detailed interactions with interest groups during the notice-and-comment process. In federal rulemaking we can observe specific organizations communicatingdetailed arguments regarding proposed policies, and we can observe how the government agenciesrespond to these comments, both in terms of how they describe the comments, and how they changethe substance of the rule from the proposed to the final version. All this information is available at a vastscale  millions of comments on thousands of regulations. This data has the potential to transform ourunderstanding of influence, particularly the kind of informational lobbying that is central in advancedeconomies. But it brings challenges too. Linking the documents together into a coherent database isnot trivial. And some of the most interesting aspects of the data take the form of complex text whichrequires new tools and techniques to fully exploit.The role of chapters 1 and 2 are therefore twofold: First, they reveal systematic evidence of apolitical strategy employed by large corporations that was previously known only from a few anecdotalaccounts. It turns out that large U.S. corporations donate millions of dollars to non-profits whoparticipate in federal rulemaking, and analysis of the timing and content of these comments suggestthat firms are able to amplify their messages to regulators with these donations. The patterns weobserve are consistent with only a few stories about how the process works: commenting is correlatedwith donations in the year prior, so we do not see evidence that non-profits are courting donors withtheir comments, instead, firms and non-profits must be coordinating in advance. Similarly, we seethe strongest relationship between commenting and donations in the set of non-profits that commentmost often, which is inconsistent with stories of hush money where firms use donations to keepactive commenters quiet. This new evidence advances our understanding of how firms interact withregulators and the tools they can use to influence policy debates in this domain. This evidence is adirect continuation and expansion of the new interpretation of corporate philanthropy as a tool forpolitical influence first introduced in Bertrand et al. (2018).Second, chapters 1 and 2 are proof of concept for further research in on influence in rulemaking.One of the challenges with exploring new data with new techniques is that it is hard to tell howwell the new techniques are working when we don't know what to expect from the data. Linkingthe rulemaking data to donations provides an external reference point. The result that the timingof comments is strongly correlated with donations in very stringent regressions with many controlsindicates that there is valuable information embedded in the comment submission data. And the factthat these donations are also correlated with our measures of textual similarity suggest that even these80relatively simple algorithms are extracting meaningful information from the content of comments andrules.The success of the empirical approaches in chapters 1 and 2 are particularly remarkable consideringthe limitations of the data. On the donations side, we only have access to donations that occur throughcorporate foundations and not direct donations from firms (or corporate executives and employees).Given the ethically and legally dubious nature of the coordination between firms and non-profits itis somewhat surprising that firms use the more transparent grant process for donating. It may bethe case that we are only observing a small fraction of the politically-motivated donations. On theregulatory side, there are many sources of noise that result from the messy nature of the raw data.Linking organizations to comments and regulatory documents to each other (and therefore commentsto rules via the preceding proposal documents) required automated, approximate matching techniques.Similarly, the measures of text similarity and sentiment provide only very rough approximationsof what a human reader would infer from the comment and rule text. Before running our analysis,it was not obvious that any meaningful signal could be extracted from the data. The results inchapters 1 and 2 demonstrate that there is a path forward. The dataset is large enough that scalecan compensate for precision in each measurement. And the panel structure allows for very stringentregression specifications that can control for many obvious confounders such as patterns within industry,year, or between particular pairs of firms and non-profits.Nevertheless, some difficult issues remain. The first is the interpretation of topic and sentimentsimilarity between comments. Because some comments are very long complicated documents, it ispossible that some comments might take opposing positions on overlapping issues and still appear tohave higher similarity than other comments that talk about unrelated issues. Document-level sentimentscores alone cannot solve this problem since the sentiment needs to be aligned with each topic. Thereare existing text analysis technologies that attempt to solve the problem of jointly estimating topics andsentiment which may be worth exploring in future work (for example see Lin and He (2009); McAuleyand Leskovec (2013); Rahman and Wang (2016); Li et al. (2010)).The second issue is the interpretation of text similarity between comments and the rule discussions.In the rule discussions, regulators must summarize comments regardless of whether they act on them.From my experience reading the rule text, I believe it is likely that regulators spend more time discussingcomments that influenced their thinking than those that did not. And when regulators make changesto the rule, it seems likely that the influential comments will also be more similar to the regulator'splain language description of the final rule. But we still need to quantify these intuitions more precisely(something I am currently working on with my co-authors).Even if the text similarity between comments and rules is correctly measuring the way that regula-tors respond to comments, there is a deeper issue of causal identification. Large firms frequently engagein many channels of influence simultaneously, in the form of coordinated multi-pronged campaigns. Forexample, in the case of the high-profile Clean Power Plan rulemaking, work by Nouri Najjar and I (un-published) found that many large energy companies simultaneously commented, lobbied, and sued topressure the Environmental Protection Agency to delay the rule. Doubtless, these large firms alsocalled and emailed any contacts they had at the agency or White House, attended public and private81meetings with EPA representatives, and asked members of Congress to pressure the agency as well.It is very difficult to disentangle the casual effects of comments from these other correlated activities.On the other hand, when we think about the causal impact of comments specifically we are perhapsfocusing on an odd counterfactual. When firms have multiple channels available to communicate thesame message, the causal effect of blocking one channel might be close to zero. But this does not meanthat the causal impact of the message content is zero. Comments are somewhat special because theyare part of the public record and are given consideration in legal challenges to rules. But in general,I think a better way to view the comments is as a sample of the information passing from interestgroups to government, and the policy preferences of interest groups on very fine-scale issues. Then theimportant questions to answer are how the communication visible in comments is biased relative tothe full set of messages, and whether it is possible to instrument for an organization's general abilityto communicate with government.In future work, my first goal is to improve the measure of influence from comments to rules, andexplore what factors predict influence. Chapter 3 is a building block for this project, as it proposesbetter estimators for the influence of comment text on rule text. In particular, the new estimators makeit possible to control for fixed aspects of writing style. In Chapter 2, we had time-varying donationswhich allowed us to include various controls for the average level of donations and commenting byfirms. But without this additional time-varying input it will be particularly important to distinguishbetween commenters that write in a similar style to the regulator, and commenters who are influentialbecause the regulator integrates content from their comments into the rule. To address how commentsmight be a biased sample of communication, it will also be useful to investigate how interest groupsconduct multi-pronged campaigns. With linked lobbying, campaign finance, and commenting data,we can explore how groups allocate resources across these channels. Court cases would be anotherarea worth exploring, both because the threat of lawsuits provides a source of leverage over regulators,and because courts take input from the public in the form of amicus briefs that parallel regulatorycomments. With improved understanding of the main empirical patterns in these activities, I hopeit will eventually be possible to find good instruments for influence. Promising areas to look includedifferences in the resources of firms due to size and market structure, different levels of past regulationleading to more or less experience interacting with government, and personal ties between corporateexecutives and politicians or government personnel. In the long run, I believe this research path willlead to a more nuanced and complete understanding of how policy is shaped by information providedby interest groups.82BibliographyAaron, D.G., Siegel, M.B., 2017. Sponsorship of national health organizations by two major sodacompanies. American Journal of Preventive Medicine 52, 2030.Acemoglu, D., 2003. Why not a political Coase theorem? Social conflict, commitment, and politics.Journal of Comparative Economics 31, 620652.Anaya, L., 2011. Comparing Latent Dirichlet Allocation and Latent Semantic Analysis as Classifiers.University of North Texas, 2011.Aumann, R.J., Hart, S., 2003. Long cheap talk. Econometrica 71, 16191660.Austen-Smith, D., 1993. Information and influence: Lobbying for agendas and votes. American Journalof Political Science , 799833.Austen-Smith, D., 1995. Campaign contributions and access. American Political Science Review 89,566581.Baron, D.P., 2001. Private politics, corporate social responsibility, and integrated strategy. Journal ofEconomics & Management Strategy 10, 745.Battaglini, M., 2002. Multiple referrals and multidimensional cheap talk. Econometrica 70, 13791401.Baumgartner, F.R., Berry, J.M., Hojnacki, M., Leech, B.L., Kimball, D.C., 2009. Lobbying and policychange: Who wins, who loses, and why. University of Chicago Press.Bertrand, M., Bombardini, M., Fisman, R., Trebbi, F., 2018. Tax-Exempt Lobbying: CorporatePhilanthropy as a Tool for Political Influence. Technical Report. National Bureau of EconomicResearch.Bertrand, M., Bombardini, M., Trebbi, F., 2014. Is it whom you know or what you know? An empiricalassessment of the lobbying process. The American Economic Review 104, 38853920.Bojanowski, P., Grave, E., Joulin, A., Mikolov, T., 2016. Enriching word vectors with subword infor-mation. CoRR abs/1607.04606. URL: http://arxiv.org/abs/1607.04606, arXiv:1607.04606.Bombardini, M., 2008. Firm heterogeneity and lobby participation. Journal of International Economics75, 329348.Bombardini, M., Trebbi, F., 2011. Votes or money? Theory and evidence from the US Congress.Journal of Public Economics 95, 587611.83Bombardini, M., Trebbi, F., 2012. Competition and political organization: Together or alone inlobbying for trade policy? Journal of International Economics 87, 1826.Brown, W.O., Helland, E., Smith, J.K., 2006. Corporate philanthropic practices. Journal of CorporateFinance 12, 855877.Chakraborty, A., Harbaugh, R., 2010. Persuasion by cheap talk. American Economic Review 100,236182.Cohen, J.E., 1986. The Dynamics of the" Revolving Door" on the FCC. American Journal of PoliticalScience , 689708.Crawford, S.P., 2013. Captive audience: The telecom industry and monopoly power in the new gildedage. Yale University Press.Crawford, V.P., Sobel, J., 1982. Strategic information transmission. Econometrica: Journal of theEconometric Society , 14311451.Dal Bó, E., 2006. Regulatory capture: A review. Oxford Review of Economic Policy 22, 203225.De Figueiredo, J.M., Richter, B.K., 2014. Advancing the empirical research on lobbying. Annual reviewof political science 17, 163185.Deerwester, S., Dumais, S., Furnas, G., Landauer, T., Harshman, R., 1990. Indexing by latent semanticanalysis. Journal of the American Society for Information Science 41, 391407.Devlin, J., Chang, M.W., Lee, K., Toutanova, K., 2018. Bert: Pre-training of deep bidirectionaltransformers for language understanding. arXiv preprint arXiv:1810.04805 .Dewatripont, M., Tirole, J., 1999. Advocates. Journal of political economy 107, 139.Dewatripont, M., Tirole, J., 2005. Modes of communication. Journal of Political Economy 113, 12171238.Drutman, L., 2015. The business of America is lobbying: How corporations became politicized andpolitics became more corporate. Oxford University Press.Dumais, S., Furnas, G., Landauer, T., Deerwester, S., 1988. Using latent semantic analysis to improveinformation retrieval. Proceedings of CHI 88 Conference on Human Factors in Computing Systems, 281285.Dumais, S.T., 2004. Latent semantic analysis. Annual review of information science and technology38, 188230.Frisch, R., Waugh, F.V., 1933. Partial time regressions as compared with individual trends. Econo-metrica: Journal of the Econometric Society , 387401.84Gentzkow, M., Shapiro, J.M., Taddy, M., 2016. Measuring polarization in high-dimensional data:Method and application to congressional speech. Technical Report. National Bureau of EconomicResearch.Goldberg, P.K., Maggi, G., 1999. Protection for sale: An empirical investigation. American EconomicReview 89, 11351155.Golden, M.M., 1998. Interest groups in the rule-making process: Who participates? Whose voices getheard? Journal of Public Administration Research and Theory 8, 245270.Grossman, G.M., Helpman, E., 1994. Protection for sale. The American Economic Review 84, 833850.Grossman, G.M., Helpman, E., 2001. Special interest politics. MIT press.Haeder, S.F., Yackee, S.W., 2015. Influence and the administrative process: Lobbying the US Presi-dent's Office of Management and Budget. American Political Science Review 109, 507522.Huneeus, F., Kim, I.S., 2018. The effects of firms' lobbying on resource misallocation .Jacobson, M.F., 2005. Lifting the veil of secrecy from industry funding of nonprofit health organizations.International Journal of Occupational and Environmental Health 11, 349355.Joulin, A., Grave, E., Bojanowski, P., Mikolov, T., 2016. Bag of tricks for efficient text classification.CoRR abs/1607.01759. URL: http://arxiv.org/abs/1607.01759, arXiv:1607.01759.Kang, K., 2016. Policy influence and private returns from lobbying in the energy sector. Review ofEconomic Studies 83, 269305.Kang, K., You, H.Y., 2016. The value of connections in lobbying. working paper .Kendall, C., Nannicini, T., Trebbi, F., 2015. How do voters respond to information? Evidence from arandomized campaign. American Economic Review 105, 32253.Klüver, H., 2009. Measuring interest group influence using quantitative text analysis. European UnionPolitics 10, 535549.Krishna, V., Morgan, J., 2001. A model of expertise. The Quarterly Journal of Economics 116, 747775.Laffont, J.J., Tirole, J., 1993. A theory of incentives in procurement and regulation. MIT press.Lando, H.A., 1991. Toward a comprehensive strategy for reducing the health burden of tobacco. BritishJournal of Addiction 86, 649652.Li, F., Huang, M., Zhu, X., 2010. Sentiment analysis with global topics and local dependency, in:Twenty-Fourth AAAI Conference on Artificial Intelligence.Libgober, B., 2018. Meetings, Comments, and the Distributive Politics of Rulemaking: Evidence fromDodd-Frank. working paper .85Libgober, B., Carpenter, D., 2018a. Lobbying with Lawyers: Financial Market Evidence for Banks'Influence on Rulemaking. working paper .Libgober, B., Carpenter, D., 2018b. What's at Stake in Rulemaking? Financial Market Evidence forBanks' Influence on Administrative Agencies. working paper .Libgober, B., Rashin, S., 2018. What public comments during rulemaking do and why. working paper.Lin, C., He, Y., 2009. Joint sentiment/topic model for sentiment analysis, in: Proceedings of the 18thACM conference on Information and knowledge management, ACM. pp. 375384.Lohmann, S., 1995. Information, access, and contributions: A signaling model of lobbying. PublicChoice 85, 267284.Lorenz, G.M., 2017. Prioritized interests: Diverse lobbying coalitions and congressional committeeagenda-setting. working paper .Lucca, D., Seru, A., Trebbi, F., 2014. The revolving door and worker flows in banking regulation.Journal of Monetary Economics 65, 1732.Lucca, D.O., Trebbi, F., 2009. Measuring central bank communication: an automated approach withapplication to FOMC statements. Technical Report. National Bureau of Economic Research.Makkai, T., Braithwaite, J., 1992. In and out of the revolving door: Making sense of regulatory capture.Journal of Public Policy 12, 6178.Mayer, J., 2017. Dark money: The hidden history of the billionaires behind the rise of the radicalright. Anchor Books.McAuley, J., Leskovec, J., 2013. Hidden factors and hidden topics: Understanding rating dimensionswith review text, in: Proceedings of the 7th ACM conference on Recommender systems, ACM. pp.165172.McKay, A., Yackee, S.W., 2007. Interest group competition on federal agency rules. American PoliticsResearch 35, 336357.Mikolov, T., Chen, K., Corrado, G., Dean, J., 2013. Efficient estimation of word representations invector space. arXiv preprint arXiv:1301.3781 .Naughton, K., Schmid, C., Yackee, S.W., Zhan, X., 2009. Understanding commenter influence duringagency rule development. Journal of Policy Analysis and Management 28, 258277.Oreskes, N., Conway, E.M., 2011. Merchants of doubt: How a handful of scientists obscured the truthon issues from tobacco smoke to global warming. Bloomsbury Publishing USA.Ottaviani, M., Sørensen, P.N., 2006. Reputational cheap talk. The Rand journal of economics 37,155175.86Peng, V., 2016. Astroturf campaigns: Transparency in telecom merger review. University of MichiganJournal of Law Reform 49.Pennington, J., Socher, R., Manning, C.D., 2014. Glove: Global vectors for word representation,in: Empirical Methods in Natural Language Processing (EMNLP), pp. 15321543. URL: http://www.aclweb.org/anthology/D14-1162.Persson, T., Tabellini, G., 2000. Political Economy: Explaining Economic Policy. MIT press Cam-bridge, MA.Pincombe, B., 2004. Comparison of human and latent semantic analysis (LSA) judgements of pair-wise document similarities for a news corpus. Technical Report. Defence Science and TechnologyOrganization Salisbury (Australia).Potters, J., Van Winden, F., 1992. Lobbying and asymmetric information. Public choice 74, 269292.Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., 2019. Language models areunsupervised multitask learners. OpenAI Blog 1, 8.Rahman, M.M., Wang, H., 2016. Hidden topic sentiment model, in: Proceedings of the 25th In-ternational Conference on World Wide Web, International World Wide Web Conferences SteeringCommittee. pp. 155165.Rodrik, D., 1995. Political economy of trade policy. Handbook of international economics 3, 14571494.Ross, L.F., Norton, J., Young, S., et al., 2000. Is academic medicine for sale. N Engl J Med 343,50810.Shapiro, S.A., 2012. The complexity of regulatory capture: Diagnosis, causality, and remediation.Roger Williams UL Rev. 17, 221.Stigler, G.J., 1971. The theory of economic regulation. The Bell journal of economics and managementscience , 321.Sunstein, C.R., 2012. The Office of Information and Regulatory Affairs: Myths and realities. Harv. L.Rev. 126, 1838.Takahashi, S., Ambrus, A., 2008. Multi-sender cheap talk with restricted state spaces. TheoreticalEconomics .Tetlock, P.C., 2007. Giving content to investor sentiment: The role of media in the stock market. TheJournal of finance 62, 11391168.Tetlock, P.C., Saar-Tsechansky, M., Macskassy, S., 2008. More than words: Quantifying language tomeasure firms' fundamentals. The Journal of Finance 63, 14371467.87Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin,I., 2017. Attention is all you need, in: Advances in neural information processing systems, pp.59986008.Vidal, J.B.I., Draca, M., Fons-Rosen, C., 2012. Revolving door lobbyists. The American EconomicReview 102, 37313748.Wagner, W., Barnes, K., Peters, L., 2011. Rulemaking in the shade: An empirical study of EPA's airtoxic emission standards. Administrative Law Review , 99158.Yackee, J.W., Yackee, S.W., 2006. A bias towards business? Assessing interest group influence on theUS bureaucracy. The Journal of Politics 68, 128139.Yackee, S.W., 2005. Sweet-talking the fourth branch: The influence of interest group comments onfederal agency rulemaking. Journal of Public Administration Research and Theory 16, 103124.Zingales, L., 2012. A capitalism for the people: Recapturing the lost genius of American prosperity.Basic books.88Appendix ARegulation commentsA.1 OverviewOur data on regulatory comments comes from regulations.gov. Under the Administrative ProceduresAct (APA), federal agencies must provide a means for the public to submit comments on proposedrules and other regulatory changes. Regulations.gov is a shared platform that is now used by mostfederal agencies to facilitate submission and public review of comments. Information about submittedcomments, including the original text and attachments, can be viewed through a web browser. The sitealso provides an API that allows more efficient data access, particularly for collecting simple commentmetadata such as the title of the comment and posted date.Our sample starts with the the complete collection of metadata for all comments posted to regu-lations.gov in the years 2003-2017 (inclusive). This is a total of 6,871,697 unique documents. Fromthese, we identify 981,232 comments that appear to be authored by organizations rather than privateindividuals (org comments). We download the complete text for all org comments using common fileformats, giving us about 90% of comment text for the org comment sample.A.2 Collecting metadataThe regulations.gov API provides a search function for document metadata. We retrieved the metadatafor all public submission documents posted since the site came online in 2003, and include all yearsup to and including 2017. Some agencies have begun digitizing older comments and posting themto regulations.gov retroactively. But an EPA spokesperson stated (in personal email correspondence)that this work is currently incomplete, and that the text of some older comments will never be releaseddigitally since the submitters were not aware of this possibility at the time. Thus we consider data onpre-2003 comments on regulations.gov unreliable and do not include them.A.3 Identifying org commentsAuthorship information can appear in three different metadata fields: title, organization, or sub-mitterName. Comments appear to fall into two main types: those that contain organization and/orsubmitterName information, and those that only contain authorship information in the title. First,we drop all comments that have submitterName information, but no organization. These appear tobe written by private individuals. For the remaining comments, we look for an organization name ineither the organization field or the title (if the organization field is blank). We use a custom neural89network-based classifier to extract organization names from the selected field (classification is neces-sary for the organization field because it contains many false positives such as self or none). Theclassifier converts each title string to ASCII characters and predicts whether each character is part ofan organization string. Contiguous chunks of characters with predicted probability greater than 0.5are counted as organization names. The classifier is multi-layer bi-directional Gated Recurrent Unit(GRU), implemented in PyTorch39. Code is available on the Brad Hackinen's github page40. Theclassifier is trained on almost 9000 manually constructed training examples. This training set wasconstructed iteratively by starting with easy to parse titles, fitting the neural network, estimating theclassifier's uncertainty from the total entropy of the character-level predicted probabilities, reviewinga sample of high-entropy titles, adding them to the training set, and repeating until the error rate wasacceptably low. We also manually classified an additional set of 1000 random titles as a test set. Theresults of the test are shown below. 93% of titles are classified without error. 83% of titles with anorganization are extracted exactly correctly, while 98.5% of titles with no org are extracted correctly(in other words, the classifier avoids 98.5% of false positives).Table A.1: Organization name extraction accuracySample Count Character Accuracy String AccuracyAll test titles 1000 0.970 0.928Test titles containing org 371 0.935 0.830Test titles with no org 629 0.991 0.985Notes:Character accuracy is the average fraction of characters classifier correctlyin each title. String accuracy is the fraction of titles with every character correctlyclassifiedA.4 Collecting comment textComments on regulations.gov can have comment text in two locations: a text field in the commentmetadata, or in one or more attachments. The text field contains text that submitters have enteredon a web form. It is often as brief as see attached. Most substantial text is contained in thecomment attachments where submitters can upload PDFs, word documents, other other file formats.We download all attachments of the following formats: PDF, MS Word 8, MS Word 12, and simple.txt files. The majority of attachments are in PDF format.We use the XpdfReader pdftotext41command-line utility to extract text from most PDFs. SomePDFs contain only images of each page. In this case we must fall back on Optical Character Recog-nition (OCR), which we implement with a combination of GhostScript42(to render page images) andTesseract-OCR43. We use Apache Tika44to extract text from MS Word formats, and the chardet45Python package to detect formatting of simple text files. All the tools are open source.39https://pytorch.org/40https://github.com/bradhackinen/subex41https://www.xpdfreader.com/pdftotext-man.html42https://www.ghostscript.com/43https://github.com/tesseract-ocr44http://tika.apache.org/45https://pypi.org/project/chardet/90Appendix BAdditional tables and figuresTable A.1: Top Agencies by Number of CommentsTop 30 agencies Number of Top 30 agencies Number ofin firms comments comments in grantees comments commentsEPA 8099 FWS 76404FAA 3870 NOAA 69171FDA 1942 HHS 60969OSHA 1245 CMS 47215PHMSA 745 EPA 13556NHTSA 724 ED 5105CMS 721 FDA 4773EERE 709 FAA 3485DOT 541 FNS 2821OCC 466 FSIS 2436FMCSA 451 APHIS 2232IRS 444 HUD 1910NLRB 366 IRS 1733USTR 336 CFPB 1361CFPB 328 AMS 1310EBSA 302 OSHA 1192HHS 276 FHWA 1095USCG 222 SSA 1064FWS 208 NHTSA 1001AMS 181 EERE 936HUD 163 DOT 925APHIS 152 BOEM 909FSIS 144 ICEB 861TSA 129 DOJ 824FRA 109 USCG 750FHWA 108 OMB 748LMSO 102 FMCSA 708BOEM 95 DOS 667BIS 94 OPM 649EIB 91 NLRB 616Notes: This table reports the 30 top agencies as ranked by the number ofcomments they receive by firms (first two columns) or by grantees (last twocolumns).91Table A.2: Co-commenting in time-varying sample - Contemporaneous and lagged donationsDependent variable Firm f and grantee g commented on the same regulation in year t× 100Mean 0.163(1) (2) (3) (4) (5)Firm f contributed to 0.746*** 0.614*** 0.587*** 0.451*** -0.010grantee g in year t (0.040) (0.041) (0.041) (0.096) (0.042)Firm f contributed to 0.964*** 0.819*** 0.798*** 0.649*** 0.188***grantee g in year t− 1 (0.042) (0.044) (0.044) (0.111) (0.045)Fixed effectsYear Y Y Y Y YGrantee Y YDonor Y YGrantee-Firm Pair YSE Clusters Grantee Firm Grantee Firm×Grantee+Firm PairObservations 125,918,520 125,918,520 125,918,520 125,918,520 125,860,865Note: The dependent variable is equal to 100 if grantee g and firm f comment on the same regulationin year t. The independent variable is equal to one if grantee g received a donation from firm f either atyear t (respectively, t − 1). Standard errors are clustered at the level indicated in each column under SEClusters.*** p<0.01, ** p<0.05, * p<0.192Table A.3: Co-commenting in time-varying sample - Future donationsDependent variable Firm f and grantee g commented on the same regulation in year t× 100Mean 0.163(1) (2) (3) (4) (5)Firm f contributed to 0.557*** 0.452*** 0.447*** 0.339*** -0.016grantee g in year t+ 1 (0.038) (0.049) (0.081) (0.087) (0.042)Firm f contributed to 0.866*** 0.715*** 0.699*** 0.543*** 0.142***grantee g in year t or t− 1 (0.032) (0.051) (0.098) (0.104) (0.040)Fixed effectsYear Y Y Y Y YGrantee Y YDonor Y YGrantee-Firm Pair YSE Clusters Grantee Firm Grantee Firm×Grantee+Firm PairObservations 125,918,520 125,918,520 125,918,520 125,918,520 125,860,865Notes: Standard errors are clustered at the level indicated in each column under SE Clusters. *** p<0.01, **p<0.05, * p<0.193Table A.4: Similarity - Contemporaneous and lagged donationsDependent variable Similarity of comments by grantee g and firm f on same regulation(1) (2) (3) (4)Grantee g received donation 0.046 0.052 -0.036 0.012from firm f at t (0.046) (0.034) (0.035) (0.017)Grantee g received donation 0.169*** 0.141*** 0.085** 0.079***from firm f at t− 1 (0.054) (0.035) (0.039) (0.027)Fixed EffectsDocket Y YFirm Y YGrantee Y YSE Clusters Docket Docket Firm+Grantee Firm+Grantee+DocketObservations 301,602 301,602 300,817 300,792Notes: The dependent variable is a similarity index between the comment of firm f and the comment ofgrantee g on regulation r, divided by the standard deviation of similarity of all comments relative to r. Theindependent variable is equal to one if grantee g received a donation from firm f in the year when the commentappears (respectively, the year before). Standard errors are clustered at the level indicated in each columnunder SE Clusters. *** p<0.01, ** p<0.05, * p<0.194Table A.5: Similarity - Sector ControlDependent variable Similarity of comments by grantee g and firm fon same regulation(1) (2) (3) (4)Grantee g received donation 0.074*** 0.058***from firm f at t or t− 1 (0.026) (0.021)Grantee g received 0.020 0.010donation from firm f at t (0.023) (0.021)Grantee g received 0.072** 0.067**donation from firm f at t− 1 (0.030) (0.030)Fixed EffectsDocket Y Y Y YFirm Y Y Y YGrantee Y Y Y YNAICS code × NTEEC code Y YSE ClustersFirm Y Y Y YGrantee Y Y Y YDocket Y Y Y YNAICS code × NTEEC code Y YObservations 162,735 162,735 162,735 162,735Notes: The dependent variable is a similarity index between the comment of firm f andthe comment of grantee g on regulation r, divided by the standard deviation of similarity ofall comments relative to r. The independent variable is equal to one if grantee g received adonation from firm f in the year when the comment appears or the year before. Standarderrors are clustered at the level indicated in each column under SE Clusters. *** p<0.01,** p<0.05, * p<0.195Table A.6: Similarity - Future DonationDependent variable Similarity of comments by grantee g and firm f on same regulation(1) (2) (3) (4) (5) (6)Grantee g received donation -0.010 0.039 0.030 0.027 0.010 -0.030from firm f at t+ 1 (0.050) (0.030) (0.058) (0.036) (0.031) (0.036)Grantee g received donation 0.069***from firm f at t or t− 1 (0.025)Fixed EffectsFirm Y Y Y Y Y YGrantee Y Y Y Y Y YDocket Y Y Y YNAICS code × NTEEC code Y YSE ClusteringFirm Y Y Y Y Y YGrantee Y Y Y Y Y YDocket Y Y Y YNAICS code × NTEEC code Y YSample with sector codes Y Y Y YObservations 300,817 300,792 175,660 175,643 162,735 162,735Notes: Standard errors are clustered at the level indicated in each column under SE Clusters. *** p<0.01,** p<0.05, * p<0.196Table A.7: Rule-comment similarity - RobustnessDependent variable Similarity of rule discussion and commentby firm f on same regulation r(1) (2) (3) (4)At least one grantee g 0.118** 0.186*** 0.131**co-commenting and receiving (0.058) (0.066) (0.061)donation from firm fin year t or t− 1Log number of pages of comments 0.405*** 0.406*** 0.405***submitted by firm f (0.024) (0.024) (0.024)At least one grantee g 0.027 -0.028 -0.027co-commenting and receiving (0.055) (0.059) (0.058)donation from firm fin any yearFixed EffectsDocket Y Y Y YFirm Y Y Y YSE Clustering Firm+Docket Firm+Docket Firm+Docket Firm+DocketObservations 4,385 4,385 4,965 4,385Notes: The dependent variable is a similarity index between the comment of firm f and the discussion ofregulation r, divided by the standard deviation of similarity of all comments relative to r and discussion ofregulation r. The independent variables are the same as in tables 2.5 and 2.6. Standard errors are clustered atthe level indicated in each column under SE Clusters. *** p<0.01, ** p<0.05, * p<0.197Table A.8: Citations of Firm in Rule DiscussionDependent variable Citation of firm f 's name inDiscussion of rule rCited(Y/N) Log (1+Citations) Cited (Y/N) Log(1+Citations)(1) (2) (3) (4)At least one grantee g 0.017 0.043* 0.021 0.059**co-commenting and receiving (0.013) (0.024) (0.014) (0.028)donation from firm fin year t or t− 1At least one grantee g -0.005 -0.032co-commenting and receiving (0.011) (0.022)donation from firm fin any yearLog number of 0.028*** 0.049***pages of comments (0.007) (0.013)submitted by firm fFixed EffectsDocket Y Y Y YFirm Y Y Y YSE Clusters Firm+Docket Firm+Docket Firm+Docket Firm+DocketObservations 4,965 4,965 4,385 4,385Note: Standard errors are clustered at the level indicated in each column under SE Clusters. *** p<0.01, **p<0.05, * p<0.198Table A.9: List of Agencies on regulations.gov (A-F)ACF Children and Families Administration DOI Interior DepartmentAHRQ Agency for Healthcare Research and Quality DOJ Justice DepartmentAID Agency for International Development DOL Employment Standards AdministrationAMS Agricultural Marketing Service DOS State DepartmentAOA Aging Administration DOT Transportation DepartmentAPHIS Animal and Plant Health Inspection Service EAB Economic Analysis BureauARS Agricultural Research Service EAC Election Assistance CommissionASC Appraisal Subcommittee EBSA Employee Benefits Security AdministrationATBCB Archit. and Transportation Barriers Compliance Board ED Education DepartmentATF Alcohol, Tobacco, Firearms, and Explosives Bureau EDA Economic Development AdministrationATSDR Agency for Toxic Substances and Disease Registry EEOC Equal Employment Opportunity CommissionBIA Indian Affairs Bureau EERE Off. Energy Efficiency and Renewable EnergyBIS Industry and Security Bureau EIB Import Export Bank of the United StatesBLM Land Management Bureau EOIR Executive Office for Immigration ReviewBOEM Ocean Energy Management Bureau EPA Environmental Protection AgencyBOP Prisons Bureau ESA Employment Standards AdministrationBOR Reclamation Bureau ETA Employment and Training AdministrationBPD Public Debt Bureau FAA Federal Aviation AdministrationBSEE Safety and Environmental Enforcement Bureau FAR Federal Acquisition Regulation SystemCCC Commodity Credit Corporation FBI Federal Bureau of InvestigationCDC Centers for Disease Control and Prevention FCIC Federal Crop Insurance CorporationCDFI Community Development Financial Institutions Fund FDA Food and Drug AdministrationCFPB Consumer Financial Protection Bureau FEMA Federal Emergency Management AgencyCMS Centers for Medicare Medicaid Services FFIEC Federal Financial Institutions Exam. CouncilCNCS Corporation for National and Security Service FHWA Federal Highway AdministrationCOE Engineers Corps FINCEN Financial Crimes Enforcement NetworkCOLC U.S. Copyright Office, Library of Congress FISCAL Bureau of the Fiscal ServiceCPSC Consumer Product Safety Commission FMCSA Federal Motor Carrier Safety AdministrationCSREES Coop. State Research, Education, and Extension Service FNS Food and Nutrition ServiceDARS Defense Acquisition Regulations System FRA Federal Railroad AdministrationDEA Drug Enforcement Administration FS Fiscal ServiceDHS Homeland Security Department FSA Farm Service AgencyDOC Commerce Department FSIS Food Safety and Inspection ServiceDOD Defense Department FSOC Financial Stability Oversight CouncilDOE Energy Department FTA Federal Transit Administration99Table A.10: List of Agencies on regulations.gov (F-Z)FTC Federal Trade Commission OJP Justice Programs OfficeFWS Fish and Wildlife Service OMB Management and Budget OfficeGIPSA Grain Inspection, Packers and Stockyards Adm. ONRR Natural Resources Revenue OfficeGSA General Services Administration OPM Personnel Management OfficeHHS Health and Human Services Department OPPM Procurement and Property Management, Office ofHHSIG Inspector General, Health and Human Serv Dept OSHA Occupational Safety and Health AdministrationHRSA Health Resources and Services Administration OSM Surface Mining Reclamation and Enforcement OfficeHUD Housing and Urban Development Department OTS Thrift Supervision OfficeICEB Immigration and Customs Enforcement Bureau PBGC Pension Benefit Guaranty CorporationIHS Indian Health Service PCLOB Privacy and Civil Liberties Oversight BoardIRS Internal Revenue Service PHMSA Pipeline and Hazardous Materials Safety Adm.ITA International Trade Administration PTO Patent and Trademark OfficeLMSO Labor-Management Standards Office RBS Rural Business-Cooperative ServiceMARAD Maritime Administration RHS Rural Housing ServiceMMS Minerals Management Service RITA Research and Innovative Technology AdministrationMSHA Mine Safety and Health Administration RUS Rural Utilities ServiceNHTSA National Highway Traffic Safety Administration SAMHSA Substance Abuse and Mental Health Services Adm.NIFA National Institute of Food and Agriculture SBA Small Business AdministrationNIGC National Indian Gaming Commission SLSDC Saint Lawrence Seaway Development CorporationNIH National Institutes of Health SSA Social Security AdministrationNIST National Institute of Standards and Technology TREAS Treasury DepartmentNLRB National Labor Relations Board TSA Transportation Security AdministrationNOAA National Oceanic and Atmospheric Administration TTB Alcohol and Tobacco Tax and Trade BureauNPS National Park Service USC United States CourtsNRC Nuclear Regulatory Commission USCBP U.S. Customs and Border ProtectionNRCS Natural Resources Conservation Service USCG Coast GuardNSF National Science Foundation USCIS U.S. Citizenship and Immigration ServicesNTIA National Telecommunications and Information Adm. USDA Agriculture DepartmentNTSB National Transportation Safety Board USPC Parole CommissionOCC Comptroller of the Currency USTR Trade Representative, Office of United StatesOFAC Foreign Assets Control Office VA Veterans Affairs DepartmentOFCCP Federal Contract Compliance Programs Office VETS Veterans Employment and Training ServiceOFPP Federal Procurement Policy Office WCPO Workers Compensation Programs OfficeOJJDP Juvenile Justice and Delinquency Prevention Office WHD Wage and Hour Division100
Thesis/Dissertation
eng
Economics
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Graduate
Essays on strategic advocacy and influence in rulemaking
Text