Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Assessing performance of animal welfare organizations to improve philanthropic decision-making McConnachie, Leanne Katherine 2007

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata


831-ubc_2007-0186a.pdf [ 7.58MB ]
JSON: 831-1.0093073.json
JSON-LD: 831-1.0093073-ld.json
RDF/XML (Pretty): 831-1.0093073-rdf.xml
RDF/JSON: 831-1.0093073-rdf.json
Turtle: 831-1.0093073-turtle.txt
N-Triples: 831-1.0093073-rdf-ntriples.txt
Original Record: 831-1.0093073-source.json
Full Text

Full Text

ASSESSING PERFORMANCE OF ANIMAL WELFARE ORGANIZATIONS TO IMPROVE PHILANTHROPIC DECISION-MAKING by Leanne Katherine McConnachie B.A., The University of Victoria, 1988 A thesis submitted in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE in THE FACULTY OF GRADUATE STUDIES (Animal Science) The University of British Columbia April 2007 © Leanne Katherine McConnachie, 2007 Abstract In 2005-06, 24 Canadian and American animal welfare organizations (AWOs) and foundations participated in an in-depth qualitative study to establish which performance criteria best determine the effectiveness of AWOs. Participants' comments resulted in the development of a framework that offers individual donors and foundations a consistent approach to evaluating organizational performance. Termed the PREP Framework, this approach categorizes performance into four key pillars, each with its own components and criteria: 1. Philosophy - the organization's beliefs and values, as reflected in its mandate, position statements, policies and practices; 2. Red Flags - indicators of problems in the organization, such as excessive personnel turnover, disproportionate management salaries, high administration and fundraising costs, lack of peer collaboration, poor financial statements; 3. Efficiencies - financial and operational conduct of the organization, including long term strategic planning, financial credibility and sustainability; and 4. People - abilities of those involved with the organization, including management, volunteers, peers, donors and board members. The study also explored the pros and cons of standardizing performance evaluation and found that many participants felt the sector would benefit from the use of more quantitative and qualitative benchmarks and standards to establish sector norms and trends. In addition, the research examined the sector's familiarity with outcome measurement and found most participants struggled to clearly define outcomes for the sector. This may suggest the sector would also gain from consultation on how to establish objectives within an outcome evaluation framework. Finally, the study investigated how funding strategies of foundations often are at odds with AWO needs and priorities. Participants identified conflicts concerning funding of: 1. short-term vs. long-term projects; 2. innovative vs. proven projects; 3. many, small grants vs. fewer, large grants; 4. new vs. established organizations; 5. emotional appeal vs. practical needs; and 6. no-kill vs. open-admission shelters. Participants provided ideas on where funding could be applied to make the most impact and achieve mutual goals for both AWOs and foundations. The findings and recommendations of this study may enable AWOs to better identify and manage their objectives while enabling donors and foundations to better assess AWO performance and outcomes to improve philanthropic decision-making. Table of Contents Abstract ii Table of Contents iii Acknowledgments v Chapter 1 - Introduction 1 Study Rat iona le 1 Trends in Philanthropy 2 Philanthropy in the Animal Welfare Sector 3 Maximizing Returns on Social Investments 4 T h e Dono r Popula t ion and Thei r N e e d s 4 Individual Donors 5 Foundations 7 T r e n d s in Non-Prof i t Per fo rmance Evaluat ion 7 Determining Effectiveness 8 Outcome Measurement in the Non-Profit Sector 10 Performance Evaluation in the Animal Welfare Sector 11 Literature Rev iew 12 R e s e a r c h G o a l s 18 Chapter 2 - Methods 21 Qualitative, Grounded Theory Approach 21 Participant Recruitment and Selection 21 Data Collection 24 Research Content 25 Data Analysis 27 Study Limitations 27 Chapter 3 - Results 29 3.1 Pe r fo rmance Indicators 2 9 P - Philosophy 32 R - R e d Flags 42 E - Efficiency 45 P - People 56 Key PREP Framework Criteria 71 3.2 O u t c o m e s 7 2 3.3 Compara t i ve Evaluat ion 78 Suggested Benchmarks 81 Accreditation Systems and Programs 87 3.4 Fund ing Conf l ic ts 89 i i i Chapter 4 - Discussion 100 4.1 Per fo rmance Indicators and the P R E P Framework 100 4.2 O u t c o m e s and Ou tcome Measuremen t 106 4 .3 Compara t i ve Eva luat ion - B e n c h m a r k s and S tandards 107 4.4 Fund ing Conf l ic ts 112 Chapter 5 - Conclusion and Recommendations 118 References 122 Appendices 127 Append i x I - V 1 : A W O Quest ionnai re (February 21 , 2005) 127 Append i x II - V 1 : Foundat ion Quest ionnai re (Nov 9, 2004) 1 130 Append i x III - V 2 : A W O Survey-Quest ionna i re (June 9, 2005) 132 Append i x IV - V 2 : Foundat ion Survey-Quest ionna i re (June 21 , 2005) 137 Append i x V - S a m p l e Interview Ques t ions to A W O s and Funders 142 Append i x VI - U B C Behav iora l Eth ics Rev iew Board Cert i f icates of Approva l 147 i v Acknowledgments First of all, I would like to gratefully acknowledge and thank those individuals who took the time to participate in my study. Their experience, insight and input were invaluable. In addition to making this thesis possible, their expertise provided me with the best "crash course" anyone wanting to pursue a career in the animal welfare sector could ever ask for. Secondly it is with incredible gratitude that I thank Dr. David Fraser, my supervisor, for having faith in my project and accepting me - a mature student with a business background, as he so often said - in the UBC Animal Welfare Program. Although my research topic was not the norm for the Program, David made me feel an integral part of the group. He put such careful thought into every one of our discussions. His reflective pauses always foreshadowed an articulate and succinct interpretation of what I was trying to say, and his fine wizardry and mastery in grammar, punctuation and phrasing (even Latin!), never ceased to amaze me. Thank you David. I also want to thank Dr. Dan Weary, whose enthusiasm, boundless energy and fine intellect made learning so much fun. I never anticipated graduating with so much knowledge about dairy cows, pig vocalizations and, his best talent, how to effectively delegate! I would also like to thank Dr. David Tindall for so graciously stepping in and joining my committee after a former member, Dr. Jim Vercammen, left on sabbatical. Thanks Jim for being there in the beginning! Dr. Susan Cox was instrumental in the early stages as well in forming my qualitative techniques and approach. She too graciously stepped in and took on the role of co-supervisor when David Fraser was on sabbatical with the United Nations in Rome. Dr. Alejandro Rojas was also supportive in fine-tuning some of my qualitative strategies when changes were required. I would also like to acknowledge those who gave me great moral support during my degree. Dr. Nina von Keyserlingk was always there to lend an ear and to encourage my progress. Dean Murray Isman was very supportive in balancing my work-study schedule during the final 7 months of writing my thesis. I also want to thank Aditi Gupta whose expertise in library science was of tremendous assistance in correctly formatting my reference table when I was pressed for time in the final few weeks. I also owe thanks to the other students and guest visitors in the Program, v from whom I learned so much about science, animal biology and welfare issues outside my particular area of interest. In terms of financial support, I want to acknowledge and thank Lydia Antoncic, Brad Goldberg and David Wolfson of Animal Welfare Trust for their foundation's contribution to my research costs, and for their patience in waiting for the final product. On the home front, I am deeply indebted to my husband, Rob, for standing by my decision to return to school and allowing me to pursue my passion. Not only did he take care of the bills, but his assistance these past 4 months with grocery shopping, cooking and walking our dog Magoo freed up valuable time to enable me to dedicate my time to writing my thesis, or sometimes, just resting. Without you Rob, this road would have been much bumpier - I thank you. I also want to thank my family and friends who constantly kept me on track with the question "Are you not done yet?" The embarrassment and guilt was more than enough to spur me on! And of course I thank and dedicate this thesis to my constant canine companion, Magoo, who was a major contributor to making this mid4ife career change possible. He inspired me to make the shift, and accompanied me for many lonely days while I was cloistered away in my office. He always knew when to "muzzle4Dunt" my elbow to remind me when I had been typing for too long and it was time for a "walky" break. He kept me healthy and fit when it would have been far too easy to just stay sitting in my chair. Thank you to everyone who was part of this personally rewarding journey. vi Chapter 1 - Introduction "Value for one's money is as important in philanthropy as it is in investing" David S. Zeidberg, Huntington Library Study Rationale I have always had a deep empathy with animals and concern for their welfare and protection. In an effort to support animal protection and conservation organizations, I became a donor in the early 1990's. Since the organizations to which I gave my charitable contributions were well known and well established, I trusted that they had good management teams that were achieving their published mandates of rescuing animals and conserving wildlife. Unfortunately, I soon became very disillusioned. Two of the organizations to which I had donated turned out not to have the animals' best interests at heart. Although their mandates to conserve wildlife and habitat sounded noble, I discovered their purpose for doing so was to ensure wild populations were sustainable enough to hunt. While for some individuals this may be a satisfactory goal, it did not meet with my personal definition of animal welfare and my ideas about what a wildlife conservation group should do. Equally, a companion animal organization that I had supported had been seriously mismanaged over the years. Their resulting deficit and poor management choices led them to make decisions about the lives of animals in their care that, in my opinion, were not in the best interests of the animals. Having lost confidence in my ability to select an animal welfare organization worthy of my charitable donations, I ceased contributing altogether. In the late 1990's I began thinking about how I could assess the abilities of animal welfare organizations (AWOs) in order to resume my donations. By AWOs, I am referring to any organization whose mandate is to improve the lives and well-being of animals and/or to protect animals from physical and emotional suffering caused directly and indirectly by humans. I thought about how stock markets assist individuals with making informed investment decisions, and that perhaps there was a way to apply similar valuation principles to non-profit groups. I felt that ill-informed donors like me needed a publicly accessible, easy-to-use service or reference tool that 1 had already analyzed and rated the various organizations on our behalf. At that time, web-based rating services were either not yet available or just in their early stages of development. Over the next few years, development and applicability of web-based charity ranking sites grew. Evidently, donors were increasingly demanding this type of information. Two of the more notable sites, Guidestarand Charity Navigator, were established in 1996 and 2001 respectively. However, they made few references to AWOs and those AWO profiles they did publish focused on high level mission statements and financial data as submitted for taxation purposes. This content did not appear to capture the true impact an organization had on the lives of animals - i.e. the number they had saved, the habitat they had protected, the laws they had amended, etc. Still feeling unable to make an informed decision, and not finding this information elsewhere, I decided I would try to develop a method, system or tool to evaluate AWOs that would address these questions. Whereas the stock market evolved to answer the question "How much is it worth?" (Johnson and Pazderka 1995, p.x), my study evolved to answer the question "Is it worthy?" Trends in Philanthropy My idea of a stock-market-style assessment approach to philanthropy, although somewhat novel in the late 1990's, is now fairly common. The non-profit industry is flush with terms such as "social investing", "venture philanthropy" and "philanthrocapitalism" - all alluding to a more entrepreneurial and business-like approach to make philanthropy more effective. In February of 2006, The Economist magazine demonstrated this changing face of philanthropy and its importance in western culture with a 16-page supplement on "The Business of Giving - A survey of wealth and philanthropy". Whether this evolution towards "social investing" grew out of a real need to change the business of philanthropy, or whether the new philanthropists with their venture capital and performance-driven backgrounds were the change agents behind this metamorphosis, is to be debated. Whatever the reason, it became clear from the discussion that tools to improve the effectiveness of philanthropy could have a major effect on society. Fulton stated that "the world now boasts 691 billionaires, 388 of them self-made, compared with 423 billionaires in 1996... If only 5-10% of the new billionaires are more imaginative in their giving, they will transform philanthropy over the next 20 years" (The Economist, 2006, p.3). 2 Furthermore, the available pool of philanthropic dollars is expected to grow for the next 20 years as the baby boom generation transfers its wealth to a declining population of individuals with relatively few heirs themselves - i.e. the "double income no kids" population. Whereas from 1979 to 1998 Americans gave roughly $2 trillion to charity, it is projected that US giving will grow to $6 trillion in 2017, and this could in turn reach $34 trillion by the middle of the century (Schervish, 2001). In Canada, the estimated $1 trillion intergenerational transfer of wealth is expected to peak in 2015 (Ketchum, 2005), although how much will go to charities is unknown. Assuming Canada mirrors US trends, it would appear that North America's non-profit sector stands to benefit from a major influx of social capital, and that it would be in society's best interest to invest and allocate these funds wisely. Philanthropy in the Animal Welfare Sector There are roughly 6400 animal protection and welfare organizations in North America (Irwin, 2003) and approximately 180 private foundations in the US offer grants to such organizations. According to the Form 990 submissions by 101 of these foundations between 2004 and 2006, they reported total revenues of nearly $21 million and an asset base of approximately $82 million (The Urban Institute, National Center for Charitable Statistics, 2006). Although there are millions of dollars contributed by millions of donors to this sector, relative to philanthropic contributions to other charitable causes, the animal welfare sector is significantly underserved. In looking at grants over $10,000 from US foundations circa 2003, giving to the Environment and Animals represented 6.2% and 6.1% of the total value and number of grants respectively, with grants specifically to Animals and Wildlife accounting for only 1.1 % of this total value and number (The Foundation Center, 2005). However, the animal welfare sector has recently seen an increase in its share of charitable dollars. Giving USA (2006) reported that donations to environmental groups and animal welfare organizations in 2005 increased 16.4% over 2004. This was a sizeable rise in comparison to overall giving, which for the same period increased 6.1%, half of which was attributable to disaster relief contributions. Although the report does not specifically break out contributions to 3 animal welfare from environment, the increase may suggest that the public is becoming more concerned in general with issues related to animals, their welfare and their environment. Maximizing Returns on Social Investments The for-profit sector's tool for efficient investment decision-making and allocation is the stock exchange. "Stock markets spread risks, raise capital and direct information to its best use, and keep managers efficient and loyal" (Johnson and Pazderka 1995, p.x). Essentially, stock markets take money from one group (investors) and give it to another group (companies) in the hope that the goals of all parties are achieved. As companies grow and become more profitable, investors receive monetary gains from their capital investment. This encourages investors to contribute more capital until companies are at a point that they can generate sufficient cash flow from their own activities and further outside investment is not needed. Investors then move on to assess and invest in other companies in need, in hopes that the same pattern will occur. Of course this is the theoretical ideal of how free markets are supposed to work. The same core principles hold true for the non-profit world. The act of giving money to an organization in order to help it achieve its goals is the same, although the reasons to give and the nature of the returns differ. Why philanthropists give "comes down to an individual's core values and experiences...the need to have an impact on society or set an example for the next generation... Philanthropy is about what excites you in the world, and what troubles you in the world" (Markin, 2004, p.54). Thus the motivations and returns are not monetary but emotional in nature. Perhaps it is because the returns are less tangible that donors tend to be less cautious about their social investment decisions and thus do not spend a lot of time researching and assessing the performance of charitable organizations. However, "why they give is not important; the act of giving, and how effectively they give, is what matters" (Economist Magazine, 2006, p.8). The Donor Population and Their Needs Donors include individuals (average donors, major donors and philanthropists), foundations (family and trust foundations, public and private grant agencies) and corporations. For simplicity, this study focuses on decision-making by two groups: individual donors and foundations. 4 Individual Donors In 2002, individual Americans gave $184 billion to non-profits (Vanguard Charitable Endowment Program, 2004) and over $199 billion in 2005 (Giving USA, 2006). In 2004, Canadian taxfilers donated nearly $7 billion to charities, representing a 6.3% increase over the previous year (Statistics Canada, 2005), and gave $7.9 billion in 2005 (Statistics Canada, 2006). Individual donor contributions represent the majority of funds raised by charities, accounting for roughly 75% of funds received in Canada (Ketchum, 2006) and the US (Giving USA, 2006). In terms of the average value of a gift, Hall, McKeown and Roberts (2001) reported that Canadian donors contributed an average gift of $259 in 2000 and that 82% of donations came from the 25% of donors who contributed a gift of $213 or more, however "half of Canadian adults who made donations in 2000 gave less than $74". In another study, LeRoy, Veldhuis and Clemens (2002) reported that Canadians gave 0.65% of their income and an average gift in US dollars of roughly $619 per year (Cdn $919) while Americans gave 1.63% of their income and an average annual gift of US$3,658. The study did not comment on the reasons for this gap, but it is possible that the higher number of millionaires and thus disposable wealth in the US combined with higher taxation rates in Canada may influence the disparity of giving between countries. Since the value of an average donation can be skewed greatly by a few very large gifts from a few donors, a more informative statistic would be one that explains the median value of what the majority of people donate to charity to eliminate the bias of a few large gifts. In 2003, the median donation made by Canadians was $220 and $230 in 2004 (Statistics Canada, 2005), and $240 in 2005 (Statistics Canada, 2006). Given the variations and spread between reported average and median gift values, for simplicity, I will use the term "average donor" to refer to those individuals who give less than $1,000 per year to charity, regardless of their net worth, and the term "major donor" to refer to individual donors giving above $1,000. The informational needs of major donors and average donors differ greatly: 1) Major Donors: A 2001 survey of U.S donors indicated the richest 7% of households account for approximately one-half of the billions given to charities each year (Schervish, 5 O'Herlihy and Havens, 2002). Because of the financial significance of this group, the majority of donor education programs that exist tend to target the top wealth holders because of their impact on the non-profit sector and society. However, affluent donors may need such programs less than average donors. Major donors have the option of donating through community-based foundations which were created to pool the funds of the communities' wealthier individuals and make philanthropic decisions on their behalf. As well, most banks and wealth management firms have now established philanthropy advisory services which guide high-net-worth individuals on how they can most effectively distribute their wealth (Markin, 2004). In 2005, the growth in donor-advised funds run by financial institutions was one of the fastest growing segments in US philanthropy (Ketchum, 2005) and the fund run by the money management firm Fidelity "is now America's fifth-biggest charity" (The Economist, 2006, p.8). As well, high-net-worth individuals often have personal relationships with the management teams or board members responsible for overseeing the non-profit organizations that they support. Their trust in and relationships with these individuals provide major donors with the inside knowledge they require to make their decisions (Cunningham and Ricks, 2004). Thus it appears major donors already have many options and sources to help them make informed philanthropic decisions. 2) Average Donors: Although average donors may not feel that considerable research of charities is warranted for their small donation amounts, collectively these individual contributions are still significant as a total dollar amount and as a reflection of the charity's membership base, and thus deserve thoughtful consideration. Unfortunately, most average donors do not typically have access to donor education services or the resources available to major donors. Left to conduct their own research, and without the time, analysis tools or experts at their disposal, average donors could be making uninformed decisions or limiting their level of donation. In fact, 46% of donors surveyed in 2000 said they do not give more because they think their money will not be used efficiently (Lasby, 2004). Without sufficient knowledge of and trust in charities, donors can also become too controlling about how their money is used. Some Canadian charities reported that 93% of their donations are restricted, 6 designated gifts which thus limit the organization's abilities to manage their own finances and priorities (Ketchum, 2006). As well, just because an individual is an average donor does not mean they are necessarily an average income earner or of average wealth. Average donors may possess significant wealth, and have the potential to become major donors in the long term, particularly in terms of bequests. Thus educating donors of all income and donation levels, be it through educational/advisory services or easy-to-use assessment tools, may contribute to the financial health and success of the charitable sector in the immediate and long term. Foundations Canadian foundations had assets of roughly $12.5 billion and granted over $1.2 billion in 2004 (Ketchum Canada, 2006). The total number of registered foundations in 2005 was approximately 8,900. Of these, just under 2,400 were considered active - i.e. main function is grant-making to multiple organizations - and 84% of these were family foundations (Philanthropic Foundations Canada, 2007). However, President and CEO of the Vancouver Foundation, Faye Wightman, said that community foundations - e.g. Vancouver Foundation, Winnipeg Foundation, etc. - are growing rapidly and that the number of such foundations increased from 32 in 1990 to 144 in 2005 (as quoted in Ketchum, 2006). In the US, foundations represented roughly 12% of all US financial contributions in 2000 to charities with half of grants benefiting educational and religious organizations (Dwan, 2002). In 2004, there were nearly 68,000 American grant-making foundations (independent, corporate, community and operating foundations). Of the total, 700 were community foundations. Total foundation assets in 2004 stood at $510.5 billion US, with grants totaling $31.8 billion. Foundation giving amounted to $33.6 billion in 2005 (The Foundation Center, 2006). Trends in Non-Profit Performance Evaluation Almost all foundations and intermediaries will conduct some level of monitoring of their recipient's use of granted funds, but others such as The Moore Foundation, The Gates Foundation, The William and Flora Hewlett Foundation, the McConnell Foundation, the Acumen 7 Fund and New Philanthropy Capital (see various web sites) have adopted more stringent monitoring and evaluation measures for the charities they assess and support. Kehrer (Council on Foundations, 1993) outlines multiple approaches to performance evaluation, including: - Monitoring - to assess whether a program is being implemented as planned and on time • Formative Evaluations - to determine how to improve and refine a program • Process Evaluations - to understand the operations and types and quantities of services delivered - i.e. how program impact and outcome were achieved • Outcome or Impact Evaluations - to assess the effectiveness of a program in producing change or making a difference • Summative Evaluations - a combination of process and outcome evaluation components to determine program quality and impact Evaluation can occur before or after a project commences. Evaluators typically want to know what is the need for the program, how it is implemented and what results or impact does the program have (Council on Foundations, 1993). Evaluations that answer these questions could help donors determine an organization's effectiveness. But what is meant by effectiveness? Determining Effectiveness Webster's dictionary defines "effective" as 1. having the intended or expected effect; serving the purpose; or 2. producing the desired impression or response. The somewhat related term "efficient" implies proven capability based on productiveness in operation and especially stresses the ability to perform well and economically (Tormont Webster's Dictionary, 1990, p.539). Therefore, when referring to an organization's impact in this study, I am looking at both the effectiveness and efficiency of an AWO. In other words, are its programs producing the desired or intended outcomes and is its management operating and performing well economically. In order for donors to measure an organization's effectiveness, one must first determine which performance outcomes and indicators will be evaluated. These indicators should not only guide donors in their decision-making but also help the organizations themselves in determining what 8 their strategic and operational priorities should be. Evaluation can occur at various levels: the organization as a whole, an organization's program(s), various individual projects that make up the program(s), or the outcomes of these projects and programs. One group that assembled a great deal of analysis on how to evaluate effectiveness was the United Way Task Force on Impact (the "Task Force") (United Way of America, 1996). The membership of this national Task Force included 30 "representatives of United Ways, human service organizations, foundations, corporations, public agencies, and research institutions" (United Way of America, 2005). They found that the performance criteria and benchmarks typically evaluated in the past were focused on the services an organization provides - the inputs, activities and outputs. These services may be defined as follows: Inputs - the materials and resources that the program uses in its activities, or processes, to serve clients: e.g. equipment, staff, volunteers, facilities, money, etc. Activities - the activities or processes that the program undertakes with/to the client in order to meet the clients' needs: e.g. teaching, counseling, sheltering, feeding, clothing, etc. Outputs - the direct products or units of service from program activities: e.g. the number of people taught, served, counseled, sheltered, fed, clothed, etc. or the number of books published, etc. Although there are benefits in measuring the above criteria to explain the scope and reach of an organization and its programs, they do not indicate much about what difference the organization and its programs have made towards solving the problem they are mandated to address. To assess the overall impact an organization is making, the Task Force recommended that outcome measurement is key in the evaluation process, and a more compelling indicator of effectiveness. Outcomes may be defined as follows: Outcomes - the actual impacts/benefits/changes for participants during or after their involvement in the program. 9 In essence, "outcomes usually are benefits or changes in participants' knowledge, attitudes, values, skills, behavior, condition or status" (Plantz, Greenway and Hendricks, 1997). "Said another way, outputs are about the program, while outcomes are about the participants" (Plantz et al., 1997). Its charge having been accomplished, the Task Force is no longer in existence; however its groundwork provides a good foundation on which to build outcome measurement models for any organization or sector. Although tying behavior and attitudinal outcomes to participants is an obvious link when it comes to human social service organizations, in the context of animal welfare organizations, the participants may include the animals, the humans or both. While the outcomes do and should benefit the animals, humane education and advocacy programs are directed at changing human attitudes and values. Thus outcome measurement can be adapted to AWOs by including outcomes for both human and animal constituents. Outcome Measurement in the Non-Profit Sector The use of outcome measurement in the non-profit sector has generally increased in recent years. The framework proposed by the Task Force focuses on establishing and measuring outcomes (i.e. actual impacts/benefits/changes), supported by outcome indicators (observable proxies to measure these outcomes, since the outcomes themselves may or may not be measurable), followed, in some cases, by outcome targets (i.e. the numerical objectives set for outcomes) to indicate the number or percentage of clients that the program intends to have achieve the outcome. However, it is recommended that organizations should not establish such targets until they gain sufficient experience with outcome measurement and its resulting data to ensure the targets they do set are reasonable (personal communication, Plantz, 2007). The data can then be supplemented with reports on an organization's inputs, activities and outputs to demonstrate how and why the organization achieved success, and where improvements can be made to enhance outcomes in the future. Measuring performance outcomes is important because it allows donors (and the organizations themselves) to determine how effective they and their programs are at achieving their mission. In contrast, measuring inputs, activities and outputs simply highlights the means of 10 reaching a goal. Given the variations in overall capacity between organizations, some may conduct certain activities more easily or on a larger scale than others. However, it is the impact these activities have on the end goal that is important. For example, a smaller organization may not have circulated as many advocacy materials, or had as many volunteers as a larger organization, but they may have achieved more impact through some other means. A focus on what was achieved (rather than how it was achieved) may help to level the playing field when assessing the performance of small or new organizations against large or well established organizations. As another example, if organizations feel they are being judged on number of school visits, they may continue to make school visits, even though an alternative approach such as an essay contest or internships may have a greater impact on changing young people's opinions and behavior. Thus focusing on outcomes could also help foster originality in terms of how organizations will reach these goals. Outcome measurement is also an important tool for auditing the results of programs. For example, The Nature Conservancy used outcome measurement to assess its conservation efforts. Their work auditing the results of the Cosumnes River Preserve in California allowed them to compare the effectiveness of their approaches on improving salmon habitat. By examining projects in a conceptual model of cause and effect, they were able to recognize which projects and strategies were unproductive and should be changed or abandoned. Rather than just measuring what they do, this approach allowed them to measure the effects of what they do, to see if they were really making a difference in conserving nature and wildlife (Christensen, 2003). Performance Evaluation in the Animal Welfare Sector There has been very little performance evaluation or outcome measurement conducted in the animal welfare sector. As the findings of this study will show, the sector has been somewhat reluctant to use quantifiable measures, choosing instead to rely on more subjective measures. This is due in part to concerns over what, if anything, should/could be measured, and in part to a 11 general lack of skills in performing evaluations. However, the Asilomar Accords provide one example of significant effort to quantify and compare results of companion animal shelters. The Asilomar Accords were created in 2004 by a group of leaders in the animal welfare sector "for the purpose of building across varying philosophies...[and] creating goals focused on significantly reducing euthanasia of healthy and treatable companion animals in the United States" (Asilomar Accords, 2006). The Accords promote transparency through standard "definitions" and a uniform method of collecting, analyzing and reporting shelter data and statistics to enable consistency and accuracy amongst organizations. Although the focus of the Accords is on very specific issues facing companion animal shelters and rescue organizations (e.g. increased adoptions, reduced euthanasia), this model shows that it is possible to standardize certain animal care measurements amongst functionally similar organizations despite the many variables and philosophies that affect these measurements. Thus while it may not be possible to reach consensus at a philosophical level, programs and models that build consensus at an operational level may offer a uniform method of assessing performance for AWO activities. Literature Review No research has been published on performance evaluation specifically in the animal welfare sector, but studies on evaluation and funding priorities in the social service sector provide general guidance for this thesis research. In a study by Hall, Phillips, Meillat and Pickering (2003), researchers studied how to improve the capacity of voluntary organizations to evaluate their work and communicate their effectiveness to funders, stakeholders and the public. Hall initiated research through focus groups with representatives from social service sector voluntary organizations and funders (government agencies and public and private foundations) from 12 communities across Canada. This was followed by a telephone survey of 1965 voluntary organizations and 322 funders. Post survey consultations were then held with participants in 10 of 12 the 12 communities to derive recommendations. Some of the main findings of the study were: • voluntary organizations use a variety of evaluation methods and find value in performing evaluations; • funders expect to increasingly conduct evaluations, particularly to assess outcomes, and will require more information than collected in previous evaluations; • a gap exists between the information funders want and what the voluntary organizations provide; • there is a lack of money and skills to perform evaluations; • there needs to be a greater emphasis on outcome versus output measurement to satisfy funders' expectations. Thus if organizations wish to better communicate their effectiveness to funders, but a gap exists between what information organizations provide versus what funders want, a logical next step would be for both groups to jointly agree upon which performance indicators and outcomes should be evaluated and communicated. Cunningham and Ricks (2004) gathered opinions from 22 major philanthropists (those they defined as donating more than $50k per year) on the importance of performance evaluation and what performance metrics they would like to see in a performance measurement tool. The researchers found that these philanthropists were skeptical and even disapproved of measuring performance or results of non-profits. Most of the study participants indicated: • they did not have time to perform assessments; • they had an inherent trust that most charities comprise good people doing good work so did not need to be assessed; • they did not have confidence in performance assessment techniques that measured just quantitative aspects; • they considered that conducting performance assessments is a waste of the charity's resources; 13 • because they could rely on other "intermediaries" to assess a charity's performance on their behalf, they did not need to do it themselves. This last point is perhaps the most important and relevant to my study. These participants claimed they did not have an interest in performing the assessments themselves, but they were interested in and able to defer assessments and decision making to intermediaries, be they foundations or fund advisors, or through peer reviews, personal connections and colleagues. This suggests that some high net-worth individuals want and actually do take guidance from other formal or informal sources or evaluation tools, be it directly or indirectly. Thus it would be interesting to see if a time-efficient evaluation tool that incorporated both quantitative and qualitative aspects would be worthwhile to donors, particularly if the evaluation was conducted by an individual or organization independent of the charity. The American Institute of Philanthropy (2006) assesses the performance of charities through its Charity Rating Guide. The Guide aims to assist donors in making responsible funding decisions by rating various financial performance criteria including how much funding a charity allocates to programs and to fundraising, and whether or not the charity holds excessive reserves. Similarly, Charity Navigator rates charities based on seven key performance categories within two broad areas of financial health: 1) their organizational efficiency, which includes program expenses, administrative expenses, fundraising expenses and fundraising efficiency; and 2) their organizational capacity, which includes primary revenue growth, program expenses growth and their working capital ratio (i.e. reserves of liquid assets). Charity Navigator allows for qualitatively distinct ranges of financial performance to establish a minimum benchmark for each performance category. In other words, if a fundraising expense of less than $0.10 per dollar is considered exceptional, all charities spending less than $0.10 will receive the same rating whether they spend $0.01 or $0.09 (Charity Navigator, 2006). Charity Navigator then separates performance into 5 rating bands: 4 stars = exceptional; 3 stars = good; 2 stars = needs improvement; 1 star = poor; and 0 stars = exceptionally poor. However, such systems focus only on financial performance indicators and do not evaluate the effectiveness of a charity in terms of the quality of its programs and services and its ability to achieve its mission and stated outcomes. 14 To enhance the value of such comparative evaluations, it would be worthwhile to see if benchmarks and standards could be established for non-financial measures as well. An article by Herman and Renz (1999) analyzed literature on organizational effectiveness and raised several points relevant to my study: 1) effectiveness is always a matter of comparison, either to another organization, some ideal, or to an organization's own performance at an earlier time; 2) the effectiveness of non-profit organizations (NPOs) can never be reduced to a single measure since the success of an NPO is measured in moral and value terms; 3) Boards of directors do impact NPO effectiveness, although exactly how they impact effectiveness is unclear; and 4) measuring effectiveness through outcome assessment is limited since it focuses on program-level outcomes and not organizational effectiveness, and outcomes are not always attributable to the program but to some other external influence. Herman and Renz suggested that further research on developing a small number of easily measured indicators and criteria would help to assess effectiveness. They stated that although size as measured by total revenues and growth would be indicators of constituency support and responsiveness, these indicators are influenced by the age of the organization and the interests of those with higher incomes who can afford to support the organization. They therefore recommended a measure of multi-constituent stakeholder satisfaction be developed as the most unbiased approach to measure effectiveness. Studies have also been done on the evaluation of foundations and the outcomes of their funding decisions and policies. Smith (2003) focused on the challenges and obstacles related to current funding policies, practices and conditions as experienced by non-profit, social agencies, and how these policies impact the non-profit organization's operations and service delivery. The study included a non-random sample of six social service agencies which completed a questionnaire and participated in an interview. Some of the main findings from this study were: • donors are increasingly tying funds to contractual agreements, but lack of standardization across funding bodies results in higher costs to non-profit organizations because of the time and resources involved in preparing different information for different funders; 15 • funders are more interested in funding short-term, new and innovative projects versus long-term operational costs; hence proven core programs are difficult to sustain; • organizations were concerned about the time and resources required to apply for and report on grant funding, particularly if the time spent was disproportionate to the amount of funds granted. Smith's participants also commented that individual funders were "becoming more and more specific and prescriptive thus reducing the agencies' independence of both strategic and operational decision-making and limiting flexibility and innovation" (p. 17). They reported that the terms of funder agreements were sometimes "too complex and difficult to understand and administer" (p. 18) and that "payment cycles rarely coincide with the [organization's] fiscal year" (p. 19). Her participants also said that even though funders say they like to see organizations demonstrate sustainability, they actually frowned upon those organizations that held healthy operating reserves. A study on venture capital practices (Letts, Ryan and Grossman, 1997) brought together a number of leaders from foundations, nonprofit groups and venture capital firms to clarify what foundations can learn from venture capital practices to help identify worthy recipients for their funds. They pointed out that a serious error made by foundations is the lack of investment in non-profit organizational capacity (management training and development, sound operating systems, etc). The foundation's focus on program efficacy and short-term grants does not support organizational strength and thus limits the impact of long-term goals and outcomes. As well, they criticized the requirement by foundations to receive only periodic reports rather than becoming more involved to help contribute to strategy and develop the management team. They explained how this approach is in stark contrast to how venture capital firms work with the start-up companies they fund, whereby in addition to advising their partners, venture capitalists help them raise additional funds so the company's CEO can concentrate on managing the growth. Thus both the Smith and Letts et al. studies indicate it would be useful for charities and funders to gain a better understanding of how their funding strategies influence the effectiveness of the organizations they support, and where their funding could be applied for greater impact. 16 A study by Ostrower (2004) asked 61 foundation leaders from 42 foundations to define effectiveness. Participants defined effectiveness in very broad terms, and had not really thought through the meaning of effectiveness in the context of their own organizations' goals. The study found that community foundations were slightly better at defining effectiveness than private family foundations since they had to communicate their vision and mission to external parties to raise funds themselves, and are expected to reflect community desires. However, most of the foundations in her study said they had no formal method of evaluating their own effectiveness or holding themselves accountable. In a book chapter written by The Center for Effective Philanthropy (Buchanan, Bolduc and Huang, 2005), the authors describe the Grantee Perception Report, an assessment tool provided to foundations that sets set out to determine how grantees perceived foundations' performance and effectiveness in terms of the grant application process and the impact the foundation was making, among other measures. Foundations said that one of the most useful qualities of the "Grantee Perception Report" is that the standardized questions posed to grantees and their subsequent responses allowed them to compare their ratings relative to the ratings given to other foundations. As well, the feedback from grantees inspired the foundations to make improvements in areas such as communications with grantees, the grantmaking process, and assisting grantees post-funding. The work by Ostrower (2004) and by Buchanan et al. (2005) suggests that a better understanding of what criteria, indicators and results define an effective organization would be helpful for both foundations and charities, not only for their own self-assessment purposes, but also for comparing their efforts to those of their peers. Finally, Chun (2001) developed and operationalized a methodology for the Roberts Enterprise Development Fund (REDF) which proposes how philanthropists could measure their social return on investment (SROI) by quantifying and monetizing the socio-economic value created by NPOs. REDF's SROI framework suggests "a nonprofit organization creates socio-economic value by making use of resources, inputs and processes; by increasing the value of these inputs; and then by generating cost savings and/or revenues for the public sector" (p.2). Using a somewhat complicated discounted cash flow analysis, the SROI Framework attempts to put monetary value 17 on the intangible benefits NPOs create. The framework focuses on the value delivered and derived when a NPO employs the homeless or those who were living on very low incomes. The resulting increase in the individuals' self esteem, the stabilization of their housing situation, their independence from the welfare system, and their contribution to the tax base all represent socio-economic value to the community. Clearly this methodology only applies to certain, specific NPOs, but nonetheless it is an attempt to quantify and monetize an NPO's social impact and value. All of these studies and methodologies indicate an interest in formalizing ways of measuring the efficiency and effectiveness of non-profit organizations and foundations, and the need to reconsider how to apply donor funds to make the most impact. As noted earlier, the studies undertaken to date focus broadly on the social service sector or on general donor and foundation attitudes and giving patterns. The focus of my research applies some of these same research strategies and goals to address issues specific to the animal welfare sector. In particular, the findings from the studies by Hall et al. (2003) and by Smith (2003) helped confirm and shape the direction of my own research in terms of sample size, research methods and questions posed to participants (see Chapter 2 - Methods). Research Goals The allocation of funds to AWOs does not always adequately reflect the merits of the respective organizations. Many notable AWOs lack the funding they appear to deserve because they have insufficient marketing skills or budget to articulate or publicize their programs and results. Meanwhile, other agencies have the marketing power, image or even name to attract funding even though their programs may contribute very little to the welfare of animals. Without a suitable assessment tool, donor decisions may simply be influenced by emotionally evocative advertising campaigns and/or familiar organizational names. But in creating an assessment tool, how would one compare the roles and outcomes of diverse organizations in order to highlight and strengthen the most effective and productive? 18 To answer these questions, the specific goals of my research covered four key areas: 1. Performance Indicators and Evaluation: What are the common criteria amongst AWOs that can be measured and evaluated to determine effectiveness, and would such evaluations inform and improve decision-making processes in humane philanthropy? 2. Outcomes and Outcome Measurement: What are the desired outcomes that AWOs should strive to achieve and do AWOs and foundations understand the difference between measuring outputs and outcomes? 3. Standardizing Evaluation for Comparative Purposes: Other than financial measures, what other criteria can be standardized and benchmarked to compare performance of AWOs, and what are the pros and cons of this approach for donors, foundations and AWOs? 4. Funding Conflicts: Are the funding strategies of donors in conflict with the needs of AWOs, and do these strategies reduce the effectiveness of AWOs? In addition to these core areas, I hoped to understand and document what information donors feel is important in their decision making, and why they may weight some criteria more heavily than others. I was also interested in how one addresses bias in the evaluation process to allow for philosophical differences amongst individuals. This is particularly challenging when funding decisions are made by a committee of foundation trustees or board members with varying personal philosophies concerning animal welfare issues such as no-kill versus open-admission shelters, whether or not to rehabilitate native versus non-native species, whether or not to hold wild animals captive in zoos and aquariums, and whether the raising of farm animals for human food consumption is acceptable. The remainder of this thesis is organized into four chapters. Chapter 2 - Methods, explains the research methodology. A qualitative, grounded theory approach was chosen in order to gain an in-depth understanding of the perspectives from organizations representing the animal welfare sector in North America. Chapter 3 - Results, highlights participant responses to the four key areas of research. It identifies their thoughts on: the criteria that donors could assess in order to 19 make informed philanthropic decisions; the performance outcomes that AWOs should strive to achieve; potential areas for benchmarking and the pros and cons of such standards; and the gap between what projects donors want to fund versus the areas and projects for which AWOs need funding, and how this impacts an AWO's effectiveness. Chapter 4 - Discussion, clarifies and comments on certain aspects of the results, and highlights how a peer-review process incorporating qualitative and quantitative indicators, combined with sector norms, trends and outcomes, provides a systematic framework that may help donors better assess performance of all types of AWOs. This chapter also examines how AWOs could use this framework to assess their own operations in order to better manage their objectives and results. Chapter 5 -Conclusion and Recommendations, summarizes the findings and analysis of the study and suggests where further work is needed to improve philanthropic decision-making with regard to the animal welfare sector. 20 Chapter 2 - Methods Qualitative, Grounded Theory Approach To fully understand the thoughts, motivations and opinions of any group of individuals, researchers typically use qualitative research and analysis methods. Qualitative research and analysis involve the use of unstructured exploratory techniques (such as group discussions and interviews) that are based on small samples in order to understand a problem in depth. Qualitative research differs from quantitative in that "it emphasizes understanding, rather than precise measurement of events" (Weiss 1998, p.83). Where the researcher has no prior knowledge about the subject area, and where very little is documented about a particular phenomenon, a common qualitative approach is inductive, grounded theory. Inductive grounded theory allows researchers to analyze and code the data they collect, and then generate theories through interpretative procedures (Haig, 1995). By using this approach, I was able to mix structure and flexibility in the data gathering process, interpreting and comparing participant responses as the process unfolded to incorporate new themes and questions while abandoning others as knowledge was gained (Weiss, 1998). An inductive, grounded theory approach enabled me to collect data on participants' feelings, attitudes, values and motivations on assessing performance of AWOs in order to generate a theory as to what criteria are most important in making informed philanthropic decisions in this sector. Participant Recruitment and Selection The "animal welfare sector" encompasses a broad spectrum of individuals and organizations whose mission is to improve the quality of life of animals, be they companion, farm, laboratory or wild animals. The size and scope of the organizations vary greatly, with some organizations working directly to rescue, shelter or advocate for animals, while others work to provide the funds necessary to enable these groups to function. For this study, I recruited participants from the following two groups: 1. Animal Welfare Organizations (AWOs) - organizations involved in caring for and/or advocating on behalf of companion, farm, wildlife and zoo/aquarium animals. These included 21 animal shelters/sanctuaries, wildlife rehabilitation centers, humane societies, and advocacy groups. 2. Animal Welfare Foundations - organizations involved in granting funds to AWOs. These included foundations that provide grants for all animal welfare concerns (i.e. companion, farm, wildlife, and zoo/aquarium) and those that grant funds only to specific types of AWOs (i.e. only wildlife rehabilitation, only companion animal shelters, only grassroots groups, only local organizations, etc.). In terms of the individual donor's perspective, I did not include average donors in this study because I felt they would not have much experience with sophisticated evaluation of organizations, and thus would not contribute greatly to the goals of my study. I had hoped to gain participation from major donors, however those contacted indicated they either did not have time to participate, or that their philanthropic endeavors were a private matter, and thus they did not wish to participate. Fortunately, foundations typically represent the interests of individual donors, and they usually conduct some level of performance evaluation of the chariites they support. Thus I felt foundations would sufficiently represent the perspectives of individual donors, albeit indirectly, and would have enough experience with performance measures to adequately contribute to this study. Rather than randomly recruiting organizations, I used purposive sampling to obtain a cross-section of participants from across North America to reflect the.diversity of the sector and to provide a broad range of experiences. Organizations varied by location and mission, with mandates that addressed local, provincial/state, national and international boundaries. I also selected organizations that ranged in size from very small to very large, both in terms of their financial capacity and quantity of staff. This diversity served to reduce social, philosophical, economic and political biases that may have occurred as a result of selecting organizations from within the same community, that followed the same mission and mandate, or that had similar organizational structures and capacity. 22 Participant recruitment and data collection were conducted in three separate phases. Recruitment took place in the spring, summer and fall of 2005. Subsequent research of these three sample sets took place in the summer, fall and winter of 2005 and early 2006. Staging of the recruitment process allowed me time between data collection to incorporate new themes, discard irrelevant themes, and improve the format of the research tools and techniques as the research progressed. The staging process also allowed me to balance the organizational makeup of my sample by recruiting new participants who could fill identified gaps or supplement emerging trends in the data that needed further validation. Recruitment was ended when data saturation was achieved. In other words, patterns of similarity were evident in the data, and no new themes emerged in response to my questions, indicating sufficient data to validate major themes (Streubert and Carpenter, 1995). I contacted potential participants from 60 organizations of which 24 organizations (40%) agreed to take part. This included 15 AWOs (of 34 contacted) and nine foundations (of 26 contacted). Some organizations wished to have more than one individual from their organization contribute to the research and thus a total of 30 individuals participated in the study. Of the 15 AWOs participating, nine were situated in Canada and six in the US. Of the nine foundations participating, four were located in Canada and five in the US. AWOs ranged in size from small organizations (i.e. only a few paid staff and budgets of less than $500,000 per year) to very large organizations (i.e. hundreds of staff and budgets in excess of $50 million per year). Foundations also ranged from small organizations with annual grants totaling less than $100,000 to major foundations with total annual grants exceeding $1 million dollars. Some participants had a single focus such as wildlife rehabilitation or companion animal sheltering only, whereas other organizations represented multiple facets of animal welfare. Representation was thus balanced amongst organization types. Many of the individuals who participated on behalf of their current organizations offered significant breadth of experience to the study. Some participants that previously worked with companion animals now worked primarily with or advocated for farm animals or wildlife, and vice versa, while some who worked 23 for AWOs now worked for foundations, and vice versa. In addition, some concurrently or previously held Board or Trustee positions with other AWOs and foundations. Furthermore, many participants personally donated to various animal-related causes. Thus the total knowledge base represented by the participants exceeded the perspectives of just the 24 organizations. For this reason, individual participant comments sometimes reflected the views of an AWO, a foundation and an individual donor. To allow participants to speak freely and openly, they were advised that their identity would be protected in the published thesis through codes and/or pseudonyms. References to participants have therefore been coded numerically by organization type, with AW01, AW02, etc. to identify animal welfare organizations and F1, F2, etc. to identify foundations. References made by participants to locations or other organizations have also been removed and/or generalized to maintain anonymity. Data Collection Data were obtained through survey-questionnaires and one-on-one interviews. Survey-Questionnaire: After they signed letters of consent, participants from the first stage of recruitment were sent a questionnaire. The questionnaire for AWOs contained 45 open-ended questions and the questionnaire for foundations contained 25 open-ended questions (see Appendix I and II). However, after reviewing the responses of the first four AWOs and three foundations, I decided to redesign the questionnaire for the remaining 17 participants. The second version of the questionnaire (see Appendix III and IV) maintained some of the original questions, but eliminated all questions about the organization itself that could be easily found in their promotional materials and annual reports. I then supplemented the questionnaire with a survey asking participants to rate the importance of 58 different criteria for determining an AWO's effectiveness. The survey was followed by 17 open-ended questions requiring a more descriptive response about performance evaluation in general, the pros and cons of this activity, standardization of assessment, and potential performance outcomes towards which all AWOs should strive. Two AWOs did not complete the survey-questionnaire because of lack of 24 time, choosing instead to participate only in the interview. Thus 15 of the remaining 17 participants completed the survey-questionnaire. Participants were asked to return their completed survey-questionnaire before the interview. This allowed me to familiarize myself with the participants' opinions and concerns and prepare interview questions that focused on specific issues they identified. Interview: Once interview questions were finalized (see sample in Appendix V), they were sent to participants at least 1-2 days in advance, and sometimes 1 week in advance, to assist participants in preparing their thoughts for the discussion. All 15 AWOs participated in the interviews, with ten interviews conducted in person and five by telephone. Eight of the nine foundation interviews were conducted in person, with one foundation withdrawing from the interview because of time constraints and thus only completing the survey-questionnaire. During the interview, participants elaborated on their responses to the survey-questionnaire, responded to the advance list of interview questions, and addressed impromptu questions on new issues that arose during the discussion. Although participants were free to refuse to answer any questions, as per the consent agreement, no participant refrained from commenting on any issue. Interviews ran from 1 hour to 3 hours in length and over 38 hours of discussion were collected: 25 hours from AWOs and 13 hours from foundations. Interviews were tape-recorded and later transcribed for data coding and analysis. Research Content Questions in the study focused on three core areas: 1. Business Fundamentals addressed philosophical values and operational aspects that are important for an organization's success. Questions addressed mandates, objectives, policy statements, management/staff/board/volunteer experience, donor base, fiscal responsibility and animal care statistics. 2. Performance Indicators and Outcomes examined relevant performance criteria and outcomes on which to evaluate the effectiveness of AWOs, and the pros and cons of 25 measuring performance criteria and outcomes. Discussions examined possible methods to achieve fair comparison of results amongst both common and diverse AWOs and what criteria could be benchmarked, if any. Questions also addressed standardization of performance evaluation, who should conduct and pay for evaluations, and what are the most desired performance outcomes for specific types of AWOs. 3. Funding Conflicts analyzed contradictions between the needs and concerns of AWOs and the practices of foundations. Questions addressed the type and terms of funding agreements, bottlenecks in the grant submission and approval process, and the role and expectations of AWOs and foundations. In addition to these core areas, other topics were raised by participants during either the survey-questionnaire or interview phase and were later incorporated into the core set of questions for all remaining participants. Inclusion of new criteria is warranted in qualitative research if the criteria are raised often enough, even if the researcher does not initially apprehend their importance (Denzin and Lincoln, 2000). Also, some questions were dropped when they were deemed irrelevant by participants. While the core questions were maintained for all participants during the interview process, not all participants were asked the same full set of questions. Thus the findings will sometimes show only a few participants responding to certain questions. This could result from either a question being dropped because of lack of interest or time, or from the topic being added later in the study because of obvious interest. Because of fluctuations in the number of respondents to any particular question, I used the following phrasing to refer to the proportion of participants that cited a common response to a question: • a few = up to 3 participants, or <15% of responses • some = 4-7 participants, or 16-30% of responses • many = 8-12 participants, or 31-50% of responses • majority = 13-19 participants, or 51-75% of responses • most = 20-24 participants, or >76% of responses 26 Data Analysis Interview recordings were transcribed, with individual participant transcripts ranging from 14 to 40 pages each in length, culminating in nearly 500 pages of data. Transcripts were read repeatedly; this allowed me to identify and code common themes in the participants' comments. Relationships were mapped among the codes, which were later refined to derive central categories or main themes (Coffey and Atkinson, 1996). This inductive method of coding allowed my criteria to evolve out of the data and avoided my own preconceived notions of what criteria might be important. However, I did use my own deductive logic and background knowledge to formulate theories and hypotheses about the codes and relationships that made sense to me. Transcripts were compared against survey-questionnaire results to detect any conflicting viewpoints from the same participant, and to supplement survey data with interview responses where the participant gave a response of "don't know", or where data were missing as a result of completing the first version of the questionnaire. Interview and survey-questionnaire responses from each participant were then combined into a single document and tabulated to see how many participants responded to a particular question, how many different criteria participants gave as a response to each question, and which criteria were considered most important or were most commonly cited. This amalgamation served as the basis of the analysis to determine which criteria would be referenced in the main body of the thesis. Study Limitations My selection of organizations was somewhat biased by geographic boundaries. While I did contact organizations throughout North America, for in-person interviews I targeted areas where I could meet with at least three participants in the same region. Not only did this justify travel costs, but it also ensured data partially reflected attitudes in the region and not just the opinions of one organization. Geography also had a minor effect on the quantity and quality of data collected. Where I was only able to attain one participant from a region, I conducted these interviews by telephone. Whereas in-person interviews typically ran 1.5 to 3 hours in length, telephone interviews ran 45-minutes to 1 hour in length and thus the full complement of questions could not be raised. As 27 well, participants typically provided shorter responses when being interviewed by telephone. Thus telephone interviews definitely affected the quantity of data collected and may have affected the quality of input from participants. While the qualitative research and staged approach to data collection allowed me to refine my questions and adjust to participant responses and needs, it also meant statistical correlation across the entire data set was not possible. Inconsistencies occurred when: 1) four AWOs and three foundations from stage one of the study received a slightly different survey-questionnaire than the remaining participants; 2) two AWOs did not complete a survey-questionnaire and thus were only able to participate in the interview; and 3) one foundation did not participate in the interview. To gain as much consistency as possible around the core questions, I used the revised survey-questionnaire as the guiding interview template for all but the first two AWO interviews. Finally, the elements I chose to investigate and my interpretation of the data will have been influenced by my own business experiences in terms of corporate and individual performance assessment, my experience in companies funded by venture capitalists, my understanding of stock markets, and my very recent experience as a board member of the BC SPCA. It is unrealistic in a qualitative study to completely ignore how the researcher's experience and judgment might shape the analysis of the data (Weiss, 1998). However, as I had never been employed by an AWO or foundation, and was new to the BCSPCA Board at the time of the study, I believe I was able to remain impartial and objective in my interpretation of the results. As well, the open-ended nature of the questions and interview format allowed participants to use their own perspectives to either embrace or reject the questions they were asked. Their suggestions therefore shaped the line of questioning to subsequent participants, allowing the data to reflect what was important to participants, and thus tempering my own personal views and opinions. 28 Chapter 3 - Results The results will be presented under four main categories: 3.1. Performance Indicators: performance criteria that participants commonly cited as useful indicators of an AWO's performance and effectiveness; 3.2. Performance Outcomes: outcomes that participants felt AWOs should strive to achieve and ultimately for which they should be held accountable; 3.3. Comparative Evaluation: aspects of an AWO's operation that participants felt were most amenable to benchmarking, their thoughts on the pros and cons of sector standards, and the process of performance evaluation to compare the effectiveness of AWOs; 4. Funding Conflicts: areas where the needs of AWOs and the interests of foundations are not aligned, where bottlenecks and challenges exist in the funding application and granting process, and how these conflicts influence the effectiveness of AWOs. 3.1 Performance Indicators The majority of comments focused on the performance indicators that participants felt donors should assess to determine an AWO's effectiveness. However, some responses highlighted indicators and activities that participants felt AWOs should monitor in order to manage themselves more effectively. Although these latter indicators may not be directly useful to donors, participants felt that if AWOs were paying attention to these indicators, they would in turn achieve better results, and thus indirectly meet performance expectations of donors. For this reason, these comments have been included in the results. When participants were asked for their general thoughts on the value of external monitoring of AWO performance, the majority were positive, stating it would make their decisions about judging worthiness and effectiveness easier, and it would help AWOs better manage their organizations. A few cautioned, however, that when it comes to animal-related causes, donors' decisions are grounded in interests that affect their "heart" rather than their "head". Thus many of the operational criteria typically assessed in performance evaluation would be meaningless to 29 donors. However, participants recognized that making decisions based on emotional appeal was a problem and that a more pragmatic evaluation method or tool would improve decision-making. A few participants were concerned that an external monitoring process would focus too much on operational criteria and not on more subjective criteria. They felt so much of what needs to be measured is intangible, or requires a gut-feel often validated by in-person visits to the organization's offices and facilities. "AW015 - besides the numbers, people ...need to go the organizations, look for yourself and see what's going on, listen to how they interact with customers, ... go through their process to see if it makes sense and evaluate on that as opposed to across-the-board [rankings on who] spay/neuters, who doesn't." A few others said they were reluctant to conduct evaluations because of a lack of skills. "F8 - what I dislike about performance evaluations is that I don't know how to do them. That's really basically it. It would really be great if there was [a professional individual or] a firm ...that specialized in this and maybe there is. So I would say, 'Oh please someone do it'. Currently, if donors wish to assess performance of an AWO, they can: 1) review the organization's tax filings; 2) refer to its annual reports (if available); or 3) refer to charity rating web sites such as Charity Navigator and Guidestar. However, some participants felt the information provided by such tax filings and websites does not provide analysis on an organization's effectiveness or the quality of its programs. "AW015 - They're basically reporting on how the finances flow, as an auditor would, not necessarily on whether the programs and services are effective. It's just guaranteed that a certain percentage is going towards programs and services but not that these are the right services or done in the right way." For this reason, many participants saw a need to supplement these data with more relevant analysis of the effectiveness of an organization and its programs. In general, participants felt donors have a responsibility to themselves and the causes they support to research the organizations they plan to support. "AW03 -1 don't think there's any one pivotal action for a donor to take. I think it's a number of different things - it's looking at the record and examining it to find out whether there's any truth to it, talking to other people who have 30 donated or organizations that have collaborated or worked with that organization. It can be people who have consulted with that organization and know its workings and what it has accomplished over the years. I think it's just basic research in a number of different areas. ..It would be nice if there was some central web site that everyone could go to that would give the truth about what everyone's doing, but to my knowledge that doesn't exist." When participants were asked for specific criteria that demonstrates effectiveness, 39 indicators were cited, with the four most commonly cited being: 1) the number of animals helped over time; 2) good strategic planning (setting realistic, achievable objectives and adhering to mandate); 3) accomplishment of past mission directives and interim short term goals; and 4) meeting community needs. Although some participants judged effectiveness based on how many animals an organization helped and whether it did do so in a responsible and efficient manner, others expressed concerns that judging organizations based on numbers creates a bias against small and new organizations with less capacity. "AW015 - I think overall the larger impact, the total number of animals that can be helped the better. But that doesn't say the smaller groups are not equally as worthy." While some felt effectiveness can be measured in terms of scope and a quantifiable impact on animals, there was recognition that effectiveness could also imply a more strategic way of accomplishing such goals (either through a novel strategy or filling a niche not currently being addressed) or through efforts to bring about significant social, behavioral and political reform and thus indirectly impact a large number of animals. "AW03 - there's always that tendency I think for organizations who grow larger to become far more conservative, because they're guarding their assets, and they're guarding their status quo, and it's the small groups and the medium sized groups that are always pushing the issues harder. They're pushing the envelope." Some participants also indicated that capacity-building and conducting internal evaluations were important indicators of effective organizations, since self-evaluation will enable them to improve their programs, processes and skills. Self-evaluation could also assist in refining long-term goals into short-term achievable steps to build success and momentum each year. 31 "AW011 - Impact is two-pronged. You might not immediately reach your goal but you can use the more logical first step [to move people's opinions along]. And I think animal welfare organizations need to do that. [People] don't just suddenly [understand] all the issues. I think people grow into [supporting]...animal welfare ... rather than suddenly becoming a flaming beneficiary for the animal welfare movement." Although some participants seemed reticent about whether one could evaluate and compare performance of AWOs, when asked what they thought was important for them to measure in their own organization, and on what criteria they like to judge and be judged by others, the indicators cited were fairly common amongst all participants. Through data analysis and coding, I was able to group the criteria into four common themes or "pillars", which I refer to as the "PREP" Framework: 1. Philosophy - the organization's beliefs and values 2. Red Flags - criteria that indicate problems in the organization 3. Efficiencies - financial and operational conduct of the organization 4. People - abilities, diversity and involvement of those who work in and support the organization At a high level, the four pillars apply to all organization types, while some of their sub-component criteria apply only to certain types of AWOs. Although participants suggested numerous criteria within each of these sub-components, the PREP Framework mainly references those criteria that were cited at least 25% of the time in order to keep the criteria broadly applicable and relatively simple. A few novel and meritorious ideas have also been included, even though they were not cited more than 25% of the time. P - Philosophy Participants raised 76 criteria that donors could assess to determine an organization's philosophy and its effectiveness with regard to upholding its philosophy and the welfare of animals. These criteria can be grouped into three broad categories: 1. Mission and Objectives; 2. Policies and Practices; and 3. Validation of Mission and Policies. 32 1. Mission and Objectives When participants were asked what was important for donors to know about an AWO's mission and objectives, 11 criteria were cited, the two most common being: i. Preventing the root cause of a problem: A few participants felt the mission and objectives of an effective organization should include preventing the root cause of underlying problems rather than only reacting to immediate situations and emergencies. "AW03 - I think that as valid as individual animal care may be, it's equally or more important that they get at the root cause of the problems that put those animals into those situations in the first place. And by focusing entirely on the animal care side of things, and not addressing the root cause, you're just perpetuating the system that allows that situation to occur. So I think for anybody that's donating ...they should look for organizations that also deal with the root cause". ii. Meeting a Community Need: Many participants felt that meeting a real community need was also important. If several organizations exist for the same cause and differ only on philosophical position, they said donors should question whether there is a really good basis for multiple groups to exist. "AW05 - In the non-profit world end up with too many philosophies... It's all good intentions but unfortunately [people start organizations up] without really having a solid view to what the market needs and what the market can bare." Despite differing philosophies on specific aspects, a few participants felt that most AWOs share the same underlying message of compassion and respect for animals, and thus there was likely room to collaborate on awareness campaigns to better serve their communities. A few also suggested that AWOs should centralize the operations of their "hot-lines" which inform the public on what to do for injured, abandoned or abused animals, particularly wildlife: "AW05 - We are hoping in the future to network all of our [region's] rehabilitation.centers and home rehabbers to be able to manage their hotline calls, helping them out, making sure the animals are going to the right person. So we want to work in cooperation with the other centers. [As for] competition, I think that's something that unfortunately is driven by economics and especially in our industry it's so important to work together." 33 Community need is relevant as an indicator of efficiency and collaboration as well, and will be discussed further under the pillars "Efficiency" and "People". Some participants said donors should also assess whether an organization's mission and objectives can be realistically achieved in a suitable time frame and with its existing budget. In other words, has the AWO set a goal that will take 20 years to accomplish, with no other interim goals to demonstrate progress? If so, the organization may be too idealistic and never achieve its mission. Also, if they have set goals that require substantial funds from new sources, some suggested it is unlikely these objectives will be realized and thus their mission is impractical. 2. Policies and Practices Participants identified 13 criteria related to policy and practices that could help donors better assess the philosophy, core values and ethics of an organization. The most common criteria fell into three main themes: i. Animal Ethics Policies and Position Statements: Some participants suggested donors review an AWO's animal ethics policies and position statements as these will outline the core values, morals and beliefs of an organization. Policies and position statements are also important because typically they are co-developed by the senior management team and the board of directors and demonstrate there has been input from multiple perspectives and that the actions of an organization are not at the whim of one person's beliefs and values. As well, a few said policies and statements provide integrity and consistency through complicated or difficult decisions. At a basic level, these statements should disclose what the organization supports and what they are against. "AW02 - I would want to know about...very generalist categories, and that would tell you right away what kind of organization they are. If they are supportive of sport hunting, hunting for subsistence, of animal use in rodeos, pets even. You'd find out really easily if this is an animal rights organization, animal welfare organization, or a conservation organization. If you are conservation related, what do you and do you not support within those frameworks? Do you support the rehabilitation of non-native species? Do you support the hunting or culling of wolves to manage deer populations? Do you support poisoning of rodents? Those type of things." 34 In fact, a few participants indicated it is with wildlife organizations where much of the conflict between missions and donor values occurs. If an AWO claims to be a wildlife conservation group, they said donors need to ask the organization by what means does it conserve wildlife. For example, some wildlife associations refer to the practice of wise-use or conservation management, meaning they support the hunting and culling of wild species to control populations. "AW02 -Your reasons as a duck hunter for conserving wetlands is to hunt ducks. My reason for conserving wetlands is to have wild spaces for wild animals to live. [So we both want to conserve ducks] but we have very different ways of achieving the same objective." A few participants said donors should also ask organizations what their policies are regarding native and non-native species. Does the AWO believe that all animal life is important, and work to rehabilitate and save all animals, or do they only support those native to the region? If they support only native species, donors should ask what the organization's policies are to manage non-native animals - do they euthanize them, trap, sterilize and release them, transfer them to an acceptable territory, or permanently house them? Some also suggested that donors ask what an AWO's policy is on quality of life. "AW03 - If it is a facility caring for wildlife, if those animals cannot be rehabilitated and released, what's their policy on long term care? Can you actually provide the animal with any quality of life, and from the donor's perspective, do they agree or disagree with the policies? And are the organizations following the policies?" One of the most controversial philosophical issues in the animal welfare sector is the "no-kill" movement for companion animal shelters. Some participants were discouraged that donors have gravitated to supporting the concept of no-kill without truly understanding the implications of doing so, referencing issues such as: "AW01 - People have to understand some organizations get to choose the animals they take, and others don't have that option...If we weren't here, who would deal with these animals?"; "AW06 - [No-kill shelters] are not an open door policy so [they] can't compare themselves to the humane society that gives every animal a chance but they only have a certain amount of room"; 35 "F8 - you know, [that shelter] would be a no-kill shelter if [they] didn't have so many pit bulls"; and "AW014 - I'm happy that it's no-kill but... I think if they stop taking animals, they're not really no-kill are they?" A few participants suggested that if a shelter states it is "no-kill", then donors should inquire further about their policies and ask if the shelter accepts all animals or do they have limits on what animals will be admitted. If they do accept all animals, including those deemed "dangerous", what steps are taken with the animals that are less adoptable? Do they train or rehabilitate them? Do they permanently house them at a sanctuary? Do they transfer them to another facility where they might be euthanized? It was implied by some participants that some no-kill shelters are simply passing on the responsibility of euthanizing unadoptable animals to other shelters. A few participants remarked, however, that open-admission shelters should not use their open-door policy as an excuse to not reduce euthanasia. "F8 - [if I fund an open-admission shelter] I have to be satisfied that they are doing everything that they can to help these animals." One participant cautioned, however, that donors really need to look at the big picture of what organizations are accomplishing, and should not penalize them because certain aspects of their philosophy differ from the donor's views. "AW07 - One of our organization's very wealthy donors... will not support the SPCA. I plead with her and say support them for what they do, don't withhold the support for what they don't do." ii. Animal Care Policies and Practices: In addition to ethical policies and position statements, some participants suggested that donors also review the organization's animal care policies and practices. When participants were asked what was important in terms of animal care, some stated the quality of care an animal receives as being the most critical aspect. "AW03 -1 think [the quality of life in a facility is] way more important than the number of animals [they help], because we've seen all kinds of dog and cat shelters where they can bring in huge numbers of animals and be adopting them out to anybody who will take them. Same holds true for wildlife rehabilitation centers [where they] will...raise and release...animals...[that] 36 stand virtually no chance of survival ... To me it would make more sense to have fewer animals that you're taking care of that have a better chance of being released and successfully surviving in the wild." When participants were asked what they look for to determine quality of care, 23 different criteria were given. The following 5 indicators were cited most frequently: a. Pre-set Limits: Many participants agreed that "saving" animals depends on funds, space, and level of care. Half of participants felt that to provide quality care, organizations must have preset limits in their animal care policies and practices. One participant from a wildlife center stated: "AW05 - Although the numbers [of animals helped] are important...a responsible center has pre-set limits on what they can and cannot accept. I think that's part of the problem we have in our industry, is that people don't know how to say no, and they take and they take and they take, and then the quality of care goes down... and then that trickles down into your people...they just get completely burnt out. People also have their limits. I don't want to release an animal that's in substandard health or that's not going to survive. I would rather euthanize it". Similar concerns were stated by an organization that shelters companion animals: "AWO10 - We're not going to just keep increasing our numbers for the sake of increasing our numbers because we [feel] it would decrease the quality of care for the animals. We believe that our ideal cap is at 4200 animals in our shelter during the course of the year and that's our goal, not to appease others in the community or funding sources who are [concerned] our numbers aren't growing. ...The quality of care that the animals are getting and the community is getting from our services is so much greater. We really did look at everything from staff burnout to the number of adoption returns that were happening...[We weren't taking the] the time to make the most perfect matches, which put stress on the animals. Also, our vet's spay-neuter surgery log was backed up for days. We realized that after a certain point the animals were just not getting as much in terms of our high standard of care that we require." One area where admission limits were regarded as critical is in permanent care facilities or sanctuaries. "AW014 - There is a fine line between sanctuary work and hoarding. It's so easy for somebody to tip over the edge and take in that one more animal that they just shouldn't be taking and people on the one hand will admire them for doing it because they're not turning it away and on the other hand there are others that say, 'Well .. okay so now you're up until 1:00 in the morning instead of 12:15. How much more can you do'?" 37 b. Number of Animals Helped: Some participants indicated the number of animals a facility takes is important, but will vary depending on location. Admission, adoption and mortality numbers will be affected by rural versus urban areas, affluent versus low-income areas and moderate versus harsh climates, as well as what species or breeds are common to that area. They also felt it was important that the absolute number admitted be presented in conjunction with the percentage of those that are rehomed, released back to the wild, transferred to another facility, die from natural causes, euthanized and, if applicable, in permanent care. Performance results for wildlife organizations are particularly affected by regional demographics. Participants from the wildlife sector stated donors need to determine if the rehabilitation center is in an area where human development and proximity to wildlife is greater. If so, one would see more conflicts and injuries. Is it an area where the likelihood of infectious disease is greater? If so, euthanasia numbers may be higher. Does the facility take in more raptors than robins? If so (raptors being much hardier birds), survival and release rates will likely be greater. Thus some felt that euthanasia numbers were important in understanding the number of animals helped, but they must be assessed in context with several other criteria such as species type and community demographics. c. Days in Care: Many participants said the number of days in care was another important indicator of quality care. "AW04 - if we're not seeing those animals being released or rehomed and the number of animals is growing, then we get concerned because we think these animals are being hoarded. So the important metric is how long are these animals staying in the shelter". Another participant elaborated on days in care, saying: "AW08 - the length of time an animal spends in the shelter I would say is one measure that should be part of the public record. And if on average an animal stays in your shelter six months or longer, that's a dangerous thing because kennel housing for dogs is not necessarily a particularly good thing for the dog." 38 Many participants cautioned that days in care will differ and be skewed based on the type and health status of the animals a facility holds, as well as whether the facility is a stray-hold, open-admission or no-kill shelter, a rehabilitation center, or a permanent sanctuary. For wildlife, a few participants commented that releasing animals too quickly may compromise the animal's health and thus survival rate, so higher days in care could be a good thing. Despite these variables, some participants indicated days in care is still a valid indicator for any AWO as long as one understands the context of the situation as it relates to the sector norm and the range between the extremes: "AW015 - if you're seeing numbers that are getting extremely high [or low] then it's something to look at and try and figure out what's behind that." d. Staff and Volunteers per Animal: Some participants also cited the number of staff and volunteers available as a factor in determining the quality care. Again, they reported that type of species, animal age and type of injuries will affect the amount of staff required for some facilities. However, some participants felt it would be possible to establish a sector standard or norm with one participant drawing an analogy to the human medical system which faces similar diversity in its patients. "AW012 - I don't know what the right answers are, but I believe that [standards are] something that should be looked at. ...There should be some kind of a ratio done. ...You know, it's like nursing. You can only look after so many patients in a day". Some also suggested that veterinary and medical expenses also indicate the quality of care administered by staff and volunteers. e. Housing Environment: Some participants commonly cited housing conditions and the environment provided at the facility as an indicator of quality care. Some of the responses related to the emotional and physical needs as stated by the "Five Freedoms", which were first adopted in 1993 by the UK Farm Animal Welfare Council (Webster, 2005). Participants felt there was no excuse for ignorance of appropriate housing standards for various species. However, a few participants stated that environmental needs would vary 39 depending on whether the facility provided permanent or temporary care. In temporary care, since animals are hopefully in the facility for a short period of time, shelters need to meet only the animals' basic needs. For sanctuaries, however, they felt ideal conditions were required because of the permanency of its animal residents, stating: "AW09 - First and foremost is freedom - space...then it's their diet and then there's certainly labour - proper staffing to attend to their needs. There's financial security so that you can afford all the veterinary expenses. ... But for us here, what's important is just that they have space, that they're not confined but provided safety with proper enclosure, that they have acreage so they can explore their environment, that they have others of their own kind to interact with, if they want to. They can have [hiding] space, if you allow the public in". iii. Business Policies and Practices: A few participants suggested that donors also consider an organization's business policies and practices. Eight criteria were cited, with half relating to fundraising policies and policies towards employees. Many participants indicated they prefer that AWOs fundraise using their own employees, and that hiring external firms to fundraise on their behalf, particularly when these firms are paid commissions on funds raised, is unethical (see Red Flags for more details). In terms of employee policies, participants felt there should be some consistency in the application of "humane treatment" towards both animals and people - i.e. that employees are paid adequate compensation and benefits, and are managed respectfully. "AW09 - It's important to look at how ethically they operate...[They] might accomplish some amazing things for non-human animals but they treat their staff poorly. They can afford to but don't provide medical insurance. They may have a CEO who has a history of being a jerk, verbally abusive...I think ethical consistency is first and foremost...You don't exploit your staff while you're trying to stop the exploitation of animals." Similar comments were cited by a foundation: "F6 - I think a lot of next generation philanthropy is going to... hopefully be more socially conscious, so that it's not just how many lives were saved or how many animals were killed but it's also what is the veterinarian being paid, what is the kennel worker getting paid, do they have health care coverage for their workers? I mean these are beyond the immediate cause and they're not what donors first think about when they're talking about giving to an animal group, but I would hope... as we look at charity we not look at it as an exploitive endeavor but we look at it as a component of society that has a 4 0 deserved place and that the players in that place deserve the same respect and acknowledgement as everybody else in society." 3. Validation of Mission and Policies When asked how donors could validate that organizations are indeed following their stated mission and policies, 21 criteria were cited, with three being commonly cited. i. References: The majority of participants said donors should look for references from credible peers within the sector (see "Peers", page 64, for more details). Some participants also suggested donors could seek validation from organizations known for their ability to assess ethical conduct such as the Better Business Bureau or data repositories and web sites which conduct and publish high level assessments and ratings of charities such as Charity Navigator, Guidestar or the American Institute for Philanthropy's Charity Watch. Some participants suggested donors "google" AWOs to see what others have said about the organization and where the AWO has been quoted in the past. A few participants also suggested checking to see if an AWO has been nominated for any awards as proof of its credentials and level of professionalism. However, some participants cautioned that stories in newspapers or on TV should not be trusted in determining an AWO's reputation since the media often use incorrect quotes or names and will sometimes speak to whoever is the most sensational rather than the most informed. Also, the media will regularly go to the same sources each time because they are the easiest to reach and always available to comment. Media coverage can also be biased by personal connections. "AW015 -1 see some agencies where they've just got media all over the place and when I delve deeper it's usually their board of directors has the local newspaper head on it or radio [manager or personality], so it's not necessarily that they're doing a better job but they just happen to have the right connections and are getting that free exposure." Media coverage can also be geographically biased. "AW015 - In [our city] to get any kind of coverage on animal welfare is not very easy to do. There are just too many competing stories. If you're in a smaller or a more localized community where you are 'it', then you might be 41 able to publicize a whole bunch of things, but it doesn't really show you're more effective at doing what you've tried." ii. Review accomplishments and financial reports: Some participants suggested donors confirm past accomplishments against past annual reports, strategic plans and financial records to see what goals were actually met. Donors could also compare how previous mandates align with the current mandate and objectives to ensure consistency over the years. iii: Site visits: In terms of validating policies and the quality of care at a facility, site visits were the preferred method. One participant said that in the case of companion animal shelters, when site visits were not possible, practices could be verified through post-adoption customer surveys. The survey could ask customers about the adoption process and the animal's health and behavior upon adoption and in the six months following. If there were any immediate illnesses, injuries or behavior problems post-adoption, it may indicate that care and veterinary treatment while at the facility were less than optimal. It was suggested that the adopter's veterinary expenditures post-adoption might also give AWOs some sense of how well the animal is being taken care of by its new family and reflect the quality of the AWO's screening process for potential adopters. R - Red Flags A few foundation participants indicated that when they assess AWOs, they are not so much looking for positive features but rather looking for obvious "red flags" or items that are out of line. One AWO participant, who also serves on a foundation board, said: "AW07- We examine that financial report looking for fraud, not looking for evidence they're doing a good job" Another participant said: "F8 - We look at all these things but they only become a deciding factor when they are out of line." When asked what red flags donors should look for, participants suggested 60 different criteria, of which five were commonly cited: 42 1. Excessive Personnel Turnover Some participants remarked that constant changes in an AWO's leadership or high turnover of staff can be an indicator that there are operational, financial or even ethical problems within an organization. "F4 - If an organization has had three executive directors in the past two years it is a very bad sign for us. It's like, 'Uh oh - problem. Nobody can stand it here.' Or [it could indicate] this is an organization that keeps making.the same mistakes because there is a conflict between someone on the board who never lets the executive directors do their job, or for some other reason it's a toxic environment that no one can really stand, or they're attracting the wrong people because they're not diligent enough in screening applicants. Whatever the reason, it's just not good." The majority of participants indicated they are more concerned with changes in leadership and management staff as opposed to line staff since new management implies a new organization. "F8 - Management is...probably the bigger issue, because I always judge an organization by the management, not by what they do so much. So if there's good management staff in place then I am confident about the organization. If they've had a big turnover, it's almost like we're funding a new organization because I don't know who they are and I don't know how they operate." However, some participants stated some level of turnover is normal for any organization, and particularly so in non-profit AWOs, because of emotional burnout, stress, lower wages and little opportunity for advancement and professional development. Therefore, they felt donors should only be alarmed if the same key positions continually turnover, or if excessive numbers of staff leave in a short time. In fact, a few participants said that moderate turnover can be considered a positive and necessary occurrence, particularly if it follows a period of ineffective activity and/or poor performance, and that an organization may hire one or two directors before they get the right skills for the position, causing 2-3 years of expected instability. 2. Disproportionate Management Salaries Although some participants stated that disproportionate salaries were a red flag, when asked what they considered disproportionate, very few could specify an amount. One participant suggested an indicator might be if the majority of employees are being paid high wages. "AW014 - I think that if you've got three employees and two of them are making over $100,000.00 a year then there is a problem there." 43 Some participants cautioned, however, that donors should also be concerned about an organization with extremely low salaries in proportion to funds raised. This might be a sign they are hiring inexperienced staff and will likely be affected by poor performance and low staff retention. As well, it may indicate they are understaffed and are having to cut priority activities, or staff may be overloaded and ultimately ineffectual. 3. High Administration and Fundraising Costs As an overall budget item, some participants indicated that fundraising costs were or concern and should not exceed 20% of functional expenses, and the lower the better. "F4 - We tend to look at 15% as a good ratio for overall administration including fundraising. Anything past 20% alarm bells are going off for us." One decision that can influence an AWO's cost of fundraising is if they hire a professional fundraising or telemarketing firm. The large majority of participants recognized that while using an external firm might benefit inexperienced or understaffed organizations, paying a commission on funds raised was unethical. Thus a flat fee for service was definitely the preferred payment method, but if a commission was to be paid, any amount exceeding 20% of funds raised was considered unacceptable. 4. Lack of Peer Collaboration Some participants felt donors should be wary of AWOs that cannot produce examples of project collaboration with, or references from, other reputable AWOs. They were particularly concerned about AWOs that discredit the reputation of other AWOs. "F6 -1 think it's very unhealthy for people who are dedicated to their cause to constantly bash the other guy. Disagree on philosophy, disagree on direction, disagree on solutions, I've got no problem with that, but to basically question motives and to put down personalities or organizational agendas I think is unproductive." A few participants also said that for the benefit of the animal welfare movement as a whole, AWOs must avoid publicly discrediting each other because of differing philosophies that the general public may not fully understand. 44 "F4 - it's irresponsible in a sense; it's counterproductive for organizations to say 'well we're a cat organization and we're going to fight against all the other cat people in order to promote this point of view.' It's like the world will freeze over and we'll still be debating the cat thing." A few participants said they were also concerned when one AWO purposely competes against another. "F8 - What I like to hear is cooperation. If I don't, it's bad. If I hear competition, it is a red flag. It's very small-minded. It's usually little tiny bureaucratic, ego-involved people." 5. No or Slow Availability of Financial Information Some participants indicated that AWOs that are slow to respond to requests for financial information and tax receipts could be questionable. They warned that if an AWO is unable to send donors an annual report, strategic plan or budget in a timely manner, this could indicate an organization with poor planning abilities at best, or unscrupulous behavior at worst. A few participants also felt that veterinary records should be available for public viewing and a red flag if the AWO won't provide them. In addition, some participants cited poor accessibility to a live person as a red flag. If phone calls to an organization are always met by an answering machine, or if no physical address is given (i.e. only a post office box), they suggested donors delve further to establish the organization's credibility. E - Efficiency Participants cited 18 indicators of efficiency in three core areas: long term planning, financial credibility and sustainability. 1. Long Term Planning For effective long-term planning, three criteria were commonly cited by participants: i. Creation of a Strategic Plan: The majority of participants agreed that a strategic plan is critical to an organization's success. "AW013 - A strategic plan is ...about getting from here to there and you have to know where 'there' is and then decide the best ways to get there." 45 While a few participants felt a 1-year plan and budget was adequate for small or new organizations, most felt 3- to 5-year plans were essential for organizations of any size. "F2 - The longer out the plan is, the better. We like to have at least a 3-year plan, or a 5-year plan would be even better provided it was updated and reviewed so you have an effective 3-year plan. Year to year just doesn't cut it." Some participants commented that by setting long-term goals, AWOs can then work backward to develop a strategy to achieve measures in the short and medium term to demonstrate they are working towards these goals. These incremental steps help AWOs achieve early indicators of success and build momentum year to year, which gives the AWO and their board some direction and focus going forward. They also felt strategic plans ensure AWOs stay within their mandate. "AW03 - We get calls every single day for issues that I could just go off in 20 directions any particular day, but that wouldn't benefit anyone. We wouldn't be doing any one thing particularly well or focusing our attention to it. So we stick within a few key areas". However, a few participants indicated that long-term strategic planning can sometimes be difficult for AWOs that provide hands-on care to animals. Participants said limited resources and the distraction of daily emergencies means planning is often pushed aside for when there is more time and/or money. However, they recognized that no business can continue to operate without some attention to planning, and at the very least, a 1-year budget with some defined objectives. "AW014 - The inability of the management to take the time to plan...or...write a grant proposal or fill out the gaming application ...because they have to take this cat to the vet. Well, you know it's an awful thing to say, but maybe you're going to have to not take that cat to the vet, and not take in so many cats, so that you can do that grant proposal so you can get that money so then you can hire somebody in the future to take that cat to the vet. And I think that that's a big mistake that a lot of the smaller organizations make they just don't do anything to plan for that." In addition to a strategic plan for the individual organization, a few participants suggested AWOs create a community strategic plan. 46 "F6 - [The AWOs] have to come together as a community and agree to some mutual goals and to talk about the various roles to be played and how each is going to contribute and how it will be monitored and built on." ii. Aligning Objectives with Budget: Some participants stated it was crucial that an AWO's objectives are aligned with its budget and are executed accordingly. "AW01 - I think [our previous business plan] set the bar so high that it was impossible for us to achieve and we set ourselves up for failure. We said we were going to do all these wonderful things. Well, the reality is we could have done those wonderful things if we had $100 million to do them". An efficient organization, therefore, should be able to show anticipated costs and revenues associated with its various goals. One situation where a few participants stated aligning objectives with budget is particularly important is when AWOs accept funding from corporations, since these relationships tend to take the form of a contractual obligation. "AW05 - Corporations ...want performance against their dollar. Some have marketing objectives, some have other objectives, but if you only meet 75% of your objectives, you will be penalized. Accountability is very important. If we don't deliver, then how can people continue to commit. But it's a tough one, because unfortunately we're selling hope. The return on investment is hope." Thus despite the sometimes intangible nature of animal welfare objectives, if AWOs accept corporate funding to pursue objectives, they must be reasonably sure they can deliver the expected results. iii. Community Need: Half of participants felt a good strategic or long-term plan should demonstrate why that AWO is needed in the community. They commented that when the need is not truly there, the duplication of efforts, coupled with the sometimes lack of collaboration among organizations, leads to an unfortunate dilution of funds for everyone. "F2 - If there are two similar groups applying for funds, we pick one and support that group to avoid diluting the funds. In general there has to be a true distinction, and it may be philosophical or emotional, but there has to be a really good basis for there to be a different way of doing it before we would even consider supporting a [duplicative] group." 47 Despite competition for donors and a finite pool of funds, many participants saw duplication as positive rather than negative since more AWOs means they can collectively reach a larger audience and thus more animals would be helped in the end. Duplication and thus competition for donors was also believed to drive creativity, enabling groups to copy the best ideas of other groups, become more effective, and propel the movement forward. "F6 - I think competition is very healthy. It can be divisive and it can be nasty and counterproductive but, generally speaking, I think competition is desperately needed in our field actually...The group-think mentality which permeates our industry is not helpful." However, it was noted that such "competition" must be balanced with the needs of the community and what the community can financially support. "AW05 - one might argue there's enough money to go around. I would argue the opposite. I think the dollars in this marketplace seem to be getting smaller from a social investment standpoint. I believe there needs to be a balance. You can't have ten big organizations serving the same area. We've got to be realistic. Look around you before you step into a region because I think you're going to cause yourself more problems than it's worth. Having said that, every year we fill up with more animals than we can handle and that's with two centers in close proximity of each other. So we're competing for funding, but do we both need to be here? Sadly we do." 2. Financial Credibility Participants cited seven criteria demonstrating financial credibility, of which the two most common were: i. Professional Financial Statements: Some participants indicated that professional financial statements are important in determining if an organization is efficient and effective. In referring to financial statements, they included expense, budget, liability and revenue statements as well as annual reports and balance sheets. "FA4 - The financial statements that an organization presents to us are one of the key things that we look at, and they relate to how the board is doing and also more broadly speaking to the health of the organization...When they're not professionally put together that's a warning sign...[They are] a tool for analyzing an organization...and [they also] in terms of where and ' how we might structure our support... I can think of one [application] where we said, 'You know what you really need is a consultant to work with you ... to help you a) strengthen your board, and ...b) help you set up better books'." 48 For AWOs that do not have the internal expertise to produce professional statements, a few participants suggested contracting an established accounting or audit firm. While some participants considered external audits mandatory for larger organizations, it was recognized that many smaller AWOs do not have the funds to pay for a professional audit every year. A few participants therefore recommended small AWOs conduct a professional audit at least once every five years. They felt an external review of finances would go a long way to strengthening donor confidence in the organization. ii. Adherence to a Budget: The majority of participants indicated adherence to budget was important, and that deficits typically demonstrated poor planning and an inability to make corrections during the year to right the deficit. "F1 - I am one of those people who looks at a deficit as a very, very serious thing and it should hardly ever happen...It is not a stable organization and they're likely just hoping that some miracle happens, and chances are they have forgotten what their mandate is." Others, however, felt it was acceptable to be over budget so long as it was within reason, publicly disclosed, and being managed. "F3 - [We need to] have a sense that there is a real plan going forward, that the board recognizes the issue, that they have steps that they are going to take to address the issue. So I guess what really raises questions or concerns are if there is a lack of transparency, a sense that things are getting hidden, that these issues aren't being taken seriously by management and the board". Some participants also stated deficits are to be expected during the initial start-up years, as well as during capital campaigns, or if an organization is trying to expand. "AW06 - Your goal is to break even every year but in the case of our organization, our board made a conscious decision knowing that we would have to have a deficit for a few years in order to grow... [We asked ourselves] 'Are we going to stay where we are and just be stagnant or are we going to try and become more relevant and do more things? If we are going to do that, we need more staff'...There are going to be growing pains there. So I think that a donor looking at that needs to look a bit more at the longer-term picture to see what is happening there". In essence, these participants felt donors should look at budgets as roadmaps to guide AWOs, recognizing that sometimes they will misjudge their anticipated revenues and fall 49 behind. However, they stated deficits should not go on for an extended period of years and AWOs should have a plan in place following the first deficit year indicating when and how they expect to correct the imbalance. 3. Sustainability Most participants felt proof of sustainability - the ability to exist and operate over the long term — was a major factor in determining an effective organization. However one participant stated that how much emphasis should be placed on sustainability depended on two factors: 1) if the facility provided hands-on care to animals; and 2) if the organization accepts bequests. Speaking as a donor herself, AW09 noted that: "[An organization's sustainability] may or may not impact my giving for that year. I can potentially see myself giving a generous donation to a local grassroots organization that I don't know is going to be around next year but they're doing a great project right now and I want them to complete it. Now if I'm going to bequest them something, yeah I want sustainability. If it's a sanctuary or shelter [that has animals in their care], I absolutely want to see [long-term] sustainability." Participants gave nine criteria as proof of sustainability, with the five most common indicators being: i. Financial Reserves: Some participants indicated that holding a financial reserve is essential even though the short-term compromise it requires is often difficult for many AWOs to accept. "F6 - they raise money to save a life and if a life gets extinguished because they wanted to create a reserve, that's a tough philosophic issue to confront." However, participants felt that holding a reserve is critical so organizations are able to withstand financial shocks such as an economic downturn, the loss of a major supporter, or unexpected catastrophes that drive costs higher or divert donor dollars to other causes. One participant added that having a reserve is also important in the event that an opportunity to expand suddenly presents itself (e.g. to purchase land or equipment, or improve infrastructure). 50 But how much should an AWO save? A few participants stated that a shelter housing animals requires 6 months at minimum on reserve. "AW08 - I think every organization should have some funding set aside for a rainy day and when you start looking around it's generally considered that you need six to twelve months of operating funds." Interestingly, a few participants commented that often holding a healthy financial reserve can work against them. They cited that donors sometimes look at their balance sheet and if they see a healthy reserve of cash, donors assume the organization doesn't need their funds. As well, they said with regard to gaming funds, their organizations have sometimes been "punished" for not spending all of the money they were given. Holding a reserve would put their organization at risk to be cut back or receive no funds at all in subsequent years for being too financially sound. ii. Revenue Generation Abilities: Some participants felt that an AWO's ability to generate revenues aside from general donations and grants should be highly valued by donors as an indicator of sustainability. This applied to mission-related income such as adoption fees as well as non-mission-related income from product sales of calendars, t-shirts and other goods. Although a few foundation participants agreed that revenue generation is important, they emphasized they were not keen on seeing a lot of time or dependency on income from activities such as lotteries and product sales. They felt it took up too much time and energy for the revenue generated. As well, many participants indicated that when generating individual donor income, they liked to see the use of monthly automated withdrawal programs. Although such systems require a significant effort and initial investment, participants agreed that the benefits far outweigh the negatives as the stable and consistent income stream allows AWOs of any size to more reliably plan ahead, and benefit from lower donation collection and processing costs. iii. Diversity of Funding Sources: The importance of having a diversity of funding sources was raised as a measure of efficiency and sustainability. Participants indicated that individual donors, foundation grants, corporate sponsorships and donations-in-kind should be the main 51 sources of funding, along with fees from mission- and non-mission-related products and services and other miscellaneous sources and promotional events. Not only does a diversified strategy reduce an AWO's risk, but a few foundations stated it made them more comfortable in their own giving knowing the AWO had other funding sources to rely upon during and after the term of their grant. Participants said the composition of these funding sources will change over time based on the history and growth stage of the AWO, with new organizations earning income from 1 or 2 of these streams (e.g. some individuals and a few large donors to get started) whereas more established organizations will generate revenue from all types of sources. They commented on the relevance of each source as follows: a) Individuals: When asked if there was an ideal breakdown of funding sources, many participants felt that individual donors should represent the largest single provider of funds, some suggesting they should account for roughly 35-40% of funds, others saying individual donors should simply represent the largest proportion of funds. "AW015 -1 always think the basic donor is your best and most secure. It's got the least fluctuation... Corporate policy...[and] grant funding can change overnight. A lot of times the other pools of money are more restricted...whereas your general donor base is allowing you to follow the mission the way you want...In our agency I don't see we'd ever want much more than the standard 20% coming from any group besides our donor base." A foundation indicated the relative order of importance of funding sources as follows: "F4 - we would rank member/donor support highest. Next would come government and foundations and third would come things like corporate sponsorship, but marketing partnerships as opposed to unattached funds". A few participants commented that although soliciting the majority of an organization's donations from individuals may take a lot more effort, they viewed it as a more consistent source and possibly a larger total source of income in the long term than foundations and corporate sponsorships which tend to restrict their grants to specified amounts for a limited period, and typically for only one year. "AW014 - You spend as much time on a $10.00 donation as you do on a $2000.00 donation. It would look like [this effort] is not efficient but you don't 52 know how many of those people are going to leave you a bequest. That might even be a better way of looking at it - in relationship to your number of donors how many bequests have you been fortunate enough to receive? - because that makes that kind of effort look a lot different, a lot more effective." b. Foundations: A few participants stated that foundation grants should comprise only a small percentage of an AWO's planned revenue sources. "F7 - I've had some AWOs that concentrate more on grants than on donors, which is crazy. ...I think anyone who depends on more than 10% from foundations is - unless it's something unusual - taking a chance. I just don't think it's wise. They're not going to get it. I think I've heard the number 20%, but I think that's high. I mean a grant is really a hit or miss". A few participants stated AWOs should also seek funding from more than one foundation at a time. "F2 - we don't want to be written into anyone's budget as a permanent contributor, and so we want to know there are others who are learning about the organization, having experience with the organization, and to whom the organization can turn when we say we have to move on." c. Corporations: Many participants noted that corporations represent important funding opportunities. They reasoned that corporations are typically able to enter into, and are accustomed to, 3- to 5-year relationships. As well, corporations have a great network of suppliers and customers that potentially represent other funding pools. "AW05 - we're very focused on corporations. Company X is a great example [of a corporate sponsor]. They talk to [Supplier Y] and that supplier is then making a significant contribution because Company X said to. So their network allows that facilitation. So they give us cash, and they in turn refer their partners over to us." Another benefit cited for corporate sponsors is the multiplier effect they have on an AWO's volunteer and financial resources. "F2 - The corporation may select an organization...simply to [create] volunteer opportunities for its employees. But the thing we like most as far as corporations are concerned is matching gift programs. That to me is much more effective [than foundation grants]." Many participants commented that corporate sponsorship is a relatively underutilized funding source in the animal welfare sector. They said this was in part due to the fact that 53 corporations are more attracted to charitable organizations benefiting humans since they are less controversial and have broader appeal to the general public. However, a few said AWOs have also been reluctant to tap into the lucrative corporate funding market due to perceived conflicts of interest and concern over negative connotations. "AW03 - you have to be very cautious when you acquire those types of funds because there's a lot of green-washing going on. Companies just want to be attached to this organization or that to make themselves look better at very low cost when in fact doing so may be detrimental to what the groups themselves are trying to achieve". However, given the financial need of most AWOs, one foundation suggested that AWOs need to be more forgiving of corporate motives, saying "The only thing about tainted money is it faint enough!" A few participants said that even if they wanted to attract corporate sponsorships, the inherent profit motive corporations attach to their sponsorships biases their decision towards larger, well known AWOs. "AW03 - The problem for us... is that most corporations, to my knowledge, [see it as] an investment. They want a return on the dollars they give out, and for small groups it's very difficult to compete with larger groups who can [give] exposure to a much larger membership". A few participants felt corporations are also biased towards organizations that are more mainstream in their philosophy and activities and less interested in organizations whose work and mandate is political and reformist. iv. Communications with Donors: The majority of participants felt that frequent communications with donors was a critical component of an effective organization, since it helps to sustain revenue sources over the long term. They said donors should ensure the organizations they support provide some sort of formal update each year. Many participants suggested organizations should provide 2-3 newsletters per year outlining where the funding has gone and what they've accomplished. One participant suggested that a truly effective AWO will use these newsletters for more than self-promotion purposes, by taking the opportunity to educate donors on issues outside the AWO's immediate mandate. 54 "AW014 - It's not just letting them know what we do but it's also letting them know other interesting things within the movement that they might not [have ever heard about]." A few participants suggested AWOs should make contact with donors through direct mail, thank-you letters and feedback requests at least 3 times per year while a few others said donors should be contacted in some way every 2 months. "AW014 - You have to keep reminding them that you're there and that you're still being effective and consistently working on issues as opposed to having them drift off. I think that it's possible to lose them if you're not in constant contact with them." v. Efficient Use of Donations in Kind: Overall, it was agreed by the majority of participants that in-kind gifts that reduce core expenses (labour, food, construction materials, medical services) are signs of efficiency and sustainability. In particular, donations of professional services and labor from accountants, lawyers, veterinarians, and trades people were highly valued. Those who doubted the relevance of donations in kind reasoned that in-kind gifts are donated because the donor no longer needs the items or because gifts are often recorded at inflated prices and thus do not accurately represent public support. Therefore a few, particularly foundations, did not like to see the corresponding cash value of in-kind gifts recorded in an AWO's financial statements. These foundations did state, however, that the value of in-kind gifts is of interest in understanding how much of an AWO's budget depends on cash donations versus in-kind gifts. Many AWOs, however, felt it is important to record both the cash and asset value of an in-kind donation in order to demonstrate the true cost of running their organization. They commented that such gifts also demonstrate efficiency since the resulting cost savings can offset budget expenses and allow them to reallocate funds to other projects and maximize existing dollars. This was particularly important for those AWOs providing direct animal care. "AW02 - we're taking in 10% more animals a year. So our food costs should go up, but it's gone down because we've been better at sourcing it. So I think donors would appreciate that we're stretching the dollar as much as possible and using local community resources to our advantage". 55 Another AWO felt that donations in kind are even more valuable when purposely sought after and earned. "AW05 - We take a very targeted approach to corporations where we say, 'You've got something we need'." This participant reasoned, therefore, that these gifts could not be considered junk or cast-offs if the AWO approached the donor first, and particularly if the gifts were new. P - People Participants cited 60 criteria relating to an organization's people — management, volunteers, peers, donors and the board of directors. 1. Management When asked what donors should know about an AWO's management (and staff), 19 criteria were mentioned as important factors to evaluate. The 4 most commonly cited were: i. Experience and Training: Many participants felt it was very important for donors to assess the background experience of the people responsible for running the organization and its programs. Some participants recommended looking at the management team's track record for examples of past successes in other organizations and companies. A few participants said management should be able to demonstrate good leadership, strong business acumen and an ability to work well with others. "F3 - I will quite often ask...a local councilor...'What's your impression of the executive director? Do you feel that he/she has good management skills? Is he/she a good communicator about the activities of the organization? Do you feel that they have the capability of pulling together the funds for a project to build a shelter in your community and the savvy to deal with the number of different issues'?" A few participants also stated that supervisory experience is particularly important. "F7 - Shelters are really high maintenance. They're highly intense, highly emotional, highly everything, and then you have to deal with the public... So in these smaller groups the managers really need to have some experience managing people. You know they can take care of the animals but it's a "dance" to manage people. And so I think that would be a very valuable skill to have. And they almost never do." 56 In addition, a few participants felt that certain key positions required individuals with professional training and credentials. They said finance directors of large and small organizations should have a Chartered Accountant or General Certified Accountant designation respectively. In terms of front-line staff providing hands-on care to animals, a few participants indicated proper animal health care training was critical for employees. They said this is particularly true in wildlife centers where staff often diagnose and medicate animals upon arrival, conduct euthanasia for severely injured animals, and are often exposed to potentially serious diseases. "AW05 - This has to be a top quality environment, because we've got the risk of the disease state...In the US, [wildlife] organizations are being funded by mainland security because they have a view to what's going on in the food chain because of the issues related to bio-terrorism...[Wildlife] organizations profile the West Nile virus and that sort of stuff...And the public's got to be aware of it. It's a safety issue...We, as an industry, have been labeled as tree huggers for many years and that is something we want to dispel." Comments supporting the need for trained animal technicians were echoed by AW02, who stated: "AW02 - there are [always] new wildlife issues. We have West Nile coming. We have avian flu coming. We have a new strain of streptococcus...We're getting more and more [serious diseases] and [while we can read about these diseases in publications, it would be more useful] if we were able to send a representative to ... conferences [or for training]... It would be a great thing for [foundations] to fund". Employees with strong computer, network administration and web development skills were also considered an advantage. A few participants commented on the need to operate good database systems to track donors, financial and operational results, and to develop and maintain up-to-date web sites in order to educate the public and raise awareness. In addition to their informational value, some participants saw web sites as a more cost-effective method of marketing and fundraising than traditional printed materials and mailings, particularly given the greatly reduced costs and turnaround time involved with processing online donations. ii. Moderate Management Turnover: As mentioned under Red Flags, excessive turnover of employees, and in particular of management, was of concern to most participants. However, a 57 few remarked that although continuity of a management team over time is important, moderate turnover can be advantageous, particularly if the current leadership is stunting an organization's growth and creative abilities. "F1 - I think people should move on... When I became President of [an organization] I wrote down my goals. I put what 3 things will mean that I'm done. And as I ticked them off, I quit...And I know it's hard for founders to let go, and I don't think they should necessarily leave the organization - that's way too much to ask - but as far as heading it up in perpetuity, I don't think it's healthy. There is huge satisfaction to leave at the right time when it's going strongly in the right direction." A related element to moderate turnover is succession planning, with a few participants saying this is especially important for organizations that are headed up by the founder. "F4 - We're often looking at situations where we've got founder's syndrome -so when we have the bright visionary coming to us and say, 'I've got this great idea' of the things we want to know is, 'Are you going to stick around? What's your plan for succession here? Who's going to replace you a year from now, five years from now? How long are you going to stay?' We are funding social entrepreneurs regularly and when they go, it's a very, very difficult time in that organization. Are the board members all his or her friends, and are they going to go too, or is there enough depth on the staff side that when this executive director or founder goes, there are people who can step into that role?" However, many participants admitted succession planning is rare to non-existent in many AWOs. iii. Paid Staff: Some participants indicated it was important that an organization have paid staff and not rely mainly on volunteers. They felt that paid staff brought a level of commitment and professionalism that cannot be achieved with volunteer-run organizations. "AW02 - You can demand a certain amount of training and expertise with paid staff... And you keep good people when you have paid staff." Comments were also made about the stress and emotional intensity of working in the area of social reform and that without pay, individuals would simply not make the level of sacrifice needed to work on the very difficult issues in this sector. Some participants also felt that, in general, volunteer-run organizations could not sustain themselves long term. 58 "AW014 - if I was looking at an organization and it was run primarily by volunteers, I would want to sit down and have a really good talk with the person who is in charge of those volunteers because what my experience has been is that the person at the top is extremely frustrated and that it's not working. It's working in a dysfunctional way but not in a sustainable way...If you're a volunteer, this is never your first commitment. It just isn't by default." iv. Competitive Wages: Although exorbitant and disproportionate salaries were earlier indicated as a red flag, many participants stated that wages should at least be competitive and are an important factor in determining an AWO's effectiveness. But what is an adequate and competitive salary? A few said it depends in part on the size and scope of the organization. "AW04 - You need to retain the best and you really get what you pay for. [Organizations should not be penalized for paying higher wages than other organizations] because it's really dependent upon the industry [and the organization]. There are some [AWOs] that are able to pay a little bit more because they have more resources available to them but they're also called upon to provide a higher level of service than other organizations would be, so I think it's all kind of relative." One participant indicated as a general rule, salaries are about 15% lower than the private sector for a position of similar responsibility and scope. Many participants cautioned that lower wages bring inexperienced staff and high turnover rates, among other problems. "F8 - I think higher staff salaries, not so much management salaries, are important. If they're at minimum wage I think that's really bad." Another participant had similar concerns. "F6 - I'm not so sure that just because you can hire people for minimum wage to clean kennels that you're doing the right thing socially or you're getting the best talent to do the job, because maybe it's more than just cleaning up the pooh. Maybe it's looking for diarrhea. Maybe it's looking for blood in the stool. Maybe it's looking for a change in behavior. Maybe it's looking for differences in interactions with the kennel workers and the volunteers. And maybe it's reporting this thing and getting the medical people or the behaviorists involved and it's not just going in there with a scooper". For a few participants, the key issue is why donors feel low pay is acceptable for charitable work in the first place.One participant suggested the issue stems more from gender-biased pay inequity. "F6 - In my view, donors should have some...sort of awareness about gifting in the social context. Animal welfare historically has exploited its workers. People come to the cause because they believe. They're not coming to get 59 rich. In the bigger organizations the pay scale is better but in the smaller organizations - and I'm going to go into dangerous ground here - but it's gender-related....We can hire you for $50,000.00 a year. We don't have to hire you at $150,000.00 a year. Why? Because I've got three other women that are willing to take this job and maybe one male...I'm not so sure, as a gifter, that that's right". For organizations that cannot afford to pay competitive salaries, a few participants suggested AWOs hire experts on contract. Although the per diem pay rate may be more costly, the employment term is shorter, and thus less expensive for the AWO on an annualized basis. Not only does contract work or outsourcing reduce the organization's overall costs, but the AWO can bring in 'the best' to work on a project and enhance credibility, which also satisfies donors' high performance expectations. As well, a few participants stated contract work is particularly effective with awareness campaigns and fundraising events which tend to require staff only for the short to medium term. Thus donors should not be alarmed to see an AWO that hires many contract workers versus full-time employees, and it may even indicate a more effective use of donor funds. 2. Volunteers Some participants felt the size of an organization's volunteer base can demonstrate community support and efficiency. However, they said certain volunteer statistics are not always the most relevant performance indicators, and are particularly irrelevant for lobby and advocacy groups. They said attracting and retaining volunteers in advocacy work was difficult since volunteers did not have the history and skill set to work on complicated campaign issues. A few participants stated that training volunteers on the issues takes too long, and therefore they often assign volunteers to menial tasks such as stuffing envelopes. Since volunteers are attracted to volunteer work in order to gain new skills or feel fulfilled, such tasks are boring and volunteer retention becomes a problem. As well, some advocacy campaigns can be short-term, 'one-off battles, again adding to the turnover rate. Therefore some participants indicated that volunteer-based criteria are valid only for facilities where hands-on animal care is required. 60 Thus discussions on volunteers focused on AWOs that house and provide direct care to animals. Participants cited 15 criteria as important indicators of effective volunteers and volunteer practices. Of those, the six most commonly cited criteria were: i. Volunteer Involvement and Commitment: The majority of participants said the number of volunteers is important in demonstrating the level of community support. However, the majority also felt that the level of volunteer involvement and commitment is more important for demonstrating community support and efficiency and impact. In other words, how many hours per year do volunteers contribute, and how many of the total number of volunteers consistently show up throughout the year? A few said a side benefit of having a consistent volunteer base is that volunteers will hold an organization accountable and report any indiscretions in order to protect the animals. "F6 -1 think the more you have volunteers...on the inside and seeing the day-to-day work, there's a better chance that any rampant deception will be caught. ...The good thing about our movement is that people actually care about the results. People actually care about whether the animals live or die". Although volunteer commitment was considered important, a few participants commented that too much involvement by volunteers can be detrimental. "F1 - You've got to stop some volunteers from being there 6 days a week, 5 hours a day. I don't like that, partly because they start running the staff ragged, and they just destroy staff so easily that way...They come to have way too much proprietary interest in the...organization." ii. Volunteer Duties: Also important was the capacity in which volunteers are used by an organization. Some participants indicated an effective volunteer program provides appropriate duties and job descriptions for volunteers, and that there is a balance between what the volunteers want to do and what they should and should not do (i.e. do they have proper training for the duty and should the duty be performed only by staff). These participants said it is crucial that staff duties are separate from volunteer duties because allowing volunteers to perform staff functions could cause confusion of responsibility in the future and an unhealthy reliance on volunteers by staff. If volunteers suddenly move on, there is no one with the background knowledge in place to perform these critical roles. 61 iii. Volunteer-to-Staff Ratio: The majority of participants considered ratios of volunteers to staff as important indicators of staff and volunteer satisfaction. However some participants stated that an AWO's volunteer-to-staff ratio would change over time, with more volunteers in the beginning stages of an organization's development as it acquires funds to pay for staff. Therefore, participants' comments on what was an "effective ratio" varied greatly. A few participants from established AWOs cited a ratio of three to four volunteers for every one staff member, while a few others gave ratios as high as 10:1 and 20:1 for smaller and newer AWOs. The nature of the work and type of animal being cared for significantly affects this equation as well, with a few participants stating direct animal care for injured animals may sometimes require a 1:1 ratio. iv. Volunteer Management: Many participants felt a formally managed volunteer program was a sign of a well-functioning AWO. Given the reliance on volunteers, and the difficulty in attracting good volunteers in the first place, it was felt that a general manager of volunteer resources was essential to ensuring volunteer efforts are maximized. "AWO10 - I think most critical is that there is a well defined training program for all volunteers and that there is a person who is in charge of the recruitment, training, hiring and firing of volunteers, and that there is an expectation for the volunteers. In our case we require our volunteers to not only go through the training program but to make a commitment of at least three or four months and that there be job descriptions for the volunteers so the volunteers know what they're supposed to do." A few participants pointed out, however, that small and new AWOs are disadvantaged with regard to volunteer management. Several participants said these AWOs likely will not have a paid manager, nor a sophisticated volunteer management and tracking system, but they will likely have a lot of volunteers to sustain operations in the short term. While many small and new AWOs elect to place a volunteer into the management position, this does not appear to be a solution. "AW014 - volunteers require time, and organizations as small as ours don't have the resources to have a volunteer coordinator. And we did try that on a volunteer basis and it just didn't work; I mean we tried it over and over again and it hasn't worked." 62 What this organization decided to do instead was become very focused and particular on which volunteers they accepted, thus managing volunteers before they came on board. Their volunteer application form enabled them to screen applicants for those who really could commit and required less supervision. v. Volunteer Tracking: The majority of participants also felt that there was real value in being able to track volunteer hours and activities, and that records and timesheet management systems, no matter how simple or sophisticated, should be kept on every volunteer. "F8 - It should go into their accounting records because volunteer hours are worth something, even at minimum wage, because you don't realize you're a bigger organization than you think you are". Not only does a tracking system allow an AWO to capture and measure their true workload and cost of business, but a few participants said it enables proper recognition of volunteers by management - an important activity in retaining volunteers. vi. Volunteer Retention Rates: Participants agreed that turnover is normal in any volunteer role, as volunteers come and go for many reasons. However, the majority commented that an effective organization pays attention to their volunteer retention rates because of the cost of attracting and training volunteers. Participants suggested looking at the historical trend an organization has experienced in its volunteer retention: is retention getting better over time, getting worse, or staying the same? If retention rates are getting worse, AWOs and donors need to address what is causing the turnover. "AW015 - If they're just coming in and working once or twice and then they're gone, the organizations are not working the system the way it should be". Therefore once a volunteer is active, the majority of participants felt it was important that management find ways to keep volunteers in the system and challenged. 63 3. Peers The majority of participants said that an AWO's relationships with its peers are a critical sign of its effectiveness. "AW03 - if I were a foundation, I would look to see who's worked with who, and over what period of time, and what the relationship has been... The most effective groups I've ever seen work well with other groups." Participants cited 22 criteria in three core areas to demonstrate and validate cooperation with peers: i. Collaborative Partnerships: Many participants recommended that donors determine whether an AWO has worked on projects with other notable AWOs, scientific experts, well-regarded academics and government agencies. For shelters, they suggested donors look for examples of where they've worked in conjunction with other rescue groups, humane societies and animal control agencies. For wildlife centers, there should be evidence of arrangements with other nearby facilities to transfer their animals in the event they are at full capacity or the other facility is better equipped for particular treatments or to house certain species. A few participants also said AWOs should make more collaborative efforts to pool staff and financial resources for the common good. "AW015 - Nowadays there seem to be so many small grassroots organizations in animal welfare that branch off for one reason or another or cause. I'm not saying that small groups aren't worthy. It's just I've never quite understood why we don't all pool together our funds and our staffing and our energies and try to [solve problems] centrally or in conjunction more." One participant spoke about how collaborative partnerships also serve community-focused philanthropy, whereby donors can contribute to multiple organizations which will collectively solve the community's problem as opposed to just supporting an individual agency. Not only does collaboration improve overall community outcomes for animals, but participants felt it also builds mutual respect within the sector. "F6 - If everybody is sitting down at the table on a monthly basis and talking about their strengths and their weaknesses - what's working and what's not -we think there is a very, very, very good chance that we are going to have 64 \ better performance, as opposed to everybody working independently. ...If I'm guessing what [the other guy is] doing, I really don't know and I'll probably think the worst of [him].... [We need to have] a better understanding of what each other is doing." Although some participants commented that collaboration is not always possible if one organization is not operating well, or is poorly perceived by the community, they concurred it would probably be better for the sector if organizations agreed to work together to build efficiency rather than independently following their own agendas. "AWO10 - We in the animal welfare field ... are pretty competitive and as I said before we don't always play well together...If I were a funder, I would try to help the organizations work together and to provide funding to help them both if they're both doing a good job, ...assuming that that community has the need and the capacity." ii. References: In addition to or in lieu of collaborative projects, a few participants suggested donors look for references from notable AWOs, other non-profit organizations, foundations, government agencies (in some cases) and academics. "AW03 - even though we're so small, our biggest accomplishment is being able to exert influence, and that's something that's hard to quantify... I think from a funder's perspective, if you really want to know what's going on, you ask someone who's in the know." A few participants said membership in national organizations and accreditation bodies to which other reputable AWOs belong can also provide assurance. As well, donors may look to see which foundations have funded the organization in the past and if those same foundations supported them for more than just one project or one year. In addition, a few participants cautioned that if an AWO claims to have a partnership with or a reference from another organization, donors should verify that the claim is reciprocated by that other party. Typically both organizations will cite each other's reference on their respective web sites, or donors can call the other party for validation. iii. Facilitation and Empowerment of Others: An interesting aspect cited by a few participants was to assess how an AWO facilitates other groups and empowers people to pursue initiatives on their own. They stated that individuals and other organizations will often contact their organizations to suggest they campaign on a particular animal welfare issue. 65 However, since these AWOs have only so many resources, staff must stick to their core strategies and plans, but they will direct the caller to another group that is already focused on this issue, or will provide callers with instructions on how to conduct the campaign or demonstration themselves. "AW03 - we encourage and teach [people who call us] on how [they can conduct their own campaign] because we feel it's more important to spread far and wide and build the movement that way. So...what we've really done is facilitate them in going off and doing their own work, rather than trying to bring them in and make them part of us." 4. Donors Various aspects of an organization's donors can also be indicative of an effective AWO. Participants cited 10 different donor criteria as possible indicators of an effective AWO. The three most commonly cited factors were: i. Number of Individual Donors: The majority of participants indicated the number of individual donors an AWO had was important because a large number of donors demonstrates a stable funding stream, community support, the ability to service community needs and the ability to influence government policies. Some considered the political influence of a large donor and membership base as particularly important since the percentage of the population that is "animal-friendly" represents a critical voice in AWO negotiations with governments over legislative reforms. "F6 - [At my former AWO, many small donors] gave us a political base, so when we would go to our elected officials to argue on behalf of an animal cause we could say,...'More people supported us than voted in the last election, so how would you like to vote on this issue? If all our contributors voted the way we wanted them to, ...we could vote you all out of office, so please pay attention to our cause'." ii. Donor Satisfaction: Some participants said that an effective organization would have satisfied donors, but when participants were asked if they had ever conducted a donor satisfaction survey, most said they had never seen or used such an instrument, although they thought it was a good idea. The few that were familiar with such surveys said they were vital management and fundraising tools. 66 "AW05 -1 think we need to focus on our donors and what makes them happy - not from a 'I'm happy to be supporting your cause' perspective but [to determine if] this is something they'll continue to invest in. 'Are you willing to increase your investment? What would you like to see more of from us? What would you like us to do more of within the space in which we operate? Would you be willing to introduce us to other like-minded individuals like yourself?'" Although the aim of such surveys would be to provide better guidance to the organization on how it is perceived and how to properly serve the community, a few participants said AWOs should always expect some level of dissatisfaction from a small number of donors to indicate that the organization is truly pushing the animal welfare movement forward. "AW09 - We're embarking on social reform. We're trying to move the public beyond where they are on an issue... The organization needs to guide and educate the public and their donors and if you're not ticking off somebody, you're not doing your job ... Often more community-based organizations will not travel down a path that I think they morally are compelled to because they don't want to have dissatisfied donors." One foundation suggested a very interesting spin on the idea of donor satisfaction or customer service surveys. "F6 - What I think would probably energize the movement the most and bring about the most revolutionary change in performance is for the donor to give feedback to some [consumer] sampling agency [as is done] on e-Bay... People who adopted animals could evaluate the shelter. Was there good customer service? Was the price reasonable? Did you get your questions answered? If we started publishing that and the shelter thought, 'Oh crap I've got bad customer service, or the price isn't right, or customers' questions weren't answered', then they're going to be prodded to do something about this, especially if that's made public...I think if we ... could get the donors' reaction to just three or five questions it would be so helpful to prod the charities to ... understand how they're being perceived... [and to motivate them] to work on [those things]". This participant went on to say that AWOs often ask only their major donors for feedback, which is not representative of their full donor base, and thus may bias the AWO's judgment of donor expectations. He also said AWOs often don't know of the missed opportunities - e.g. when a donor assesses the organization and then decides not to support them, not to increase their support, or not to adopt an animal. In short, many participants agreed that a donor satisfaction and customer service survey sent to a percentage of all donor types and levels 67 would help organizations identify areas where they can improve to ensure they are meeting donor expectations. iii. Donor Retention and Trends: Many participants agreed that retention and growth of donor numbers, revenues and relationships over time are desired outcomes. However, they felt that although the trends might be interesting to donors and foundations, they were more useful to the AWO for self-evaluation purposes. "AW015 - [Donor retention] has to be monitored, not only for the cost associated with losing donors and the cost of getting a new donor in the system, which is much higher, but I think it also tells you, if you analyze it correctly, your impact on the community. It's one thing to be losing donors because they move out of the area...but it's another to be losing donors because they no longer believe you are doing your mission. ... Also how your donors move through your system [is important]... Are [individual donation levels] coming in high and dropping lower? That just tells you a lot of things about your marketing." 5. Board of Directors The majority of participants stated AWOs need board members to provide strategic direction, create policy, ensure good governance and assist with fundraising. Although some foundations stated they did not spend a lot of time during funding decisions assessing an AWO's board of directors, they did provide ample commentary about what they expect from a board and its members. In some cases, foundations simply wanted assurances that the AWO had wise, knowledgeable, capable people attached to the organization. This would ensure good governance and sound strategy without the board interfering in the day-to-day operations or management of the organization. AWOs felt their boards were crucial to their success, but recognized they don't always have the board they would like to have. In general though, participants felt certain board attributes can make the difference between a well run or poorly run AWO. Thirteen different criteria were cited as important indicators of an effective board. Of those, the five most commonly cited were: i. Board Member Backgrounds: Board member backgrounds, work experience and skills can have a critical impact on an organization's effectiveness. The majority of participants 68 agreed that a diversity of professional skills should be present on the board to provide a broad perspective of opinions. "F1 - Without depth and breadth of talent, you're going to get sucked into emotional decision-making instead of good, practical business decisions". Participants said a board should first and foremost be able to provide legal and financial advice. Lawyers were considered essential to oversee contractual agreements and regulatory matters, and chartered accountants were crucial to overseeing financial reports and audits. Once these skills are secured, AWOs then require a diverse set of professional skills to complement staff roles and fill in any gaps. Individuals with backgrounds in business management, marketing, fundraising, sales and education were cited most often. Having a veterinarian or biologist on the board was mentioned by only a few participants as important for overseeing animal care policy, and thus only relevant for shelters. It was also felt that these latter skills were sufficiently available within most AWOs, either through staff or volunteers, and hence that they were not necessarily required at the board level. A few participants also indicated it was important to have some board members who had been involved in the animal welfare movement for some time since they would help organizations to avoid making political mistakes or mistakes of ignorance. When asked whether a grassroots versus professional background was more important, only a few said grassroots was important but that it must be balanced with professionals as well. "F7 - A lot of these boards are run by people who are zealots and they're, really wonderful with animals but they...don't grasp the importance of [being] organized like a company [and the importance of operating it like a business] is going to benefit the animals... Anybody who thinks it's not a business is really making a mistake...But I think you need both the zealots and the business people". ii. Board Member Connections: Some participants said board members with political and fundraising connections are important, as the board really represents an AWO's network in the community. Participant F4 referenced the example of the CEO of a particular AWO: "F4 - [She] was a real firebrand and distributed a great newsletter, but didn't have any networks with which to work that connected her to the system she was trying to change, so she was ultimately ineffectual." 69 It was suggested by some participants that the smaller an AWO, the more they need professionals on their board to supplement their activities. However the larger organizations should be able to finance such positions as paid employees, and thus it is more important for them to recruit members with financial and political connections that can exert influence in the right circles. iii. Active Involvement by Board Members: Some participants stated that they like to see board members actively involved in committees and events and that a "rubber stamp" board was perceived negatively. A sign of a rubber stamp board is one that is mainly composed of high profile names that don't have the time to be really committed and are just lending their name. "F2 - We'd rather have 5 dedicated no-name people who are out there doing the work and getting things done". When asked what demonstrates an active board, participants cited evidence of a clear strategic plan with board input, assistance with fundraising, existence of policy statements, and a list of committees in which members take part. Some participants also felt that personal donations from board members demonstrate true commitment. If members are unable to give due to personal finances, then there should be evidence that these individuals are very active in fundraising. iv. Board Member Election Process and Terms: Some participants stated that it is important that the election process for board members be open and transparent. The process should allow for anyone with a genuine interest in the organization to run for election, and that members are not hand-selected by the founder or president. Although longevity of board and executive committee members can represent stability, some participants felt there should be some degree of healthy, managed turnover. One participant suggested this could be achieved through staggered board and executive committee terms to enable an AWO to maintain a core group of familiar people at any given time. Another foundation suggested board members should be limited to a 3-year term, renewable once, for a total of 6 years. In general, it was felt that a managed turnover of all board members at pre-determined time limits was important, 70 even if the directors were extremely effective. This would allow for an exchange of individuals, at least temporarily. v. Number of Board Members: Half of participants said the number of board members is important, and the ideal size depends in part on the size of the organization, with recognition that boards can grow with the organization as it develops. A few participants suggested that an ideal board should have around 8 people and some said the minimum should be 6 people. This ensures the board is large enough to have informed discussions and not be driven by one person. One comment was made that when boards go beyond 15 individuals, these members are not likely active. A few participants felt that having the minimum number of three board members was inadequate and a big red flag (and an even bigger red flag if they are family members of the Director). As discussed previously, however, the ideal should really reflect a number representing the right mix of individuals to provide diversity of input. Key P R E P Framework Criteria In attempt to see which criteria were of highest priority and the most likely to be used by donors, the following hypothetical question was posed to 17 of the 24 participants at the close of most interviews: "You are an individual donor with $3,000 to give to an AWO. What information could that AWO provide you that would convince you they are the right organization to which you should donate your funds. In other words, how should they demonstrate their productivity, effectiveness and ability to make a difference? If possible, pick three to five criteria that you would want to hear about or see listed on a web site that rates charity performance." Since the hypothetical question was posed at the end of the interview, participants had time to think through all the issues raised during the discussion and better understand and articulate what was most important in determining effectiveness. Participants cited 40 criteria in response to the hypothetical question. The most common nine criteria were: 1. demonstration of past accomplishments and precedent-setting work 2. support from, and of, others (i.e. peer collaboration, references, partnerships, funding) 3. a clear mission statement and objectives 71 4. experienced management background and abilities 5. practical and measurable evaluation methods and criteria 6. sound finances 7. community support and involvement 8. existence of and adherence to core values and policies 9. an intelligent, achievable and usable strategic plan In analyzing and coding these responses, three of the four PREP pillars were captured as follows: Philosophy - clear mission statement and existence of core values and policies Efficiencies - usable strategic plan, sound finances and practical evaluation methods People - experienced management, proven accomplishments, support to/from others, and community involvement. Note that Red Flags were not mentioned because the hypothetical question asked participants only for positive indicators of performance. 3.2 Outcomes In addition to performance indicators, participants were asked to comment on which outcomes they felt AWOs should ultimately strive to achieve. Participants were asked to provide ideas about 1) general outcomes that applied to all AWOs; 2) outcomes for specific activities such as humane education and advocacy; and 3) outcomes for specific types of AWOs - i.e. shelters, wildlife rehabilitation, farm animal advocacy and sanctuaries, etc. In total, participants cited 106 outcomes across nine different types of AWOs and AWO activities. Participants appeared to have difficulty understanding how outcomes differed from outputs and other performance indicators. The outcomes they cited did not always meet the definition of an outcome as explained in the introduction (i.e. the actual impacts/benefits/changes resulting during or after program participation). Nonetheless, I have listed their most common responses to provide an understanding of how participants understood outcomes for their sector. 72 Percentage of respondents has also been included since questions pertaining to performance outcomes were posed to all but one participant (either in the questionnaire, the interview, or both) and thus were suitable for statistical tabulation. i 1. General AWO Programs and Activities: Participants identified 16 general outcomes that apply to all AWOs. The four cited most commonly, and the percentage of respondents who cited this response, were: i. education and outreach - 25% ii. campaign success on short and long term goals - 21% iii. evidence of past accomplishments -17% iv. change in public opinion or behavior - 13% Participants were asked to be more specific about what outcomes educational and outreach programs should try to achieve. 2. Educational Programs: Participants identified four outcomes for education programs. The response to this activity was low, and thus the two most "commonly" cited were: i. change in public behavior - 8% ii. post-education follow up - 8% When participants were asked to further comment on what kind of "change in behavior" they were seeking, discussions focused on the difficulty in measuring the effect of educational programs since changes in behavior and opinion were long term in nature. "F6 -1 think [humane education] is the biggest con job in the world because I think there are very limited ways to measure what is your impact on teaching a third grader a more compassionate view on life. And so they throw money at it and they can't measure the results. I'm not [saying good things aren't] happening, but you don't know...But [humane education is] also one of the cornerstones of our movement ...I mean I understand the message, I just don't see how it's getting quantified and measured". When asked what post-education follow-up outcomes they would like to see, some participants cited approaches that would help identify that a change in behavior occurred in the short term. Examples included the use of essays, surveys, contests, and written reports pre- and post-education to examine changes in the individual's thought processes, opinions and behavior. 73 Another participant suggested looking to see if those involved in the educational event later became active in volunteering, such as collecting money, helping at events, writing and signing petitions and distributing literature. Another said a sign of an effective education program is if there is interest and collaboration from other organizations and charitable groups to incorporate an AWO's educational programs into their own programs (e.g. if an AWO's youth-oriented educational programs are adopted by other organizations such as the Boy Scouts and Big Brothers/Big Sisters groups). 3. Advocacy Programs: Participants identified 14 outcomes for advocacy work, with the two most commonly cited outcomes being: i. amendments to legislation and case law - 25% ii. increased public awareness - 17% Some participants commented, however, that legislative change is typically a long term goal and if achieved, one for which many AWOs could claim credit so no single organization could link the change directly to their own efforts. To assess which AWOs may have contributed to this goal, some participants suggested that donors verify that an AWO has the necessary connections to the media, academia and political arena in order to influence government. 4. Companion Animal Shelters and Adoption Centers: The outcome most participants cited for any shelter offering adoption services is the number of animals helped. To refine this response, participants were asked what they meant by "helped", which resulted in 15 different outcomes. The five most commonly cited outcomes were: i. number of animals adopted (relative to numbers admitted) - 58% ii. number of successful adoptions - 50% iii. reduction in number of animals euthanized - 42% iv. number of animals spayed/neutered (both pre-adoption and of community's pets) - 38% v. number of animals admitted - 33% 74 A few participants also cited proactive outcomes, notably: the enactment of laws and restrictions surrounding breeding and importation of dogs, and improvements in pre-adoption screening of owners and animal behavior training to reduce returns. Another item that was cited as an outcome by some participants was quality animal care and quality of life during an animal's stay at a shelter. "AW015 - When an animal comes into this system...are the basic necessities of care...being provided in the way of veterinary checks, feeding, quality of the food, cage cleaning, and exercise."" Many participants also wanted to see a decrease in trends related to animal surrender, stray populations, and adoption returns. 5. Animal Rescue and Seizure Organizations: Animal rescue and seizure was assigned its own category because of the specialized nature of this work (investigations, arrests) and the typically compromised condition of the animals being seized. Participants cited four outcomes, the two most common being: i. rehoming of seized and rescued animals - 29% ii. increased investigations, prosecutions and convictions - 29% However, a few participants recognized that in the long term, if the advocacy and education efforts of other groups are effective, we should see the number of investigations, prosecutions and convictions diminish over time. 6 . Wildlife Rehabilitation and Conservation Organizations: Participants cited 17 outcomes for wildlife rehabilitation and conservation organizations. The four most commonly cited outcomes were: i. increased animal survival/ release rates - 58% ii. education - 42% iii. reduced human-wildlife conflict through public outreach - 33% iv. increase in wild population and species recovery - 29% 75 To ensure positive survival/release outcomes, some participants stated AWOs must also demonstrate they are involved and working with government and environmental groups to protect and stabilize habitat and wilderness areas to which the animals were being returned. This would improve "real" survival rates and the ultimate goal of recovery of wild populations. When asked about public outreach and education to reduce human-wildlife conflict, participants indicated they would like to see increased person-visits to the facility as well as increased calls to the "abandoned animal" hotline. This would demonstrate their educational efforts were reaching a broad audience and should thus reduce the number of conflicts and animals brought to the center. Although some centers do not allow visitors to the facility if they believe it interferes with their animals' rehabilitation efforts, a few participants commented that every facility should provide some level of visitation or viewing opportunity as a public education initiative. 7. Farm Animal Welfare Organizations: Most participants were unable to provide adequate responses to this category because they were not informed on the issues. Those participants that did respond suggested ten outcomes. The three most commonly cited outcomes were: i. improved animal health and housing conditions on farms - 33% ii. increased consumer education and awareness about farm animal issues - 33% iii. increased consumer purchases of organic products - 29% Many participants felt that increased purchases of organic products demonstrated consumers were concerned about how farm animals were raised, both for their own health reasons and the animals' sake. When the question was posed regarding outcomes for accreditation organizations involved in humanely raised food labeling programs, those who responded cited two outcomes: i. increased number of animals protected/certified under the program- 25% ii. increased number of producers joining the program - 21% 8. Zoos, Aquariums and Related Accreditation Organizations: When participants were asked to recommend outcomes for zoos and aquariums and the groups that accredit them, responses 76 were quite divergent. Some participants were vehemently opposed to the existence of zoos and aquariums, making comments such as: "F1 - the very act of capturing that animal...the makes me crazy...shut 'em down". Others felt zoos and aquariums served an important educational purpose and conservation function. Still others, while opposed to the concept of zoos and aquariums, recognized such facilities were here to stay and thus efforts to ensure the animals' quality of life were of utmost importance. Fifteen outcomes were cited with the four most common being: i. improved housing, quality of care and staff training - 50% ii. increased accreditation/regulation of zoos and aquariums - 42% iii. in-situ conservation efforts to improve release rates and survival - 17% iv. education - 17% Some participants strongly felt that education was a vital component of any aquarium/zoo. Participant F2 felt all zoos and aquariums should have to conduct and publish research to improve our understanding of the animals' needs and how to conserve them in the wild. He felt this gives the zoo and its animals' lives purpose, rather existing purely for entertainment purposes. Participant F1 suggested that effective zoos and aquariums should strive towards the use of technology to teach people about wild animals and marine life, thereby reducing the "need" for captive animals in the first place. 9. Laboratory Animal Welfare and Protection Organizations: Questionnaire and initial interview results showed that most participants were not familiar enough with issues relating to laboratory animal welfare outcomes, so the question was dropped from some participant interviews. For those who did respond to this question, eleven outcomes were cited. The two most commonly cited were: i. fewer animals used in research - 29% ii. humane conditions and quality care for laboratory animals - 17% 77 To reduce the animals used in research, one participant suggested that, to be effective, animal welfare and animal rights organizations should better educate the public on what brands and products are tested on animals, and publicize exactly what the tests entail. She felt this would motivate consumers to purchase substitute products that are not tested on animals and the resulting loss in consumer demand for animal-tested products would then motivate companies to more quickly adopt alternative methods to testing on animals. 3.3 Comparative Evaluation The performance indicators and outcomes noted in 3.1 and 3.2 suggest how donors can evaluate an organization's effectiveness. However, how might one compare an organization's performance against another? More importantly, should one compare performance of AWOs? Participants were asked for their input on the concepts of standards, benchmarks and accreditation in order to ascertain the pros and cons of such comparative measures. The majority of participants supported the idea of comparative evaluation, standards and benchmarks, and were curious as to which criteria would present the best possible measures for both external and internal self-evaluation of effectiveness. "AW01 -1 think that benchmarks would be very, very helpful and I wish we did more of that...I think it would be great to see how we stack up against other charitable organizations, [and not just] in the animal welfare sector." Some cautioned, however, that the impact of an AWO's geographic region would have to be factored into the results. "AW015 - I think it would be nice if there was some kind of standard. I think the closest we could get would be to agencies that are very similar. But so much is dependent on your community and your funding and the government and attitudes and ideologies. Southern US is going to have completely different attitudes than the Northwest." Many participants agreed that donors would benefit from some basis for comparing effectiveness amongst organizations, and that standards would make this comparison easier. "F3 - [Standardization] is something within the foundation world that has been talked about a lot... and how it would make things easier for everyone, but 78 then it's been just that - talk. I think it's worth pursuing. Whether it's doable is a real question... I think we would try to take advantage of standardization if someone else led it." Some participants felt standardized performance measurement would also bring about a more transparent and professional approach to demonstrating and proving effectiveness. "AW08 - Performance metrics or outcome measures are very important. I think that for too long, people have existed on their anecdotes and their stories." A few others indicated it would save time, set expectations and improve the decision-making process, and thus lead to more informed funding decisions. "F1 - The benefit [would be] less room for subjective/political judgments by the decision-makers and that would also be the disadvantage. What I'm hoping we can get out of this research is a final questionnaire to save time but still leave the options wide open that we can ask the right questions based on what we think at this time." Although the majority of participants felt standardized performance evaluation and benchmarks would be worthwhile, some added that the sector would likely shun such comparative methods. "F6 - [Our sector] is not very quantifiable. We potentially try to disassociate with measurable results and I personally think we should have more measures". It was suggested that the reluctance stems from self-protection. "F6 - The Asilomar Accords was an interesting exercise because it does offer an opportunity for independent analysis, and the organizations that are not doing well that have a dominant place in our industry don't want that comparison. They don't want to be embarrassed. They don't want to be put on the defensive. And it's totally understandable. I wouldn't want to be if I was on the losing side of the comparison. It will be interesting to see how far the Asilomar Accord goes because of that basic flaw." In general, it was recognized by many participants that, if used in conjunction with other assessment tools and techniques, standardization provides an additional framework within which they can establish some baseline expectations in order to assess and compare organizations. Participant F4 indicated that if foundations instituted a benchmarking exercise, the important next step would be to incorporate the capacity to improve within the program. In other words, once an 7 9 AWO has been "evaluated", if they fall below the baseline or the average in a certain area, there needs to be some guidance or resources available to help that AWO figure out how they will address their shortcomings over time. This ensures AWOs derive value from the benchmarking and assessment process rather than being penalized for participating. He added that this would ensure the lucky few who perform well initially aren't the privileged few forever, and it would allow other AWOs to grow and join the top ranks as they improve. Despite the support for establishing benchmarks, the majority also expressed concern about standards being too broad and that, in assuming the same benchmark for all AWOs, there would be major gaps in the evaluation process and outcomes. Some participants were skeptical about the use of comparative evaluation in their sector given the subjective nature of certain criteria and the diverse missions, ideologies and constituents of AWOs. "AW015 -1 think the hardest thing on effectiveness is getting a good measure, a measurement that can be compared, an outcome that is not just number of animals through a system but how they went through the system or how you were educated." Some reluctance was also related to the difficulty in knowing how and what to measure. "F8 - We do look for bang for the buck and it's adjusted by organization by both size and type. But really this is a very organic process. I mean [with this research] you're trying to quantify [performance], and congratulations [to you] for trying to quantify it, but you know it is a very difficult process to quantify." Some participants commented that the different missions and motives of donors also add to the complexity of deciding which indicators should be compared. "AW05 - it's all about who it's relevant to and why. To the general public there might be different measurables than to a corporation than to a foundation." Still others stated they really did not know which benchmarks would have the most impact on outcomes. "AW08 - I've been looking for practical benchmarks now for a while and seeing very little that's actually out there...yeah I know I need financial statistics, but what financial statistics? What works, what doesn't work? If you measure "x", does it produce the outcome "y" that you're looking for or does it produce a different outcome?" 80 A few participants also worried that a framework for standards and benchmarks might encourage AWOs to tailor their programs to the criteria, thereby stifling creativity. Suggested Benchmarks Despite the skepticism expressed, all but four participants were able to give examples of criteria they felt would make good performance benchmarks. As discussion progressed, many participants expressed a keen interest in using benchmarks and were anxious to see the final results of this study. "AW012 - There are just gut feels and no industry benchmarks exist. That's why I'm dying to really see this pull together at the end." When asked which figures and indicators would provide useful benchmarks for guiding management decisions, for making comparisons amongst AWOs and for demonstrating an AWO's effectiveness to donors, participants cited 18 criteria that they felt were common to many AWOs. Participants' responses fell into three primary benchmark categories: 1. Overhead; 2. Fundraising; and 3. Animal Care. 1. Overhead Benchmarks With regard to benchmarking overhead costs, participants focused on two main themes: i) administrative costs; and ii) employee salaries. i) Administrative Costs: All participants were familiar with the existing non-profit industry norms and guidelines regarding the 80/20 split between program and administrative expenses respectively. Those participants who referenced their own organizations' breakdown of administrative costs as a percentage of budget cited a low of 9.5% and a high of 20%. A few participants commented that any organization running below 10% might not be allocating sufficient funds to their administrative area and thus not managing as effectively as they could. ii) Employee Salaries: Some participants indicated employee salaries, numbers of employees, and the programs they conduct are helpful in giving donors a sense of the overall size and scope of an organization. A few other participants suggested salaries and numbers could be a potential benchmark for donors to assess the balance between the amount of 81 income an organization raises versus where the money is going. In other words, is the ratio of employees appropriate for the income and budget of an organization? Furthermore, a few other participants commented that a salary and staffing benchmark could also be used by management to compare their relative effectiveness on certain activities against other organizations. In other words, is the group with fewer resources actually more effective on a cost per outcome or output basis? 2. Fundraising Benchmarks The majority of participants supported the idea of fundraising benchmarks since they were already commonly used in the sector. However some said a single standard could not be established because of the diversity amongst AWOs and the activities themselves which have drastically different costs (e.g. auctions versus lotteries versus direct mail campaigns versus grant proposals). Some participants also indicated that fundraising costs are affected by an AWO's market niche, stating that certain causes are more appealing to the general public than others (e.g. dogs and cats versus farm animals) and thus attracting donors for companion animal issues would be less costly. One foundation commented that rather than a specific benchmark, they simply look to see that fundraising costs are "pretty low", and if they seem to be far out of line, that is when this number becomes important. Nonetheless, participants did indicate three areas where they were aware of their own internal and/or industry benchmarks: i) costs to raise funds; ii) marketing returns; and iii) diversification of funding sources. i) Costs to Raise Funds per Dollars Raised: It was suggested that rather than focusing on total cost of fundraising, or the percentage of total expenses allocated to fundraising, a more fair comparable for donors to assess would include a calculation based on total overhead and fundraising expense divided by the organization's total funds raised for the year. This would account for the additional staff and systems costs that are associated with more established AWOs while acknowledging that larger AWOs are able to raise more funds. Thus judging the return on investment provides a more fair comparison of effectiveness amongst organizations of different sizes and history. 82 Some participants revealed that fundraising costs should range between 15-35% of total funds raised, and that 20% is the accepted industry average. The high end of 30-35% was stated as being more realistic, given the many competing causes and the feeling that it is harder to get through to donors now than before. Some also stated that market penetration and the age of the organization greatly affect this figure because more established AWOs typically have more staff supporting fundraising activities, have a larger audience to reach and have to invent new ways of interesting their existing audience, thus increasing their costs. "F6 - Once you get greater and greater market penetration, capturing the untapped market becomes more and more difficult and I would say more and more costly....If you are a start-up, your fundraising costs will be low, so your return on investment should be very, very high". When participants were asked if they knew what ratio would indicate an effective fundraising program, only a few were able to give specifics. Examples included the benchmark of employing 1 fundraising staff person for every $1 million raised, fundraising costs of 15-20 cents for every dollar raised for an established program, and break even on costs for dollars raised for a new program. For a specific fundraising activity such as direct mail, one participant stated a cost of 60 cents for every $1 dollar raised. ii) Marketing Returns: When participants were asked what demonstrates an effective fundraising and marketing campaign, half of their comments referenced response rates to direct mail efforts. It was recognized that new acquisition mailings would always have a much lower response rate, and that it generally fell between 1-3% of recipients responding, and hence why acquisition mailings are more costly and typically only break even at best. For existing donors (i.e. renewal mailings), some participants cited response rates in the range of 5-20%, with 10% being the most commonly cited result. iii) Diversification of Funds Raised: As mentioned in the section on Efficiency, participants felt it was important that AWOs diversify their funding sources to ensure they do not rely on one or two key contributors. Most participants indicated AWOs should strive to have funding coming from individual, corporate, foundation, and in-kind donations, with individual donations representing the largest percentage. Only a few participants cited a benchmark, stating that on 8 3 average, 10-20% of funds should come from foundations, and that the majority of funds should come from individuals, with no other source representing any more than 20% of total funds. Although these benchmarks help the marketing and fundraising staff of an AWO judge their own effectiveness, participants felt they typically were not very useful to donors. In fact, some foundations stated that it would be nice to hear about these figures, and they agreed they were important in terms of verifying anticipated revenues, but they really were not expecting AWOs to be able to measure and report on these results. They believed only those AWOs with a professional marketer on staff would know of such benchmarks and what was considered successful. Thus while foundations found these figures useful and interesting if provided, they did not view these figures as a critical component or deciding factor in awarding grants. 3. Animal Care Benchmarks Half of participants indicated an interest in establishing benchmarks for animal care. In fact, many of the quality of care indicators discussed in the section on Philosophy offered criteria well suited for benchmarks and baseline standards. Some participants, however, felt benchmarking animal care would be too controversial given the effect of one's ideological stance on acceptable levels of care. Despite participants' initial doubts that this undertaking could be accomplished, there was still a desire to see if in fact it could be done, particularly by foundations. "F3 - I look at [adoption numbers, days in care, euthanasia rates] but I don't look at it in a systemic way ... We don't have a benchmark from which to go, and actually I don't know if they exist. I [look to see if] the percentage just intuitively seems really high or low...Standards on that...would be interesting to see... It seems to make sense to have understanding of what the sector as a whole thinks is appropriate". Another foundation commented on their frustration with the lack of specifics and relevant information AWOs provide on animal care figures. "F6 - I am just absolutely amazed at how hard it is to find information about how many lives they're saving and I guess what I want to know is [at the most basic level] how many animals are you taking and how many lives are you saving?...Some will say we're saving all our adoptable animals or some will say we saved 800 animals last year but I...would like to know the context of 84 what realm are you operating in, and how successful are you in that realm, and you just can't find it." Yet another foundation stated: "F7 - We have a spay-neuter program and it's made a difference for sure,... so it's helping, but how much real difference is it making? The only way you can do that is statistics, and interestingly enough, people who work in the shelters don't realize the importance of statistics." When referring back to the comments on Philosophy from the PREP framework, combined with comments on standards and benchmarks for animal care, ten criteria presented themselves as areas where results could be easily quantified, with three of them cited most commonly, and four others, although referenced by only a few, worth mentioning: i. Adoption Rates and Returns: For companion animals, half of participants cited adoption rates as being the most important statistic which an animal shelter could report. One participant expressed frustration with the fact that although AWOs often state how many adoptions they have made, they often fail to state how many successful and unsuccessful adoptions occur. He indicated that on average, about 15-20% of adoptions fail, although no analysis has been done to figure out why, or to see if this is normal for most shelters. ii. Death Rates: Half of participants supported benchmarks for death rates (natural death, survival rates and euthanasia), despite the sensitivity of the issue. With regard to euthanasia, some felt it was important to determine benchmarks as a percentage of animals brought into the shelter. Others, however, cautioned that such benchmarks would have to be adjusted to account for whether animals in the facility are companion, wildlife or farm animals, and for their state of health upon admission. Others stated it would be unfair to compare euthanasia benchmarks between no-kill and open admission companion animal shelters and therefore benchmarks would have to indicate whether or not the facility has a choice in the animals they receive. Participants from wildlife rehabilitation centers worried that if their survival rates and euthanasia statistics were benchmarked, they could be greatly misunderstood by the general public who might think their death numbers are too high. Centers often have to euthanize wild 85 animals upon arrival because of their compromised health and injuries, if the animals cannot survive without their mothers, or if the animals are carrying an infectious disease that would put the other animals (and people) in the facility at risk. They said, therefore, without properly understanding the context, the publication of such statistics could be detrimental to the organization's image. Participant F8 suggested, however, that the average survival rate at a wildlife rehabilitation center is around 58%, and anything below 50% is not good. iii. Days in Care: Days in care was one of the most important criteria cited by participants when they were asked for indicators of quality care, and half of participants thought it would be a useful benchmark. However, only one participant quoted a figure, suggesting 6 months was too long for shelter stays, and that such a benchmark is confounded by ideology, particularly when it comes to no-kill versus open-admission shelters. "AW08 - If on average an animal stays in your shelter six months or longer, that's a dangerous thing, because kennel housing for dogs is not...a particularly good thing for the dog. Many people will think it's fine,...[and] the tension between the no-kill community and the open admission shelter tends to be based upon the idea that, on one side, euthanasia is the worst possible thing that could happen to an animal, versus on the other side that animal suffering is the worst possible thing that could happen." iv. Space per Animal: A few also a suggested benchmarking the maximum number of animals kept at a shelter or sanctuary depending on space availability. They said one could look at how many animals are on the property, the size of the enclosure(s), and the number of staff, and roughly estimate whether the figure is intuitively too high. v. Staff per Animal: A few participants indicated benchmarks for staff per animal could indicate whether adequate care and attention is being provided to animals in the facility. Another participant raised the issue of the human medical system where standards and benchmarks are frequently used to establish appropriate nursing care per patient. vi. Cost per Animal: One participant suggested that a useful benchmark for shelters and rescue groups would be the cost per animal. This participant recommended adding up an AWO's total expenses and dividing this total by the number of animals helped. If the ratio comes in at $100 or less per animal, this organization would be considered to be very 86 effective, whereas a ratio of $1500 per animal was extremely bad, unless the group delivered a lot of educational programs which would inflate the cost per animal. Thus the participant stated this ratio was really better applied to smaller AWOs where humane education and ancillary activities to animal care are not provided. vii. Shelters per Community: An interesting idea raised by a few participants was a benchmark surrounding the number of shelters per community, based on the size of the community, to determine whether there is a need to fund additional or duplicate facilities. Accreditation Systems and Programs Following discussion on standards and benchmarks, participants were asked for their comments on accreditation systems and programs. Some participants were in favor of such a system. "AW01 - I'm a very strong supporter of the idea of a credible, national organization that can speak for all humane organizations by going around and doing a type of quality control of humane organizations in the country and be able to grade them. I think that would go a long way to helping organizations and helping donors...recognize which organizations are ethically sound for one thing, and [which] are actually involved in delivering programs for a second thing. There would be some real value attached to being a 3-star organization. I don't know if we'll ever get to that point, which would be a pity. But I do think that would make donors feel much more comfortable and it would give much more purpose to the humane movement if there was something like that." Another participant summarized the need for accreditation and standards as follows: "F1 -1 had such trouble filling this [questionnaire] out ...[and] knowing where I was going until I got to this end bit here and I started writing down accreditation all over the place which is related to performance evaluation, which is the key." This participant went on to speak about the need for the accreditation standard 1) to have national input and approval to remove personal, organizational and regional biases and standards; 2) to be client-centered - i.e. centered on the needs of the animal; and 3) to be reviewed by peers. "F1 - It has enormous potential. Eventually with the right data, which comes out of the accreditation process, you can say 'This is the way it should be done' and it makes our job of providing funding much easier. It would cut out a 87 lot of this enormous problem about who's doing a good job. We haven't got a clue, unless you actually go and visit. And what right have I got to impose my feelings upon anybody? What right have I got to make my beliefs the standard? That's why you need a national standard [derived and inspected through] a peer process. The manager from [one region's] humane society can come and inspect another. It's not punitive; they're there to help, to make it better". A few participants felt accreditation was particularly important for AWOs providing hands-on care to animals. A few participants from the wildlife sector stated they are looking to set up an accreditation system for wildlife rehabilitation in their regions. In one case, the system will ensure through inspections that certain standards of animal care are adhered to. In another case, they are looking to set standards for training and education requirements of those who work directly with the animals. Conversely a few participants cautioned that an accreditation body would likely never work because of the diversity of the membership: "F9 - The danger in what you're suggesting, having a validation body, is that it becomes too big and the criteria too broad because it has to fit everyone". Participants were asked who they felt should conduct performance evaluations. They were given suggestions of the AWO themselves, a national animal welfare organization, the funding agency or an independent third-party audit firm. Many participants said the AWOs themselves should conduct the evaluation, many said it should be the role of foundations, and many said it should be conducted by a third-party individual who is independent of the AWO. "AW01 - [It should be] someone strong enough in terms of understanding our organization [and who] has grounding in animal welfare, but someone who is not directly affected by the plan." In other words, evaluations should be conducted by an academic or an expert in the field who is not a member of the AWO's staff or board of directors. Similarly, some said a peer review would be meaningful. "AW03 -1 don't think there are any business/industry background type people that really get what it is that we do. A lot of it is hard for someone from the outside to really quantify, particularly around legislative campaigns." 8 8 When participants were asked who should pay for the evaluation, half of participants felt that if the foundations were the ones requesting the results, they should be the ones to pay for it. Some recognized, however, that the AWOs themselves would benefit from the planning and review process, and so perhaps they should pay for part of the cost. However, a few participants stated that with budgets as tight as they are, and lives to save, it would be hard to justify allocating donor dollars to this endeavor. 3.4 Funding Conflicts To determine where funding conflicts exist, participants were asked to identify gaps between what AWOs need versus what donors and foundations typically fund. Specifically regarding foundation funding, participants were also asked to identify bottlenecks in the grant submission and approval process. Participants also raised issues concerning the challenges they face in deciding what projects and organizations to fund. Participants cited 74 different issues that fell into four main categories: 1. funding policies and limitations; 2. bottlenecks in the grant process; 3. challenges in the decision-making process; and 4. funding needs. 1. Funding Policies and Limitations There were two main concerns pertaining to funding policies and the limitations on the types of funds available: i. Emphasis on new programs - Many of the AWOs applauded the efforts of foundations to fund new ideas rather than funding the same things or same organizations year after year. However, of particular concern to some AWOs was the common policy of foundations to fund new ideas and new programs to the exclusion of existing ones. AWOs stated that although they are keen to take on new programs, their first responsibility is to keep the operation going and pay for the basic items like maintenance, telephone and salaries. Once that is stable, they can look at new activities. Participants felt that putting too much emphasis on funding new programs forces AWOs to constantly reinvent themselves rather than concentrating on 89 existing programs and what they are doing well. This unnecessarily redirects resources and » creates inefficiencies. "AW015 - We would like to think that we've got some of the programs and services down now, and the funding really needs to support and enhance those, not to invent something new so that the funder feels [they are] going forward. So to me one of the biggest challenges is what the funders want to be financing, and restricting that money for something new and different as opposed to improving the system. We could ...provide more services, but it's the basic service we already know is needed". Some AWOs felt the interest in new program funding was driven by the foundations' boredom with existing activities and their interest in being involved with something new and exciting. "AW014 - They get tired of you after three years so they don't want to fund you anymore and it's not that you're not doing good work, it's just that they're tired of funding you and they see something else and off they go." A related frustration concerned foundation policies that prevent AWOs from re-applying to the foundation in subsequent years. "AWO10 - With a lot of funders you apply once, they give you money,...and they say 'don't ever talk to us again.'...When there is a shrinking pool of foundations and sources supporting animal welfare organizations in the first place and then if out of that limited universe you can only go to them once, within a few years you're tapped out". A few participants also noted that the short 1- to 3-year time limits associated with new program funding do not fit with what the movement and its campaigns need. They stated that campaigns to change opinion require long-term repetition of the message. "AW014 - [Our campaign on the egg industry is] a ten year project. We're hoping that it will get funding again [from the same or other groups] but we're going to have to come up with something new and innovative within the project itself, a sub-project I guess, before it will be funded again. That's frustrating...because's long term projects like ours, ten to fifteen years even, [that get the message across]." AWOs felt the only reason foundations favored short-term projects was because their results were easier to measure, and thus donors could gain immediate feedback and a sense of satisfaction and comfort with their choices. 90 When foundations were asked to explain their preference for funding short-term and new projects, a few admitted that short-term programs were in fact easier to measure than operational items or long-term activities. However, they added that they liked funding short-term projects because such projects set AWOs up for success by allowing them to develop a proven track record which would allow AWOs to secure funds from other foundations in the future. As well, they said long-term funding carried too many uncertainties and changes, especially changes in leadership and natural disasters or recessions which could put the program and the AWO in jeopardy. Hence foundations felt the long term presents too many unknowns. ii. Lack of general funding - Some AWOs also commented on the lack of general funding for core operations, capital and capacity building. "AWO10 - there's no one who's interested in helping you pay salaries, keep the lights on, clean the floors, all that sort of stuff. And if no one is willing to help support or acknowledge or honor the nuts and bolts of the organization, then none of us are ever going to move forward". Some participants felt foundations should consider contributing towards employee salaries in order to attract top talent and raise the sector's leadership abilities. "AW03 - a big part of the. funding we do need is for salaries, and it takes people to do this work and you can't expect people to live in poverty". As for capital funding, one participant suggested it was inherently biased towards large organizations and precludes small organizations since most capital grants require the organization to own the land or building on which improvements are being made - something most small organizations cannot afford. When foundations were asked why they disliked funding general, operational costs, mainly they worried the AWO would be placed into a position of perpetual dependency on the foundation. They said that when the time came to withdraw financial support, pulling support for a program was one thing, but no foundation wanted to be responsible for shutting off the phone and electricity or laying off employees. It was also felt that financing operations was not 91 an appropriate role for a foundation, as AWOs should be able to raise basic operational costs from their community. "F7 - [Unless you're in a remote area,] if you can't support yourself locally with the local population, then you're doing something wrong, or you're not needed. So many of these organizations are duplicates...Everyone is taking money away from the ones that really need the money and who are really making a...difference." Another foundation said that it depends on their relationship with the AWO. "F3 - I think it just depends on the individual decision maker's feelings and relationship with the organization. I think that the longer we've known an organization, the more likely we are to provide general support. There's that trust being built." Yet another foundation stated that while they don't advertise it, they will sometimes fund general operations up to 20% of the grant to help launch a great idea, so long as they feel the organization will garner donor support from the general public in a few years. 2. Bottlenecks in the Grant Process When foundation participants were asked what areas they felt presented the most serious bottleneck in the grant process, 27 items were cited that can be summarized under one broad category - poor grant submissions. i. Poor grant submissions: Some foundations said a combination of poor writing skills and a misunderstanding of the foundation's mandate were the main problems. Proposals appear to be thrown in to meet a deadline and the resulting lack of clarity makes it difficult for foundations to establish the goals and merits of the proposals. Not only is this frustrating and time-consuming for foundations, but it is the main cause of applications being rejected since foundations believe that if AWOs lack the resources to properly complete the application, they likely lack the resources to complete the proposed project. AWOs stated they often do not have time to sufficiently research and write up applications. This was primarily an issue of having adequate and knowledgeable resources to meet some of the complex requirements of foundations. 92 "AW06 - we were talking to a foundation that ...wanted to fund things that had a measurable impact on social views with regard to animal welfare ...We really had to put our heads together [to figure out how we could achieve and demonstrate that] and we just didn't have the resources to follow that through". Many participants agreed that improvements in grant writing skills were needed in the sector, especially for grassroots organizations with minimal staff or those run by volunteers. However, AWOs were concerned that foundations were paying too much attention to the quality of the writing rather than the merit of the proposed idea. "AW014 - The organizations that know how to write a grant are probably the ones that have the money to hire a grant writer... I think that it's entirely possible for somebody to write a grant proposal for a really lousy project and make it look fantastic, just because they know how to articulate." Related to writing quality is knowing what information should and should not be included in proposals. A few foundations stated that they do not want lengthy financial printouts and that one to two pages showing budget and revenues/expenses are sufficient. A few others said they wanted more comprehensive financial statements with accompanying detailed explanations. A few also said that values for donations in kind should be eliminated since the value is often inflated. A few said they do not want a lot of media clips, magazine articles and brochures, and they do not want anecdotal or emotional stories about rescued animals. "F8 - they should leave out the individual stories of animals they've helped. I know the story, little Missy the dog that was saved, and of course it's great, but it just does nothing to tell me about an organization...In fact it's almost insulting to my intelligence that they think that this is going to change something." A few also said they disliked seeing AWOs spend a lot of money on producing a fancy, expensive proposal as it only shows the AWO's priorities are out of line. As well, a few foundations said AWOs should not assume that foundations only award grants equal to or less than the amounts shown for previous recipients. Some foundations will step up for an innovative program with big impact; therefore, AWOs should not gear their submissions around the previous years' funding amounts. 93 As to the content of proposals, foundations said that AWOs should focus on providing a realistic, detailed project cost with appropriate statistics and community demographics, along with an objective assessment of past organizational performance and program effectiveness. They indicated AWOs should always include the background and experience of the team that will be managing the project, particularly if they have demonstrated success on similar programs elsewhere. As well, if an AWO has approached other foundations for funds, they would like them to include these foundations' names and the amounts requested in their application because having another funding organization or major donor(s) involved provides funders with a sense of comfort knowing they will not be carrying the project on their own. 3. Other Challenges in the Decision-Making Process When it comes to selecting which projects and organizations to fund, participants commented on four areas that challenge and complicate the decision-making process: i. Impact (numbers) versus emotional appeal - Many participants said one of the most difficult decisions for any funder is choosing between a program or organization that will affect the most animals versus what emotionally appeals to them. "AW014 - [If I were] a funder something that I would look at is how many animals is it really helping and how big is the problem. Is it really a problem?...Relatively speaking in our city, relative to North America, we don't have a problem in terms of cats, and yet we have hundreds of people helping cats." AW07 noted the contrast between the hundreds of millions of farm animals killed each year versus the several hundred thousand seal pups that are killed, yet the public has greater interest in banning the seal hunt despite the fact that, relatively, one could argue they suffer less. "AW07 - Almost all those [farm] animals suffer more stress than the seals will ever suffer...People give to things that don't affect them personally. If you think of it, those seals live a completely free life until seconds before they're killed. No trucking, no cages, to which all these other animals are subjected." This participant implied that ending the intensive farming of animals would impact the general population directly and personally (i.e., it would require a change in our diet and cost of food) 94 whereas protecting wildlife generally does not require a direct, personal sacrifice, and hence why people, particularly those in urban settings, find wildlife causes more appealing. Similarly, a few participants suggested that although "saving" wildlife is appealing to donors, the results may be very short term and therefore an inefficient use of funds. Participant AW014 commented on the fact that deciding to fund an owl's rehabilitation is "safe" because it is not controversial, and everyone (the public, foundations, and corporations) will feel good about funding it. She said: "AW014 - [Such decisions] might be completely ineffective except for that one animal, and there's something to be said for that but...what good does it do if we are destroying their habitat?" A few other participants felt the reason why we wish to save wild animals is more important than which animals and how many we are actually saving. "F6 - [People] fund these things because of what it says about us [as human beings]... What is the role of the human species?...I like the fact that we should be a nurturing, caring, compassionate species and so the fact that we try to do good and we fail, to me, is not a terrible disappointment. What would be a more terrible disappointment is not to try." It also noted that what donors fund comes down to what they most identify with. " F6 - I think when you're talking about a cause, it's whether you identify with that cause. Do I identify with wildlife? Do I identify with factory farmed animals? Do I identify with anti-fur campaigns? I think it's mostly very, very personal...It comes down to the spin - what are you doing this for? Are you doing it just for the life that you saved, or are you doing it to send a message? Are you doing it to educate the general population on the importance of life, or are we doing it to basically say that mankind has a stronger responsibility to wildlife than just allowing them to live in the [mountains]? A few other participants suggested donors need to consider projects that are about the relationship humans have with animals and should focus on projects that combine benefits to animals, humans and the environment. This could help make animal welfare more mainstream and closer to the center of society's concerns, and thus more fundable. "F4 - For the size of the funding pie for animal welfare to increase...[the sector] has to learn how to phrase and make its case in terms that are understandable and relevant to broader concerns about community sustainability... Currently animal welfare is off the radar for most [citizens] and... governments, etc. It's just not an issue and it needs to be repositioned, 95 or re-branded in a sense, in order to be understood. I think it's the movement's responsibility to convey how it is that animal welfare is important for early child development... for child obesity, ...for crime rate. It's important for you-name-it....For animal welfare to be responsible it has to articulate its role in that broader picture." ii. Providing fewer, larger grants versus more, smaller grants - A few foundations stated they struggled with whether their grant amounts are really making a difference. Some foundations will have a preset, standard amount for all grants and ensure everyone is treated equally. Other foundations will provide a funding range, but up to a maximum amount per applicant to ensure they fund multiple organizations each year. "F7 - I'm giving a few thousand dollars to people wherever. I get a nice letter or a report [from the grant recipient, but] really doesn't affect me. It doesn't make me feel good. I kind of think that I'm not having any impact, and either I have to give it all away, or I have to reduce the number of grants and forget about all these rinky dink things...And that's hard to judge. I suppose that's one of your questions. How do you value the $5,000 grant to pay vet bills for 30 cats to be altered? You know it helps the people who aren't able to pay the vet bill, but does it have any impact? It's a hard question to answer, so I struggle with that." One foundation suggested that to encourage AWOs to submit more realistic proposals, grant amounts should not be pre-determined per recipient because a foundation may want to consider directing all its funding to one or two key projects. "F1 - Organizations will try to think up [projects] that come up to $5,000, because [that's what the majority of our grants are].... It's problematic. I would rather just look at the merits of a proposal, and then if some barnburner of an application comes across our desk that needs the whole shebang, then we have some decisions to make, but I would rather people not be pre-disposed to certain numbers." iii. Funding new versus known entities and programs - The decision to fund new and smaller AWOs versus known, established entities was another topic of disagreement. One participant commented that although the well known entities may be better able to absorb the investment and have the capacity to get bigger and stronger, the grassroots organizations may have the ability to network better with other AWOs and engage more people in their community in what they are trying to accomplish. A few other participants disagreed, feeling that grassroots organizations are more isolated and less aware of others doing similar work. 96 Still others felt large and small AWOs needed to be brought together to fully benefit the sector as a whole. "F4 - There is a tension between funding large and funding small groups. What we're trying to do as a foundation is to say, without excluding or tipping one way or the other, is there a process by which ... we can bring everybody together and we can hire a facilitator and do consultations beforehand to create an agenda and then say, 'Okay, so what are your priorities as a movement here?' It's about thinking like a movement." iv. Funding "no-kill" versus open-admission shelters - Many participants felt that the controversy over no-kill shelters creates dissent within the animal welfare movement and prevents the animal welfare agenda from moving forward. "AW07 - Until we stop breeding animals or importing them, a no-kill is impractical. Nothing else. It's very misleading. But it's also a very good fundraiser. People give a vast amount of money to no-kill operations." Many participants spoke about the lack of understanding in the general public about the repercussions of funding only no-kill shelters, and that in re-directing their funds from open-admission shelters, donors may in fact be inadvertently driving up the euthanasia rates since animals will continue arriving, but open-admission shelters will have less funds to support the increasing volume. Participants applauded the goal of no-kill but recognized that it is a goal to work towards, and that becoming no-kill in the short term is not feasible without a strategy to address the root causes of animal abandonment in the first place. 4. Funding Needs Participants cited five key areas where they felt donor and foundation funding could be applied for maximum benefit in the sector: i. Collaborative Projects - Throughout the interviews, many participants commented on the lack of collaboration between AWOs and how more collaboration was needed in order to maximize the efficiency of available resources. Participants suggested collaboration would be easiest and most beneficial on public awareness campaigns, on core operational areas such as call centers and hotlines, and on transferring animals to the location best equipped to provide treatment, facilitate adoption, or provide permanent care. 97 ii. Long-term Projects - Most participants agreed that programs that truly make an impact on public opinion and actions tend to be long term in nature, although most foundations are interested in short-term projects with immediate results. A few suggested that if foundations were unable to provide continuous funding over the long term, they may wish to recommend successful programs and AWOs to other foundations and individual philanthropists to secure funding for the AWO as the foundation's own funding term is ending, essentially brokering financial assistance with other foundations and philanthropists to ensure the program has the funds to succeed in the long term. iii. Capacity Building and Organizational Development - Many participants noted that a major weakness in the animal welfare sector is the inability to attract and retain qualified staff and management, and that low pay and little opportunity for training and advancement limits the level of professionalism in the sector. AWOs said individual donors are rarely interested in funding the "human" and operational needs of an AWO, and therefore some suggested foundations could help AWOs build capacity by augmenting the compensation of AWO employees. This could take the form of higher salaries, improved benefits, and/or funding professional training and development of staff through attendance at conferences and courses. They could also help AWOs hire specific expertise, either full-time or on contract. Furthermore, some AWOs require funding for more technical staff to improve overall efficiency through deployment of cost-effective computer-based management and tracking systems and enhancements to website design and content to better educate the public, and to attract and process donations. Related to increased donations, some participants also indicated that financial support to hire or contract more experienced fundraising staff would help them enhance their revenue generation and build reserves so they can become less dependent on foundation and government grants. As well, a few said foundations could assist AWOs with underwriting the costs of advertising or conducting donor surveys, all with a view to increasing donations from the general public. Related to capacity building, a few participants indicated that funding was needed for various kinds of equipment to improve accountability and productivity. Much- needed 98 equipment included: 1) computers and software programs to track operational statistics; 2) digital cameras and projection units to improve educational presentations or to collect/present evidence; 3) medical equipment; and 4) vehicles to transport animals between facilities. iv. Performance Evaluation and Benchmarking - Most participants agreed that performance evaluation, even if only internal, is critical to managing an effective organization. However, since this activity is costly and time consuming, and biased if performed by staff or the board, many participants suggested foundations fund the cost of hiring external evaluators in order to help AWOs identify where they can become more productive while at the same time helping foundations become better at conducting their own evaluations of AWOs. One foundation suggested that foundations consider mentoring newer foundations and philanthropists to help them more easily evaluate and select worthy recipients for their funds. v. Adoption Process Improvements - Given that successful adoptions were cited as one of the key outcomes of any shelter, a few participants felt improving the adoption process, and particularly post-adoption follow up, could significantly decrease the number of returned animals. A few also said customer service surveys would also help AWOs know how their practices and policies are contributing to adoptions, returns and "missed opportunities" (i.e., potential adopters who changed their mind after arriving at the shelter). 99 Chapter 4 - Discussion The following discussion provides an interpretation of the results, from both my perspective and within the context of existing literature, and highlights points that require further clarification. 4.1 Performance Indicators and the PREP Framework The PREP pillars and criteria form an analysis framework that could provide a practical approach for determining an AWO's effectiveness. The PREP Framework's components are summarized in the following checklist: PREP Framework Philosophy Red Flags Efficiency People 1. Mission and Objectives 1. prevent root cause ii. meet community need 2. Policies & Practices i. animal ethics policies and position statements ii. animal care policies and practices a. pre-set limits b. # animals helped c. days in care d. staff & volunteers per animal e. housing environment iii. business policies and practices 3. Validation of Mission, Policies & Practices i. references ii. past accomplishments iii. site visits & customer surveys 1. Excessive Personnel 1. Turnover i. 2. Disproportionate " Management -Salaries 3. High Administration 2. & Fundraising Costs j. 4. Lack of Peer Collaboration " 3. 5. No/Slow Availability j of Financial Records jj iii. iv. v. Long Term Planning strategic plan objectives aligned with budget community need and niche Financial Credibility professional financial statements budget adherence Sustainability financial reserves revenue generation abilities diversity of funding sources a. individuals b. foundations c. corporations communications with donors efficient use of donations in kind 1. Management i. experience & training ii. moderate management turnover iii. paid staff iv. competitive wages 2. Volunteers i. involvement ii. duties iii. volunteer-to-staff ratio iv. management v. tracking vi. retention rates 3. Peers i. collaborative partnerships ii. references iii. facilitation & empowerment 4. Donors i. # of individual donors ii. donor satisfaction iii. retention & trends 5. Board of Directors i. backgrounds ii. connections iii. active involvement iv. election process/terms v. number of members As a first step, participants indicated that donors need to fully understand an organization's philosophy and mission in order to make informed funding decisions. While the name of an organization and even its stated mission may initially seem to fit with a donor's philanthropic interests, examining its policies and position statements would help donors validate an AWO's 100 actual mandate and actions, and ensure they match with the donors' own values and expectations. It has been shown that names and mandates may, in some cases, be misleading or may misrepresent the actual activities of an organization. For example, confusion has occurred because of the name of the Canadian SPCA which, in fact, is not a national but a regional organization serving only the province of Quebec. The Canadian SPCA has been accused of using the impression given by its name to solicit funds from other provinces, with many donors believing they are supporting a national organization and that their donations are being directed to local SPCAs (Ha, 2006 and 2007). To add to the confusion two other groups have similar names — the Canadian Federation of Humane Societies (CFHS) and the Humane Society of Canada. The CFHS is, in fact, the national body that represents SPCAs and other humane societies across the country (2007). The Humane Society of Canada is a Toronto-based advocacy organization which lists only one employee, the Executive Director, and maintains mainly post office boxes in other provinces (2007). Another example is Doctors and Lawyers for Responsible Medicine, whose goal is the "the immediate and total abolition of all animal experiments, on medical and scientific grounds" (2007). However, one does not need to be either a doctor or a lawyer to become a member of this organization, and the term "responsible medicine" is open to interpretation. These examples illustrate the need for donors to analyze organizations carefully to see whether the name, philosophy, mandate and policies accord with the donors' intentions. Participants also cited animal care policies and practices as indicators of an AWO's philosophy. Of the five indicators cited under quality of animal care, two - number of animals helped and days in care - warrant further discussion. Although the number of animals helped is the statistic that AWOs most often use to indicate their impact, it is only a descriptive statistic which indicates the scope and capacity of the organization. If "number helped" is to be used as an indicator of an AWO's philosophy, it should be presented in conjunction with re-homing/release and death rates. As well, how these numbers 101 are changing from year to year - i.e. are the trends increasing or decreasing, and by how much? - would provide donors with an understanding of the AWO's progress within the context of its own philosophy and community. Days in care, although a useful comparative statistic and indicator of philosophy and practices, needs to be interpreted carefully. For example, in the case of companion animals, a very low number of days in care could mean that the community is extremely supportive, or alternatively that the AWO is not taking the time to properly assess adoptive families. On the other hand, very high days in care may reflect an adoption policy that is overly restrictive, a facility that hoards animals, or a facility which lacks community support. In addition to researching an AWO's philosophy, participants indicated that donors should look for "red flags" in order to quickly identify anomalous items that warrant further investigation. The large number of red flags cited by participants supports the idea that it is often easier to look for what is wrong than what is right. However, it is important to understand when some deviations from the norm may be justifiable. Although excessive turnover of management and employees was indicated as a red flag, and although participants indicated that meeting objectives and consistency in an AWO's management team were positive indicators, very few foundations expressed willingness to fund higher wages or improvements to management structure to prevent turnover, foster stability and meet expectations. A similar observation was made by Smith (2003) in a study of the social service sector in Canada. Participants in the present study pointed out that given the emotional stress, the often low pay, lack of advancement and minimal professional development that managers in the animal welfare sector receive, donors should not be surprised to see some significant turnover. Conversely, a complete lack of turnover can be a problem if performance results have been historically poor. Thus it appears that judgment is needed to assess the level of turnover relative to the AWO's performance to determine whether donors should be concerned. Under the pillar of "Efficiency", three elements - community need, holding a reserve and donations in kind - require further clarification and discussion. 102 Many non-profit organizations establish themselves in response to a perceived community need, but even when there is direct duplication, very few organizations merge in order to achieve economic and operational efficiencies. Even where differing philosophies prevent groups from merging, there are areas where collaboration and amalgamation would make sense. Participants stated that advocacy work would be well suited to joint initiatives. Companion animal shelters, for example, whether no-kill or open-admission, can all share in raising public awareness concerning spay/neuter, adoption and preventing cruelty. All wildlife organizations strive to reduce human-wildlife conflicts and educate the public on dealing with injured or seemingly abandoned animals. Organizations that advocate for farm animals want to raise consumer awareness concerning food production systems and want to amend regulations addressing the raising, transport and slaughter of farm animals. By sharing costs related to advocacy activities, AWOs maximize their impact, reduce costs and gain efficiencies, and thus limit the dilution of funds. Many participants said holding a reserve is an important factor for long term sustainability of an organization. However, reserves may be depleted if they are used to fund emergency situations. Therefore it is important for donors to understand the reason for depleting reserves and how the organization plans to rebuild them. AWOs and foundations held differing views on donations in kind as indicators of efficiency. The disagreement appeared to stem mainly from cases where the value of such donations had been recorded inaccurately. Since a targeted program to solicit donations in kind program can result in significant efficiencies for an AWO, it would be worthwhile for the animal welfare sector to consider a standard practice to accurately depict the value of such gifts, such that everyone is reporting fair market value for the goods and services they receive. Last but not least, participants agreed that the elements that make up the pillar on "People" are major indicators of an organization's effectiveness. Every organization, no matter how new, small, large or well established, can be judged on the basis of their people. The results suggest the abilities of the managers should instill faith that they will be able to fulfill the mandate and objectives of the organization. Interestingly, however, most AWOs do not publish much 103 information about these individuals in their reports or on their web sites. Many AWOs will list the names of the management team and board of directors, but provide no further details on their experience and why they became involved with the organization or the sector. In fact, of the 15 AWO participants in my study, only half listed the members of their management team and/or board of directors on their web sites. Of those, only half provided short biographies, the other half only listing individuals' names. The decision not to publish the names of managers may result from a perceived need to protect the identities of employees and board members, given the sometimes controversial issues facing animal protection groups. It may also be a purposeful strategy to put emphasis on the animals, not the people. Whatever the motive, the common failure to promote the talents and experience of the people involved in an organization appears to be a missed opportunity to foster donor confidence. Related to the quality and abilities of the people is the compensation paid to employees. Although salaries that are disproportionate to the size and scope of the organization constitute a red flag, attracting successful people with the necessary skills is important and requires appropriate compensation. However, it appears that workers in the animal welfare sector are underpaid relative to the level of responsibility required. For example, the average salary for a companion animal shelter manager in BC is roughly Cdn $45,000 (personal communication, General Manager of Operations, BCSPCA, 2007) yet the job description requires educated, qualified professionals with several years experience in staff and volunteer management, public relations, customer service, fundraising, financial management and budget planning (BCSPCA, 2002). This pay approximates many entry-level professional jobs for people with no relevant management experience. Therefore it is not surprising that candidates with the required experience and skills are not always attracted to these positions, and those that do apply, often do not stay long. Thus a conflict occurs between donor expectations of strong performance by AWOs and the quality of staff that AWOs can afford to hire and retain. 104 The sheer number of performance indicators and criteria cited by participants in this study demonstrates a lack of consistency in assessing organizations and the need to identify the most relevant criteria. The PREP Framework provides a systematic approach to ensure that all organizations are being assessed on a common set of relevant criteria - e.g. valid mission, achievable objectives, financial stability, donor support, quality of their management team, and credibility with peers. This should reduce the influence of emotional appeals, but it leaves room for subjective judgment as to whether the AWOs' results are acceptable given its circumstances. If donors combine their research of all four pillars of the PREP framework, they should garner enough information to ensure they focus their charitable dollars on capable, fiscally responsible, effective organizations with philosophies that align with their own values. The depth of analysis that donors conduct will obviously vary depending on the donor and the size of the donation. To address the complexities involved in evaluating performance of such highly varied organizations, a peer review and accreditation process might enable a more comprehensive assessment of performance by introducing a level of qualitative judgment on measurements or organizations that are difficult to compare. One possible model for executing performance evaluations of diverse organizations is that used by the Canadian Council on Animal Care (CCAC). The CCAC organizes peer reviews "which involve an assessment of the effectiveness of an institution's animal care committee and the appropriateness of its animal care facilities, practices and procedures" (Canadian Council on Animal Care, 2007). The review is conducted by an assessment panel composed of at least four representatives - a scientist, a veterinarian, a community representative often drawn from an SPCA/humane society in the local area of the institution to be visited, and a CCAC Assessment Director. Although this model may seem time and labor intensive to some, it does appear to represent the range of skills, knowledge and neutrality required to conduct an informed and unbiased assessment. CCAC's international equivalent, the Association for Assessment and Accreditation of Laboratory Animal Care International (AAALAC), operates similarly, although its accreditation and panel members do not include community representatives (2007). In terms of who should conduct evaluations, although some participants felt AWOs should conduct their own evaluations, given the lack of time and 105 skills AWO employees have to fully devote to this activity, peer assessments by a third party would provide the unbiased expertise and practicality participants seemed to favor. 4.2 Outcomes and Outcome Measurement Although outcome measurement is increasingly being used in the non-profit sector to assess the effectiveness of programs and organizations, most of the participants in this study had difficulty identifying and defining outcomes. It appeared they were either not familiar with outcome measurement, or were simply more familiar with the use of other performance indicators such as inputs, activities and outputs. When participants were asked for general outcomes applicable to all AWOs, most responses were not outcomes at all. The only true common outcome they cited was a change in public opinion or behavior. However, this outcome is very broad and would be difficult to attribute to any one project or organization. Outcomes were easier to define for companion animal shelters. They included a reduction in euthanized animals and an increase in successful adoptions. However, many of the outcomes cited were reactive, whereas participants generally described effective AWOs as those that proactively address the root cause of problems. Only a few participants cited examples of proactive outcomes, including programs to control breeding and importation to reduce the over-supply of animals. For organizations related to farm animal welfare, proposed outcomes tended to focus on the purchase of organic products. However, since the increased purchase of organic products may be more related to the consumers' concerns about health, this outcome would be difficult to attribute to the effect of an AWO. Increased purchases of humanely labeled products might be more appropriate. In summary, participants were able to come up with only a few clear and significant performance outcomes for evaluation purposes. Some of the suggested outcomes such as changes to laws could only be realized in the long term, and thus would be irrelevant for donor 106 evaluation today. Other outcomes, such as changes in public opinion and consumer behavior, cannot always be attributed to the work of any one particular group. Still other outcomes, such as successful wildlife rehabilitation and release, cannot fully be measured due to the lack of post-release monitoring. Hence outcome-based assessment of organizations would be challenging in many cases. The sector might benefit from further discussions on what outcomes would be more appropriate and measurable. 4.3 Comparative Evaluation - Benchmarks and Standards The majority of participants felt there was a real need for comparative evaluation of organizations, but were not sure if specific standards are feasible. Doubts surrounding the applicability of benchmarks seemed more related to a lack of knowledge of what could be benchmarked, and how one would go about it, rather than an absolute disagreement with the concept of benchmarking. Participants were, in fact, able to suggest common criteria that could be standardized. However, the actual benchmark values will depend greatly on the type of operation, and would require in-depth analysis of various activities to set appropriate benchmarks. Alternatively, instead of standardizing quantitative values, a simpler approach would involve standardizing only the elements to be evaluated, such that every AWO reports on, and is measured on, the same relevant criteria. One foundation highlighted a recognition program for the social service sector whereby applicants compete to win the top prize. All applicants must complete a standard template which assesses strategic thinking, good governance and general performance through good stewardship of their financial and volunteer resources. In addition to providing a fair and uncomplicated evaluation framework, the applicants have stated that they found the process and the template very valuable as a means of highlighting what criteria were important to monitor, evaluate and improve upon. Standardizing the elements (and their definitions), ensures donors assess the same areas for each AWO, but are using their judgment to assess whether the result is appropriate for that particular organization or type of operation. Over time, donors could assess the organization's trends to see how its performance compares 107 to previous years, and the results of similar organizations. This would provide donors with a general sense of the organization's relative effectiveness, rather than forcing AWOs to meet a specific performance marker. As well, in comparing results to general sector norms, this method could assist AWOs with their own self-evaluation processes, and better focus their improvement efforts on areas where they are relatively weak. Such an approach may also avoid penalizing organizations that have legitimate reasons as to why they fall outside the norms. Nonetheless, this study has identified some candidate areas for quantitative benchmarking. Financial benchmarks would likely be the easiest to develop and test for effectiveness, given there are already some standards being used by the sector for overhead and fundraising expenses. In Canada, federal tax policies require registered charities to spend at least 80% of their tax-receiptable donations on charitable activities. The remaining 20% can be spent on management, administration and other operating expenses. In the US, the Internal Revenue Service does not regulate the amounts; however, according to Charity Navigator (2006), 7 out of 10 charities evaluated spend at least 75% of their budget on programs and services and 9 out of 10 spend at least 65%, the remainder going towards administrative, fundraising and operational costs. These figures align closely with those reported by the American Institute of Philanthropy (2006), which states that spending 60% or more on programs and services is reasonable for most charities. Although the difference in administrative costs between Canadian and US charities seems significant, the reality may be more similar than these figures suggest. Charitable organizations sometimes "re-classify" administrative and fundraising expenses as program expenses to stay within governmental and generally accepted guidelines. This is accomplished through the use of project or research contracts to classify staff salaries as program expenses, and by incorporating an educational component in all fundraising materials so that charities can categorize some fundraising expenses as educational costs (American Institute of Philanthropy, 2006). Although this practice is legitimate to some degree, many charities push the boundaries and risk losing their charitable status (Donovan, 2006; Wren, 2006; Pringle, 2007). There is some debate on whether this pressure to keep overhead costs low sacrifices an organization's effectiveness by encouraging them to overload management or eliminate key functions and 108 services in an effort to reduce overhead expenses (National Post, 2006). Ironically, even though the Canadian government legally requires the 80/20 ratio, 47% of Canadians feel charities should decide for themselves what the appropriate cost structure is for them to effectively achieve their missions (Muttart Foundation, 2006). Fundraising benchmarks are commonly monitored in the non-profit sector. It is possible, however, that fundraising benchmarks generally applied to all non-profit organizations may be unreasonable for AWOs. Animal-related charities solicit funds from the same population base as do other charities, and so their costs on direct mail, and print and TV advertising typically equal those of other charities conducting similar campaigns. However, it was reported in Foundation Giving Trends that animal-related charities receive only a small percentage (approximately 1%) of the public's donations (The Foundation Center, 2005) and thus their return on this investment may be much lower, and their cost to acquire donors much greater. Having a clearer understanding of what fundraising cost/return ratios are normal for the animal welfare sector would assist AWOs in more fairly assessing the effectiveness of their own campaigns, and would determine whether AWOs are pursuing achievable benchmarks. Participants were not specific about what level of compensation is reasonable in the animal welfare sector, and thus benchmarking employee salaries - e.g. as a percentage of funds raised, as a percentage of budget, or in relation to scope and responsibility - may be helpful. In 2003, the average compensation for a CEO in the Canadian non-profit sector was $97,349, with the average male CEO earning $109,478 and the average female CEO earning $83,250 - roughly 1/4 lower than her male counterpart (The Canadian Society of Association Executives, 2003). In the US, a study of 300 non-profit organizations (Hrywna, 2006) indicated that the average executive director's salary was expected to exceed $100,000 in 2006, up 1.6% from 2005 levels, but less than the rate of inflation at 3.5%. That study also indicated that female executive directors earned 25% less than the average male director's salary and reported that executive directors in the health category received the highest average salary, earning US$132,707 in 2005. Thus the salary ranges can be significant, and without any benchmarks for the animal welfare sector, it is difficult for donors to determine what is reasonable. It might be possible to establish a benchmark 109 such that management and executive salaries should not exceed a certain percentage of an AWO's total revenue, presuming revenue is an indicator of an organization's size, scope and responsibility, but this has not yet been attempted. Benchmarks for staff salaries in relation to total expenses, funds raised and budget would not only assist donors in determining the reasonableness of salaries but may also provide guidance to AWOs on where more or less staff resources are needed. Benchmarks or standards for animal care could also be feasible, although many participants stated such standards would pose significant challenges, since philosophies and community demographics greatly influence results. Some participants raised concerns about benchmarks encouraging AWOs to make decisions simply to achieve required benchmarks versus doing what is right for animal welfare. While these points are valid, they may not be reason enough to avoid some form of standardization. Animal care standards have already been adopted in other facilities where variables are equally diverse and complex. As mentioned previously, AAALAC (2007) and the CCAC (2007) have been able to set standards and conduct certification to raise the competency and professionalism of laboratory managers and technicians, and to ensure responsible animal care and use, even though the experiments, facilities and animals used in research, testing and education vary greatly. The American Boarding Kennels Association (2007) has an accreditation program that requires pet care facilities to demonstrate their adherence to a comprehensive set of operational standards for animal care and management. The Asilomar Accords (2004) offer a framework to standardize the reporting of shelter statistics. The Royal Society for the Prevention of Cruelty to Animals (2007) has established humane farming standards which have been adopted by other humane farming certification programs in other countries. Furthermore, a few participants commented that there is already a movement to standardize care protocols and certify workers in wildlife rehabilitation. Thus there are many examples of standards and benchmarks in other areas of animal welfare. In fact, it seems an odd omission that dog and cat shelters do not have similar guidelines to ensure consistent, minimum quality of care across operations. 110 In terms of specific animal care benchmarks, although euthanasia rates will differ greatly across AWOs depending on the operation and ideology, it is important to see what benchmarks AWOs set for themselves in terms of reducing euthanasia numbers over time. For companion animal shelters, the "death reduction" program established under the Asilomar Accords demonstrates how a community's AWOs are making progress towards reducing deaths. For the wildlife sector, benchmarks may serve them well in helping donors realize that their euthanasia rates will always be much higher than those of shelters or sanctuaries, but that on average, their death rates approximate other centers facing similar issues (e.g., similar region, species, injuries, diseases). Some might argue that benchmarking euthanasia would reduce animal welfare by encouraging some organizations to release more animals in sub-standard health in order to reduce euthanasia rates. However, if used as a guide, such benchmarks could benefit the sector by helping centers and donors compare euthanasia practices to norms for the sector. Participants, particularly foundations, also indicated interest in understanding the sector norm for days in care. The only days-in-care figure cited by participants in the study was for companion animals, and the figure provided was a worst-case scenario of 6 months. Many factors contribute to an animal's well-being while in a shelter; these include physical and emotional enrichment with toys and other animals, proper bedding, food, and space (Roy, 1992; Smith, Durman, Roy and Bradshaw, 1994; Hennessy, Williams, Miller, Douglas and Voith, 1998). Longer shelter stays and over-crowding can adversely affect the animals' welfare and adoptability (Foley, 2001). Stephen and Ledger (2005) found that for dogs kenneled in rescue shelters over a period of 6 weeks, "the prevalence of behaviors associated with frustration... increases in a environment that does not facilitate the performance of behavioral needs" (p.91), although the dog's breed and age, as well as the presence of observers, may have affected the onset and level of frustration. Thus it appears the sector may benefit from a better understanding of some general shelter benchmarks and averages for days in care, with recognition that additional parameters such as emotional and physical enrichment and housing standards may also be important in evaluating days in care benchmarks at no-kill shelters where the average days-in-care are greater. I l l Other benchmarks such as space per animal and staff per animal were also identified. Space allowance (square meters or square foot per animal) is a common approach used in the codes of practice for the care and handling of farm animals (Canadian Agri-Food Research Council, 2003; and Agriculture Canada, 1990) and could potentially be a useful tool for shelters and facilities. The number of staff per animal might also serve as a benchmark if adjusted for the species at a facility, much like the human health-care system adjusts nurse-per-patient ratios by patient illness, vulnerability, or staff skill (Agency for Healthcare Research and Quality, 2003; Institute of Medicine, 2004). A cost-per-animal benchmark might also be achievable. If complex, human welfare standards can successfully be applied in the medical system, it seems reasonable that benchmarks could be applied in the animal welfare system as well. In summary, benchmarks, if used properly, could guide organizations towards accepted norms of effectiveness; they could allow organizations to identify and track their performance trends over time and manage accordingly, and they could help AWOs and their donors identify where organizations excel and where they need to improve. However, benchmarks would need to be assessed in the context of the community demographics and the size of the organization. What might seem like tremendous strides for a small rural community may be trivial in an animal-friendly urban center. Benchmarks, therefore, are a tool to facilitate discussion about organizations, not the definitive means to assess or compare organizations. 4.4 Funding Conflicts Many of the funding needs, conflicts and ideas identified by participants in this study echoed those found in the studies highlighted in Chapter 1 by Letts et al. (1997), Hall et. al (2003) and Smith (2003). To reiterate, Letts et al. found that foundations' focus on short-term grants and their lack of investment in organizational capacity limited the long-term goals and outcomes of non-profit organizations. Hall et al. found that a gap exists between what information funders want to receive on grant applications, and what the voluntary organizations provide. Smith found that foundations' interests in short-term, innovative projects meant that the proven, core programs of non-profits were more difficult to sustain, and that non-profits found the complex and unique 112 requirements of each funding body were too costly and time-consuming for the amount of funds being granted. In my study, the results identified several contradictions between what foundations said they hoped to accomplish and how they actually allocated funds. First, foundation participants said they wanted to offer short-term funding because the long term carried too many uncertainties and risks, yet they favored new and innovative projects which inherently involve more risk than established, proven projects. AWO participants noted that although they have existing programs of proven effectiveness and low risk, foundations were not interested in funding ongoing activities. Second, foundations said they favored innovative projects because these propel the animal welfare movement forward and make a greater difference in the long term. However, they also said they prefer short-term projects because these are more easily measured and tangible. At the same time, many participants stated that real program benefits and outcomes are not measurable or even realized until well into the medium to long term and require long-term commitment. A third paradox surrounding foundations' preference for new and innovative projects was the fact that some foundations cap the amount of funds they will provide to any single recipient. This cap forces AWOs to craft proposals that meet the funding limit and parameters, rather than develop a more comprehensive and creative project that reflects the true cost of solving a problem. As well, having to be creative and constantly come up with new ideas takes valuable staff time away from executing existing programs, which in turn may impact their performance on existing programs. These contradictions highlight the fact that all projects carry differing levels of risk and return. Short-term projects may involve lower risks for funders, but they may not deliver significant benefits or reforms for the animal welfare movement. Equally, although long-term projects may hold more uncertainties, their potential benefits may be greater. It seems, then, that similar to a mutual fund's diversified investment portfolio, funding a mix of innovative projects and basic operations over varying timelines from a combination of organization types and sizes might provide foundations with a balanced approach and more predictable return on social investment. 113 Based on participant comments, foundations may also want to consider graduated or phased social investments. In other words, first time grants to AWOs are awarded for short-term programs or projects with clearly defined and measurable goals. This would provide foundations with the opportunity to learn about an organization and its management team. A second phase of grants with this AWO may include projects with strategic, longer-term outcomes that carry less measurable goals in the immediate term. This might provide foundations with a better sense of an AWO's strategic management capabilities and its track record in achieving objectives over the medium to long term. Once the trust has been built, a final round of funding may provide general operating support to sustain successful programs developed in earlier years. Foundations also commented that they are frustrated with the poor quality of proposals they receive. Nonetheless, very few foundations had considered funding grant/proposal writing seminars to improve this skill set or to hire a professional grant writer to submit proposals on the AWO's behalf. Some foundations have program officers to assist applicants with the proposal writing process for their own grants, but this does not necessarily improve the AWO's ability to complete proposals for other funding sources. In addition, foundation participants said they were frustrated by the proposals they receive for projects that are outside their stated mandate (e.g., submissions for capital versus program funds, for operations versus projects, etc.). However, some foundations admitted that they will often provide funding for projects outside their normal mandate if the project has great merit or if they have confidence in the organization's abilities based on previous successes. In bending their own rules and accepting proposals for projects outside their mandate, foundations may be contributing to the very problem they stated they wish to avoid, and perpetuating the unnecessary workload for both themselves and AWOs. Most foundations stated they wanted AWOs to become self-reliant and not to come back to them year after year for funds. Once again, however, only a few foundations stated an interest in funding capacity-building to strengthen the general skill sets of the organization, and none stated an interest in granting funds to develop marketing and revenue-development aspects to help 114 organizations become self-reliant. In particular, very few foundations funded web design costs, even though a well designed and up-to-date web site with online donation capabilities can significantly enhance revenue development and reduce operational costs related to marketing, communications and donations processing. According to President and CEO of e-Philanthropy Foundation, Ted Hart, on-line donations increased from $10 million in 1999 to approximately $1.9 billion in 2003, and are expected to account for 1/3 of all donations by 2010 (quoted in Legace, 2004) . Therefore it appears that developing web content and functionality is critical for an AWO's communications strategy with individual donors and for relying less upon foundation funding. Further contradictions arise when foundations pre-determine whether to support small, large, grassroots or established organizations and to limit their support to specific regions. Although having different foundations focus their efforts on different organization types has its merits in broadly growing the animal welfare sector, such decisions do not always result in the most effective programs being funded, since many organizations would not fit the pre-determined limits. Again, to meet their stated priorities of funding programs that will make a difference, foundations may wish to consider assessing proposals on the grounds of their ability to solve problems, and not turning them away because of a lack of fit to pre-determined criteria of organization size or geographic location. Finally, funding only AWOs with certain practices or ideologies can also limit the sector's results. Most participants felt that donors' interests in funding only no-kill shelters has had a positive impact on pressuring open-admission shelters to reduce their euthanasia of healthy and adoptable animals. However, in deciding not to fund open-admission shelters, the goal of reducing euthanasia and providing quality care for animals in those facilities is compromised as these organizations struggle to pay for the costs associated with the arrival of more and more animals. Furthermore, in adopting no-kill policies, such shelters often must reject more animals than they can house or adopt in their community, or transfer them to other shelters. This simply shifts the euthanasia of these animals to another organization (City of Houston Task Force, 2005) . There was a general understanding that both shelters and foundations need to address 115 the number of animals being abandoned in the first place by resolving irresponsible breeding, purchasing and guardianship of animals. In discussing the effective allocation of funds, some participants highlighted an extended role for foundations. A few AWO participants indicated they would like foundations to promote successful programs and projects with which they have been involved to other AWOs and foundations in the sector. Not only would this allow information sharing between communities so others can copy effective programs in their communities, but it would also assist effective AWOs in raising further funds, and provide assurances to donors and other foundations who may be considering donating to these organizations. Another option might be for foundations to "opt out" in certain years, but with a plan to fund costs on and off throughout a 10-year period, so long as other funders are secured for their opt-out years. This effectively would meet the need of long-term funding without the individual foundation bearing the financial responsibility for the full term. It was also suggested that rather than tossing ineligible proposals aside, foundations may consider taking on a more active, intermediary role by recommending the proposal to a foundation or philanthropist with a specific interest in such projects. Given that some foundations do not accept unsolicited proposals, and that there are a limited number of foundations in the first place, this would assist many AWOs tap into a larger funding pool to which they currently do not have access, particularly with regard to individual philanthropists. Even if foundations decide they wish to restrict the number of years they support a project, the idea of "brokering" support from another foundation or philanthropist to keep successful programs going would indirectly achieve the longer funding terms that many projects need. It was suggested that foundations could also consider mentoring other foundations. For example, established foundations could share their evaluation techniques and data with less experienced foundations and philanthropists in order to help them more effectively allocate their funds. Many grantmaker associations and networks have been created throughout the US (e.g., Philanthropy Northwest, San Diego GrantMakers Online, Animal Grantmakers) with the idea of sharing best funding practices and promoting effective philanthropy. The growth of these types of 116 organizations demonstrates that foundations see a benefit in giving and receiving such advice amongst their peer group. In summary, there are many disconnects between the funding practices of foundations and the needs of AWOs. AWOs seem to indicate that major donors and foundations should take a more diversified approach to their social investments rather than focusing on organizations or projects that fit within specific, pre-determined parameters. By funding the best operations, programs and ideas, and by removing unnecessary restrictions, AWOs felt this would help foundations (and the AWOs themselves) achieve their ultimate goal of improving the lives of animals. Foundations seemed somewhat receptive to changing certain practices to increase the impact of their giving, although they were averse to providing more support to general operating funds or ongoing multi-year grants, despite these being AWO priorities. These gaps appear to be common across the non-profit sector; this illustrates a potentially debilitating impact on the overall effectiveness of the sector as funding priorities and funding interests fail to align. The similarity in findings between Letts et al. (1997), Hall et al. (2003), Smith (2003) and my own study several years later indicates a real need to address and reconcile these conflicts within animal welfare and the non-profit sector in general. 117 Chapter 5 - Conclusion and Recommendations "Earning that money took thought and planning. Giving it away deserves just as much attention." (Markin, 2004) Although outcome measurement is increasingly being used in the non-profit sector to assess the effectiveness of programs and organizations, participants in this study had difficulty identifying outcomes for AWOs and, for the most part, misinterpreted outputs for outcomes. What they did accurately identify as key outcomes of the animal protection movement were generally too long-term and diffuse (public attitude change, legislative change) to be useful for comparing the effectiveness of a given organization or program. While they did identify some plausible outcomes for companion animal facilities and a few other categories, many of these would not apply to other types of AWO facilities and activities. It appears, therefore, that the sector would benefit from further discussions to better define appropriate outcomes for various AWO activities. In terms of standardizing performance assessment, benchmarks are commonly used for some non-profit services to assess the adequacy of an organization's performance. For example, the health sector uses benchmarks such as number of nurses per hundred patients in a critical care ward as measures of efficiency. However, some participants in this study, although wishing that animal welfare organizations could be compared by such quantitative performance criteria, were skeptical of this approach, and they identified various reasons why it might not be appropriate. Certain business aspects (overhead, fund-raising costs) were suggested as being amenable to benchmarking, but with some provisos. Although participants expressed an interest in animal care standards, and although these could be modeled after standards that exist for farms, research laboratories, zoos and kennel operations, indicators such as adoption rate, euthanasia rate and days in care were seen as confounded by philosophy (no-kill versus open admission shelters), species in care, and regional factors, to the point that the use of standard benchmarks could be misleading, and pursuit of benchmarks could actually be detrimental to animal welfare. 118 Because of the diversity within the animal welfare sector (advocacy organizations, pet rescue shelters, wildlife rehabilitation centers) many quantitative indicators would likely need to be viewed as norms or ranges that would be useful to identify anomalies, rather than benchmarks that should be met in all cases. If developed with appropriate sector input, these could be of value in helping AWOs assess their own operations, and in assisting sophisticated donors such as foundations to make better funding decisions. As with other complex sectors, the most practical approach to evaluating AWOs would likely be a form of peer review and accreditation, and perhaps modeled after the programs used for laboratory animals and zoos. As accreditation is often used to demonstrate that organizations meet the norms or expectations of a sector, such a program could make a major contribution to animal welfare and to donor confidence in animal welfare organizations. A "red flag" approach also showed promise. That is, participants could identify certain indicators that warranted further investigation if they were out of line. For example, although a benchmark euthanasia rate for wildlife rehabilitation facilities may be too difficult to specify, if an agency's euthanasia rate falls outside a broad normal range, it may indicate problems. Participants also agreed that the human side of organizations - the people involved with an AWO - provides useful means of identifying worthy organizations. They indicated that the abilities, commitment and connections of an AWO's management, volunteers and peers were critical to an organization's success, and that its board of directors and donors can also play a pivotal role in helping an organization achieve its objectives. Of these, the relationship an AWO has with peer organizations was seen as the most revealing indicator of an AWO's effectiveness. Similarly, there were efficiencies that any AWO could be expected to achieve. Participants suggested that effective organizations serve the needs of the community, follow credible financial management practices, and establish realistic, long-term strategic plans that demonstrate that the operation is sustainable. An important caveat, however, is that long-term sustainability may require some unpopular decisions in the short term, and thus donors must consider the bigger 119 picture - the impact the organization is trying to accomplish in the long run - in addition to short-term results. Finally, because AWO's are extremely diverse in their philosophy, there is a risk that basic philosophies of an AWO will not align with those of the donor. If donors feel they have been purposely misled, or the AWO has inappropriately represented itself, the sector as a whole can be harmed by the loss in donor confidence. By reviewing established mission statements, position statements and policies, donors can verify that AWOs are in line with their own goals and interests. Therefore, philosophical fit between the donor and the AWO is a useful part of the evaluation process. While the majority of the study addressed effectiveness of AWOs, equally important is the effectiveness of foundations in the allocation of their funds. This study highlighted many conflicts between foundation funding practices and the needs of AWOs. Resolving these conflicts could make a significant contribution to ensuring more effective programs, better funding decisions, and better use of staff time on both sides. If foundations deem that the proposed outcome is worthy, and have faith in the abilities of the applicant's management team, how the AWO allocates the funds to best achieve this outcome should be immaterial. Thus if proposals are assessed on merit rather than how they fit with certain pre-determined parameters, foundations could maximize the impact of their social investments and better serve the needs of the AWO community and its constituents. In conclusion, participants were mostly supportive of the idea and benefits of standards-based approaches for assessing and comparing performance of AWOs, but were skeptical about the use, applicability and interpretation of some of these results given the diversity of the sector. The insights they provided suggest that an assessment framework based on four common elements -Philosophy, Red Flags, Efficiencies, and People (the PREP Framework) - shows much more promise for determining effectiveness of organizations in the animal welfare sector. Because it is a checklist of elements to be considered, rather than a set of quantitative goals or standards to be met, it can be applied to organizations of different sizes, types and locations. However, the results 120 need to be interpreted intelligently, and the context of the specific organization needs to be taken into account. Because of the complexity of the sector, an approach incorporating peer review and accreditation shows the most promise. A logical next step would be for AWOs and foundations to develop a template that incorporates what they deem to be the most relevant PREP criteria, benchmarks, and outcomes identified in this study. If used as an additional tool to assess an AWO's performance and abilities, this template should enable donors to make more informed and effective philanthropic decisions. 121 References Agency for Healthcare Research and Quality. 2003. Evidence Report/Technology Assessment, no. 74. The Effect of Health Care Working Conditions on Patient Safety. AHRQ Publication: Rockville, MD. Retrieved March 12, 2007 from Agriculture Canada. 1990. Recommended code of practice for care and handling of pigs. Agriculture Canada: Ottawa, Ont. American Institute of Philanthropy. 2006. Retrieved January 15, 2007 from American Institute of Philanthropy. 2006. Retrieved August 15, 2006 from http: //www, ch a ritvwatch. org/crite ria. htm I Animal Grantmakers. 2006. Retrieved May 13, 2005 from Asilomar Accords. Retrieved January 21, 2007 from Association for Assessment and Accreditation of Laboratory Animal Care International (AAALAC). Retrieved March 11, 2007 from http://www.aaalac.Org/accreditation/rules.cfm#standards Buchanan,P., Bolduc, K., Huang, J. 2005. "Turning the Table on Assessment: The Grantee Perception Report." The Center for Effective Philanthropy. Excerpted from the book, A Funder's Guide to Organizational Assessment. Fieldstone Alliance: St. Paul, MN. Retrieved December 3, 2006 from GEO GPR chapter.pdf Canadian Agri-Food Research Council. 2003. Recommended code of practice for the care and handling of pullet, layers and spent fowl. Canadian Agri-Food Research Council: Ottawa, Ont. Canadian Council on Animal Care. Retrieved March 14, 2007 from Main.htm Canadian Federation of Humane Societies. 2007. Retrieved March 14, 2007 from we are/ Canadian Society of Association Executives. 2003. Backgrounder: The 2003 Association Executive Benefits and Compensation Report. The Canadian Society of Association Executives and the Association Research Centre Inc.: Toronto, Ont. Retrieved February 6, 2006 from Charity Navigator. 2006. Retrieved January 15, 2007 from Charity Navigator. 2006. Retrieved June 13, 2006 from http://www.charitvnavigator.Org/index.cfm/bav/content.view/catid/2/cpid/48/print/1.htm Christensen, J. 2003. Auditing Conservation in an Age of Accountability. Conservation In Practice, a publication of the Society for Conservation Biology, Summer 2003, Vol 4, no.3 122 Chun, S. et al. 2001. SROI Methodology. Analyzing the Value of Social Purpose Enterprise within a Social Return on Investment Framework. Chapter 1, p.2. The Roberts Foundation: San Francisco, CA. Retrieved February 5, 2006 from method 1.pdf City of Houston. 2005. Report of the Mayor's Animal Protection Task Force, pp. 13-14. Retrieved November 22, 2005 from Coffey, A. and Atkinson, P. 1996. Making Sense of Qualitative Data. Chapter 2: Concepts and Coding. Sage Publications: Thousand Oaks, CA. p. 32 Council on Foundations. 1993. Evaluation for Foundations: concepts, cases, guidelines and resources. Jossey-Bass Inc.: San Francisco, CA. pp. 12-13 and p. 235. Cunningham, K. and Ricks, M. 2004. "Why Measure? Nonprofits Use Metrics to Show That They Are Efficient. But What If Donors Don't Care?" Stanford Social Innovation Review, Vol. 2, Summer 2004. pp. 44-51. Retrieved November 27, 2006 from measure/ Denzin, N.K., and Lincoln, Y.S. (Eds). 2000. Handbook of Qualitative Research. Sage Publications: Thousand Oaks, CA. pp. 6-22. Doctors and Lawyers for Responsible Medicine. 2007. Retrieved March 14, 2007 from Donovan, K. 2006. December 10, 2006. MADD Rejects "Disgruntled" Critics: Charity's CEO dismisses volunteers' complaints that so little of donations go to programs. The Toronto Star. Retrieved March 14, 2007 from Dwan, P. 2002. Citigroup Private Bank -US Marketing. Retrieved October 13, 2004 from Foley, J. 2001. CCAH Update. Vol. 6, no. 1, Spring 2001. Center for Companion Animal Health: University of California, Davis. Retrieved February 17 2007 from Sheltermedicine.html Giving USA. 2006. Charitable Giving Rises 6 Percent to More than $260 Billion in 2005. Giving USA Foundation. Melissa S. Brown (ed). Retrieved September 21, 2006 from releases/trustreleases/0606 PR.pdf Ha, T. November 8, 2006. Montreal SPCA Fundraising Draws Criticism. Globe and Mail. p. A8 Ha, T. February 6, 2007. Montreal SPCA's Funding Queried. Globe and Mail. p. A6 Haig, B. 1995. University of Canterbury. "Grounded Theory as Scientific Method". Philosophy of Education Society 1995. Copyright 1996 - 2004. Retrieved January 27, 2007 from docs/haiq.html Hall, M., McKeown, L , Roberts, K. 2001. Caring Canadians, Involved Canadians. Highlights from the 2000 National Survey of Giving Volunteering and Participating SGVP. Statistics Canada: Ottawa, Ont.. Retrieved October 13, 2004 from 123 Hall, M. Phillips, S. Meillat, C. Pickering, D. 2003. Assessing Performance: Evaluation Practices and Perspectives in Canada's Voluntary Sector. Canadian Centre for Philanthropy: Toronto, Ontario; and Centre for Voluntary Sector Research and Development: Ottawa, Ont. Norah McClintock (ed). Retrieved October 3, 2005 from en/paqe111Q.cftn Hennessy, M.B., Williams, M.T., Miller, D.D., Douglas, C.W., and Voith, V I . 1998. Influence of male and female petters on plasma Cortisol and behavior: can human interaction reduce stress of dog in a public shelter. Applied Animal Behavior Science, Vol. 61, Issue 1. pp. 63-77. Herman R., Renz, D. 1999. Theses on Nonprofit Organizational Effectivenesss. Nonprofit and Voluntary Sector Quarterly, Vol. 28, no. 2. Sage Publications: Thousand Oaks, CA. pp. 107-126. Retrieved February 21, 2006 from http://nvs.saqepub.eom/cqi/reprint/28/2/107 Hrywna, M. 2006. NPT 2006 Salary Survey. Women Catching Up With Men, But Neither With Inflation. The Nonprofit Times. February 1, 2006. Special Report, p. 23. Retrieved October 5, 2006 from Institute of Medicine (IOM). 2004. Keeping Patients Safe: Transforming the Work Environment of Nurses. The National Academies Press: Washington, DC. pp. 10-13. Accessed on August 15, 2006 at: pdf/10851.pdf Irwin, P. 2003. A Strategic Review of International Animal Protection. Chapter 1. The State of the Animals II. Humane Society Press: Washington, DC. Johnson, L.D., and Pazderka, B. 1995. The Economic and Social Benefits of Stock Markets - It's No Gamble, p. x. The Fraser Institute: Vancouver, BC. Ketchum. September 2005. Philanthropic Trends. Ketchum Canada Inc: Toronto, Ontario. Retrieved October 15, 2006 from 05.pdf Ketchum. Spring 2006. Philanthropic Trends. Ketchum Canada Inc. Toronto, Ontario. Retrieved October 15, 2006 from 06.pdf Lasby, D. 2004. The Philanthropic Spirit in Canada: Motivations and Barriers. Canadian Centre for Philanthropy: Ottawa, Ontario. Retrieved February 21, 2007 from Spirit.pdf Legace, M. 2004. How Non-Profits use the Internet to get Ahead. 3/22/2004. Harvard Business School Working Knowledge: Boston, MA. Retrieved November 18, 2006 from LeRoy, S., Veldhuis, N., Clemens, J. (2002). The 2002 Generosity Index: Comparing Charitable Giving in Canada and the US. The Fraser Institute: Vancouver, BC. Retrieved September 4, 2006 from Letts, C.W., Ryan, W., Grossman, A. 1997. Virtuous Capital: What Foundations Can Learn from Venture Capitalists - Harvard Business Review, Vol.75, Issue 2. Harvard Business School Publishing: Boston, MA. Markin, A. Spring 2004. Why Give? Nuvo Magazine, Vol. 7, no. 1. p. 54. Muttart Foundation. 2006. Talking about Charities 2006 - Report. Retrieved February 25, 2007 from http://www.muttart.orq/download/TAC2006/TAC2006-ExecSummary.pdf 124 Ostrower, F. 2004. Foundation Effectiveness: Definitions and Challenges. Urban Institute Center on Non-Profits and Philanthropy: Washington, DC. Retrieved December 3, 2006 from foundation effectiveness.pdf Philanthropic Foundations Canada. 2007. Montreal, Quebec. Retrieved February 17, 2007 from en/paqe1087.cfm Philanthropy Northwest. Retrieved August 17, 2005 from Plantz, M.C., Greenway, M.T., Hendricks, M. 1997. Outcome Measurement: Showing Results in the Nonprofit Sector. Using Performance Measurement to Improve Public and Nonprofit Programs. In New Directions in Evaluation. Kathryn E. Newcomer (ed). Issue 75. pp. 15-30. Retrieved June 16, 2005 from Pringle, P. February 24, 2007. Reviews Are Less Than Charitable for Nonprofit Film Bash. Los Angeles Times. Retrieved March 14, 2007 from Raising Funds has a Cost. 2006. National Post. November 14, 2006. p. JV8 Roy, D. Environment Enrichment for Cats in Rescue Shelters. Undergraduate Thesis, University of Southampton: Southampton, U.K. Royal Society for the Prevention of Cruelty to Animals. 2007. Retrieved March 15, 2007 from Homepage San Diego Grantmakers Online. 2006. Retrieved February 15, 2005 from Schervish, P. 2001. Transcript from Ben Wattenberg Think Tank special on PBS "How the new philanthropy will change America" July 12, 2001. The Giving Boom: How the New Philanthropy Will Change America. Retrieved September 13, 2006 from transcript.html Schervish, P., O'Herlihy, M., Havens, J. 2002. "Charitable Giving: How Much, By Whom, To What and How?" The Non Profit Sector: A Research Handbook. Woodrow Powell and Richard Steinberg (eds). Yale Press: New Haven, CT. p. 11, pp. 14-15. Smith, A. 2003. The Dance: Seeking, Administering and Accounting for Funding in the Voluntary Sector. The Muttart Foundation. Edmonton, Alberta . pp. 17-21. Retrieved October 13, 2004 from Smith, D.F.E., Durman K.J., Roy, D.B., and Bradshaw, J.W.S. 1994. Behavioral aspects of the welfare of rescued cats. Journal of Feline Advisory Bureau, Vol. 31. pp. 25-28, p. 39. Stephen, J. and Ledger, R. 2005. An Audit of Behavioral Indicators of Poor Welfare in Kenneled Dogs in the United Kingdom. Journal of Applied Animal Welfare Science, Vol. 8, no. 2. pp. 79-96. Statistics Canada. 2005. The Daily, Tuesday November 1, 2005. Statistics Canada: Ottawa, Ont. Retrieved April 11, 2006 from Statistics Canada. 2006. The Daily, Thursday November 23, 2006. Statistics Canada: Ottawa, Ont. Retrieved April 2, 2007 from 125 Streubert, H. and Carpenter, D. 1995. Qualitative Research in Nursing: Advancing the Humanistic Imperative. Lippincott: Philadelphia, PA. The American Boarding Kennels Association. Retrieved March 11, 2007 from The Foundation Center. 2006. Highlights of Foundation Yearbook. Foundations Today Series. Retrieved February 21, 2007 from The Foundation Center. 2005. Foundation Giving Trends. Foundations Today Series. Retrieved February 18, 2006 from The Foundation Center. 2005. Distribution of Foundation Grants by Subject Categories, circa 2003. The Foundation Center's Statistical Information Service. Retrieved February 18, 2006 from fund sub/2003/10 03.pdf The Humane Society of Canada. 2007. Retrieved March 11, 2007 from: The Urban Institute, National Center for Charitable Statistics. 2006. Retrieved October 31, 2006 from United Way of America. 1996. Measuring Program Outcomes: A Practical Approach. United Way of America: Alexandria, VA. Excerpt retrieved June 16, 2005 from http.V/national. United Way of America. 2005. Retrieved June 16, 2005 from http://national.unitedway.Org/outcomes/faq.cfm#Q1 Vanguard Charitable Endowment Program. 2004. Briefings. Spring 2004. Do Americans Give Enough to Charity. Vol. XI p. 1. Retrieved February 25, 2007 from http://www.vanquardcharitable.Org/newsletter/b spring2004.pdf Webster, J. 2005. UFAW Animal Welfare Series. Animal Welfare: Limping Towards Eden. Blackwell Publishing: Oxford, UK. Weiss, C. 1998.Evaluaton. 2 n d Edition. Prentice-Hall, Inc. Simon and Schuster/A Viacom company: Upper Saddle River, NJ. pp. 83-86 Wren, D. 2006. Nonprofit experts question Five Rivers' tax filings - Program's expenses could be inflated. The Sun News. Myrtle Beach, SC. Retrieved March 14, 2007 from packages/investigating f ive rivers/15317721 .htm?template=contentModules/printstory.isp Ziedberg, S. 2006. Poor Charlie's Almanac: The Wit and Wisdom of Charles T. Munger. Expanded Second Edition. Peter D. Kaufman (ed). The Donning Company Publishers: Virginia Beach, VA. p. 25 126 Appendices Appendix I - V 1 : AWO Questionnaire (February 21,2005) Questionnaire for Animal Welfare Organizations (AWOs) Employees: 1. What is your ratio o f management to staff? 2. What is your staff to volunteer ratio? 3. What is your staff-turnover rate? 4. What is your volunteer-turnover rate? 5. H o w often does management meet w i t h the staff? 6. H o w often does management meet w i th volunteers? 7. D o y o u prov ide external tra ining for your staff? 8. What is the Presidentia l turnover rate over the past 10 years? 9. H o w long has your current Pres ident/CEO held his/her pos i t ion? 10. B y what percentage do you increase salary compensat ion each year? 11. H o w do your staff salaries compare w i th industry? 12. H o w do your management salaries compare w i th industry? 13. What is the background o f your C h i e f F inanc ia l Off icer/Finance Manager ? Board of Directors 14. H o w many ind iv idua ls sit on your board o f directors? 15. What is the ratio o f internal management representatives versus external board members? 16. H o w are they elected (i.e. by what criteria), and h o w long is their term before re -elect ion? 127 17. What ski l ls/background do you require external board members to have? 18. H o w often does the management team meet w i th the Board? 19. H o w often does the board meet independent o f management? 20. H o w w o u l d you describe the relat ionship between management and the Boa rd? 21 . W h e n was the last change-over o f board members? F i n a n c e s 22. F r o m what sources (and h o w many sources) do y o u receive your funding? 23 . What is the percentage breakdown o f your fund ing - please show area that consumes the most funds, and the least? 24. What is your annual budget for: a. A n i m a l hous ing improvements? b. A n i m a l toys and accessories c. A n i m a l veterinary care and medic ines? d. A n i m a l food 25. What area o f your operat ion is the most under-funded? 26. I f you cou ld double your fund ing, what w o u l d you do dif ferent ly? 27 . H o w do you manage your current funds to sustain and generate cash f l o w (i.e. invest i n mutual fund, bonds, savings account, etc.)? 28. A r e your funds managed external ly b y a cert i f ied accountant or f inanc ia l analyst? 29. H o w was this f i rm selected? L i a b i l i t y 30. H o w many lawsuits are f i l ed against you per year? 31 . H o w many lawsuits do y o u win/lose? 32. H o w m u c h have these lawsuits, i f any, cost you over the past 5 years? 33 . H o w m u c h have y o u budgeted, i f any, for lawsuits i n the com ing years? 128 Performance Objectives: 34. D o y o u set 1/3/5 year objectives for your organizat ion? I f so, what were they? 35. D o y o u assess achievement o f these objectives us ing internal staff or an external , independent 3 r d party? 36. Have y o u achieved your objectives over the past year, 3 years and 5 years? I f not, w h y not? 37. What is the achievement y o u are most proud of? 38. What object ive has most eluded you? 39. D o y o u currently conduct annual performance evaluations? 40. W h o currently pays for these performance evaluations? W h o should pay? 4 1 . What other statistics or audited reports do you measure and mainta in? Animal Care 42. H o w many animals are under your care or protected by your organizat ion per year? 43 . H o w many animals has your organizat ion released/adopted/relocated over the past 1 year and 3 years? 44. H o w many animals d ied or were euthanized under your care or protect ion i n the past 1 year and 3 years? 45 . What is your organizat ion 's po l i c y towards an imal care (i.e. this is a broad category and inc ludes, among others, pol ic ies on rehabi l i tat ing and/or re leasing in jured animals , euthanasia o f unwanted or injured animals , capturing and hous ing w i l d an imals , hous ing and cu l l i ng o f farm animals , and car ing for native versus non-native species)? 129 Appendix II-V1: Foundation Questionnaire (Nov 9,2004) Questionnaire for Funding Agencies and Philanthropists Finances: 1. H o w many A n i m a l Wel fare Organizat ions ( A W O s ) do you fund per year? 2. What is the average term o f your funding agreements? 3. D o y o u allocate funds evenly amongst a l l A W O s that apply? I f so, what is the average amount o f your fund ing agreements per A W O ? I f not, what is your m i n i m u m and m a x i m u m donat ion to a single organizat ion? 4. D o y o u allocate fund ing based upon special projects be ing undertaken b y that A W O ? 5. D o y o u fund general operations o f an A W O ? 6. What do y o u require A W O s do w i th the funds once granted (i.e. must they invest them, put them in a trust, be held by an independent party, spend them w i th in the year, etc.)? A W O Employees: 7. H o w often do you meet w i th an A W O s management team? 8. H o w often do you meet w i th the A W O s board o f directors? 9. D o y o u assess the educat ion and experience o f the staff/management o f A W O s you fund? 10. D o y o u assess staff, management and volunteer turnover rates? 11. D o y o u assess the relat ionship between management and the board? 130 Performance Objectives: 12. D o y o u expect the A W O s you fund to set annual strategic objectives and business plans? 13. What statistics and reports do you expect and/or receive f r om A W O s ? 14. D o y o u pay for performance evaluations o f A W O s ? 15. What in format ion about A W O s is important i n mak ing your fund ing dec is ion? 16. W h i c h cr i ter ia do you weight more heav i ly than others? W h y ? 17. H o w do you compare s imi lar A W O s seeking the same funds? 18. H o w do you compare diverse A W O s seeking the same funds? General: 19. What other aspects o f phi lanthropy are important to y o u (e.g. tax benefits, desire to do good, etc.)? 20. What is the biggest bottleneck i n the dec is ion-making process? 2 1 . H o w do y o u account for ind iv idua l biases, mora l ph i losophies and cultural values i n determining what A W O s get funding? 22. H o w do y o u deem one cause as be ing more important than another? 23 . A r e your fund ing decis ions impacted by an A W O s legal history (i.e. the number or nature o f l aw suits f i l ed against them, or f i l ed b y them against others)? Animal Care: 24. D o y o u assess an imal care statistics (e.g. euthanasia rates, release rates, adopt ion rates, hous ing environments, veterinary budget, f ood budget, etc.) before mak ing fund ing decis ions? 25. What are the three most important an imal care statistics or issues y o u assess i n mak ing fund ing decis ions? 131 Appendix III - V2: AWO Survey-Questionnaire (June 9,2005) Questionnaire for Animal Welfare Organizations (AWOs) A) Relevance of Possible Performance Criteria Please indicate with an "x" whether the following evaluation criteria would be very important, important, or not important in providing a fair and accurate assessment of an AWO's performance level (both good and poor performance). If you are not sure of its importance, or unsure of the question, please place an "x" in the "Don't Know" column: Evaluation Criteria Board of Directors Very Important Important Not Important Don't Know Election process Number of Board members Length of Board member term Volunteer (not paid) Board Education and experience Involvement in AWO's strategic direction & planning Frequency of Board meetings Public access to Board minutes Volunteers Very Important Important Not Important Don't Know Total # of volunteers # of active volunteers (i.e. 2 hrs per week every week for 1 year) # of volunteer hours per year # of volunteer hours per week # of volunteers on site per day Volunteer to staff ratio Average length of term served by volunteers # of volunteer-run community events Auditable tracking system of volunteer activities 132 Employees Very Important Important Not Important Don't Know Total # of employees # of staff to management ratio Staff to volunteer ratio Staff and management salaries Staff turnover rate Management turnover rate Average employee term Staff & management training/professional development budget Strategic Planning Production of a formal business plan with 1, 3 and 5 year objectives >75% achievement of objectives Funding Sources Very Important Important Not Important Don't Know Total # of donors # of donors < $100/yr # of funders granting >$1000/yr Donor satisfaction surveys Communications/Updates to Donors Meetings with Donors Donations in kind (e.g. food, supplies, services) Active monthly donation program (automated withdrawal from donors) Variety of funding programs, options and projects External investment firm used to manage/invest funds 133 Fundraising & Marketing Very Important Important Not Important Don't Know Internal versus external telemarketing effort Commissioned versus flat-fee for service telemarketing firm AWO product sales (self-generated cash flow) Response or return rate on marketing campaigns Address a variety of animal species and welfare issues Conduct multiple programs -hands-on care, advocacy, education, etc. Ratio of fundraising staff to funds raised Total cost of fundraising as a percentage of funds raised # of media events (coverage in TV, newspaper and magazines) Animal Care (for shelters, sanctuaries and rehab centers) Very Important Important Not Important Don't Know # of animals brought in # of animals seized # of natural deaths # of animals euthanized # of animals housed per year # of days in care prior to death Min, max and avg days in care # of animals released/adopted # of animals in permanent care Public Outreach (for lobbyists, advocacy and education groups) Very Important Important Not Important Don't Know # of people reached (via circulation / viewership) # of public events # of meetings with government/industry officials Note: * A later version of this survey on June 21, 2005 changed one question under Strategic Planning from ">75% achievement of objectives" to "Adherence to previous year's budget(s)" 134 B) Performance Criteria In answer ing the f o l l ow ing questions, consider the cr iter ia f r om the prev ious table, a long w i t h your o w n experience i n speaking w i th major donors/philanthropists and i n comple t ing grant appl icat ions for foundations or grant agencies. 1. What cr i ter ia best demonstrates an effective and product ive A W O ? 2. What cr i ter ia best indicates an ineffect ive A W O ? 3. What in format ion do funding sources ask for that is unnecessary (i.e. what should be e l iminated f rom the grant appl icat ion)? 4. What in format ion don't fund ing sources ask for that they should be requesting? 5. What in format ion are you required to prov ide that misrepresents the true effectiveness or stabi l i ty o f your A W O ? C) Performance Evaluation 6. What are your thoughts on performance evaluations (t ime, value, costs)? 7. What are the c o m m o n attributes amongst diverse A W O s that can be fa i r l y d isc losed, assessed and compared? 8. W o u l d a c o m m o n set o f cr i ter ia or evaluat ion template rank ing business pr inc ip les and operational aspects only (to a l l ow compar ison across diverse A W O s ) be useful? 9. C a n performance standards on ly be developed amongst c o m m o n types o f A W O s (i.e. shelters vs. shelters, advocacy vs. advocacy, rehab vs rehab, etc.)? 10. What w o u l d be the benefit o f c o m m o n , standard evaluat ion criteria? 11. What w o u l d be the disadvantage o f c o m m o n , standard evaluat ion criteria? 135 12. W h o should conduct performance evaluations - the A W O itself, fund ing agencies, a national/local representative o f A W O s , an independent audit f i rm? 13. Shou ld those ind iv idua ls assessing performance at year-end also be i nvo l ved i n in i t ia l d iscussions to set the objectives? 14. W h o should pay for the evaluation? 15. H o w does an A W O demonstrate that it has truly made a difference - i.e. h o w do y o u measure outcome or impact? 16. Is di f ferent iat ion essential, or should A W O s re-invent the whee l , repeating the w o r k o f other A W O s in the same jur i sd ic t ion -i.e. is compet i t ion i n the A n i m a l Wel fare sector good or bad? 17. Identify at least one desired performance outcome for organizat ions i nvo l ved i n the f o l l o w i n g act iv i t ies: Activity/Organization Desired Performance Outcome C o m p a n i o n an imal shelter/adoption A n i m a l seizure/rescue W i l d l i f e rehabi l i tat ion W i l d l i f e conservat ion F a r m an imal welfare advocacy Humane fa rming cert i f icat ion Aquar ium/Zoo fac i l i ty Aquar ium/Zoo accreditat ion L a b an imal research L a b an imal advocacy Additional Comments? 136 Appendix IV - V2: Foundation Survey-Questionnaire (June 21,2005) Questionnaire for Funding Agents and Philanthropists A) Relevance of Possible Performance Criteria Please indicate w i t h an " x " whether the f o l l o w i n g evaluat ion cr i ter ia w o u l d be very important, important, or not important i n p rov id ing a fair and accurate assessment o f an A W O ' s performance leve l (both good and poor performance). I f y o u are not sure o f its importance, or unsure o f the question, please place an " x " i n the " D o n ' t K n o w " c o l u m n : Evaluation Criteria Board of Directors Very Important Important Not Important Don't Know Election process Number of Board members Length of Board member term Volunteer (not paid) Board Education and experience Board involvement in AWO's strategic direction & planning Frequency of Board meetings Public access to Board minutes Volunteers Very Important Important Not Important Don't Know Total # of volunteers # of active volunteers (i.e. 2 hrs per week every week for 1 year) # of volunteer hours per year # of volunteer hours per week # of volunteers on site per day Volunteer to staff ratio Average length of term served by volunteers # of volunteer-run community events Auditable tracking system of volunteer activities 137 Employees Very Important Important Not Important Don't Know Total # of employees # of staff to management ratio Staff to volunteer ratio Staff and management salaries Staff turnover rate Management turnover rate Average employee term Staff & management training/professional development budget Strategic Planning i Production of a formal business plan with 1, 3 & 5 yr objectives Adherence to previous year's budget(s) Funding Sources Very Important Important Not Important Don't Know Total # of donors # of donors < $100/yr # of funders granting >$1000/yr Donor satisfaction surveys Communications/Updates to Donors Meetings with Donors Donations in kind (e.g. food, supplies, services) Active monthly donation program (automated withdrawal from donors) Variety of funding programs, options and projects External investment firm used to manage/invest funds 1 3 8 Fundraising & Marketing Very Important Important Not Important Don't Know* Internal versus external telemarketing effort Commissioned versus flat-fee for service telemarketing firm AWO product sales (self-generated cash flow) Response or return rate on marketing campaigns Address a variety of animal species and welfare issues Conduct multiple programs -hands-on care, advocacy, education, etc. Ratio of fundraising staff to funds raised Total cost of fundraising as a percentage of funds raised # of media events (coverage in TV, newspaper and magazines) Animal Care (for shelters, sanctuaries and rehab centers) Very Important Important Not Important Don't Know # of animals brought in # of animals seized # of natural deaths # of animals euthanized # of animals housed per year # of days in care prior to death Min, max and avg days in care # of animals released/adopted # of animals in permanent care Public Outreach (for lobbyists, advocacy and education groups) Very Important j Important Not Important Don't Know # of people reached (via circulation / viewership) # of public events # of meetings with government/industry officials 139 B) Performance Criteria (inputs, activities and outputs) In answer ing the f o l l o w i n g questions, consider the cr i ter ia f r om the prev ious table, a long w i t h your o w n experience i n speaking w i th A W O s and rev iew ing grant appl icat ions by A W O s requesting funds. 1. What cr i ter ia best demonstrates an effective and product ive A W O ? 2. What cr i ter ia best indicates an ineffect ive A W O ? 3. What in format ion do A W O s prov ide that is unnecessary (i.e. what should be e l iminated f rom the grant appl icat ion)? 4. What in format ion is m iss ing f rom the grant appl icat ion (what should be inc luded that i sn ' t be ing prov ided for reasons o f conf ident ia l i ty , d i f f i cu l t l y i n measur ing, etc.)? 5. What in format ion do A W O s prov ide that misrepresents the true effectiveness or stabi l i ty o f an A W O (e.g. income statement amort izat ion or goodwi l l ) ? C) Performance Evaluation 6. What are the c o m m o n attributes amongst diverse A W O s that can be fa i r l y d isc losed, assessed and compared? 7. W o u l d a c o m m o n set o f cr i ter ia or evaluat ion template rank ing business pr inc ip les and operational aspects only (to a l l ow compar ison across diverse A W O s ) be useful? 8. C a n performance standards on ly be developed amongst c o m m o n types o f A W O s (i.e. shelters vs. shelters, advocacy vs. advocacy, rehab vs rehab, etc.)? 9. What w o u l d be the benefit o f c o m m o n , standard evaluat ion criteria? 10. What w o u l d be the disadvantage o f common , standard evaluat ion criteria? 11. What are your thoughts on performance evaluations (t ime, value, costs)? 140 12. W h o should conduct performance evaluations - the A W O itself, fund ing agencies, a national/local representative o f A W O s , an independent audit f i rm? 13. Shou ld those ind iv idua ls assessing performance at year-end also be i nvo l ved i n in i t ia l d iscussions to set the objectives? 14. W h o should pay for the evaluation? D) Performance Outcomes 15. Is di f ferent iat ion essential, or should A W O s re-invent the whee l , repeating the w o r k o f other A W O s in the same ju r i sd i c t ion -i.e. is compet i t ion i n the A n i m a l We l fare sector good or bad? 16. H o w does an A W O demonstrate that it has truly made a difference - i.e. h o w do y o u measure outcome or impact? 17. Identify at least one desired performance outcome for organizat ions i nvo l ved i n the f o l l o w i n g act iv i t ies: Activity/Organization Desired Performance Outcome C o m p a n i o n an imal shelter/adoption A n i m a l seizure/rescue W i l d l i f e rehabi l i tat ion W i l d l i f e conservat ion F a r m an imal welfare advocacy Humane fa rming cert i f icat ion Aquar ium/Zoo fac i l i ty Aqua r ium/Zoo accreditat ion L a b an imal research L a b an imal advocacy Additional Comments? 141 Appendix V - Sample Interview Questions to AWOs and Funders To AWOs - brief version In preparation for the interview, give some thought to the following areas: First of all, what is the biggest bottleneck in the grant decision-making process (i.e. applying for and receiving grants or donations). How it could be improved? Secondly, we will talk generally about financial statements. Do you need to alter your financial statements and budgets in any way to meet the needs of funding applications and requirements -i.e. do you have to "repackage" staffing and operational costs as program expenses, etc.? What do you feel is an acceptable over-budget percentage, if any? Thirdly, we will spend some time on identifying performance outcomes (as opposed to inputs, outputs and activities) for the various AWO types (see question 17 in the attached document). In particular, I'm interested in your thoughts on how an advocacy organization like yourself should be measured, given that many of your results (i.e. legislative change, bans, change in public opinion, etc.) are hard to measure and take a long time to come into effect. Fourthly, we will go through the attached survey/questionnaire and get your opinion on the importance of the various criteria. I am actually using this document for all new participants in place of the original questionnaire you completed. Perhaps you could review it prior to our meeting and highlight the items you most wish to discuss, either because you deem them to be very important, or not sure of what importance they play. Finally we'll close with your thoughts on the following hypothetical situation: You are an individual donor with $3,000 to give to an AWO. What information could that AWO provide you that would convince you they are the right organization to which you should donate your funds? In other words, how should they demonstrate their productivity, effectiveness and ability to make a difference? If possible, pick 5 criteria (sort of an "elevator pitch") that you would want to hear about. To Funders - brief version: 1. Elaborate further about how you would recognize a proposal or project that would bring about "systems-level" changes, a term you used in your survey response? 2. With regards to AWO financial statements and budgets, what are the red flags you look for, what is the optimal way for AWOs to lay out their revenues and program expenses, and what is an acceptable over-budget percentage, if any? 3. We will spend some time on identifying performance outcomes (as opposed to inputs, outputs and activities) for the various AWO types (see question 17 in the attached document). 4. We will go through the attached survey/questionnaire and get your opinion on the importance of the various criteria. I am actually using this document for all new participants in place of the original questionnaire you completed. Perhaps you could review it prior to our meeting and 142 highlight the items you most wish to discuss, either because you deem them to be very important, or not sure of what importance they play. 5. Many funding agencies and AWOs say the way to identify effective from ineffective organizations is in their ability to achieve the goals stated in their mission statement, and in keeping their activities in line with their stated objectives (i.e. not changing mid-way through the year to chase a grant, or chase a trend). How do you know when the reported performance is factual, how do you know when the mission statement truly reflects their goals and activities and how do you identify when an AWO is misrepresenting itself and it's objectives? 6. Finally we'll close with your thoughts on the following hypothetical situation: You are an individual donor with $3,000 to give to an AWO. What information could that AWO provide you that would convince you they are the right organization to which you should donate your funds? In other words, how should they demonstrate their productivity, effectiveness and ability to make a difference? If possible, pick 5 criteria (sort of an "elevator pitch") that you would want to hear about. To AWOs - Detailed Version (note: interview questions to participants varied depending on survey-questionnaire responses) 1. Board Members a) What type of education and experience is most important for Board members to bring to the table? b) Are Board member donations to the AWO important? c) Do you feel there has been an over-professionalization of AWO Boards (and lack of grassroots people), or are professionals needed at the Board level to support the organization properly? d) What kind/level of involvement and activity should Board members have in an AWO? 2. Volunteers a) What is an appropriate benchmark for a volunteer to staff ratio? b) What indicates a good volunteer program? c) How and why do volunteer ratios and their importance differ between different types of AWOs -i.e. shelters, versus wildlife rehab, versus advocacy groups? 3. Employees a) What should donors look for in the management team of an AWO? b) How do the skill sets and/or strategies differ between the for-profit and non-profit world? c) What is a good benchmark for a staff to management ratio? d) What is the appropriate amount of time for a founder to remain in the Executive position? 4. Funding Sources a) Is there an ideal breakdown or benchmark of funding sources - i.e. how much should come from individual donors/members, foundations, corporate sponsorships and program generated revenues? b) What types of communications with donors are important? c) Why are donations in kind not important? d) Should all medium-to-large AWOs use an external investment firm to manage their funds versus a simple savings account? e) What reserve level (% of total income) should any AWO have on hand? 5. Fundraising and Marketing a) What is a good response or return rate on marketing campaigns? b) What is a good staff-to-funds ratio or benchmark to strive for - i.e. how many staff for every $1m raised? 143 c) What is a good benchmark for fundraising cost per funds raised - i.e. how many cents per $1.00 raised? d) So long as these benchmarks are achieved, does it matter if they pay sizeable commissions to a telemarketing firm? 6. Animal Care a) What do statistics like average days in care, number of animals released and/or in permanent care tell you? b) Can you provide some examples of what would be important for any AWO in terms of quality of care? c) Would number of staff to animals "housed" be important? 7. Mission Statements and Objectives a) In addition to adequate staffing and length of service, how can donors be sure that the stated missions, objectives and achievements of AWOs are viable and and accurate? What indicators, sources of information should they research to back up these claims? b) Elaborate on why donors should be cautious about family members on a board, or AWOs where they use their home as the facility c) What are some other red flags? d) Are peer reviews effective? e) How should one fairly compare results of a large, well-established AWO against a new, small, start-up AWO? 8. Grant Process a) What is the biggest bottleneck in the grant making process? b) Is there a better way for AWOs to layout program expenses versus true overhead costs? 9. Competition a) Is there any way as a funder you could encourage group collaboration to centralize administration, with satellite offices for community shelters? b) What should/could be centralized? 10. Performance Outcomes a) Can you think of any other outcomes, short and long term, for the AWOs listed in question 17? b) What measurements or indicators could an AWO show you to prove they succeeded at increasing public awareness and educating the public? Finally we'll close with your thoughts on the following hypothetical situation: You are an individual donor with $3,000 to give to an AWO. What information could that AWO provide you that would convince you they are the right organization to which you should donate your funds? In other words, how should they demonstrate their productivity, effectiveness and ability to make a difference? If possible, pick 5 criteria (sort of an "elevator pitch") that you would want to hear about or read about on a charity ranking web site. To Funders - Detailed version (note: interview questions to participants varied depending on survey-questionnaire responses) 1. Board Members a) What type of education and experience is most important for Board members to bring to the table? b) Are Board member donations to the AWO important? c) Do you feel there has been an over-professionalization of AWO Boards (and lack of grassroots people), or are professionals needed at the Board level to support the organization properly? d) What kind/level of involvement and activity should Board members have in an AWO? 144 2. Volunteers a) What is an appropriate benchmark for a volunteer to staff ratio? b) What indicates a good volunteer program? c) How and why do volunteer ratios and their importance differ between different types of AWOs -i.e. shelters, versus wildlife rehab, versus advocacy groups? 3. Employees a) What should donors look for in the management team of an AWO? b) How do the skill sets and/or strategies differ between the for-profit and non-profit world? c) What is a good benchmark for a staff to management ratio? d) What is the appropriate amount of time for a founder to remain in the Executive position? 4. Funding Sources a) Is there an ideal breakdown or benchmark of funding sources - i.e. how much should come from individual donors/members, foundations, corporate sponsorships and program generated revenues? b) What types of communications with donors are important? c) Why are donations in kind not important? d) Should all medium-to-large AWOs use an external investment firm to manage their funds versus a simple savings account? e) What reserve level (% of total income) should any AWO have on hand? 5. Fundraising and Marketing a) What is a good response or return rate on marketing campaigns? b) What is a good staff-to-funds ratio or benchmark to strive for - i.e. how many staff for every $1m raised? c) What is a good benchmark for fundraising cost per funds raised - i.e. how many cents per $ 1 . 0 0 raised? d) So long as these benchmarks are achieved, does it matter if they pay sizeable commissions to a telemarketing firm? 6. Animal Care a) What do statistics like average days in care, number of animals released and/or in permanent care tell you? b) Can you provide some examples of what would be important for any AWO in terms of quality of care? c) Would number of staff to animals "housed" be important? 7. Mission Statements and Objectives a) In addition to adequate staffing and length of service, how can donors be sure that the stated missions, objectives and achievements of AWOs are viable and and accurate? What indicators, sources of information should they research to back up these claims? b) Elaborate on why donors should be cautious about family members on a board, or AWOs where they use their home as the facility c) What are some other red flags? d) Are peer reviews effective? e) How should one fairly compare results of a large, well-established AWO against a new, small, start-up AWO? 8. Grant Process a) What is the biggest bottleneck in the grant making process? b) Is there a better way for AWOs to layout program expenses versus true overhead costs? 9. Competition a) Is there any way as a funder you could encourage group collaboration to centralize administration, with satellite offices for community shelters? 145 b) What should/could be centralized? 10. Performance Outcomes a) Can you think of any other outcomes, short and long term, for the AWOs listed in question 17? b) What measurements or indicators could an AWO show you to prove they succeeded at increasing public awareness and educating the public? Finally we'll close with your thoughts on the following hypothetical situation: You are an individual donor with $3,000 to give to an AWO. What information could that AWO provide you that would convince you they are the right organization to which you should donate your funds? In other words, how should they demonstrate their productivity, effectiveness and ability to make a difference? If possible, pick 5 criteria (sort of an "elevator pitch") that you would want to hear about or read about on a charity ranking web site. 146 Appendix VI - UBC Behavioral Ethics Review Board Certificates of Approval February 21, 2005 - Original Approval Certificate (see page 148) May 30, 2005 - Re-approval for changes to questionnaire format and content (see page 149) April 27, 2006 - Continuance approval for 1 additional year (see page 150) 147 


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items