Annapolis Valley Event & Sport Tourism Association

SECTION 9: COMMON ERRORS IN ECONOMIC IMPACT STUDIES

This section identifies and discusses some of the common errors and omissions committed in economic impact assessment. Some of the errors are quantified using a case study of recreation-based tourism events held in Kentville, Nova Scotia.

There exists a variety of methodological approaches and levels of detail applicable in studying economic impacts. As a result of this variability, suspicion and skepticism of the accuracy in assessing economic impacts has become prevalent. Such skepticism is understandable given that two groups studying the same or similar impact scenario can come up with significantly different results. For example, the economic impact of professional football was estimated to be approximately $13 million in Atlanta in 1972 (Shaeffer and Davidson, 1972) and $200,000 in Baltimore in 1983 (Baade and Dye, 1990).

Errors in conducting impact analysis can occur in a wide variety of ways and at various stages of the assessment process. In what follows four common errors that reduce the efficacy of economic impact analysis will be discussed in turn. The errors to be discussed relate to definition of study area, inclusion of local resident expenditures, focusing on expenditure impacts instead of income impacts, and data collection.

Error 1

Inadequate Definition of the Study Area

The appropriate region for an impact analysis is dependent upon the size of the event. Event size represents a critical element in identifying the area that will likely be impacted. Choosing the appropriate area for the impact analysis necessitates equating the demand for goods and services related to the event, to the supply of these goods and services available at given levels of political aggregation (town(s), county(s), provinces(s) etc.). Often, however, the designation of the impact region is dependent upon the party commissioning the study.

While the effects of the festival (event) may be spread over some distance, local supporters may want only to show the effects of this event on the local community or even on a narrow segment of the community. In other cases, they may want to emphasize the spillover of benefits into other communities in an attempt to gain broader support (Davidson & Shaffer, 1980, p.14).

The availability of data is also a consideration. Published multipliers are likely available at the provincial level (state or county level in the USA) but not for sub-county regions or for single cities or communities. However, an available multiplier for a province (or even a county) may simply not be suitable for a city or town where the event actually takes place. The applicability depends upon the size of the event and the spatial distribution of the impacts.

The definition of the impact area can have a significant effect on the analysis by having an effect on the following:

The level of non-local spending - As the impact area is reduced, non-local spending increases as participants formerly designated as locals, become visitors.
Direct spending accruing to impact area.- When the impact area is reduced, the direct expenditures captured by local firms are reduced as more sales fall outside the local region.
The multiplier.- If the impact area is reduced, the multiplier is reduced as the economic linkages within the designated area are diminished so secondary spending within the study region declines and secondary spending outside the region rises.

The magnitude of error associated with including local expenditures in the impact analysis was calculated for a 100 team slow pitch event held in Kentville in 1986 (Van Blarcom, 1996). A reduction of the impact area, for this study to include only the town of Kentville (eliminating the surrounding communities) would have the following effects:

Number of non-local teams increases from 83 to 92. Non-local visitor expenditure would rise by $5225, an increase of 3.8 percent of the total direct expenditure.
All accommodation expenditures would fall outside the impact area, as would an estimated 50 percent of all other expenditures with the exception of registration fees. Total (impact area) direct expenditure would fall to $65,433 from 128,944. A reduction of 51 percent.
Reducing the impact area leads to reductions in the size of the multiplier. This occurs because local firms, who sell directly to visitors, will now purchase fewer goods and services (inputs) within the condensed local area. Further, the number of local employers and employees deriving income from visitor expenditures (directly and indirectly) as well as respending their income in the local area will be reduced. As a result, the level of economic activity within the specified local area shrinks. Quantifying the reduction in the multiplier for the reduced local area is beyond the scope of this study, as it would require re-surveying businesses and residents and delineating revenue and expenditure flows among these communities. The income multiplier for the original study area should not be used to calculate the total income effect for the condensed study area. This is because the former reflects the economic structure unique to the original study area.

Error 2

Inclusion of Local Resident Expenditures

Event-related expenditures made by local residents do not represent an injection of new money into the local economy and therefore should not be included in the impact assessment. For this group of participants/spectators, event related expenditures represent a re-circulation of money that would have been spent locally in absence of the event. To the extent that local participants/spectators may increase their weekend expenditures given the presence of the event, they are likely to simply reduce their expenditures in the future; thereby eliminating any net increase in local spending by this group. As Crompton (1995) noted, the economic impacts attributable to special events relate only to new money injected into the economy by visitors from outside the community. Inclusion of local participant/spectator expenditures exaggerates the impact of the event.

One exception to the rule of excluding local resident expenditures in the impact analysis should be noted. If the event keeps residents at home that would have otherwise left the area (perhaps to attend a similar event held elsewhere) then their expenditures should be included in the impact analysis. Such expenditures are considered to be incremental (represent new money) in the sense that it is money retained in the host community that otherwise would have been lost. Identification of such expenditures requires specific survey questions related to this issue to be completed by local residents. Such questions are complicated by the fact that the impact on the local economy relates not to what these locals spent at home during the event but to what they would have spent in going elsewhere. These questions are hypothetical in nature and require projections on behalf of the respondents as to what would have been spent in going to an outside event. Using residents' expenditures related to the event held at home as a substitute for spending at an event outside the community is inappropriate in that they may or may not be equivalent. Differences in spending could result from a number of factors including the nature of the events, the need for transportation and accommodation, etc. The reliability of projected spending at outside events is contentious and many economists recommend that all expenditures by local residents be ignored (Baade and Dye, 1990; Smith, 1989).

The error associated with the inclusion of local participant expenditures for the previously mentioned 100 team slow pitch tournament in Kentville is calculated below. Table E2.1shows local participant direct expenditures were $7,319. Total expenditures with the inclusion of local participants were $136,263 compared to $128,944 without this group. Inclusion of local participant expenditures inflates the impact by approximately six percent. Although locals made up 17% of total participants for this event, they had significantly lower weekend expenditures relative to the other visitor types. Events with a larger percentage of local participants would see an increase in the relative magnitude of such an error.

Error 3

Concentrating on the Expenditure Multiplier Instead of Income Multiplier

There exists a great deal of confusion over the use of expenditure (sales) multipliers versus income multipliers in economic impact analysis. An expenditure multiplier relates the direct, indirect and induced effects of visitor spending to the total volume of expenditures (sales transactions) generated in the impact area. An income multiplier relates the direct, indirect and induced effects of visitor spending to income generation in the local area. Fridgin (1991) clearly related that it is misleading to multiply visitor expenditures by an expenditure multiplier and call the resulting figure the economic impact of that injected money on residents of the host community.

In economic impact analysis, sales multipliers are often not of primary concern. The variable of interest is the impact of those sales on local household incomes. The key element is how much extra income the local area receives from visitor expenditures. However, since sales multipliers are significantly greater than income multipliers, there is a tendency for those looking to maximize the impacts, to use the former. The implication of using an expenditure multiplier versus an income multiplier for the slow pitch tournament in Kentville is calculated below. It must be noted that no input-output model and associated multipliers are available for the study area (Kentville and surrounding area). The impact analysis was conducted using the basic multiplier format put forth by Archer and Owen (1972).

Use of an Expenditure Multiplier Gives a Total Impact of:

Where:

S= Expenditure Multiplier
B= that proportion of income earned by local area residents re-spent on local goods and services.
C= proportion of local business sales paid out to local employers and employees.
$128,944= direct visitor expenditures
$164,974= total (direct, indirect and induced) expenditures

Use of an Income Multiplier Gives a Total Impact of:

Where:

I= income multiplier.
A= the proportion of original visitor expenditures remaining in the impact area.
B and C as shown above.
$128 944= direct visitor expenditures
$51 142= total (direct, indirect and induced) income

Note: A, B and C were calculated using information from a survey of local businesses, residents and published data.

The total expenditure impact of $164,974 is over three times the total income impact of $51,142. These figures clearly illustrate the danger of using expenditure multipliers when the variable of interest is the impact of visitor expenditures on local incomes.

The large difference in the impacts occurs because the expenditure multiplier incorporates total visitor expenditures in the calculation, whereas the income multiplier includes only the fraction of total visitor expenditure remaining in the local area. The income multiplier allows for the percentage of visitor receipts that flow to firms outside the region in the form of payments for material inputs, as well as payments to persons outside the region in the form of wages and profits. This money is not re-circulated (multiplied) because it does not remain in the local economy.

"Residents are interested in knowing how much extra income the host community will receive from the injection of funds from visitors. They are not interested in the value of sales per se because it has no impact on their standard of living". (Crompton, 1995, p.21).

Sales figures are not truly indicative of the economic benefits accruing to a community to the extent that receipts from visitor expenditures flow immediately to businesses and individuals outside the local economy.

Error 4

Data Collection

There are numerous pitfalls in data collection efforts that can significantly reduce or destroy an economic impact study's ability to produce credible results. Below are some of the issues related to data collection that frequently create problems in economic impact studies.

Sample Size

The sample must be of sufficient size to generate reliable estimates of visitor spending. Choose what you consider to be an acceptable confidence level and error in estimation and an accepted formula to calculate required sample size. Note that segmenting the visitors by type (for example those requiring accommodation versus those who do not) can effectively reduce variation in responses within visitor type and reduce the total required sample size by reducing the sample size necessary to get reliable estimates for each of the visitor types. If visitor types are not identified until after the surveying has taken place, extreme caution must be used in estimating average expenditure for visitor types that have a limited number of observations.

Representative Sample

The premise which underlies being able to use a sample to estimate the characteristics of a larger group (population from which the sample was taken) is that the sample is reflective of the population. There is a variety of sampling strategies aimed at achieving this goal, for example random versus stratified quota (segmented) approaches. Every effort must be made selecting a sample (i. e. who, where, when) that will accurately portray the survey population.

Purpose of Visit

When assessing the economic impact of an event, only expenditures by visitors who are in the local area because of the event are relevant. A question that identifies whether or not the specified event is the reason for the respondents visit must be included.

Definition of Party Making Expenditures

A "party" is usually defined as a group of persons who travel in the same vehicle or stay in the same room when requiring accommodation. It is critical when recording party expenditure figures on the surveys that the respondent and the researcher are in agreement as to exactly whom the expenditure totals include. The "party" concept is most applicable for families where one parent (or both together) is responsible for expenditures and therefore can provide accurate estimates of spending for the group as a whole. Adults, while traveling and staying together may be responsible solely for their own expenditures (including a share of gas/accommodation spending) and are less likely to be able to provide accurate measures of spending for their party. It is generally recommended that adults, without any accompanying family, provide spending estimates for only themselves. Accuracy in identifying exactly for whom the expenditure are applicable, is critical.