9. ESTIMATING CHARACTERISTICS OF THE POPULATION

POOLING DATA

We have several options as to how to use the responses from the two strata (Apparent Minority and Others) to estimate characteristics of the populations of small, emerging, minority and rural museums.

The first option is to estimate the characteristics separately for the two strata, and take a weighted average to estimate the characteristic for all museums of the type. For example, we could analyze separately the most frequently cited needs for personnel among small museums in the Apparent Minority stratum and small museums in the Others stratum, and take a weighted average of the results to determine the most frequently cited needs for personnel among all small museums.

The problem with this option is that there are so few eligible responses of each type of museum in the Apparent Minority stratum (from nine rural museums to fifteen small museums), that we could not produce useful estimates for many characteristics.

The second option is to weight each response by the appropriate weight for its stratum, as described in Section, and estimate characteristics for the weighted data from both strata pooled together. This option would produce the most accurate estimates, but it would make tests of statistical significance very difficult. Since we consider it inherently misleading to present estimates of differences between types of museums without a measure of their significance, we will not use this option.

The option we have chosen is to select a subset of the eligible responses representing a random sample of the sampling frame, without stratification, and analyze this subset as representing all museums. Therefore, for purposes of analysis, we have grouped our responses as follows:

ALL RESPONSES
This group includes the responses from all respondents which we determined to be museums, as discussed in section.
ALL MUSEUMS
This is the subset of ALL RESPONSES representing a random sample of the sampling frame, without stratification. The initial 504 records IMS selected from the sampling frame included 21 Apparent Minority museums, while the total sample for the Apparent Minority stratum was 45 (see Section). We therefore include in the ALL MUSEUMS group all eligible responses from stratum Others and a randomly selected 21/45 of the eligible responses from stratum Apparent Minority (that is, nine responses). While this approach excludes eleven otherwise usable responses from our analysis, and thus slightly increases the variance of our estimates, we think this is less important than gaining the ability to apply straightforward tests of statistical significance.
SMALL MUSEUMS
This is a subset of ALL MUSEUMS consisting of respondents we classified as small (see Section).
EMERGING MUSEUMS
This is a subset of ALL MUSEUMS consisting of respondents we classified as emerging (see Section).
MINORITY MUSEUMS
This is a subset of ALL RESPONSES consisting of all respondents we classified as minority (see Section). That is, this group consists of all eligible responses from minority museums, regardless of whether they come from the Apparent Minority stratum or the Others stratum. The implicit assumption in this approach is that there is no systematic difference in the characteristics we will estimate between minority museums whose names make them recognizable as such (and therefore belong to stratum Apparent Minority) and other minority museums.
RURAL MUSEUMS
This is a subset of ALL MUSEUMS consisting of respondents we classified as rural (see Section).

9.2 ADDRESS

We classified each respondent in one of six geographic regions, based on the State listed in the respondent's address. The States comprising each region are listed in footnote.

We tested for differences in distribution by region between all museums and each group of museums using the normal test for difference of two proportions, as described in section.

We estimated the number of museums of each group in each region by multiplying the share of a group's respondents that were in a region by our estimated population for the group. We estimated the Sampling Error for each group as follows:

    Sampling Error = Population / Responses^.5

9.3 QUESTION 1

This question asks the respondent to classify itself as one of thirteen museum disciplines.

Because there were few or no respondents in some museum disciplines (for example, no respondents were planetariums) we could not produce usefully precise estimates of the number of museums in each discipline.

9.4 QUESTIONS 2 AND 3

Each of these questions asks how many people carry out the activities of the museum (full-time for question 2 and part-time for question 3), and how many are paid.

For each question, we took the mean of the usable responses for each group of museums. The response rate for the part of each question asking about paid people is much lower than for the part asking about total people. We surmise that many respondents who have no paid staff left this space blank instead of writing in a zero. This would explain the fact that for every group of museums, the mean response for total full-time personnel is less than the mean response for paid full-time personnel. Because of the low response rates and apparent non-representativeness of respondents, we have not drawn any findings from the part of each question asking about paid personnel.

We tested for statistically significant differences between each group of museums and all museums using the normal approximation to the permutation test for difference of means, as follows:

mG is the mean response for the group (for example, SMALL MUSEUMS) mC is the mean response for the complement, that is, for respondents not in the group N is the number of usable responses for the group and its complement combined nG and nC are the numbers of usable responses for the group and for its complement, respectively v is the variance of all the usable responses for the group and its complement combined 

    Variance(mG-mC) = N/(N-1) (1/nG + 1/nC) v

    z = (mG-mC) / Standard Deviation(mG-mC)
We counted a difference as statistically significant if |z|>1.64.

For the Small, Emerging, and Rural museum groups, the complement consisted of all responses in the All Museums group that were not also in the group being examined. For the Minority museums group, the complement consisted of all responses in the All Responses group that were not also in the Minority museum group.

9.5 QUESTIONS 4, 5, 9, 10 AND 11

Each of these questions asks the respondent to identify which of seven activities are its highest, second and third priority need for a given resource. Respondents may also answer that they have no need for the resource.

For each question, we tested for differences between groups by comparing the share of respondents who selected each activity as one of the three highest priority needs. We arranged the data as follows: 

                   Selected   Didn't
                   activity   select

Group                    Gy       Gn
Complement               Cy       Cn
where: For the groups SMALL MUSEUMS, EMERGING MUSEUMS and RURAL MUSEUMS, the complement consisted of the subset of ALL MUSEUMS that were not also in the group being tested. For MINORITY MUSEUMS, the complement consisted of the subset of ALL RESPONSES that were not also in MINORITY MUSEUMS.

We applied the normal test for difference of two proportions, that is: 

    pG = Gy / (Gy+Gn)

    pC = Cy / (Cy+Cn)

    p = (Gy + Cy) / (Gy + Gn + Cy + Cn)

    v = p (1-p) ( (1/(Gy+Gn) + (1/(Cy+Cn)))

    z = (pG - pC) / v^.5
We considered a difference as statistically significant if the result had a one-tailed descriptive level of significance less than.05 (that is, |z| > 1.64).

9.6 QUESTION 6

This question asks the respondent to check the box for each activity it has performed since January 1990.

We counted a response as usable if the respondent provided a usable response regarding any activity, but even by this standard the response rate was only 50%. Moreover, we cannot distinguish between respondents who intended to answer "no" with regard to an activity and respondents who merely failed to respond.

Because of these problems, we have drawn no conclusions from the responses to this question.

9.7 QUESTION 7

This question presents the respondent with 13 activities (including "other"), and asks the respondent to enter an estimated number for each it had done in the past two years, that is, since January 1990.

We counted a response as usable if it was an integer greater than zero. We will discuss analysis of the responses regarding various activities separately.

9.7.1 General Visits by the Public (Attendance)

We have assumed that all museums had attendance, and that therefore the attendance for respondents that provided a usable response is representative of all museums. We tested for differences in mean reported attendance between all museums and each group using the normal approximation to the permutation test for difference of means, as described in section.

We found a mean attendance for all museums of 77,085 for the two-year period referred to in the question. By comparison, the 1989 AAM survey of museums found a mean annual attendance of 69,181 for all museums in 1988. American Association of Museums, Data Report from the 1989 National Museum Survey (Washington: American Association of Museums, January 1992), Table E:47-A. We expect that there are three major causes for this discrepancy.

First, the attendance figures from the 1992 IMS survey refer to general visits by the public, and do not include scheduled school visits or scheduled adult visits (see question 7 of questionnaire in Appendix A). By contrast, the figures from the 1989 AAM survey include "individuals attending in prearranged groups".American Association of Museums, Appendix A, part E, question 47.

Second, the 1992 IMS survey intentionally defined "museum" to include smaller institutions that are excluded from the definition used for the 1989 AAM survey. The 1992 IMS survey did not require that a "museum" have a professional staff equal to at least one full-time worker, or be open to the general public at least 120 days per year, which were eligibility requirements for the 1989 AAM survey.American Association of Museums, pp. 4-5.

Third, we expect that some respondents to the 1992 IMS survey mistakenly entered annual attendance figures, even though the questionnaire asks for the number of visits over a two-year period.

9.7.2 Scheduled School Visits or Programs
Scheduled Adult Visits or Programs
Held a Special Event for General Public

These questions are ambiguous as to whether the respondent should enter the number of groups or programs, or the number of persons participating in the groups or programs. Therefore, we have not analyzed the numbers entered, but only the share of respondents that provided a response greater than zero for each question, and thereby indicated they had one or more groups or programs of a particular type during the two year period.

Moreover, the question directs respondents to enter a value only if they have conducted the activity in question. Therefore, we cannot distinguish between respondents who intended to indicate they did not conduct the activity, and those who merely failed to respond to the question. Therefore, we can use the proportion of respondents that indicated they conducted the activity only as a low-end estimate of the proportion of the population that conducted the activity.

We tested for differences between all museums and each group in the proportion that indicated they had one or more groups or programs of a particular type, using the normal test for difference of two proportions, as described in section.

9.7.3

We did not draw any findings from these questions, because the response rate was less than 50%.

9.8 QUESTION 8

We did not draw any findings from the responses to this question, because the response rate is less than 50%.

9.9 QUESTION 12

This question asks the respondent to identify which of 11 items (including "other") would be its first, second, and third highest priority for usefulness in expanding its building or moving to a new building. Respondents may also answer that they do not need to expand or move.

We analyzed responses to this question the same way we analyzed questions 4, 5, 9, 10 and 11, as described in section.

9.10 QUESTION 13

We analyzed responses to this question the same way we analyzed questions 4, 5, 9, 10 and 11, as described in section.

9.11 QUESTION 14

The question asks the respondent to report its operating expenditure by choosing one of fourteen ranges. The question omits a choice for the range $200,000 to $225,000. If a respondent chooses the top range, "Over $300,000", then he is to also enter a numeric amount.

We counted a response as usable if:

We assigned each response a value equal to the midpoint of the range the respondent had selected, or equal to the numeric amount the respondent entered.

Unfortunately, 2.4% of eligible respondents selected the "Over $300,000" range but did not enter a numeric amount. Excluding these responses would bias our findings downward, so we assigned each of these responses a value equal to the mean value for all numeric responses over $300,000.

We tested for statistically significant differences between each group of museums and all museums using the normal approximation to the permutation test for difference of means, as described in section.

9.12 QUESTION 15

We tested for statistically significant differences between each group of museums and all museums using the normal approximation to the permutation test for difference of means, as described in section.

We found 95% confidence intervals using Student's t distribution.

9.13 QUESTIONS 16 AND 17

Each of these questions asks the respondent to select the first, second and third most important from a set of potential sources of income.

For each question, we tested for differences between groups by comparing the share of respondents who selected each potential source as one of the three most important. We applied the normal test for difference of two proportions, as described in Section.

9.14 QUESTION 18

This question asks the respondent, for each of several potential funding sources, whether the respondent did not apply to that source since January 1990, applied and received funding from the source, or applied but did not receive funding from the source.

We compared groups as to the share of museums that applied, and the share of applicants that received funding, for each source, using the normal test for difference of two proportions, as described in Section.

9.15 QUESTION 20

This question asks the respondent to choose the first, second, and third most useful improvements for federal funding programs from a list of 11 options, including "other".

We counted a response as usable if it had exactly one choice for most useful, no more than one choice for second most useful, and no more than one choice for third most useful, except that we counted a response as unusable if it had a choice for third most useful but not for second most useful.

We tested for differences between groups in the proportion of usable respondents who chose each option as their first, second or third choice, using the normal test for difference of two proportions, as described in Section.

9.16 QUESTIONS 21, 22, 23

Each of these questions asks the respondent to answer "yes" or "no" regarding an aspect of its minority involvement. We counted a response as usable if the respondent selected "yes" or "no", but not both.

We tested for differences between groups in the proportion of usable respondents who answered "yes" to each question, using the normal test for difference of two proportions, as described in Section.

We estimated the number of minority museums in the population that would report each type of minority involvement by multiplying the proportion of usable minority respondents who answered "yes" to each question by our estimate of the total population of minority museums from Section. We also found the proportion answering "yes" to each question among all respondents that are not minority museums, and multiplied this by the difference between our population estimates of all museums and minority museums to obtain the number of non-minority museums in the population that would report each type of minority involvement.


[Search all CoOL documents]