PB1200 Statistics Research

PB1200: Information Resources in Public Justice
Statistics Research

It is easy to become overwhelmed by the amount of available statistics in public justice. Even the word statistics has its origin in the word "state," implying the collection of facts for political purposes, often for taxation. Science recognizes the practice of statistical manipulation as a branch of applied mathematics and the use of statistics plays a major role in nearly all branches of the sciences and even the humanities. Computers have improved the speed and accuracy of statistical calculations without necessarily improving our understanding of what the statistics mean. Statistics, then, can be the numbers or facts collected or the methods used to analyze them. We will concentrate on how to find the statistical facts which have already been compiled and analyzed and how to evaluate them.

Statistics are compiled by government agencies (such as the FBI, the Census Bureau, the Bureau of Justice Statistics), state and local government agencies, professional organizations, academic institutions, marketing research organizations, and many others. They are collected from varying sources by varying methods and cover varying time periods. One thing you can be sure of when you are looking for statistics is that the number you want is almost never available in exactly the way you want it.

When you examine any kind of statistic, it's easy to trust the result because they are numbers and they look so precise and correct. Remember that statistics are collected by people and analyzed by people and the same errors and biases can creep into them as into anything else. Even if everything appears to be done by computer, somewhere along the way a person was involved in collecting or inputting or evaluating the data. There are even people and organizations that deliberately manipulate the data they collect to prove what they want.

Validity and Reliability

Two things you need to consider are the validity and reliability of the results and these can be affected by a number of things. If the results of a survey are reliable, it means that the same survey given again in the same manner (not necessarily using the same people), will achieve similar results. To say that the results of a survey are valid tells us that the numbers mean what they say they do, that they answer the question that was asked. Validity and reliability are very important in the collection of statistics. If the numbers you get are not valid and reliable, then they are of little use.

Following this there are a few questions to consider when you examine statistics. These things can help you determine how to think about the numbers you get. It is always important to critically examine every piece of information we use. Statistics seem to be particularly prone to misuse and misinterpretation.

Who collected the statistics?

People and organizations that go to the trouble of collecting statistics generally have some sort of reason to do so. This can often lead to bias and error in the collection, whether deliberate or not. When statistics are used by lobbying groups to gain public support and public funding, it behooves us to examine the motives of the groups paying for the surveys and collection of statistics. This affects the validity of the statistics.

In addition to the obvious bias that might be introduced by the motives of the group conducting the survey, there is also the question of whether a professional social researcher or an amateur wrote the survey questions and conducted the data collection. The professional is trained to examine questions for ambiguity, relevance, and ease of analysis as well as using the correct techniques for administration of the survey and analysis of the data. Amateur surveyors often introduce error into the end product through ignorance of the appropriate techniques to use. This affects both the validity and reliability of the statistics.

What was the wording of the questions?

The wording of questions can make a big difference in responses. Consider the two following questions:

a. Individual freedom is the cornerstone of American values. Do you think that people should have the freedom to choose whether they want to smoke in public places?
b. Second-hand smoke has been shown to be detrimental to the health of non-smokers who breathe it. Do you think that non-smokers should be subjected to the harmful effects of second-hand smoke in public places?

Both questions are about the freedom to smoke in public places. The wording of the question affects the validity of the result. In other words, do the numbers mean what they seem to mean? Do you think both would result in the same answers even though they deal with the same subject? What if the survey were done anonymously by mail? What if the survey were done in person and the person conducting the survey was smoking? What would the result be if only smokers were surveyed? What if only non-smokers were surveyed? Which brings us to other problems with collecting information.

How was the survey done?

There is research to show that people do not always answer truthfully on surveys and the more sensitive the question is, the more often they answer untruthfully. There are a number of ways to reduce this tendency. One way is to assure the respondent he or she will not be connected with his answers, that the survey is being done anonymously. It is quite difficult to assure people that they will be anonymous, particularly when surveys are done by phone or in person.

People tend to react to social cues and other cues on the survey (and cues from the interviewer if there is one) when they answer. The name of the organization conducting the survey, the wording of questions on the survey, the appearance or facial expressions of the interviewer, and other very small cues have been shown to influence respondents. It is much easier for a respondent to answer in a way he or she feels will be socially acceptable to the interviewer than to say something which might be unacceptable. This also affects the validity and reliability of the survey.

If the answers to the survey may affect the person or organization in the future, it is possible that some slanting of the data will occur. For example, if data is being collected to determine what cities are the most attractive to live in, fear of crime can be a factor. People who answer this survey and want their city to do well in the survey may tend to minimize their fear of crime in their answers. However, if data is being collected in order to determine how much grant money a police department will receive depending on how fearful the community is of crime, some people may tend to maximize their fear of crime in their answers in order to bring more money into the department. Of course, this all depends on whether those who are questioned know (or think they know) the true purpose of the survey.

What sort of sample was examined?

There are various kinds of samples that can be drawn in order to get valid and reliable results from surveys. One thing you can do is get an answer for every person you want to know about. This is called taking a census instead of a sample and it is occasionally done. When the government wants to know how many people are in prison and what offenses they are there for, a census is usually done. Every prisoner is counted and their offenses enumerated. There is also a census of the entire population of the US once every ten years. When work Is done using a census technique that there is no sampling error involved. However, other errors arise involving ordinary human error, i.e., miscounting, misreading numbers and so forth.

When it is uneconomical or unfeasible do a census, a sample is used. Samples which are scientifically drawn from the population give very good results reflecting the population as a whole quite well. However, there is error associated with using samples and it varies with the sample size. The sample must be fairly large so that the sampling error can be small. If the sampling error is 10% and the difference between the opinions in the sample is only 5%, you can't draw any firm conclusions. In fact, the difference between two opinions needs to be twice the sampling error before you can firmly say that one is larger than the other. If not, the difference could be solely due to sampling error.

Generally, the larger the sample, the smaller the sampling error. However, the benefits of larger samples diminish quickly after you reach about 1000 (when sampling error is about 3%). To get to 1% sampling error the sample size would need to reach nearly 10,000. Most national polls rarely sample more than about 1000 people.

However, samples must be carefully drawn if you want to be able to calculate sampling error accurately. This generally means that the sample must involve some sort of randomization process. It may seem very simple to draw a random sample but in fact it can be quite complex when you are dealing with real people and real sampling problems.

It is much easier to get a poor sample. Samples of convenience are samples which are easily accessible. This is akin to asking questions of people you meet at the mall. There is no way to know if your sample is representative of any particular group, except the group of people who visit that mall. Another kind of sample is the judgment sample, which is a group chosen by some expert to be representative of the entire population. Again, there is no way to calculate what sampling error might be associated with such a group. The error might be very small or it might be very large, but the problem is that it cannot be calculated. With a well-drawn random sample, the error associated with it can be calculated quite simply. The type of sample used affects the validity and reliability of the results.

More about samples

Reading the newspaper or listening to news reports, one might be led to believe that violent crime is running rampant. In most news reporting, "if it bleeds, it leads." This tends to inflate the average person’s view of the crime rate. But news reports do not present a scientifically drawn sample of crime.

If you have a barrel of black and red beans, there is only one way to find out exactly how many of each are in the barrel. You have to count each and every one, i.e., take a census of the beans. However, there is an easy way to find out approximately how many of each kind there are: take a sample. Scoop out a cup full of beans and count only that cup. If the sample is large enough and selected properly, it should fairly represent the whole.

Lots of things can contribute to bias in samples. For instance, if a survey is done by mail, first of all there will be a number of people whose addresses are wrong or unknown. These will not make the cut. Others will simply not be returned because some people are too busy to answer, don’t want to take the trouble, or consider it an invasion of privacy to answer surveys. Finally, some people may simply answer untruthfully.

For example, if you hear that the average American brushes his teeth twice a day, you may wonder about how such a thing could be counted. Are non-toothbrushers going to confess to a stranger that they never brush regularly? How can we be sure that the numbers we look at have been honestly reported? Allow yourself skepticism about bias in statistical numbers since there is nearly always some present.

The basic kind of sample is called the "random" sample. It is selected purely by chance from the entire universe of possible respondents. The test of the random sample is this: Does everyone or every thing have an equal chance to get in the sample? If not, then it is not truly a random sample. The random sample is the only one that can be examined with full confidence by statistical theory but the problem with it is that it is usually very difficult and expensive to get.

A stratified random sample is more economical to obtain. In this sample, you divide the universe into several groups and draw a sample with the same proportions as the original groups. But how do you make sure what the proportions of the original groups are? You just can’t win in sampling. Every time you collect statistics, you are in a running battle with all the many different sources of bias.

Inadequate sample size is another that must be examined as a source of bias. You all know that if you toss a coin enough times, that it will come up heads 50% of the time and tails 50% of the time. But what are "enough times?" If I toss a coin ten times and it comes up heads 8 times and tailes 2 times, does that mean our conventional wisdom about coin tossing is wrong?

No, it only means the sample is too small. Toss the coin a thousand times and you will generally come very close to a 50/50 split. Is a thousand always a large enough sample? Sometimes, but not always. It depends on how large and varied the population is that you are sampling and how likely it is that the thing you are studying will occur. If your purpose is to select a group of people and track them from childhood to adulthood to see which of them will develop into mass murderers, a sample of 1000 from the whole world of several billion would probably not be enough. It is possible you may find no mass murderers at all in your sample and your years of study would have gone to waste. Good sampling is the key to good statistics.

Does the survey measure facts or attitudes?

Data can be collected which will show the actual crime rate for various crimes in a particular community and the government does this on a regular basis. This is an example of a survey for facts. Even these can contain error. First of all there are coding errors where someone just writes down the wrong number accidentally. Random errors such as this do tend to cancel out over the long run. However, there are other errors which tend to be more systematic and don't cancel out. For instance, errors where statistics were not kept accurately in the first place so they can't be reported accurately.

If the government asks for a breakdown of burglaries occurring during the day and after dark, perhaps you have the number of burglaries and the hours during which they occurred but you don't have the manpower to look at each day of the year and figure out when it got dark that day. And what about those burglaries that happen just at sundown or sunrise? So you estimate or you don't report at all and that adds error to the figures. And what about crimes that are never reported to the police? They can’t be included in this type of report at all, though there are estimates made of unreported crime. Thus, even when you examine statistics that are facts be aware that there can be some error associated with them.

Surveys which request opinions can also have problems associated with them as we have noted in the previous cautions above. Examine statistics to see whether they are surveys of facts or attitudes. A survey of attitudes would tell us whether fear of crime has increased or decreased in a particular area. We could compare this with the increase or decrease in the actual rate of crime for the area but the two things are not the same.

When was the information collected?

Dates are important in following trends and opinions can change quickly after major events. Many people who may have had more favorable attitudes toward private militia groups changed their attitudes following the bombing of the federal building in Oklahoma City. There can also be problems with memories of events that happened long before a survey was taken.

Regardless of the statistics you gather and use, be aware of who gathered them, how, when and for what purpose. When you use statistics, cite the source so that others can investigate further if desired.

Statistics Resources

Most of these items are produced by the U.S. government and, in the future, some of these may be available only in CD-ROM format. These items do not describe statistical methodology or teach how to do statistics.

The bibliography below cites statistical resources in paper in the Academic Library, but there are also many statistical resources online.

Indexes to Statistical Sources

(These indexes do not contain the statistics themselves but, like other indexes, point to the proper resources to consult.)

American Statistics Index (ASI). Index Tables: Indexes statistics in publications of the federal government. The first issue (1974) attempted to cover 10 years retrospectively into the early sixties. Issued monthly and cumulated annually. Gives a summary of each publication covered and what types of information and statistics may be found in it. possible subject headings for crime statistics include crime and criminals, correctional institutions, prisoners, parole and probation, police, courts, law enforcement, juvenile delinquency, terrorism, and so forth. Very detailed.
Government Documents Index (also known as the Monthly Catalog) on CD-ROM Library Network: Content: similar to ASI but does not provide the in-depth coverage of statistics within documents; try general headings such as crime and criminals, criminals, and crime statistics. Also, try combining the keyword "statistics" with keywords for your topic.
NCJRS Library Network: Content: citations & abstracts for criminal justice books, reports, articles, documents and audiovisuals in the NCJRS library

General Statistical Sources

Book of the States Ref/JK/2403/B6/yr.: Statistical tables with sources noted accompany the text on trends in state activities. Statistics can be somewhat limited but are valuable for gathering all information on all states in a single item.
Compendium of Social Statistics Ref/HA/12.5/A1/U5/yr.: Provides social statistics on demography, health, nutrition, housing, education, working conditions, and employment for all nations.
County and City Data Book U.S. Docs/C 3.134/2:C83/2/yr.: Presents the latest available census figures for each state, county, and for the larger cities in the U.S. Also summary figures for states, geographical regions, urban areas, standard metropolitan areas, and unincorporated places. Includes: number of serious crimes known, crime rate, number of violent crimes
Directory of Federal Statistics for Local Areas: A Guide to Sources U.S. Docs/C 3.6/2:St2/2/yr.: Provides tabular descriptions of statistical information on areas smaller than states. Arranged alphabetically by broad topic (agriculture, banking, trade). Within each topic, information is presented in table format covering subject, data items included, areas to which data apply, frequency, and bibliographical source.
Historical Statistics of the United States, Colonial Times to 1970 U.S. Docs/C 3.134/2:H62/970/pt.1-2: Provides statistical information on the United States dating to 1610. Information is arranged in 24 chapters on major subjects. Content: Types of crime & crime rates, homicides, arrests, criminal justice system expenditures, court cases filed, prisoners & executions. Detailed subject index.
Municipal Yearbook Ref/JS/342/A2/I52/yr.: Detailed statistical tables on police with personnel and budgetary information. Expenditures and salaries presented for groups of cities clustered by size of population.
Social Work Almanac Ref/HV/90/G53/1995: Information and statistics on demography of the U.S., children, crime & delinquency, education, health & mortality, mental illness, older adults, social welfare, economic assistance, housing, and homelessness. A bibliography and index are included.
State and Metropolitan Area Data Book US Docs C 3.134/5:yr.: Covers the U.S., regions, divisions, states, metropolitan areas with information on number and types of crimes, crime rate, correctional facilities, prisoners, arrests, number on probation
State of Texas Children Ref/HV/742/T4/S827/1994: Introductory essays plus statistical data on each Texas county for child poverty rate, percent low birthweight babies, infant mortality rates, child death rates, percent of all births to teenage mothers, juvenile violent crime arrests, teen violent death rates, percent teens not in school and not in labor force, and percent of children in single parent families.
*Statistical Abstract of the United States U.S. Docs/C 3.134:yr. (latest at reference desk): Presents summary statistics on the political, social and economic aspects of the U.S. Serves as a first source of information and as a guide to further information. Tables generally cover a period of several years for comparison purposes, usually 15-20, though some go much further. Covers the U.S, regions, divisions, states, selected cities with information on types of crime & crime rates, victims & victimization rates, child abuse, arrests, criminal justice employment & expenditures, numbers and characteristics of prisoners, parole population
Taylor, Charles Lewis. World Handbook of Political and Social Indicators Ref/HN/25/T39/1983/v.1-2: Assembles political, social, economic and cultural series of quantitative data for nations of the world. Covers mid-twentieth century to the present. Information in tabular form with some text.
U.N. Statistical Office. Demographic Yearbook Ref/HA/17/D45/yr.: International demographic data including official statistics from 250 geographic areas of the world. Covers population characteristics, birth, death, marriage and divorce statistics.
U.N. Statistical Office. Statistical Yearbook Ref/HA/12.5/U62/yr.: Contains international statistical tables on education, educational expenditures, science and technology, libraries, book production, newspapers and periodicals, film and cinema, and radio and television broadcasting.
USA Counties (CD-ROM)
Reserve; Covers U.S., states, counties and includes serious crimes known to police, types of crimes, crime rate for serious crimes
Wasserman, Paul. Statistics Sources Ref/HA/1/S7/yr./vol: Subject index to data on industrial, business, social, educational, financial, and other topics for the U.S. and the world. Citations for U.S. sources directly under the topical heading with sources for other countries under the name of the country.
World in Figures Ref/HA/155/W66/1988: Part 1 consists of general world data with countries ranked according to subject headings and indicators. Part 2 consists of regions and countries statistically described regarding population, geography, resources, finance and trade.

General Public Justice Statistics Resources

BJS Data Report US Docs J 29.19: yr: Information from BJS publications on crime and its characteristics, drugs, the cost of crime, police response to crime, adjudication and sentencing, corrections
Bureau of Justice Statistics Bulletin US Docs/J 29.11: Different issues cover different topics including crime, criminal victimization, capital punishment, prisons, prisoners, probation & parole, sentences
Bureau of Justice Statistics Special Report US Docs/J 29.13: Different issues cover various areas including other countries, U.S., regions, states, selected cities with reports on crime topics
Compendium of Federal Justice Statistics US Docs/J 29.20:yr: Covers U.S. judicial districts including number of suspects by offense, disposition of suspects, processing time, pretrial release or detention, disposition of cases, length and type of sentences, probation & parole, incarceration rate, average time served
Criminal Victimization in the United States US Docs/J29.9/2:yr: Information on types of crime, victim characteristics, offender characteristics, crime characteristics, reporting of crime to police
Criminal Victimization in the United States: 73-88 Trends
US Docs/J 29.9/2:973-88/trends; Information on the number of victimizations and rate for selected personal and household crimes, number and percent reported to police, data by age, race, sex, and household characteristics
Law Enforcement Officers Killed and Assaulted US Docs J 1.14/7-6:yr: Covers the U.S., regions, divisions, states, agencies regarding number of law officers killed, location, type of weapon, time of day, type of assignment, profile of victims, profile & disposition of offenders, circumstances, summary of incidents, accidental deaths, similar data on assaults of law officers and federal agency officers
National Economic, Social, and Environmental Data Bank (CD-ROM) Reserve: Selected information from the Bulletin and Special Report series and other BJS publications
Report to the Nation on Crime and Justice US Docs/J 29.2:R29/2: Covers the U.S, regions, states, counties, selected cities with information on the number and types of crimes, trends, circumstances, victimization, characteristics of offenders, criminal justice system response, court proceedings, sentences, corrections, costs
*Sourcebook of Criminal Justice Statistics Ref/HV/7245/N37/yr.: Covers the U.S., regions, divisions, states, cities over 100,000 with information on characteristics of the criminal justice system, public attitudes toward crime topics, nature and distribution of known offenses, characteristics and distribution of persons arrested, judicial processing of defendants, persons under correctional supervision
Statistical Handbook on Violence in America Ref/HN/90/V5/S833/1996: Topics include fatal violence in America, other interpersonal violence, vulnerable groups such as youth, elderly, women & minorities, and the impact of violence.
*Uniform Crime Reports for the United States: Crime in the United States US Docs/J 1.14/7:yr: Covers the U.S., regions, divisions, states, metropolitan areas, selected counties, cities, individual universities and colleges with information on number and type of offenses, crime index, crime rate, offenses cleared by arrest, number and characteristics of persons arrested, number of law enforcement personnel
Violent Crime in the United States US Docs/J29.2:V81: Information on amount and characteristics of violent crime, characteristics of victims and offenders, arrests, convictions, sentences, drug involvement

Sources on Prisons and Prisoners

Census of Local Jails US Docs/J 29.21: Covers U.S. regions, states, counties, individual jails regarding inmate populations, jail capacity, inmate detention status, race & sex of inmates, admission & releases, inmate deaths, number & capacity of confinement units, facility age, planned changes, jail employees by type, expenditures, alternative programs, medical facilities
Correctional Populations in the Unites States US Docs/J 29.17:yr: Covers the U.S., regions, & states with information on jail inmates, probation, prisoners, parole, capital punishment, number of persons in each category by various characteristics (sex, race, type of offense, sentence length), capacity and condition of correctional facilities
Historical Corrections Statistics in the United States, 1850-1984 US Docs/J 29.2: C 81/850-984: Covers the U.S., regions, divisions, states regarding capital punishment, prisoners, prison facilities, jail inmates, jail facilities, juvenile correctional facilities, parole, probation.

Court Statistics Sources

Federal Judicial Workload Statistics Law Library: Covers the U.S., individual court circuits and districts with information on the number & type of federal court cases filed, pending & terminated; jurors selected; persons on parole or probation
Federal Court Management Statistics Law Library: Covers individual U.S. appellate & district courts with information on workload profiles for individual courts; filings, terminations and pending cases; judgeships, actions per judgeship; median times; types of cases; national profiles; court rankings
Juvenile Court Statistics US Docs/J 32.15:: Covers U.S. counties regarding the number of cases by type, age, sex, and race; disposition
Reports of the Proceedings of the Judicial Conference of the United States Held in Washington, DC [dates]: Annual Report of the Directory of the Administrative Office of the United States Courts US Docs/Ju 10.1: yr: Covers the U.S., individual court circuits and districts including information on the number and disposition of cases by court; cases commenced, terminated & pending by type; origin of appeals; time involved; number & length of trials; length of sentences; federal probation system; bankruptcy petitions; jurors selected

Drugs Statistics Sources

Annual Data [year]: Data from the Drug Abuse Warning Network. US Docs/HE 20.8212/11:: Covers U.S. regions regarding drugs incidents reported by emergency rooms and medical examiners; number and percentage of incidents by sex, age, race, motive & disposition; if applicable manner & cause of death; drug category; form of drug & how administered; drug combinations; source of drug.
Drug Use Forecasting Law Library: Covers selected metropolitan areas regarding drug use among persons arrested by type of drugs, type of crime, sex, age, race
International Narcotics Control Strategy Report US Docs/S 1.146:yr.: Covers world regions and countries regarding the production, cultivation, and eradication of specific drugs; refining, seizures, arrests, labs destroyed, consumption, users
National Household Survey on Drug Abuse US Docs/HE 20.8202: H81/: Covers U.S. regions regarding the prevalence, trends & frequency of drug use for marijuana, cocaine, inhalants, hallucinogens, heroin, psychotherapeutic drugs, alcohol, cigarettes, & smokeless tobacco; by age, sex, race, education, and employment status.
NNICC Report [year]: The Supply of Illicit Drugs to the United States US Docs/J 24.22: Covers world regions and selected countries regarding the availability & use of cocaine, opiates, and other dangerous drugs; quantities available, prices, purity, cultivation & production, control, seizure, sources
Smoking, Drinking & Illicit Drug Use Among American Secondary School Students, College Students, and Young Adults, 1975-1991 US Docs/HE 20.8202: SM 7/2/v.1-2: Covers U.S. regions regarding the prevalence & frequency of drug use for various drugs, alcohol & cigarettes; differences in use by race, sex, and selected socioeconomic factors; grade of first use; degree & duration of highs; attitudes & beliefs; social milieu

Terrorism, Bombs, and Explosives Statistics

Bomb Summary US Docs/J 1.14/7-7:yr: Covers: U.S. regions and states regarding the number of bombings, injuries, & deaths, type of targets, amount of damage, time of day and month, bomb characteristics
Explosives Incidents Reports US Docs/T 70.11:yr - some on microfiche: Covers the states regarding the number and types of bombings and explosives incidents; deaths, injuries & property damage; targets; motives; types of containers & fillers; accidental explosions; explosives thefts and recoveries
Patterns of Global Terrorism US Docs/S 1.138:: Covers the world by regions regarding terrorist incidents by type of facility, type of victim, type of event; number wounded & killed
Terrorism in the United States US Docs/J1.14/22:: Covers U.S. regions regarding the number & types of incidents; number of suspected incidents & preventions; number killed & injured; terrorist groups; targets; summary of specific incidents for current year; five-year list of incidents

Polls and Opinion Surveys

Gallup Poll Ref/HN/90/P8/G35/yr.: National opinions on various aspects of crime, its handling by politicians, death penalty, handgun control
Official Guide to American Attitudes Ref/HN/90/P8/M58/1996: Most of the information here is taken from the General Social Survey conducted by the National Opinion Research Center. Topics include the environment, race & immigration, religion, work & money, marriage & family, women’s roles, personal outlook, and sex & morality.
Opinions '90 Ref/HM/261/06/1990: Covers the US, regions, & selected cities regarding opinions on the death penalty, plea bargaining, supervision of probationers, shoplifting, police protection satisfaction, soundness of the criminal justice system, future of the criminal justice system
Public Opinion Quarterly Periodicals/First Floor: Specific polls and poll results are presented as well as articles on survey methodology.

Back to the Table of Contents for PB1200

This web site was created by Margaret Sylvia (msylvia@stmarytx.edu) to educate Public Justice students at St. Mary's University in the use of information resources.
Created August 1998. Last modified

PB1200: Information Resources in Public Justice Statistics Research