War, Perception, and Social History

by Fred Ray on January 15, 2009 · 2 comments

J. D. Petruzzi has a thoughtful and perceptive post about how historical events are perceived by authors and readers. I would add that it’s all to easy to substitute opinion for fact, and deplore that many modern authors feel the need to put themselves on a higher moral plane so as to pass judgment on their subjects and at times even their readers.

War, more than perhaps any other type of human event, is a complex, chaotic process that often defies any sort of neat understanding or retelling. Reading after action reports you really do sometimes wonder if the authors were on the same battlefield.

That bring us to the latest rage in historical studies, war as a social event. While social profiling has been popular in regimental histories for some time, the latest trend is to use huge data sets and apply the tools of social science to reach some sort of conclusion. I, frankly, have some doubts about this approach, not because it’s necessarily invalid but because authors invariably promise far more than they can deliver and frequently stray off into social determinism. This is particularly true when the authors are not that entirely familiar with the subject matter.

I say all this to preface a massive project by two UCLA academics, Dora L.  Costa and Matthew E. Kahn: Heroes and Cowards: the Social Face of War.

Our book would not have been possible without a monumental data collection effort that first began in 1981, led by Robert Fogel, the 1993 Nobel Laureate in Economics. How do you construct a longitudinal dataset from disparate sources and from the free-form letters, affidavits, and other documents that constitute a soldier’s record in the National Archives? The data are publicly available here.

Beginning with one list of white volunteer units and one list of U.S. Colored Infantry Units, both sorted in random order, inputters collected basic descriptive information from the “Regimental Books” in the National Archives on all of the enlisted men in a company until the two samples consisted of roughly 1.6 percent of all whites (almost 6,000 men) and 1.6 percent of all blacks (almost 6,000 men) mustered into the Union Army. The men were then linked to their army records, stored in the National Archives. These records consist of compiled military service records and of cards containing medical records and vital statistics.


From today’s vantage point, the Fogel data set is extraordinary. Concern about identity theft and insurance companies using confidential information to cherry-pick healthy and low-risk patients would make it extremely difficult to build a similar dataset today. A researcher attempting to build an analogous data set for Vietnam veterans would need to obtain the consent of each man. But the types of individuals willing to grant permission would probably not be a random sample of the population. And, despite the well-publicized stories of men buying their way out of the draft, the white sample is representative of the Northern population of military age in terms of wealth and literacy rates. Only in World War II were service rates higher.

The data set is available for examination or download at the Center for Population Economics at the University of Chicago, and Costa’s work can be seen on her web page, which includes papers on topics like unit cohesion and survival in Andersonville prison.

The authors have made a series of posts about the book and their methodology on the Volokh Conspiracy blog (lawyers, but otherwise okay) here, here, and here. The latest post looks at the treatment deserters received after returning home and punishment, both formal (disenfranchisement, forfeiture of pay) and informal (social ostracism).

I am skeptical but willing to be convinced, and have ordered a copy of the book. How many horses did the authors see? How many will I see?

UPDATE: the authors have posted their last post on the subject.

UPDATE 2: For a bracing corrective about the validity of scientific studies in general, read John P. A. Ioannidis’ article on why most published studies are false, plus an interview in New Scientist.

{ 2 comments… read them below or add one }

Jim Schmidt January 16, 2009 at 5:05 pm

Good post…I saw the review in the Wall Street Journal and may get a copy as well…the authors used the “Early Indicators” database pioneered by Nobel laureate Robert Fogel…a lot of other folks have used that database for some interesting Civil War-related research…I’ve written columns about it:



Of course, the danger in relying too much on the database is that it can become a “lies, damn lies, and statistics” exercise…if the authors of Heroes/Cowards really have used other sources – soldier journals, etc. – then it could be a good study and an interesting read.

Keep up the good work.



Fred Ray January 16, 2009 at 10:19 pm

Thanks, Jim. All these studies have some common problems:

1) the tendency to conflate correlation with causation

2) confirmation bias

3) failure to consider alternative explanations

Not saying that’s the case here, but it’s easy to “prove” what you go looking for


Leave a Comment

Previous post:

Next post: