Guidelines for a Literature Survey and an Annotated Bibliography

An annotated bibliography is an organized list of sources, each of which is followed by a brief note or “annotation.” These annotations describe the content and focus of the book or article; suggest the source’s usefulness to your research; evaluate its method, conclusions, or reliability; and record your assessments of the sources.

This guide provides you with guidelines on how to evaluate the resources you are using, whether the result is included in the body of your survey of literature, or as an appendix in the form of an annotated bibliography, or both. Either way the purpose is evaluative: i.e. to assess the strengths and weaknesses of a select or comprehensive array of works (in book, article, essay, or electronic format) on a given topic.

This guide provides you with guidelines on what areas to address and questions to ask, the answers to which will provide you with the material from which your survey of literature can be composed; and, as a derivative, how to construct annotations. Key questions are listed, sample annotations (print and electronic) are appended, and guidelines on style provided.

I Introduction

The most important part of your literature survey is the Introduction. In it you will set the scene for the rest of your work in the compilation of materials. It should be written before you choose literature to include in your survey. Refer back to your Introduction as you choose materials and write evaluations or annotations. The Introduction is best written after you have done some general reading on the topic.

The Introduction should define your topic and should include the following:

• One sentence summarizing the topic
• Statement of how the topic fits into the larger context of the subject area
• Definitions of any terms used
• Statement as to how selective or comprehensive your survey and accompanying bibliography is. Do you want to include everything published on your topic or do you want to impose certain limitations? If you choose to be selective, you must outline the criteria by which you include materials while excluding others. Limitations may consist of:
  • Years of publication: e.g. deal with literature published between 1960 and 2000.
  • Geographical: e.g. research done in North America only
  • Language(s) of materials, e.g. only works in English
  • Aspects of a topic: e.g. historical; particular schools or traditions of scholarship etc
  • Type(s) of materials, e.g. books, articles, essays, theses, online resources

Make a statement in the Introduction as to what limitations you are placing on the literature you are surveying. What is the scope of the literature review? What types of publications are you using? See further below under VIII Structure.

II Goals

The objectives in doing a review of literature and its accompanying annotated bibliography include the following:

• To choose and evaluate current literature, print and electronic, relevant to your topic.
• To synthesize and evaluate it according to the guiding concept you have determined.
• To demonstrate the quality of your own research.
• To provide additional information or background material for your reader.
• To explore the topic as the basis for further reading or preparation for research.
• To give your research historical perspective or context.
• To place your project into the context of established work in the field.

III Methods

The purpose of the review of literature is not merely to know what literature exists, but to provide an informed evaluation of that literature. To achieve both purposes, two methodological skills are required:

1. Information Literacy: the ability to locate relevant literature (books, articles, essays, theses, or electronic resources) using efficient information seeking skills.
   • How good are your information seeking skills?
   • Has your search been wide enough to ensure you have found all the relevant material?
   • Has it been narrow enough to exclude irrelevant material?
   • Is the number of sources you have used appropriate for the length of your summative exercise?
   • Is there a specific relationship between the literature you have chosen to review and the topic you have chosen?

   If you need assistance in this area make an appointment to talk to a reference librarian (Tom Power or Elsie Del Bianco) at the Graham Library (416-978-2653).

2. Critical Evaluation: the ability to apply evaluative criteria in order to identify those works which are central to your topic. The rest of this guide addresses this area.

IV Evaluation of Books/Articles/Essays

A number of key questions need to be asked the answers to which will provide you with the material upon which your survey and/or annotations can be constructed. Checkpoints indicate the potential answers to the questions.

Purpose

Why was the book/article written? To:

• inform? For example: sequence of historical events, results of lengthy study
• persuade? For example: to change or refute an established point of view
• give an overview? For example: textbooks, encyclopaedias

Checkpoints:

The purpose is usually presented in the introduction.

Publisher

• Who published the book?
• A university press? (University theses and dissertations are considered published by the university that granted the degree to the student who wrote it.)
• Commercial publisher?
• Professional Association, Institution, or Research Centre?
• What can you tell about the book from the publisher?
• What theological tradition is the publisher in? Catholic, liberal, evangelical, reformed, or broad academic?
• Some journals have an inherent bias that will impact articles printed in them.
• Is the journal: liberal or conservative, centrist, feminist in stance? Can you determine what its orientation is?

Content and Organization

Examine the table of contents and/or headings to determine if the book is organized in a logical and understandable manner.

• Do the contents indicate that the book contains the information you need?
• How thoroughly does the material cover the topic?
• Is there added material such as appendices?
• Is the material organized, focused, and clearly presented?
• Is the argument or presentation understandable?
• Is this original research, a review of previous research, or an informative piece?

Checkpoints:

1. Table of contents
2. Index
3. Introduction/Preface
4. Book reviews
5. Reviews of the literature

Date of Publication

• Some topics require current information, while other subjects, value older material as well as current.
• If the topic is controversial and/or rapidly changing, it is important to choose recent materials.
• Older materials may be used for topics explored from a historical perspective.
• Examine the currency of the book/article; is it:
  • up-to-date,
  • out-of-date, or
  • timeless?

Authority/author

• Is the author an expert in this field?
• Is the author reliable?
• Where is the author employed?
• What else has he/she written?

Checkpoints:

1. Preface of book
2. Biographical sources in library (e.g. P. W. Carey & J.T. Lienhard, Biographical dictionary of Christian theologians, 2000).
3. Type of publisher

Sources of Information/Perspective/Bias

• Where is the author getting information?
• What types of research methods are used?
• What are the author’s assumptions?
• From what perspective or school of thought is the author writing?
• How does the book/article fit into the “big picture”?
• In what context is the book/article written?
• In what time period?
• Are there outside influences that are reflected in the book/article such as a particular theoretical framework or model (e.g. a feminist interpretation, post-liberal, denominational)?
• Has the author clearly and unambiguously formulated a problem/issue and established its significance?
• What are the strengths and weaknesses of the way the author has done so?
• Could the problem have been approached from another perspective?
• Has the author evaluated the literature relevant to the topic? Does the author include literature taking positions s/he does not agree with?
• Is the analysis accurate and relevant to the research question?
• Are the conclusions validly based upon the evidence and analysis?
• Is the author objective, or is s/he merely ‘proving’ what s/he already believes?

Other publications

• Have similar materials been published?
• How do they compare?
• How important is this publication for the subject?

Bibliography

• Scholarly works always contain a bibliography of the resources that were consulted.
• Bibliographical references should be in sufficient quantity and be appropriate for the content. Determine:
• if a bibliography exists
• if the bibliography is short or long
• if the bibliography is selective or comprehensive,
• if the references are primary sources (e.g. journal articles) or only secondary sources (e.g. encyclopedias)
• if the references are contemporary to the book or much older, and if the citation style is clear and consistent.

Usefulness

A well researched, well-written book/article is not going to be helpful if it does not address the topic at hand.

• Is the book/article relevant to the current research project?
• Ask, “is this book/article useful to me”? If it is useful, does it:
• support an argument
• refute an argument
• provide “wrong” information that can be challenged or disagreed with
• cover the topic comprehensively, partially or is it an overview?
• How does the author structure his or her argument? Can you ‘deconstruct’ the flow of the argument to analyze if/where it breaks down?
• Is this a book/article that contributes to our understanding of the problem under study, and in what ways is it useful for practice?
• What are the strengths and limitations?
• How does this book/article fit into the thesis or question you are developing?

Audience

• For what type of reader is the author writing?
• Is the level of the book/article appropriate for your needs?
• Is the book/article intended for a scholarly or general audience?

Checkpoints:

1. Introduction
2. Level of language used
3. Type of publication/publisher the material is found in

Illustrations

• Are charts, graphs, maps, photographs, etc. used to illustrate concepts?
• Are the illustrations relevant?
• Are they clear and professional looking?
• Do they enhance the text?

V Evaluating Internet Resources

To evaluate a web page (and determine if you want to use it for your research) look for:

Purpose

Why was the page created?

Organization and Content

• Is the page organized and focused?
• Is it well designed?
• Is the text well written?
• Are the links relevant and appropriate?
• Are the links evaluated?
• Is the scope or purpose of the web page clearly stated?
• Is the document written for an academic audience?
• Are the sources for any factual information clearly listed?
• Is a bibliography or list of works cited provided?
• Can you verify citations from your own academic resources?
• Is the information free of spelling and grammatical errors?

Objectivity/Bias of the author

• Distinguish between fact and opinion. Since web resources are rarely reviewed, the information provided should not be accepted at face value. Some web pages have an inherent bias that will impact everything that appears on them.
• What is the theological orientation of the author or sponsor?
• Is there objective analysis as opposed to mere expression of opinion?
• Is the information presented with a minimum of bias?
• Is there use of bibliographical references to other credible sources?

Date of Production/Revision

• The value of a web page is closely related to currency and timeliness.
• When was it last revised?
• Are the links still viable?
• Is the date of creation of the page provided?
• Are the links in the document current (i.e. have not expired or moved)?

Usefulness

• Is the web page relevant to the current research project?
• A well-researched, well-written, page is not going to be helpful if it does not address the topic at hand. Ask, “is this useful to me?” If it is useful, does it:
• support an argument
• refute an argument
• provide “wrong” information that can be challenged or disagreed with productively
• What is the relative value of the website compared to the range of information resource (print and non-print) available on the topic?

Authorship

• What is the provenance of the website?
• Does the page have an identifiable, respectable author? If not, the page must be used with caution.
• What is the institutional affiliation of the author? Does this imply a particular theological perspective?
• Who is responsible for the page?
• Is the author an expert in this field?
• What else has he/she written or produced?
• How accurate is the provided information?
• Is the author of the document clearly stated?
• Are the credentials of the author(s) provided (i.e. position, education, occupation)?
• Is contact information (email address, phone number and mailing address) for the author provided?
• Do you believe this person is qualified to write on the given topic? (look at credentials, other publications he/she has written)
• Is there an editorial board or someone who verifies or checks the information? Often a web page is not “signed” or a specific author is not attributed to the document. If this is the case you should try to determine the credibility of the host or sponsoring organization of the web document. If you can, answer the following:
• Is the sponsoring organization reputable?

Audience

• To what type of reader is the web page directed?
• Is the level appropriate for your needs?
• Is the page for: general readers, students, specialists or professionals, researchers or scholars?

Coverage

• Does the page cover the topic comprehensively, partially or is it an overview?
• Are the links relevant and appropriate for this site?
• What do the links offer that is not easily available in other sources?
• Are the links evaluated in any way? Are annotations included?

Illustrations

• Are the graphics clear in intent, relevant and professional looking?
• Do the graphics add to or enhance the content?

VI Style: Printed Works

Use the Chicago A style for your annotations. Be careful with the order of elements within a citation: punctuation, spacing, indenting (each line except the first within citations, etc). Use the format outlined in K.L. Turabian, A manual for writers of term papers, theses, and dissertations 7th ed. (Chicago, 2007).

Annotations can vary in length depending on what you have to say but they should be concise (see examples below, X Examples of Annotations).

VII Style: Electronic Resources

Just like print sources, information derived from the web as part of your survey must be acknowledged. Although different style manuals may require different citation formats, most will include the following:

• Author’s last name, first name
• Title of document
• Title of complete work (if different from title of document)
• Document date (if known and if different from date accessed)
• Protocol and full address
• Date of access

The Chicago A style for electronic resources is as follows:

Footnotes/Endnotes:

using the Chicago Manual of Style to include Internet sources produces the following model: author’s name (in normal order), document title, date of Internet publication, or other retrieval information (date of access), text division (if applicable).

World Wide Web (WWW):

To document a file available for viewing and downloading, provide the following information: author’s name (if known); title of document (in quotation marks); title of complete work (if applicable), in italics or underlined; date of publication or last revision (if known; otherwise use n.d.); URL in angle brackets; date of access (in parentheses).

Example: Leslie R. Shade, “Gender Issues in Computer Networking,” 1993, http://www.mit.edu:8001/people/sorokin/women/lrs.html(28 May 1996).

Email and Listserv Messages:

Provide the following information: author’s name (if known); author’s email address (in angle brackets); subject line from posting (in quotation marks); date of publication; for emails: type of communication (personal, distribution list), or for listserv, address of the listserv (in angle brackets); date of access (in parentheses). To document a file that can be retrieved from a list serve or web address, provide the following information after the publication date: address of listserv (in angle brackets); address or URL for list’s archive (preceded by via and enclosed ion angle brackets); date of access (in parentheses).

Example (Email): Danny Robinette, robinetted@ccmail.gate.edu”Epiphany Project,” 30 April 2000, personal email (29 May 2000).

Example (Listserv): Victor Parette, vrparent@mailbox.syr.edu “On Expectations of Class Participation,” 27 May 2000 (29 May, 2000).

Example (Listserv Archive): Nick Carbone, ncikc@english.umass.edu”NN 960126: Follow-up to Don’s Comments about Citing URLs, ” 26 January 1996, acw-l@unicorn.acs.ttu.edu>via http://www.ttu.edu/lists/acw-l(17 February 1996).

Bibliography:

Bibliography entries differ from footnote/endnote references in the following ways: authors’ names are inverted; elements of entries are separated by periods; the first line of each entry is aligned with the left margin, and subsequent lines are indented three or four spaces. If your paper is double-spaced, double-space the bibliography as well.

Example: Hoemann, George H. “Electronic Style-Elements of Citation.” Electronic Style Page. 3 November 1995. http://funnelweb.utcc.utk.edu/~hoemann/style.html (29 May 1996).

VIII Structure

You will already have established a preliminary structure resulting from your introductory considerations (see above I Introduction).

When composing and writing your literature review and annotated bibliographies certain structural considerations are worth noting:

• Your review should be organized into practical sections that present themes or identify trends.
• Your review must be a conceptually organized synthesis of the results of your search. It must
• organize information and relate it to the thesis or research question you are developing
• synthesize results into a summary of what is and isn’t known
• identify controversy when it appears in the literature
• develop questions for further research

IX Examples of Surveys of Literature

1. Waldman, Nahum M. The recent study of Hebrew: a survey of the literature with selected bibliography. Cincinnati: Eisenbrauns, 1989.
2. Oesterley, W. O. E. A short survey of the literature of rabbinical and mediæval Judaism. London : Society for Promoting Christian Knowledge, 1920.
3. Hahn, Herbert Ferdinand. The Old Testament in modern research. With a survey of recent literature. Philadelphia: Fortress Press [1970].
4. Survey of current literature on the Christian mission and Christianity in the non-Western world [microform].No. 1 (1982)- Aberdeen: Scottish Institute of Missionary Studies, 1982-.
5. Runia, David T. Philo in early Christian literature : a survey. Minneapolis: Fortress Press,1993.

X Examples of Annotations

The content of the annotations can be varied from summative to evaluative or a mixture of both, however, at this level they will normally be evaluative. An annotation that is purely a summary is merely a summary of the source. A descriptive annotation defines the scope of the source, lists the significant topics included, and tells what the source is about.

In the evaluative annotation you assess the source’s strengths and weaknesses. You assess why the source is interesting or helpful, or why it is not. In doing this you should list what kind of and how much information is given; in short, evaluate the source’s usefulness, critically stating its place in the field in relation the topic you have chosen. Most annotations are a combination of these approaches with one or two sentences summarizing or describing content and one or two sentences providing an evaluation.

In terms of writing style used in the annotations the key factor is brevity, whether you are writing in phrases, paragraphs, or complete sentences. Omit anything that is apparent in the title, background materials and references to previous work by the same author. You are recommended to use full sentences in your annotations. Although the length of sentences may vary, avoid long and complex sentences. Paragraph length is expected, and they should be full and coherent.

Sample annotations on the topic of New Testament theology follow below:

Goppelt, L. A Theology of the New Testament. 2 vols. Grand Rapids: Eerdmans, 1981, 1982.
The first volume includes a great deal of material on the history of and methods of the discipline of biblical theology. Goppelt is especially insightful when he treats salvation history. Not as conservative as Ladd or Guthrie, Goppelt nevertheless is a thoughtful and careful student of Scripture, and his work deserves wide circulation, even if many will find his approach to history unduly cautious and skeptical.

Guthrie, D. New Testament Theology. Downers Grove: InterVarsity Press, 1981.
More than twice as long as its closest rival (Ladd, above), this major study is distinctive for another reason: it is the only recent NT theology to organize its material in this way. Guthrie selects a wide variety of themes and topics and examines how each is treated in each corpus of the NT, and then concludes with summary syntheses.

Kummel, W. G. The Theology of the New Testament according to its Major Witnesses: Jesus-Paul-John. Nashville: Abingdon, 1973.
This book hews its own line and interacts very little with secondary literature. Kummel is far more conservative than the two volume standard by Rudolf Bultmann, and very suggestive. The material is organized as in Ladd.

Ladd, G. E. A Theology of the New Testament. Grand Rapids: Eerdmans, 1974.
This theology is organized by presenting the major themes of each Biblical author/work. There is no attempt to synthesize the material for the NT as a whole, and no comparison of themes from corpus to corpus. Nevertheless, although some coverage is patchy (e.g. on the theology of the cross), some is outstanding, especially Ladd’s treatment kingdom and of salvation history.

XI Examples of Published Annotated Bibliographies

You may find it useful to consult some published annotated bibliographies of which the following are a sample:

1. Joseph A. Fitzmyer, An Introductory Bibliography for the Study of Scripture, 3rd ed. (Rome, 1990).
2. Douglas Moo (ed.), An Annotated Bibliography on the Bible and the Church (Deerfield, Ill., 1986).
3. Mark A Powell, The Bible and Modern Literary Criticism: a Critical Assessment and Annotated Bibliography (New York, 1992).
4. Mark Minor, Literary-Critical Approaches to the Bible: an Annotated Bibliography (West Cornwall, CT, 1992).
5. Robert J. Kepple & John R. Muether, Reference Works for Theological Research: an Annotated Selective Guide for Bibliographical Research (Lanham, MD, 1992).
6. Henry O. Thompson, The Book of Amos: an Annotated Bibliography (Lanham, MD, 1997).
7. Frank E. Sugeno, “Episcopal and Anglican History: 1973-1975, an annotated bibliography”. Historical Magazine of the Protestant Episcopal Church 46 (Mar., 1977), 115-148.

XII Examples of Online Annotated Bibliographies

• Church History
• Book of Genesis

T. Power, Nov. 2012.