Cabaniss Usability Study - Revision history

Jlderidder at 15:51, 8 March 2011

Jlderidder — Tue, 08 Mar 2011 15:51:52 GMT

←Older revision		Revision as of 15:51, 8 March 2011
Line 45:		Line 45:

	'''We determined, however, that a true test of learnability should cover a longer period of time and far more queries. '''		'''We determined, however, that a true test of learnability should cover a longer period of time and far more queries. '''
		+

	We also decided that it is truly not fair to compare the finding aid interface to digital content against the item-level interface for a short-term test. The finding aid presents far more information and context, requiring more time to read and become acquainted with the organization of the content. It is primarily a browsing interface rather than a searching one.		We also decided that it is truly not fair to compare the finding aid interface to digital content against the item-level interface for a short-term test. The finding aid presents far more information and context, requiring more time to read and become acquainted with the organization of the content. It is primarily a browsing interface rather than a searching one.
Line 56:		Line 57:


-	[[Previous research has shown that the finding aid as interface to digitized manuscript collections is preferred by scholars and experienced researchers, whereas undergraduates prefer item-level description]] (see footnotes below). Given that our study focused on undergraduates, our results are not surprising.	+	<u>Previous research has shown that the finding aid as interface to digitized manuscript collections is preferred by scholars and experienced researchers, whereas undergraduates prefer item-level description</u> (see footnotes below). Given that our study focused on undergraduates, our results are not surprising.
		+
		+
		+	Since delivery of digitized content via the finding aid is by far the cheaper method for digitization and web delivery, we believe this is the preferred path for future digitization of large manuscript collections. More research needs to be done on the learnability of finding aid interfaces, modifications to those interfaces to improve navigation and terminology (replacing archival terms with user-friendly ones).
		+
		+
		+	'''Suggested modifications to web delivery based on usability results:'''

		+	# Revise current EAD terminology for the web display to exchange archivist terms for user-friendly labels. This would require further research and user testing.
		+	# Add a “help” option explaining how to navigate the typical finding aid, for those who have never encountered one.
		+	# Add search within file feature for (at least) large finding aids.
		+	# Add a sidebar navigation box for EADs similar to that used by Princeton [http://diglib.princeton.edu/ead/getEad?eadid=C0290&kw=] to facilitate location of items, particularly the dropdown list of series titles.
		+	# Add a “recently viewed” list on the side or bottom of the screen, so the user can return to previously viewed items.
		+	# Enlarge the thumbnails, as they are not large enough to really preview the item.
		+	# Allow user to input search term by hitting the enter key instead of requiring user to click on the “search” button.
		+	# Add a browse option for subject headings, with a method of narrowing down a topic under a broader heading (perhaps a “search within results” option).

-	Since delivery of digitized content via the finding aid is by far the cheaper method for digitization and web delivery, we believe this is the preferred path for future digitization of large manuscript collections. More research needs to be done on the learnability of finding aid interfaces, modifications to those interfaces to improve navigation and terminology (replacing archival terms with user-friendly ones)

Jlderidder at 15:47, 8 March 2011

Jlderidder — Tue, 08 Mar 2011 15:47:01 GMT

←Older revision		Revision as of 15:47, 8 March 2011
Line 25:		Line 25:


-	'''Short Discussion on Results'''	+
		+	== '''Short Discussion on Results''' ==
		+


Line 31:		Line 33:


-	One of the more interesting findings was that participants without previous digital collection experience found the finding aid interface significantly easier than those who claimed familiarity with the traditional digital library interface.	+	'''One of the more interesting findings was that participants without previous digital collection experience found the finding aid interface significantly easier than those who claimed familiarity with the traditional digital library interface.'''


Line 37:		Line 39:


-	Not surprisingly, non-native speakers had difficulty with both interfaces.	+	''Not surprisingly, non-native speakers had difficulty with both interfaces. ''


-	~~There was very little difference~~ in the apparent learnability of the two interfaces, implying that the finding aid interface ''may'' be as learnable as the usual digital library interface (based on item-level description searching)~~. We determined, however, that a true test of learnability should cover a longer period of time and far more queries~~.	+	We did note differences in the apparent learnability of the two interfaces, implying that the finding aid interface may not be as learnable as the usual digital library interface (based on item-level description searching).

		+	'''We determined, however, that a true test of learnability should cover a longer period of time and far more queries. '''

-	~~The question arose as~~ to ~~where~~ the ~~boundary is between testing~~ the ~~interfaces~~, and ~~testing~~ the ~~delivered information?~~	+	We also decided that it is truly not fair to compare the finding aid interface to digital content against the item-level interface for a short-term test. The finding aid presents far more information and context, requiring more time to read and become acquainted with the organization of the content. It is primarily a browsing interface rather than a searching one.

		+	Once acquainted with a collection, however, we expect that locating appropriate content would be as speedy (or more so) than via searching item level metadata which may or may not contain the necessary search terms.

-	The majority of (primarily undergraduate) participants preferred the item-level description interface. However, 40% of those with a background in history, and a third of those with a background in special collections, preferred the finding aid interface.

		+	'''The question arose as to where the boundary is between testing the interfaces, and testing the delivered information?'''

-	(~~Previous research has shown that the finding aid as interface to digitized manuscript collections is~~ preferred ~~by scholars and experienced researchers, whereas undergraduates prefer~~ item-level description. ~~Given that our study focused on undergraduates~~, ~~our results are not surprising~~.)	+	The majority of (primarily undergraduate) participants preferred the item-level description interface. However, 40% of those with a background in history, and a third of those with a background in special collections, preferred the finding aid interface.


-	~~We also decided~~ that ~~it is truly not fair to compare~~ the finding aid interface to ~~digital content against the~~ item-level ~~interface for a short-term test~~. ~~The finding aid presents far more information and context~~, requiring more time to read and become acquainted with the organization of the content. Once acquainted with a collection, however, we expect that locating appropriate content would be as speedy (or more so) than via searching item level metadata which may or may not ~~contain the necessary search terms~~.	+	[[Previous research has shown that the finding aid as interface to digitized manuscript collections is preferred by scholars and experienced researchers, whereas undergraduates prefer item-level description]] (see footnotes below). Given that our study focused on undergraduates, our results are not surprising.


-	~~Research has indicated that presentation~~ of digitized content via the finding aid is ~~indeed the preferred delivery method for scholars. Since this is also~~ by far the cheaper method for digitization and web delivery, we believe this is ~~by far~~ the preferred path for future digitization of large manuscript collections.	+	Since delivery of digitized content via the finding aid is by far the cheaper method for digitization and web delivery, we believe this is the preferred path for future digitization of large manuscript collections. More research needs to be done on the learnability of finding aid interfaces, modifications to those interfaces to improve navigation and terminology (replacing archival terms with user-friendly ones)


Line 64:		Line 68:


		+
		+	'''Footnotes:'''
		+
		+
		+	Nimer, C. and Daines III, J. G. 2008. "What do you mean it doesn't make sense? Redesigning finding aids from the user's perspective.” Journal of Archival Organization 6:4 (2008), 216-232. DOI=http://dx.doi.org/10.1080/15332740802533214


		+	West, T., Fesenko, K., and Brown, L. C. 2009. “Extending the Reach of Southern Sources: Proceeding to Large-Scale Digitization of Manuscript Collections”. Final Grant Report for the Andrew W. Mellon Foundation, Southern Historical Collection, University Library, University of NC at Chapel Hill (June, 2009).



	[[User:Jlderidder\|Jlderidder]]		[[User:Jlderidder\|Jlderidder]]

Jlderidder at 20:01, 4 March 2011

Jlderidder — Fri, 04 Mar 2011 20:01:59 GMT

←Older revision		Revision as of 20:01, 4 March 2011
Line 1:		Line 1:
	*IRB approval for the Cabaniss usability study was obtained in May 2010.		*IRB approval for the Cabaniss usability study was obtained in May 2010.

		+
		+	'''Usability Study Flier and FAQs:'''
	*Advertising began on August 6 and ran through August 30, 2010. Ads were placed on the University Libraries main web page, as well as on the student information portal called MyBama and on the Digital Collections web page. In addition, information was emailed to the History, American Studies, and Political Science Departments. These departments forwarded the information to their graduate students and faculty members. Fliers were also displayed in the reading room, and those work work at the reference desk were told about the study and were asked to encourage patrons to take a flier.		*Advertising began on August 6 and ran through August 30, 2010. Ads were placed on the University Libraries main web page, as well as on the student information portal called MyBama and on the Digital Collections web page. In addition, information was emailed to the History, American Studies, and Political Science Departments. These departments forwarded the information to their graduate students and faculty members. Fliers were also displayed in the reading room, and those work work at the reference desk were told about the study and were asked to encourage patrons to take a flier.
		+
		+	http://www.lib.ua.edu/wiki/digcoll/index.php/Image:Usability_Flyer_w_FAQs.pdf
		+

	*Response to the advertising was fairly positive. We received more volunteers than we needed, but all volunteers were contacted and invited to come and participate in the study, in hopes that enough volunteers would participate to reach our goal of 20 participants. We recruited nearly 30 volunteers before taking down the ads, but due to no-shows, recruited a second time. We finally tested 21 participants, though one set of results was disqualified from analysis due to unwillingness to complete the test.		*Response to the advertising was fairly positive. We received more volunteers than we needed, but all volunteers were contacted and invited to come and participate in the study, in hopes that enough volunteers would participate to reach our goal of 20 participants. We recruited nearly 30 volunteers before taking down the ads, but due to no-shows, recruited a second time. We finally tested 21 participants, though one set of results was disqualified from analysis due to unwillingness to complete the test.

-	*We are using the Morae software to perform the usability studies. More information about this software can be found at [http:~~//www.techsmith.com/morae.asp http://www.techsmith.com/morae.asp].~~	+	'''Usability Study Informed Consent:'''

-	*Usability testing was completed in September and analysis is currently in progress. ~~We hope to have the results available early in 2011~~.	+	http://www.lib.ua.edu/wiki/digcoll/images/1/1c/Informed_Consent_Cabaniss.docx

-	~~Data Tables from Usability Study~~:	+	*We used the Morae software to perform the usability studies. More information about this software can be found at [http://www.techsmith.com/morae.asp http://www.techsmith.com/morae.asp].

-	~~[[Image~~:~~Data_Tables_from_Usability_Study~~.~~pdf]]~~	+	*Usability testing was completed in September 2010. The study consisted of asking participants to perform a total of 8 tasks, 4 within the Cabaniss collection (http://u0003_0000252), and 4 within a similar collection which was laid out in our "traditional" fashion (The Robert Jemison Jr. Papers Collection: http://acumen.lib.ua.edu/c/u0003_0000753 ).

-	Usability Study ~~Informed Consent~~:	+	'''Usability Study Questions:'''

-	http://www.lib.ua.edu/wiki/digcoll/images/1/1c/~~Informed_Consent_Cabaniss~~.docx	+	http://www.lib.ua.edu/wiki/digcoll/images/e/e4/Usability_Test_Questions.docx

		+	After the participants performed these 8 tasks, they were asked to fill out a survey which gave them the opportunity to give us feedback on their experience and their opinions about the interfaces in general and the Cabaniss collection in particular.
		+
		+
		+	'''Short Discussion on Results'''
		+
		+
		+	The usability study and its results are detailed in the an article (currently submitted to JCDL 2011) entitled "Low Cost Digitization of Manuscript Collections: Trading Usability for Access?"
		+
		+
		+	One of the more interesting findings was that participants without previous digital collection experience found the finding aid interface significantly easier than those who claimed familiarity with the traditional digital library interface.
		+
		+
		+	Educational experience clearly improved ability for participants to locate content via the finding aid, as did a background in history or familiarity with special collections.
		+
		+
		+	Not surprisingly, non-native speakers had difficulty with both interfaces.
		+
		+
		+	There was very little difference in the apparent learnability of the two interfaces, implying that the finding aid interface ''may'' be as learnable as the usual digital library interface (based on item-level description searching). We determined, however, that a true test of learnability should cover a longer period of time and far more queries.
		+
		+
		+	The question arose as to where the boundary is between testing the interfaces, and testing the delivered information?
		+
		+
		+	The majority of (primarily undergraduate) participants preferred the item-level description interface. However, 40% of those with a background in history, and a third of those with a background in special collections, preferred the finding aid interface.
		+
		+
		+	(Previous research has shown that the finding aid as interface to digitized manuscript collections is preferred by scholars and experienced researchers, whereas undergraduates prefer item-level description. Given that our study focused on undergraduates, our results are not surprising.)
		+
		+
		+	We also decided that it is truly not fair to compare the finding aid interface to digital content against the item-level interface for a short-term test. The finding aid presents far more information and context, requiring more time to read and become acquainted with the organization of the content. Once acquainted with a collection, however, we expect that locating appropriate content would be as speedy (or more so) than via searching item level metadata which may or may not contain the necessary search terms.
		+
		+
		+	Research has indicated that presentation of digitized content via the finding aid is indeed the preferred delivery method for scholars. Since this is also by far the cheaper method for digitization and web delivery, we believe this is by far the preferred path for future digitization of large manuscript collections.
		+
		+
		+	'''Data Tables from Usability Study:'''
		+	(Note: Kevin W. Walker performed the statistical analyses of the results.)
		+
		+	http://www.lib.ua.edu/wiki/digcoll/images/3/33/Data_Tables_from_Usability_Study.pdf

-	Usability Study Questions:

-	http://www.lib.ua.edu/wiki/digcoll/images/e/e4/Usability_Test_Questions.docx


-	Usability Study Flier and FAQs:

-	http://www.lib.ua.edu/wiki/digcoll/index.php/Image:Usability_Flyer_w_FAQs.pdf


-	~~''Updated September 7, 2010.''~~	+	[[User:Jlderidder\|Jlderidder]]

Jlderidder at 19:35, 4 March 2011

Jlderidder — Fri, 04 Mar 2011 19:35:01 GMT

←Older revision		Revision as of 19:35, 4 March 2011
Line 9:		Line 9:
	*Usability testing was completed in September and analysis is currently in progress. We hope to have the results available early in 2011.		*Usability testing was completed in September and analysis is currently in progress. We hope to have the results available early in 2011.

		+	Data Tables from Usability Study:
		+
		+	[[Image:Data_Tables_from_Usability_Study.pdf]]

	Usability Study Informed Consent:		Usability Study Informed Consent:

Jlderidder at 16:29, 3 January 2011

Jlderidder — Mon, 03 Jan 2011 16:29:51 GMT

←Older revision		Revision as of 16:29, 3 January 2011
Line 3:		Line 3:
	*Advertising began on August 6 and ran through August 30, 2010. Ads were placed on the University Libraries main web page, as well as on the student information portal called MyBama and on the Digital Collections web page. In addition, information was emailed to the History, American Studies, and Political Science Departments. These departments forwarded the information to their graduate students and faculty members. Fliers were also displayed in the reading room, and those work work at the reference desk were told about the study and were asked to encourage patrons to take a flier.		*Advertising began on August 6 and ran through August 30, 2010. Ads were placed on the University Libraries main web page, as well as on the student information portal called MyBama and on the Digital Collections web page. In addition, information was emailed to the History, American Studies, and Political Science Departments. These departments forwarded the information to their graduate students and faculty members. Fliers were also displayed in the reading room, and those work work at the reference desk were told about the study and were asked to encourage patrons to take a flier.

-	*Response to the advertising was fairly positive. We received more volunteers than we needed, but all volunteers were contacted and invited to come and participate in the study, in hopes that enough volunteers would participate to reach our goal of 20 participants. We recruited nearly 30 volunteers before taking down the ads~~. If we are unable to get 20 of those volunteers to participate~~, ~~we may need~~ to ~~recruit~~ a second ~~round of~~ participants.	+	*Response to the advertising was fairly positive. We received more volunteers than we needed, but all volunteers were contacted and invited to come and participate in the study, in hopes that enough volunteers would participate to reach our goal of 20 participants. We recruited nearly 30 volunteers before taking down the ads, but due to no-shows, recruited a second time. We finally tested 21 participants, though one set of results was disqualified from analysis due to unwillingness to complete the test.

	*We are using the Morae software to perform the usability studies. More information about this software can be found at [http://www.techsmith.com/morae.asp http://www.techsmith.com/morae.asp].		*We are using the Morae software to perform the usability studies. More information about this software can be found at [http://www.techsmith.com/morae.asp http://www.techsmith.com/morae.asp].

-	*Usability testing is currently in progress. We hope to have ~~testing completed by the end of September. At that time we will begin to compile and analyze~~ the results.	+	*Usability testing was completed in September and analysis is currently in progress. We hope to have the results available early in 2011.

Scpstu32 at 20:58, 7 September 2010

Scpstu32 — Tue, 07 Sep 2010 20:58:16 GMT

←Older revision		Revision as of 20:58, 7 September 2010
Line 1:		Line 1:
-	IRB approval for the Cabaniss usability study was obtained in May 2010.	+	*IRB approval for the Cabaniss usability study was obtained in May 2010.

-	Advertising began on August 6 and ran through August 30. Ads were placed on the University Libraries main web page, as well as on the student information portal called MyBama and on the Digital Collections web page. In addition, information was emailed to the History, American Studies, and Political Science Departments. These departments forwarded the information to their graduate students and faculty members. Fliers were also displayed in the reading room, and those work work at the reference desk were told about the study and were asked to encourage patrons to take a flier.	+	*Advertising began on August 6 and ran through August 30, 2010. Ads were placed on the University Libraries main web page, as well as on the student information portal called MyBama and on the Digital Collections web page. In addition, information was emailed to the History, American Studies, and Political Science Departments. These departments forwarded the information to their graduate students and faculty members. Fliers were also displayed in the reading room, and those work work at the reference desk were told about the study and were asked to encourage patrons to take a flier.

-	~~The response~~ was fairly positive. We received more volunteers than we needed, but all volunteers were contacted and invited to come and participate in the study, in hopes that enough volunteers would participate to reach our goal of 20 participants. We recruited nearly 30 volunteers before taking down the ads. If we are unable to get 20 of those volunteers to participate, we may need to recruit a second round of participants.	+	*Response to the advertising was fairly positive. We received more volunteers than we needed, but all volunteers were contacted and invited to come and participate in the study, in hopes that enough volunteers would participate to reach our goal of 20 participants. We recruited nearly 30 volunteers before taking down the ads. If we are unable to get 20 of those volunteers to participate, we may need to recruit a second round of participants.

-	We are using the Morae software to perform the usability studies. More information about this software can be found at [http://www.techsmith.com/morae.asp http://www.techsmith.com/morae.asp].	+	*We are using the Morae software to perform the usability studies. More information about this software can be found at [http://www.techsmith.com/morae.asp http://www.techsmith.com/morae.asp].

-	Usability testing is currently in progress. We hope to have testing completed by the end of September. At that time we will begin to compile and analyze the results.	+	*Usability testing is currently in progress. We hope to have testing completed by the end of September. At that time we will begin to compile and analyze the results.


-	Usability ~~Test~~ Informed Consent:	+	Usability Study Informed Consent:

	http://www.lib.ua.edu/wiki/digcoll/images/1/1c/Informed_Consent_Cabaniss.docx		http://www.lib.ua.edu/wiki/digcoll/images/1/1c/Informed_Consent_Cabaniss.docx


-	Usability ~~Test~~ Questions:	+	Usability Study Questions:

	http://www.lib.ua.edu/wiki/digcoll/images/e/e4/Usability_Test_Questions.docx		http://www.lib.ua.edu/wiki/digcoll/images/e/e4/Usability_Test_Questions.docx
		+
		+
		+	Usability Study Flier and FAQs:
		+
		+	http://www.lib.ua.edu/wiki/digcoll/index.php/Image:Usability_Flyer_w_FAQs.pdf
		+
		+
		+	''Updated September 7, 2010.''

Scpstu32 at 20:52, 7 September 2010

Scpstu32 — Tue, 07 Sep 2010 20:52:56 GMT

←Older revision		Revision as of 20:52, 7 September 2010
Line 5:		Line 5:
	The response was fairly positive. We received more volunteers than we needed, but all volunteers were contacted and invited to come and participate in the study, in hopes that enough volunteers would participate to reach our goal of 20 participants. We recruited nearly 30 volunteers before taking down the ads. If we are unable to get 20 of those volunteers to participate, we may need to recruit a second round of participants.		The response was fairly positive. We received more volunteers than we needed, but all volunteers were contacted and invited to come and participate in the study, in hopes that enough volunteers would participate to reach our goal of 20 participants. We recruited nearly 30 volunteers before taking down the ads. If we are unable to get 20 of those volunteers to participate, we may need to recruit a second round of participants.

-	We are using the Morae software to perform the usability studies. More information about this software can be found at [http://www.techsmith.com/morae.asp].	+	We are using the Morae software to perform the usability studies. More information about this software can be found at [http://www.techsmith.com/morae.asp http://www.techsmith.com/morae.asp].

	Usability testing is currently in progress. We hope to have testing completed by the end of September. At that time we will begin to compile and analyze the results.		Usability testing is currently in progress. We hope to have testing completed by the end of September. At that time we will begin to compile and analyze the results.

Scpstu32 at 20:51, 7 September 2010

Scpstu32 — Tue, 07 Sep 2010 20:51:01 GMT

←Older revision		Revision as of 20:51, 7 September 2010
Line 1:		Line 1:
-	IRB approval for the Cabaniss usability study was obtained in May 2010. ~~The following decisions have been made in how to proceed in planning the study:~~	+	IRB approval for the Cabaniss usability study was obtained in May 2010.

-	*We will start advertising via the ~~website for~~ the ~~study this summer. Jason Battles will be consulted~~ on ~~how this advertisement should appear~~.	+	Advertising began on August 6 and ran through August 30. Ads were placed on the University Libraries main web page, as well as on the student information portal called MyBama and on the Digital Collections web page. In addition, information was emailed to the History, American Studies, and Political Science Departments. These departments forwarded the information to their graduate students and faculty members. Fliers were also displayed in the reading room, and those work work at the reference desk were told about the study and were asked to encourage patrons to take a flier.

-	*In late August when classes start back, we ~~will advertise via listservs~~ and ~~on bulletin boards~~ in the ~~the History~~, ~~American Studies, and Political Science departments~~. ~~This will require Amanda~~ to ~~gather information from departmental secretaries in early August~~.	+	The response was fairly positive. We received more volunteers than we needed, but all volunteers were contacted and invited to come and participate in the study, in hopes that enough volunteers would participate to reach our goal of 20 participants. We recruited nearly 30 volunteers before taking down the ads. If we are unable to get 20 of those volunteers to participate, we may need to recruit a second round of participants.

-	*Clark Center was consulted about how to ~~best advertise in~~ the ~~Reading Room~~. ~~He suggested displaying the fliers in the middle of each table in the Reading Room as well as providing a list of FAQs to hand out to interested parties~~. ~~These FAQs should answer most questions that people have, but those working at the reference desk will at least need to be made aware of the study~~.	+	We are using the Morae software to perform the usability studies. More information about this software can be found at [http://www.techsmith.com/morae.asp].

-	*Our goal is to have ~~all subjects tested before mid-term exams which usually occur around~~ the ~~first~~ of ~~October~~.	+	Usability testing is currently in progress. We hope to have testing completed by the end of September. At that time we will begin to compile and analyze the results.

Scpstu32 at 18:43, 11 May 2010

Scpstu32 — Tue, 11 May 2010 18:43:49 GMT

←Older revision		Revision as of 18:43, 11 May 2010
Line 8:		Line 8:

	*Our goal is to have all subjects tested before mid-term exams which usually occur around the first of October.		*Our goal is to have all subjects tested before mid-term exams which usually occur around the first of October.
		+

	Usability Test Informed Consent:		Usability Test Informed Consent:

Scpstu32 at 18:43, 11 May 2010

Scpstu32 — Tue, 11 May 2010 18:43:41 GMT

←Older revision		Revision as of 18:43, 11 May 2010
Line 8:		Line 8:

	*Our goal is to have all subjects tested before mid-term exams which usually occur around the first of October.		*Our goal is to have all subjects tested before mid-term exams which usually occur around the first of October.
		+
		+	Usability Test Informed Consent:
		+
		+	http://www.lib.ua.edu/wiki/digcoll/images/1/1c/Informed_Consent_Cabaniss.docx
		+
		+
		+	Usability Test Questions:

	http://www.lib.ua.edu/wiki/digcoll/images/e/e4/Usability_Test_Questions.docx		http://www.lib.ua.edu/wiki/digcoll/images/e/e4/Usability_Test_Questions.docx